The point of deep learning theory (and this blog)
What’s the point of doing deep learning theory in 2026? Was there even a point to doing it before language models blew up (e.g., in the era of ResNets and CIFAR-10)? Has theory actually contributed anything to practice? If I were to jump into deep learning theory right now, where should I start? Does {OpenAI, Anthropic, Google, xAI, etc} care about theory? Is this skillset worth learning during a PhD?
These are all questions that I’ve received and grappled with as I have had to argue my case on the faculty job market and, later on, set my research agenda to build a group. I don’t know the answers to these questions, but I have come to understand that they’re dramatically underspecified. So I’m starting this blog without a thesis and just hoping that thinking/writing out loud will be useful to me and to others.
Some posts will be about papers or ideas. Some will be about concepts from philosophy of science that seem useful for thinking about the role of theory. Some will be about examples where theory seems to have actually guided or refined practice, and others where theory seems to have mostly organized something practitioners already knew. I’ll also ask other people to write posts on these topics.
