15 January 2025 · 2 min
Almost all of academic science has moved away from actual (empirical) science. It is higher status to work on theories and models. I believe that it is closely related to well documented scientific stagnation as theory is often ultimately sterile.
This tendency is quite natural in academia if there is no outside pressure… And is the main reason why academia should be ruthlessly judged by practitioners and users. As soon as academia can isolate itself in a bubble, it is bound to degrade.
It is worth trying to understand some of the factors driving this degradation… Theoretical work can sometimes be seen as more complex. This complexity can be mistakenly equated with higher intelligence or prestige. Empirical work, while also complex, often deals with tangible, observable data, which might seem more straightforward to the uninitiated.
Empirical work is more likely to lead to nuanced or inconclusive results while theory is often seemingly more direct and definitive. Theoretical research often requires fewer resources than large-scale empirical studies which might need extensive funding for equipment, data collection, and personnel. Thus you get to do more research with less using models and theory.
Theoretical work is often seen as requiring a high level of creativity to devise new frameworks or models. While empirical work also requires creativity in design, execution, and interpretation, the creativity in data collection or experimental design might be less recognized or appreciated.
The educational system often glorifies theoretical knowledge over practical skills until one reaches higher education or specialized training. E.g., we eagerly make calculus compulsory even if it has modest relevance in most practical fields. This educational bias can carry over into professional work.
We still have laboratories. We still collect data, but it is too often in service of theory rather than a primary activity.
Society must demand actual results. We must reject work that is said ‘to improve our understanding’ or ‘to lay a foundation for further work’. We must demand cheaper rockets, cures for cancer, software that is efficient. As long as academic researchers are left to their own devices, they will continue to fill the minds of the young with unnecessary models. They must be held accountable.
Daniel Lemire, "The ivory tower’s drift: how academia’s preference for theory over empiricism fuels scientific stagnation," in Daniel Lemire's blog, January 15, 2025, https://lemire.me/blog/2025/01/15/the-ivory-towers-drift-how-academias-preference-for-theory-over-empiricism-fuels-scientific-stagnation/.
[BibTeX]
Is there a field or sub-field you’re specifically worried about here?
I mean, and Biology majors all have backgrounds in stats and wet lab work, and an economics student can’t even smell a PhD without heavy work in econometrics.
We are in the middle of a decade-long stagnation where, despite an exponential increase in funding, scientific fields are producing less and less. Emblematic of this trend are the 2024 Nobel prizes where they could not even find a credible ‘Physics’ advance. Instead, they rewarded purely theoretical work from artificial intelligence.
Lee Smolin in The Trouble with Physics: The Rise of String Theory, the Fall of a Science makes the point clearly:
Though it is true that high school students spend time in a room that looks like a science laboratory, their intellectual work there is perfunctory.
Yes, there is data crunching and ‘experiments’ but it is shallow and disconnected.
The BBC in 2017 summed it up…
Science is facing a “reproducibility crisis” where more than two-thirds of researchers have tried and failed to reproduce another scientist’s experiments, research suggests.
I know many econ PhDs which were not econometrics. But I don’t think there lies a worry with economics.
In economics, the ‘too theoretical’ problem is that: Even in econ 101 we already start with teaching market models that are so remote from the actual market & market questions that one almost has to forgive the careless student if she treats much of it as abstract theory instead of anything they could ever link to in their day to day life.
I increasingly appreciate the superiority of contact with reality over clever theorizing. (It has been a hard lesson. I am an intellectual by instinct and still have to use conscious force of will to make prototypes instead of building castles.)
But in the best case, theoretical and applied work inform and drive each other forward. We need both, and we need them on speaking terms.
In AI I’d argue the opposite is true.
The majority of published works is crap, because it’s just doing yet another minor variant on the same data again and again. And by million monkeys and if you are not very stringent about not overfitting, eventually some result by chance looks good if you try enough random seeds. And because all the reviewers care about are the benchmark tables, the crap gets published. But is completely not reproducible or transferable to other data sets or random seeds. None of this will have a lasting impact.
We have way too few theoretical advances.
AI research as of late tends to be grounded in practice. Furthermore, AI accounts for a large fraction of all the recent scientific progress.
But only a small sunset. The majority is crap science to produce papers.