Overconfident individuals often win by claiming more resources than they could defend (Johnson and Fowler). If nobody knows who is strongest, whoever thinks he is the strongest might win by default. That is, there is no better way to fool others than to first fool yourself.
Accordingly, human beings are often overconfident:
- The teachers know that lecturing is highly effective (even when it is not).
- Sex partners know that they are great lovers (even when they are not).
- Star scientists know that they are more often right than others (even if that’s not true).
- Economists know how they economy work (even when they don’t).
- The entrepreneur knows his new business will succeed (even when most fails).
In scholarship, overconfidence is a great asset. I routinely fool myself into thinking that I can make a lasting contribution with my next research projects. By being overconfident, I take risks that I would not take otherwise. If I were more cautious, I would go work in the financial industry, make a lot of money and retire early. Instead, I keep thinking that my next research project might advance Science.
Thus, scientists often develop a God complex:Â they somehow know that whatever theories they hold must be true. They also believe that they can accurately assess the significance of the work of others. This is not entirely conscious, of course. They know that it is impossible to make fair assessments. Yet they fool themselves long enough to believe that their assessments are largely correct (even when they are not).
This is somewhat ironic in that Science’s ultimate purpose is to tame our God complex. Yet the very people who pursue science have gigantic egos. They would never admit that Science is built on trial and error. They want you to believe that Science is driven by superior intellects. How can such pompous people be in charge of moderating our God complex? Because while the scientist might be driven by overconfidence, his tools keep him honest. The scientist is like a high speed train: he might have a huge engine, he is (mostly) forced to follow tracks. A scientist operating entirely based on his inflated ego will eventually be derailed.
The God complex leads us straight to the conventional peer review system: write a paper, send it to a journal where a handful of review will assess it. Can reviewers really tell whether the work is significant? Of course, they can’t! But they believe so, for a time. The scientific way is to try and verify. That is how, ultimately, scientists deal with their God complex. Hence all (apparently) correct research papers should be published. Work that has stood the test of time should form the foundation of a field, not work that has been selected by (false) Gods.
But surely, scientists cannot try all possible theories? They must someone select the most promising directions. However, once we acknowledge the God complex, some natural strategies emerge:
- We should reject central planning. We must reject the authority of the few to decide the worthwhile research directions. We should choose to support many small projects rather than fund one large initiative driven by a handful of individuals. Results matter, not authority.
- We should test as many different ideas as we can. Scientists should be encouraged to try many different things.
If you think that these strategies sound sensible, consider that they are almost entirely the opposite of what is being done:
- Science is increasingly planned centrally around large government bureaucracies. We want to build scientific heroes who will direct other (lesser) scientists. Large teams focus on a narrow set of problems and directed by experienced researchers are the gold standard. Independent thinkers who try different things “waste resources”.
- We encourage specialization. One argument which is often raised is that if a professor tries his hand at a new topic, he will be unable to train new Ph.D.s in this new area, thus limiting our ability to produce more Ph.D.s. It is also frowned upon to try an idea, see it fail, and write up the results as a research report: every project should lead to a publication in a respected venue.
By analogy between Science and capitalism, it is as if the government was funding large corporations at the expense of small shops. It is as if the government funded companies that are highly specific and inflexible at the expense of agile ventures because more stable companies can better train their employees. And, more importantly, it is as if the fate of start-up companies was determined by small secret committees of experts: unless such a committee concludes that the company will work, it cannot launch. It is as if we believed it best to direct the consumer to a few trusted companies, for fear that he might choose to deal with a young unpolished start-up.
In effect, the modern scientist has rejected the free market of ideas. He prefers state-directed communism where a few choose for the masses.
Wikipedia opportunity: There is no entry in Wikipedia for Garage science where independent researchers do bona fide research with modest means.
Further reading: The perils of filter-then-publish and Improve your impact with abundance-based design.
Update: The original title of this post was “scientists are communists”.
Love the idea of garage science. Hard to publish that way though. You would have to be happy with other ways of having an impact, I’d think.
@Greg
All calculus students learn about “Green’s theorem”. Ever wondered who was Green?
He was very much a garage scientist:
http://en.wikipedia.org/wiki/George_Green
(Though they did not have garages back then.)
OK, you asked for it: I will disagree.
In a great paper, Michael Mulkay talked about three models of science. The open model is the mythical “science is open”. The closed model is much closer to what you describe, essentially the Kuhnian model where science gets stuck, followed by a crisis and a revolution.
Mulkay’s third model is the interesting one: science evolves by branching. People move between disciplines, and this is where the interesting stuff happens. There’s a lot to be said for this: many classic paradigm shifts (in a Kuhnian sense) were really ideas from one field transforming another. For Mulkay, these scientific communities were relatively small (a hundred or so people at most) working in a focused area. The classic disciplines of physics, etc., are higher-level constructs, but not really scientific communities. Most funding actually happens in these smaller fields.
But of course, there isn’t a free market, quite. Just as there isn’t for, say, kitchen appliances. Some appliances are popular, other less so. Some satisfy niche demands. There are big providers who try to do everything, and many smaller ones who don’t. The modern scientist is just trying to make a contribution, and in doing so, yes, is likely to overvalue the contribution they can make. But they are generally happy to compete within the bounds of the fields they know.
@Stuart
I embrace the idea of branching by a small group of people. But the competition scientists currently favor is nothing like free market competition. It is more like the competition between bureaucrats. The main goal is to climb higher in the hierarchy.
There are numerous issues with you argument. The major issue being that Peer Review actually prevents the God Complex and Halo effect. The fact that no matter how influential you might be in your field, your work wont be accepted unless it is passed by an anonymous panel of reviewers, actually prevents the God Complex.
If we had a completely open system, similar to say blogging, people would just arbitrarily follow the leader. Also the reason people flock to prestigious journals or conferences because everyone cannot read all the literature even in a relatively small area, the peer review process provides a good mechanism to rank the literature.
Finally you wish that there should be fewer “Star” scientists directing lesser ones rather you want more independent scientists. There are several issues with your argument. The first one being what is the distribution of the research skill in the population who want to be scientists? While you might like it to be near uniform, it might not necessarily be the case.
Finally consider literature, even though blogging has made sure that anyone with access to internet can access a large number of audience, yet it has not lead to increase in quality of literature being published.
@Bhat
All good points. Please keep in mind that my concerns had to do with two things: central planning, and discouraging diversity. Thus, I don’t necessarily disagree with the points you raise. They don’t contradict my post (in my view). Let me make my point of view precise.
The major issue being that Peer Review actually prevents the God Complex and Halo effect.
Realize that what you just did by leaving a comment is effectively peer review. And sure enough, I can write whatever I want on my blog (thinking of myself as a God), but people like you will come and tear apart my theories.
So, you are absolutely correct, peer review is a great way to tame the God complex. If you have been reading this blog, you know that I am very keen on peer review. However, I don’t believe that a small set of secret reviewers can assess accurately the long term significance of a piece of work. Thinking so is falling prey to the God complex.
What I am saying is not so controversial. Journals like PLoS One do not ask reviewers to assess the long term significance of the papers: they only ask whether the paper is correct.
What I oppose is what I call “conventional peer review” where reviewers are asked to guess whether the work is important.
(…) everyone cannot read all the literature even in a relatively small area, the peer review process provides a good mechanism to rank the literature.
Oddly enough people never complain that we lack a way to rank blogs.
Anyhow, if your filters are based on the journals, then you are probably overwhelmed. In fact, you have probably been overwhelmed since before WWII.
Most people I know have far more sophisticated filters. They follow specific individuals, citation networks and keywords.
I rely exclusively on Google Scholar, and it does not filter based on journals. I have no problem narrowing down my search. I often have the opposite difficulty: I often miss important related work because it appeared in venues I don’t know, written by authors I don’t know, using keywords I don’t know.
This is not to say that curated list of papers such as those generated by journals have no value. They do. But while they were essential in the pre-Internet era, they are now only one signal out of many. And maybe not the most important signal.
Some of the best work I have ever read was presented in small workshops. In fact, it is quite common for people to notice that their most cited work appeared in minor venues. It is certainly my case: there seems to be an inverse relationship between the impact of my work (as measured by citations) and the prestige of the venue. (Before you say that this is impossible, look at the distribution of citations among papers in prestigious journals. You will find that most of them are rarely cited.)
The first one being what is the distribution of the research skill in the population who want to be scientists? While you might like it to be near uniform, it might not necessarily be the case.
I’m not assuming that the productivity of scientists is uniform. In fact, there is a substantial statistical dispersion. However, we do not know how much of this dispersion is due to the system and how much is due to innate skills.
I believe we greatly overestimate the effect of innate skills. Consider that in the early XXth century, German scientists had sometimes a substantial edge over other scientists. Then after WWII, American scientists dominated. It is quite obvious that the “great scientists” somehow appear whenever a country becomes powerful. Strange, no? Clearly, it is not the genetics that changes… but the working conditions. Good working conditions tend to generate great scientists.
There is a Matthew effect at work. By trying to select the most productive people and giving them more resources, you create a self-fulfilling prophecy. It is unclear whether this form of attribution of resources is optimal. Do you get more science by giving one million dollars to one man, or 100 thousand dollars to ten men? The answer is maybe not obvious.
@Phil
The fragmentation is nothing new. Physics has many interpretations of quantum mechanics, that’s not so very different from different interpretations of climate data. The main change is a huge transformation in media and communication, which has opened all this up to external scrutiny, and which often has a political or economic agenda it uses to selectively draw on scientific results.
However, as scientists, we still have to adhere to basic rules. Creationism is not a science — and this has been tested legally. Conspiracy theories are not scientifically testable. (Perpetual motion: more interesting as the laws of thermodynamics are descriptive, not causal, so research on *why* entropy increases could actually be interesting.)
I think you nicely demonstrate why more extreme versions of relativism are basically useless.
Even in a free market, products have to be useful. Without that, they will fail. Novelty items can exist, but are inevitably marginal. I’d consider conspiracy theories et al. as novelty items, some people will buy them, they can be fun, but you’d never use one to cook your dinner.
The interesting thought is that the free market tends towards pluralism. As in more competing niche products.
And the implication for “science” is that a truly free market of ideas (eg. the internet) would similarly lead to fragmentation, large numbers of niches of people who believe their own incompatible theories, each of which is backed by its own canon of evidence, that its community considers trustworthy.
I think we’re seeing the beginnings of this with the proliferation of global warming denialists, perpetual motion machinists, creationists, conspiracy theorists etc. And fragmentation will continue.
Lurking behind our idea of “Science” is a picture of a consistent nature which acts as arbitrator of such disputes. But the better picture of the future might be the blind-men analysing an elephant : multiple perspectives based on small windows of observation, failing to grok an overall coherent picture.
In this sense, a “free market” of science might actually spell the end of anything that we could recognise as science at all. Leaving us with nothing but a multitude of local belief systems.
I take it that you subscribe to an “Evolutionary Epistemology of Theories” approach to understanding how science works (or ought to work.) see:
http://plato.stanford.edu/entries/epistemology-evolutionary/
There may be a process of Natural Selection that goes on (or ought to be going on), but I’m not sure that the “truth of the way things are” is a fitness function. It might have been in the 17th C. but I don’t think it is now. “Is there a revenue stream from this idea” or “can I patent it” seems to be the modus operandi now.
But your point that the Stalinist 5-year plan isn’t any good at *that* either is well-taken.