We like to have an idealized view of the scientist. He or she is someone who chose against a high-paying career to pursue the ideals of science and academia. Unlike the greedy engineer or entrepreneur, he works not for himself, but for the greater good.
We often hold similar views of politicians, journalists and medical doctors. Politicians went into politics “to make a difference”. Journalists aim to expose the truth. All medical doctors, of course, chose their profession to “help others”.
As a model, the disinterested individual is a poor one. In fact, an entire academic school is based around the realization that politicians and civil servants seek to advance their own interests: public choice theory. And despite having entire TV shows dedicated to exposing this truth, most people have a hard time thinking of politicians as self-interested. But they are, of course!
Scientific research is even more problematic because most people cannot even imagine what a researcher does. However, once you start thinking about scientists as greedy for status, many things make sense. If scientists were primarily motivated by the public good, they would make their research papers accessible and they would carefully publish the data and software. There would be no need for funding agencies to try to compel scientists to share.
In practice, that is not what you find. Scientists are often silently pleased when their results are difficult for others to reproduce:
The people who dominate both the natural and social sciences primarily think in terms of reputation and career. They think that the point of making a scientific discovery is to (…) further your career. (…) they lament the additional year it’ll take just to confirm that something is true.
[They are] irritated with the idea that ordinary scientists should be able to replicate easily, several times, what they’ve done once, for the first time in human history. (source: Andrew Gelman’s blog)
Of course, being self-interested does not make you evil. A lot of scientists, in their quest for the Nobel prize, have made tremendously important advances.
Still, we cannot count on scientists to police themselves fairly. That some researcher is highly regarded among his peers might simply mean that he contributed greatly to the advancement of these peers… To illustrate this point, consider Marc Hauser. He was a star Harvard professor who earned high praises from his peers (including Peter Singer and Noam Chomsky). He only fell from grace when his research assistants (who were mere undergraduate students) challenged him. As it turns out, much of Hauser’s career was based on fabricated science.
How do we keep politicians in check? By limiting their power. That’s why we have constitutions. We don’t let other politicians be the judge of politicians: we ask independent experts (judges) to assess their actions.
Scientists similarly need to be assessed by independent experts, and it cannot be their immediate peers. You cannot trust Peter Singer and Noam Chomsky to tell you whether Marc Hauser is doing solid work. (They falsely told us that he was!)
In computer science, we have these independent experts: for every computer scientist, there are hundreds of highly qualified engineers who can put the science in practice.
Want to know whether the science is any good? Ask someone who independently put it in practice.
If I had my way, funding agencies would systematically rely on practitioners to assess the proposals. Sadly, that is not how it works.