Most scientific papers are probably wrong

According to this New Scientist article, most scientific papers are probably wrong:

John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false.

In my opinion, there is no question that the weak review process most conferences enjoy and the pressure to publish lots of new results, as opposed to verifying what others have done, contribute to this. In my experience, very few people will ever try to reproduce your results, very few people will check your proofs, very few people will reimplement your algorithms.

A few years ago, I tried to publish an errata to one of my paper. It took some doing since the editors had never received such a request. Quite clearly, our tolerance for unaccurate or wrong results is quite high.

Is it a problem? If you are an engineer trying to implement a new system, and you are naïve enough to trust a few scientific papers, then you can get in big trouble. It usually isn’t much of a problem because the average engineer doesn’t read cutting edge scientific papers. It seems obvious that the problem can get more serious if medical doctors read cutting edge articles… do they?

Published by

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).

Leave a Reply

Your email address will not be published.

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks: 

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax

Or, you can subscribe without commenting.

You may subscribe to this blog by email.