Michael Nielsen has an excellent post today: Three myths about scientific peer review. The myths are:
- Scientists have always used peer review: it seems it became widespread only during the second half of the XXth century.
- Peer review is reliable. In fact, we know litte about the reliability of the current peer review system. How do you know how many innovations were suppressed by peer review? Any experienced scientist knows that work rejected by a peer-reviewed journal can be of high importance and high quality.
- Peer review is the way we determine what’s right and wrong in science. Of course, that is very wrong. I routinely rely on work which appeared on arxiv, whether it has been peer reviewed or not. I also routinely ignore or dismiss peer-reviewed papers—often because I determine the paper to be wrong.
If you read the reactions to Michael’s post, you find Peter Turney’s response:
I’m sympathetic to much of what you’re saying, but, on the other hand, I know that peer review has immensely improved many of my own papers.
My take: I entirely agree with Peter Turney. Many of my papers benefited tremendously from peer reviewed. I could not imagine doing my work without getting feedback from my peers! However, peer review is an honor-based system. It is meant to help the authors get a sense for the value of their work, and help them improve. You can cheat the system. And people do it routinely to get promotions and jobs.
I just (a couple of days ago) posted a similar article regarding criticism of the peer review process here:
Skeptics: Be Skeptical of Skeptics!
http://ithinktherefore.org/philosophy/skeptics-be-skeptical-of-skeptics/
It’s also linked in the Reddit science section. Please feel free to link, track, comment, etc. I’d love to have some peer review myself! 😉
In short, I’m concerned about tenacious adherence to the “knowledge” published through the peer-review process, seeing as how it can often be a political process, rather than a democratic or scientific one.
Peer review is very much like democracy. The majority gets to vote down an idea even though it might be useful. Publishing in a peer reviewed journal is like a bunch of senators deciding on a policy issue. With the internet we can perhaps make is a bit closer to general elections. 🙂
NEVER criticise Society. Only people who can’t get into it do that.
I agree with your views in that peer review allows others to provide feedback to the author. I have often sought such comments from others, whether it’s teaching, learning or research. By having an open model of review, one could unfold the “blind spot” that is often associated with research.
Thanks again for sharing this important message.
John