Peer review is not the gold standard in science

Peer review as we know it today was introduced very late, over a century after the scientific revolution. It happened after Einstein’s time… arguably the most productive era in science. Current scientists often equate a success with the publication in a selective peer-reviewed venue. But that was never the scientific paradigm. In fact, it is pre-scientific thinking. Back in Einstein’s time, many scientists believed in the ether. It would have been difficult to dismiss the ether as a concept. The prudent approach would have been to pay lip service to the ether. Similarly, most scientists believed in eugenics. They believed in forced sterilization for the greater good. Many of the racist laws in the US followed straight from progressive science. Opposing eugenics would have been difficult in the context of peer review. It would have been difficult to challenge eugenics openly as a scientists.

Siler et al. (2014) looked at published manuscripts that were initially rejected. They find:

Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research.

Recently, people like Matt Ridley challenged the idea that the SARS-Cov2 virus originated from nature. Back when he published his book on the topic, it would have been difficult to pass peer review.

You may not remember, but early on, it would widely accepted that the lab origin of SARS-Cov2 was only for far-right conspiracy theorists. The Canadian State broadcaster (CBC) told us, in its ‘science’ section:

One of the most persistent and widespread pieces of disinformation during the COVID-19 pandemic has been the conspiracy theory that the novel coronavirus that causes the disease was created in a lab — and was let loose either by accident or on purpose by some nefarious actor.

In the US Senator Cotton suggested that thespread of a coronavirus is connected to research at the Wuhan institute of virology. In response, the Washington Post wrote:

Sen. Tom Cotton (R-Ark.) keeps repeating a coronavirus conspiracy theory that was already debunked. (…) In response to Cotton’s remarks, as well as in previous interviews with The Washington Post, numerous experts dismissed the possibility the coronavirus may be man-made.

Here is what one of the most reputed medical journal (The Lancet) published:

We stand together to strongly condemn conspiracy theories suggesting that COVID-19 does not have a natural origin.

The article omits the fact that the authors have glaring conflicts of interest (undisclosed).

Thacker describes some of the event in a piece for BMJ:

But the effort to brand serious consideration of a lab leak a “conspiracy theory” only ramped up. Filippa Lentzos, codirector of the Centre for Science and Security Studies at King’s College, London, told the Wall Street Journal, “Some of the scientists in this area very quickly closed ranks.” She added, “There were people that did not talk about this, because they feared for their careers. They feared for their grants.

Daszak had support. After he wrote an essay for the Guardian in June 2020 attacking the former head of MI6 for saying that the pandemic could have “started as an accident,” Jeremy Farrar, director of the Wellcome Trust [a major funder] and co-signer of the Lancet letter, promoted Daszak’s essay on Twitter, saying that Daszak was “always worth reading.”

Daszak’s behind-the-scenes role in orchestrating the statement in the Lancet came to light in November 2020 in emails obtained through freedom of information requests by the watchdog group US Right To Know.

“Please note that this statement will not have EcoHealth Alliance logo on it and will not be identifiable as coming from any one organization or person,” wrote Daszak in a February email, while sending around a draft of the statement for signatories. In another email, Daszak considered removing his name from the statement “so it has some distance from us and therefore doesn’t work in a counterproductive way.”

Several of the 27 scientists who signed the letter Daszak circulated did so using other professional affiliations and omitted reporting their ties to EcoHealth Alliance.

For Richard Ebright, professor of molecular biology at Rutgers University in New Jersey and a biosafety expert, scientific journals were complicit in helping to shout down any mention of a lab leak. “That means Nature, Science, and the Lancet,” he says. In recent months he and dozens of academics have signed several open letters rejecting conspiracy theory accusations and calling for an open investigation of the pandemic’s origins.

Efforts to characterise the lab leak scenario as unworthy of serious consideration were far reaching, sometimes affecting reporting that had first appeared well before the covid-19 pandemic. For example, in March 2020 Nature Medicine added an editor’s note (“Scientists believe that an animal is the most likely source of the coronavirus”) to a 2015 paper on the creation of a hybrid version of a SARS virus, co-written by Shi.

Here are the facts as we knew them back then… as anyone could know…

  • There was an outbreak caused by a bat sarbecovirus, in the one city in the world that had been collecting hundreds of bat sarbecoviruses and experimenting on them.
  • It happened one year after that lab proposed inserting the one feature that distinguishes SARS‑CoV‑2 from all other viruses.
  • The lab in question refuses to this day to release the database of the viruses it had been working on.
  • Virus leaks have been common.

It was always sensible to ask whether SARS-CoV-2 came from the Wuhan lab. Yet this was openly censored. As is often the case, instead of reflecting on this failure, many people rewrite history. “We never denied it could have come from a lab”, they say. “We never denied that it could have been human-made,” they say. But they very explicitly and strongly did so. They specifically and repeatedly said that this virus could not have been made in a laboratory:  yet a funding application to do exactly that, a few years before, had been submitted to the US government by Daszak, the very man who insisted that the lab origin was a conspiracy theory.

Of course, knowledgeable scientists knew that the lab origin was a possibility. They did not dare to speak up. Would you speak up when it could mean the end of your career?

This was not at all an isolated incident. Dr. Scott Atlas was censored by Stanford for questioning the covid dogma. The Stanford Review writes:

This censure, now a black mark on the University, was unquestionably motivated by political animosity. Atlas, a health policy expert who worked as a professor at Stanford Medical School for fourteen years, chose to serve his country by taking an advisory role in the Trump Administration’s White House Coronavirus Task Force. As an advisor, Atlas suggested reopening schools and businesses and pushed back against draconian lockdown policies.

You might answer… « Attacking people for getting closer to the truth isn’t new » But science seeks to address this very point. In fact, it is the very essence of the epistemology of science: the recognition that truth is not arrived by social consensus or by following the powerful. There are many ways to describe science, but to a first approximation…  Science is the process whereas anyone can post ideas and results for others to replicate, and everyone get to fail in public, and, hopefully correct themselves. Science the opposite of a gatekeeping process, it is, by its very nature, a progressive and open process.

It does not mean you should not use peer review publication. But you need to recognize that it is not the reference in science. Evidence is evidence. Consensus is not evidence.

Remember: ‘The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.’

Daniel Lemire, "Peer review is not the gold standard in science," in Daniel Lemire's blog, May 11, 2024.

Published by

Daniel Lemire

A computer science professor at the University of Quebec (TELUQ).

3 thoughts on “Peer review is not the gold standard in science”

  1. What do you think of open/blind peer review? From my point of view, it’s a shame sometimes even useful feedback comes so late in the process with little chance to improve. Of course you’ll be in a better position the next time you submit somewhere in case your topic still has relevance and you didn’t miss your slot to publish something novel.

  2. ‘The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.’

    Well I would say replace “the world” with “human society,” and the quote becomes true. But as written, I would rather say very, very few are reasonable, and it is on them that progress depends.

    Or at least I guess we should distinguish scientific progress form engineering progress. 😂

Leave a Reply

Your email address will not be published.

You may subscribe to this blog by email.