On the upcoming collapse of peer review

I just read Is Peer Review in Decline? (PDF warning!) by Glenn Ellison. Here is how the author starts:

Comparing the early 1990’s with the early 2000’s, there is a decline in the share of papers in the top general-interest journals (and the absolute number) written by faculty members from the Harvard economics department.

To follow the author, let us assume that the quality of the work at Harvard remains top-notch. If so, how do we explain this decline?

His general conclusion seems to be that there is fairly little to gain for a famous author in publishing in a top journal. Famous authors get citations and recognition no matter where they publish. They also get annoyed by the lengthier and more arduous peer review process.

He makes the following “prediction”:

The peer-review process may also be subject to unravelling: as more top economists withdraw from the process, the signal that publication
in a given journal provides is devalued and this may lead to further withdrawals.

Indeed, if this trends generalizes itself, if top authors stop playing the peer review game, by getting their papers through backdoors (invited papers, more specialized or lesser journals, and so on), why would the little guy put up with harsh peer review?

Disclaimer. I think peer review is absolutely essential and it is not going away. This blog, for example, is peer reviewed. If I write about something I do not know well, I can be sure to get comments criticizing my post. It happens regularly that someone will go out of his way and create another blog post to criticize my own. What is probably going away is the formal and lengthy process by which you go through to get a paper in a prestigious venue. Naturally, this will mean that your publication count will also become much less meaningful and you will have to find another way to justify the quality of your work. Seems like some top notch researchers are on top of this trend already.

See also my post Are we destroying research by evaluating it?

An upcoming revolution in science? The end of academic journals?

Adam Rogers makes a bold prediction:

Eventually, printed journal articles will be quaint artifacts. Scientific papers will be living documents with data published on Web pages – commented on, linked to, and mirrored by labs doing the same work 6,000 miles away. Every research effort will have thousands of reviewers working in real time. Today’s undergrads have never thought about the world any differently – they’ve never functioned without IM and Wikipedia and arXiv, and they’re going to demand different kinds of review for different kinds of papers. It’s in their nature.

(Source: Wired, September 2006)

I think that the science media of the future will be electronic and read/write. We will annotate, link, cross-reference more than ever before in the coming years. We are in the middle of a paradigm change.

I do not think that publishing houses need to go out of business. I do not think that traditional universities and research laboratories will disappear. However, everything is growing more distributed. Physical location and physical support is becoming irrelevant.

Actually, by 2015, I will not even bother to get up of bed. I will just sit there by my computer all day. Come to think of it, that is precisely what I do right now except that I don’t stay in bed.

Prestige is overrated?

Grigori Iakovlevitch Perelman proved the longstanding Poincaré conjecture and posted the solution on arXiv. One of the most difficult problems in Mathematics today. However, instead of publishing his work in a prestigious journal, he simply dropped it on an Internet archive. Maybe the Perelman story is meant to teach us something:

If your ideas are important enough and you get them out, people will pay attention to them, whether you publish in a high prestige peer-reviewed journal or not.

For more insights, see what Downes had to say.

Get an RSS feed of your favorite researcher

Want to monitor the publications of a researcher? As long as he submits his papers to arXiv.org and/or Cogprints, you can use citebase to get a RSS feed: enter the author’s name, do a search, then click on the RSS link. ArXiv also has RSS feeds if you are only interested in this particular repository.

Source: Peter Turney.

Further reading: See my earlier post on this topic.

Springer Online Mathematical Encyclopedia

The Springer Online Mathematical Encyclopedia is really very interesting. The quality is amazing.

The Online Encyclopaedia of Mathematics is the most up-to-date and comprehensive English-language graduate-level reference work in the field of mathematics today. This online edition comprises more than 8,000 entries and illuminates nearly 50,000 notions in mathematics. The Encyclopaedia of Mathematics is updated on a regular basis to remain a quick, precise source of reference to mathematical definitions, concepts, explanations, surveys, examples, terminology and methods, which will prove useful for all mathematicians and other scientists who encounter mathematics in their work.

Make your publications available as a RSS feed

A long time ago, I turned my publication list into an RSS feed. To my knowledge, I’m the first researcher to have done so. It came to me today that I should probably claim this innovation now before other people start doing it.

If everyone I care about did it, I could monitor publications in a researcher-centric way, not in a publisher-centric way. It would be much better for me, and much better for the researchers I care about.

SSHRC – National Consultation on Research Data Archiving, Management and Access Systems

One of the big founding councils in Canada, SSHRC, is looking at an open acces policy and Stephen Downes is asking us to voice our opinions.

If funding agencies start requiring open access, publishers could be in for a big change. They might be powerful compared to the average scientist trying to publish-or-perish, but the little scientist who must offer open access to his publications to keep his grant will suddenly grow some teeth.

Would I risk my federal grant for the whims of a publisher like IEEE? Never.

Once we have open access to most significant publications, the research game may change dramatically. In fields like Computer Science, it would make little difference, but I hear that philosophers are still very old fashioned about their publications.

Journals are already dead! Long live eprint servers!

For researchers who actually want to be read, there are several good eprints servers including arxiv.org (which I don’t use, but many physicists seem to like it) and cogprints (great for AI-related stuff). Of course, you can simply post your papers on your web site and let Google find them (my favorite solution).

On this topic, Suresh cites Cosmic Variance:

Most people these days post to the arxiv before they even send their paper to a journal, and some have stopped submitting to journals altogether. (I wish they all would, it would cut down on that annoying refereeing we all have to do.) And nobody actually reads the journals they serve exclusively as ways to verify that your work has passed peer review.

I think we are slowly getting at the point paper-based publications are going to be completly unecessary. Right now, people still ask me for page numbers when I say I published a given paper. I was even asked for photocopies of the journal issue. These people will soon die and we will be finally free to let the trees in the forest.