Peer review and its alternatives

Waiting, waiting …

You delivered your manuscript for peer review five months ago and nothing seems to be happening. The commissioning editor you originally dealt with has passed you on to an editorial assistant who is apologetic but no review reports have materialized. So, should you just sit there and take it, simply fretting? Start thumping the publisher’s table? Cut your losses and approach another publisher? Indeed, can you avoid this tiresome business altogether?

Unfortunately, as we shall see, peer review is a stage of the academic publication process that is hard to avoid (unless you are looking to self-publish your work, but that’s another story). Let’s take a stroll through this subject and find out why.

Peer review – the basics

Peer review is used in various areas outside publishing (e.g. by funding authorities to assess applications for research grants) but as far as academic publishing is concerned it performs a gatekeeper function – i.e. it is the process by which a book publisher or journal subjects a scholarly work intended for publication to the scrutiny of others who are experts in the same field.

In addition, peer review is supposed to encourage authors to meet the high standards of scholarship and conduct that are accepted in their disciplines.

Most publishers recruit two or more experts in the field to undertake this review (journals more commonly one). Among other things, these reviewers will be asked to make an evaluation of the text’s theoretical, methodological and empirical merits and a judgement of its literary style and readability. In addition to a general assessment of the text, they may also be requested to answer specific questions.

Normally, referees undertake the review on the basis of anonymity, but they may choose to sign their reports and even subsequently engage in a dialogue with the author. This can this lead to a far better book or article, and just as importantly in the former case can result in an endorsement from a well-known scholar that can be used to promote the book.

The peer review occurs at an awkward moment when the author is relatively powerless and hasn’t yet had a chance to build the sort of relationship with an editor that could protect the author from a reviewer in a rotten mood. But it is also a vitally significant moment. Perhaps the best way to look at the peer-review process is to liken it to the tempering of steel rather than as an ordeal by fire. Authors whose texts survive this process (and, in the case of books, most do) usually find that working closely with a good editor to incorporate reviewers’ advice and other editorial feedback into their text can be one of the most positive and productive aspects of creating a scholarly book/article.

Problems with peer review

While there is a lot going for peer review, it has its critics. Some scholars criticize it for being elitist, prone to bias, and overly slow. Certainly, there is a likelihood for elite scientists to be sought out as referees than less established ones, and for the process to support the dominant discourse, smother innovation and suppress politically incorrect arguments. And delays are common (though horror stories of journals taking five years to review an article probably attract such attention because the delay is so extraordinary).

Moreover, although peer review is generally considered essential to academic quality, it does not reliably prevent plagiarism or fraud, and indeed often fails to detect errors. On rare occasions, scandals involving outright fraud have struck even publications with the highest peer-review credentials.

The anonymity of reviewers is blamed for many of the problems of peer review. From the author’s point of view, an anonymous reviewer is in a position of great and unquestionable power while the author is utterly dependent on a good review. It is not unknown for reviewers to behave badly and quite common for them – by a process of ‘criticism creep’ – ending up becoming virtual co-authors of a work without having to reveal their identity (though in fairness this means they also don’t get any recognition for this input).

Alternatives to peer review

Some scholars (and journals) see the answer to these problems with peer review in open peer review, where the reviewer’s identity is made known to the author (and perhaps to eventual readers). Others argue for applying the ‘wisdom of crowds’ concept to peer review, arguing that the system could be radically improved by the adoption of ‘soft peer review’, i.e. using the new ‘Web 2.0’ social networking tools – commenting, collaborative annotation and using tagging, bookmarking and hits – to measure popularity. Not everyone is convinced (some authors have a very understandable fear of losing ownership of their material, for instance), but new experiments are regularly launched in this area. One high-profile experiment by the journal Nature involved submitted articles being put up on the journal’s website and comments invited from readers. With less feedback received than expected and some authors unhappy at such exposure to public criticism/ridicule, the initiative was terminated. By no means has this been the only experiment but to date no credible alternative to peer review has emerged. In short, what Churchill said about democracy applies equally to peer review: it is a lousy system, but to date all the alternatives have been even worse.

If you are the author of a book, you are unlikely to be affected by these developments in the near future, but it is a different matter if you are also writing journal articles. Journals publishing is very often far ahead of book publishing in testing and adopting new ideas, techniques and technologies.

Surviving peer review

It is of course all very well to explore peer review and its alternatives in such a general discussion. But what of your own situation? How best can you survive this uncomfortable process? This I shall follow up in a later post (and if you are impatient you might like to take a look at our book, which devotes a whole chapter to this issue).

Advertisements

3 Responses to Peer review and its alternatives

  1. […] “original research” in the sense of not having been peer reviewed before publication. Peer review is indeed an important aspect of scholarly communication, as it paves the way towards the […]

    • This is a particularly vexing topic for scholars in the Arts and Humanities. Research produced for publication by such scholars may be a subjective work of creativity rather than an article based on objective standards such as in the hard sciences.

      Certainly any article or book intended for publishing benefits from editing and criticism. Many such pieces, however, have been subject to extensive review by thesis committees and colleagues. Furthermore Design or Creative Writing faculty have professional level publishing skills (layout, editing, print-production) which dovetail perfectly with emerging and relatively inexpensive self-publishing technology.

      Furthermore publishing, whether peer-reviewed or not, is a business which has suffered greatly from the one-two punch of the ongoing ubiquity of the internet and the recession. Look at the fate of major-daily newspapers over the last five years! Those are mass-market commercial ventures. Imagine the suffering of niche academic presses!

      If academia is unable to develop some sort of new crowd sourced peer-review standards to validate self-published materials we run the risk of sticking our arms and shoulders in the sand along with our heads.

      The publishing industry is undergoing a media revolution that is savaging traditional print-based top-down edited-and-vetted information. Obviously there are larger problems with the credibility of direct-to-the-reader information. Journalism is only just coming to grips with blogs as a viable method of reporting even though blogs are over a decade old. Good or bad, however, the move toward the primacy of such self-published information is irreversible. Academia needs to adapt a beneficial peer-review concept to emerging technology or risk (arguably further) irrelevance in the marketplace of information.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: