Reviewing papers as you would like your papers to be reviewed

As research scientists, most of our energy and time should be devoted toward trying to discover new things about the world in which we live. This is why most of us go into science, and it is by discovering new things that we make our contribution to a wider society. Despite this, many of us find ourselves preoccupied with the problems of publishing papers that describe our latest findings. This is because publishing often involves a battle with a system that is unpredictable, difficult to navigate, and slow (Raff et al., 2008; Ploegh, 2011; Drubin, 2017). It is now common for junior researchers to leave a lab before the work they have done there is published.

It didn’t use to be this way (Siegel, 2008), so why is the current process of publication often painful? Many blame the editors of journals who make the final decision on publication. There have been many recent proposals to change the path to publication, including making the reviews and/or the names of referees public, limiting revision times, the establishment of a review commons (www.reviewcommons.org), and asking reviewers to come to a consensus before a decision is made.

It is clear, however, that the community of reviewers should shoulder much of the blame for the current problems with peer review—and that means us! Although some reviews offer valuable suggestions to improve a paper, it is not uncommon for a reviewer’s comments to be unreasonable, unfair, and even offensive. This is a serious cultural problem. Nasty reviews erode trust in the objectivity of the system. They encourage researchers to worry about publications being delayed or blocked by reviewers working in the same area. And, having suffered during the process of peer review, some may be tempted to give as good as they get, thereby propagating the problem. Over time, our sense of working in a community with whom we share a common purpose erodes. Experts in their field start to be seen as competitors who should be excluded from the review process, even though they are best placed to identify pitfalls in the work that the authors might have been missed. And there are wider implications too. In such a climate, it is harder for scientists to publish work in a new field or work that uses new or unconventional methods or approaches. This can stifle innovation and cause fields to ossify; it can also discourage young individuals from becoming scientists, undermining the whole scientific endeavour.

Many aspects of life, from movies to holidays, are also evaluated by reviewers who are just as likely to be imperfect as are reviewers of science. Why then are scientists so hung up on peer review? Peer review is not the bedrock on which science rests, and it is not a substitute for good scientific practice. One of the great strengths of science is that it proceeds in steps toward progressive understanding: while most steps are small and require modification, the sum of steps taken by the scientific community progressively leads to a better understanding of the way the world works. As Max Perutz said, “In science, the truth wins.” Because of this process, the “importance” of any particular piece of work can only properly be assessed long after it is produced.

This Perspective takes some pressure off being a peer reviewer. If something we have reviewed gets published against considered advice, it is the authors and editors who should worry, not the reviewer. In the end, the truth will out. At its best, peer review should help authors to correct and improve their papers, so that their findings can be rapidly communicated to the community and the public. This involves reading the paper carefully, identifying errors, suggesting simple ways to improve the paper, and advising the editor, who makes the final decision about publication, how best to proceed.

My perspective on reviewing papers has changed over the years without my realizing it. I have not always reviewed as I would like to have done. So, like several other researchers (Bienz and Weston, 2012; http://rajlaboratory.blogspot.com/2014/04/how-to-review-paper.html; https://aninfinityofhypotheses.wordpress.com/2014/07/01/my-reviewers-oath/), I thought it worthwhile to write down a set of principles that I aspire to follow. I have found the exercise useful and encourage others to do the same. By changing the way we review, perhaps we can raise the overall quality of the papers that are published, assist researchers trying their hand at reviewing papers for the first time, and do our bit to make working in science and publishing a more pleasurable experience for all.

Oath: As a reviewer

1. I will review papers as I would like my papers to be reviewed.

2. I will read the paper carefully, in a timely manner, and will, where necessary, suggest ways for the authors to correct and improve it—focusing on specific aspects of the study I consider critical to address before publication (flaws/controls).

3. I will review each paper without trying to change its focus and without insisting that authors use methods I would use or do an experiment I would do in their place.

4. I will review with the understanding that each useful paper represents a small step forward and that the true impact of work can only be assessed long afterward. I will work to ensure that important papers make it into the journals the relevant community reads.

5. I will ensure that claims made are supported by the data. If they are not, I will ask the authors to restate their claims or to provide new data to support them. If claims are not supported by the data and/or if data are flawed, I will recommend a paper be rejected.

6. I will not recommend revising papers that I think will require substantial (>3 months) additional work, or that require the validation of a key hypothesis prior to being published.

7. I will aim to be constructive even when recommending rejection of a paper. I will always aim to be unbiased, will never be rude or patronizing, and will not make personal comments about the authors.

8. I will make the limits of my expertise clear. I will not recommend that the same paper be rejected from more than one journal (except in cases of potential fraud), since authors should be able to benefit from a range of reviewers’ opinions.

9. I will be willing to discuss my assessment with other reviewers and the editor. I will inform the editor if I suspect fraud.

10. I will treat all information in a paper I review as confidential. I shall not allow my team to make use of data in a paper we have reviewed. I shall never share a paper I am reviewing, except for the named individuals who helped with the review.

FOOTNOTES

ACKNOWLEDGMENTS

REFERENCES