by Carl
Reviewers have a lot of sway on how papers evolve, so it is critical that we, as Reviewers, wield this power with care. To ensure your reviews are helpful and constructive, it is important to provide well caveated advice, delivered with humility. Your goal should be to maximize the paper’s utility to the scientific community so that others will be able to learn the most from it, as easily and clearly as possible. Humility is critical in the process and the recognition that the authors are different people with different goals and objectives from yourself.
Remember that there are people on the other side of this who will be reading your review. Those people have feelings. Many will be junior trainees. Your review could be the thing that convinces them to stay in or leave science. Accordingly, you should review as you would want to be reviewed, keeping things constructive. Write a review that delivers constructive criticism in a careful and considerate way, as you would do to a close friend or family member. Refer to the manuscript and not the authors in your review (e.g. “The manuscript stipulates that X depends on Y, but …”). It is likely that not all the authors agreed with everything in the manuscript, so referring to the authors is potentially inaccurate, whereas the manuscript did objectively say what it said. Further, referring to the authors may read like an ad hominem attack and people will take “the authors” more personally than “the manuscript”. This is hard to avoid in writing, so we often do a final sweep to ensure we avoid referring to the authors unnecessarily.
You should try your best to review the paper that was written, not the paper that you wanted. Recognize that the authors may have different priorities than you. Don’t tell the authors to do something that they may consider not germane to the paper because that effectively gives them no choice and now the message of the paper will likely become muddied. However, it is okay to suggest alternative framings or discussion points that would help people with a similar perspective to you understand and appreciate the paper. Just leave this as a choice to the authors.
Humility is important when suggesting improvements. Acknowledge where you might be wrong, and give authors a way out if so. If you are wrong, others may be too, so the issue may nonetheless need addressing but in a different way. (e.g. “The manuscript uses a T-test, but, to my understanding, the data are not normally distributed. The authors should consider doing a test that does not assume normality, or explain why the T-test is a good choice for their data”).
Give a lot of freedom of choice to the authors, with multiple ways to address issues, which could include altering the text, alternate analyses, or additional experiments to address the concern. Make authors note caveats and limitations where they exist (e.g. “The manuscript concludes based on their gene expression analysis that X regulates Y, but since this was not shown directly, the authors should acknowledge that other possibilities exist (e.g. Y regulating X, Z regulating X and Y).”). If the issue can be resolved with an experiment, you can suggest that as one of the options. Don’t tell the authors to perform “nice to have” experiments that will not change the conclusions of the paper or may distract from its central message. If the outcome is sound already, this will not change it, it will just waste the authors’ time. You can suggest experiments as future directions that are not needed for publication and if they think it is worth pursuing they may do it later.
Don’t tell the author to cite papers. If you think it’s an omission, you can suggest that the authors consider it. They may already know of the paper and have chosen not to cite it for some reason. The literature you are most familiar with (especially your own papers!) will always seem more relevant to their work than it objectively is.
Try not to recommend rejection of the paper. Determining whether the paper is suitable for publication is the Editor’s job (although not all do this well). In your review, the first paragraph is typically a summary of the manuscript, detailing what they did, caveats, why it is important, and who will care about it. This section is what will help the Editor to make a decision about the paper. Some journals request that you recommend an outcome. If the Editor is doing their job, the paper should only have gone out for review if they thought it was a strong candidate for the journal. Given this, I think that the reviewers should really only select the “reject” option if there are such critical flaws in the paper that it would be unsuitable for publication at all.
Constructive peer review is critical to science. Not only does it ensure the integrity of the scientific literature, it helps us build and maintain a helpful community. In contrast, adversarial peer review achieves neither goal: the community is harmed by suspicion of who the ‘evil’ reviewer might be, and the flawed papers may simply be resubmitted to a different journal without the flaws being addressed, hoping for a different set of reviewers. The more constructive reviewers there are, the more the norms of the field shift to maintain a constructive style. People receiving constructive and well-worded reviews will pay it forward. Receiving adversarial reviews as a trainee is likely to lead to some trainees dropping out of science altogether. In many cases, the people we would lose are the very people we should be trying to retain – careful, collaborative, considerate people who have better things to do than to stay where they feel unappreciated. By focusing on improving the science and recognizing that people are behind that science, we can ensure that our science is robust and our communities are thriving.
Constructive feedback on this guide is of course appreciated! Please send directly to Carl via email.