5 Reviewing Steps of an Outstanding Peer Reviewer

I was both surprised and honored to receive the 2015 Outstanding Reviewer – Radiological Technology Award from the Journal of Medical Imaging and Radiological Sciences (JMIRS). JMIRS is an international peer-reviewed journal for medical radiation technologies and the publication of Canadian Association of Medical Radiation Technologists (CAMRT).

According to the Peer Reviewer Awards website, the initial selection criteria for an outstanding reviewer are to complete a minimum of one review on time. Also, the reviewer provides feedbacks and recommendations that are useful for improving the manuscript. Then, the editors will choose from nominees.

Trained to appraise medical journal articles, I have been volunteering as a peer-reviewer since 2014. I have no clue why I got the award, but it is certainly a positive way of telling me that I have done it right. Below, I will describe how I review a manuscript.

1. Set a Reminder

Once I accept an invitation for review, I have three weeks to complete my assignment. The first action after accepting is to set a reminder at least a week before the deadline. Usually, I review within a few days. However, an alarm will ensure that I will not miss the date.

2. Perform the Review

Then, I download the pdf file on my desktop for easy retrieval. Once I finish with it, I will move it to a permanent location. For the first pass, I quickly read through the manuscript. I highlight anything I find relevant, such as research methodology, sample, and anything unique about the study. Also, I comment as I go, using the comment bubbles. For research that employs complicated methods, or a manuscript with many issues, I may jot notes on a piece of paper.

It appears to be funny, but I do a lot of literature search during the review process. It is because I need to verify some contents, especially research methodology. Sometimes I have to consult statistic books to confirm the choice of methods and assumptions are correct.

I examine all the elements, such as the rationale, literature search, methodology, results, discussions, conclusions, and references using a holistic approach. I need to answer each comment – not that I will provide answers to them, but to verify that each comment or question is legitimate and accurate. I offer the comments, but it is up to the authors to respond. Sometimes they make revisions and sometimes they defend their manuscript.

3. Write the Recommendations

If I were an author receiving feedback from the reviewers (there are usually two per manuscript), I would feel apprehensive before I even read the comments. It is because I expect criticisms that undermine my efforts. After writing and revising my manuscript, I think it is flawless and should not be touched. Unfortunately, there are always comments that will bruise my confidence. Even worse, the reviewers who criticize my work do not know as much as I do…

My role here is the reviewer. My job is to appraise the manuscript critically and confirm that the final product reaches the journal’s standard. I use a modified version of feedback sandwich (Dohrenwend, 2002). Before criticizing, I commend the good work. I organize my comments with a Word document.

As a fellow researcher, I acknowledge the authors’ effort. I consider this gesture importance because I need to show appreciation. The authors use their time and effort to conduct research and write up papers, and they apply to our journal. I would not have the chance to read the manuscript if they had not done it.

Furthermore, the authors will be more likely to accept my comments and less likely to become defensive if a “rapport” emerges. Moreover, there must be some strength in the manuscript. I just need to recognize and put it down in writing. Therefore, the first part of the review is always applauding the good points.

No matter how much I like or dislike the manuscript (or the research), I feel my responsibility is to help the authors succeed. The end point is simply, for me, that the manuscript is accepted for publication in JMIRS. This is the objective of my review.

I strive to give constructive feedback that is objective, unambiguous, and concise. Usually, I give reasons for the comments. Even if I criticize, I do it with respect, and I always suggest some actions. Sometimes the suggestion is “please rephrase”, or “please clarify”. For instance, instead of writing “poor choice of words” I write “did you mean ______?” The latter comment is more palatable than the former one.

Reviewing the tables and diagrams is my favorite. It is interesting to see how the authors present their findings in charts. If I disagree with their presentation, I consult statistics books before I comment. It is because I may remember the details incorrectly, or the authors are using advanced techniques that I am unaware of. This is an educational activity for me.

I don’t always praise at the end of the review. That’s why it is not a genuine feedback sandwich. However, the tone and wording are intended to minimize resentment and maximize communication.

Last but not the least, I look at the references. References are crucial for the article’s credibility. Some naïve authors have not embraced this concept, but I do. I check every reference carefully against where it appears in the context. I check that the references are correct, and all the components are present (for instance, reference number, authors’ names, and year). The journal provides databases to reviewers but they are not exhaustive. Although I may not be able to access all the references that appear in a manuscript, I use all resources available and confirm to my best effort that they are used correctly.

4. Sleep on It

This is another important step. After I finish the review, I do not submit immediately. Usually, I pick it up the next day and revisit the whole process. It is almost guaranteed that I will make some changes.

 

Save the work and sleep on it

 

5. Submit!

Although my personal goal is to help with publishing the manuscripts, I also recommend rejection if the work is unsuitable for the journal. After reviewing my review, I copy the document and paste it into the reviewer’s comment box. When I hit submit, I sigh a breath of relieve. I know the job is not done yet. I will receive updates, and I look forward to reading the article in print.

In conclusion…

It is humbling to read others’ research. The authors are knowledgeable and generous to share their work. I learn a lot from reviewing them. I am aware that the authors are the subject experts. Although I may be a research methodology expert, I am also a learner and a helper. This is why I comment with respect.

There are still a lot of reviewing techniques that I need to learn. I am happy to receive the award. It is a reinforcement for my future development as a reviewer.

Acknowledgement

Special thanks to Tara Rosewall, PhD., for recommending me to become a reviewer. Also, thanks to the JMIRS for recruiting me. Thank you for giving me the award. I am grateful for the opportunity to contribute to the profession.

Reference

Dohrenwend, A. (2002). Serving up the feedback sandwich. Family Practice Management, 9(10), 43-46.