Peer review training with preprints [Guest post]

About Guest Posts

The openRxiv blog includes occasional guest posts on different aspects of science communication and open research. This allows us to provide space for new perspectives from groups piloting new approaches and building on the work we do at openRxiv.

Our first guest post is from the leaders of HHMI’s new TAP program and showcases that potential. The HHMI team is using bioRxiv and medRxiv preprints in a new approach to peer review training. This is empowering the next generation of scientists and is a model for other initiatives that seek to provide training and improve the peer review process.

Note: Guest posts reflect the authors’ perspectives, not openRxiv’s organizational positions.

Review by peers is an important part of the scientific process. Several million articles are published in journals each year, and each one has, at least in theory, been thoroughly scrutinized by two or more experts in the field. These peer reviewers assess the article’s strengths and weaknesses, identify areas requiring revision, and provide feedback to authors and editors. Despite its importance, academics traditionally receive little formal training in peer review. It is typically something they learn as a postgraduate student or post-doc from their mentors or simply pick up when they obtain an independent position and start receiving invitations to review for journals. As a consequence, peer reviews can vary in quality, and bad habits may persist and be passed on. 

The increasing public availability of research articles as preprints provides an opportunity to address this issue. Since preprints are typically posted before formal peer review, they are a great resource for training programs that teach peer review and provide opportunities for early-career scientists to get credit for peer review. 

The Howard Hughes Medical Institute (HHMI) is among the early adopters of peer review training using preprints on bioRxiv. In the Transparent and Accountable Peer Review (TAP) program, early-career scientists from HHMI labs learn best practices for producing high-quality peer review reports and help advance research by providing collegial, constructive feedback to preprint authors and useful context for readers. To complete the program, the participants post a public peer review report on a preprint of their choosing with the support and collaboration of their research mentor. Because preprints exist outside the journal system, TAP participants—and their mentors—must assess the importance and value of the work without relying on journal prestige as a proxy for quality. TAP trains reviewers to focus on how well the main claims of an article are supported by the evidence and where the findings fit in the context of the research field.

TAP training at HHMI

The TAP program differs from many other peer review training initiatives in that participants produce a public review of a preprint. To prepare for writing their report, the participants attend a one-day workshop on the principles and practice of peer review, which is facilitated by HHMI lab heads and other experts. The workshop combines instructional modules with small group discussions that introduce participants to the core principles of peer review, such as how to appraise an article and construct their report. Breakout sessions are used to discuss emerging topics, such as careful use of AI in peer review, disclosure of competing interests, and how to evaluate bias.

The workshop also includes a hands-on session facilitated by HHMI lab heads. The TAP participants read an assigned preprint in advance and, during the session, work with a facilitator to identify the main claims of the article, draft and categorize major or minor points of feedback, and contextualize the findings of the paper within the research field. They are encouraged to use structured vocabulary adapted from eLife’s assessment vocabulary to comment on the strength of evidence presented and the significance of the work.

The participants then draft a practice peer review report based on the preprint discussed in the practice session and receive personalized feedback from HHMI staff. The practice reports are not posted publicly, but participants apply the lessons learnt when they later develop their public peer review report. 90% of participants who completed a wrap-up survey said that drafting a practice report was valuable.  

Writing a public TAP report

After completing the workshop, TAP participants select a preprint for public review and work with their mentors to draft their report. The reports are posted on PREreview, an open platform for preprint peer review. Before participants publicly post their reports, HHMI first sends the report to the preprint authors with the name of the reviewer redacted. This allows the authors to receive the feedback, prepare for the report’s public posting, and, if they choose, respond. Several authors have now written formal responses to TAP reports by posting comments on PREreview.

The public nature of the TAP reports allows both the preprint authors and the scientific community to benefit from the feedback and insights. Another advantage of transparent peer review is that reviewers can receive credit for their contributions. The public peer review report is a tangible, citable scholarly output that provides a path to future peer review opportunities. However, since posting a public report can be intimidating and some peer reviewers may be reluctant to reveal their identities, especially those with less experience, TAP participants can choose to sign their report with a pseudonym. They are encouraged to discuss this decision with their mentor. About half of the TAP participants in 2025 opted to use a pseudonym; the remainder used their real names.

TAP training has an impact

TAP is making a difference. The project launched as a pilot in 2024 with 16 participants. Since then, 76 early career scientists have posted 85 peer review reports of bioRxiv preprints.

In the wrap-up survey, 89% of participants who completed the program in 2025 said it was a good use of their time and 92% reported that TAP training prepared them to appraise manuscripts independently and develop constructive, focused feedback. All the mentors who responded to the survey said they would consider participating again.

“I feel much more confident in conducting peer review and being able to provide constructive feedback moving forward,” said TAP participant Yasmeen Hamed. 84% of participants said they are likely to pursue other peer review opportunities in the future. To support that next step, the organizers of TAP at HHMI are connecting participants with preprint peer review services, such as eLife, Review Commons, Peer Community In, and Rapid Reviews.

The impact of TAP extends beyond peer review. The early career scientists who have benefited from the program are applying the skills they developed to their own scientific writing. It is also helping them to improve their manuscripts, grant applications, and conference abstracts.

Photo credit: Toby Hayman / HHMI

What’s next for TAP? 

HHMI is exploring how to share the material developed for the TAP program with the broader scientific community, and we are encouraging research labs, academic departments, and graduate programs to use journal clubs for peer review training. 

Journal clubs are an important component of graduate training in the life sciences and are already used to help early-career scientists learn how to critically appraise papers.  Many journal clubs already discuss preprints, and evolving them to produce public peer review reports on preprints will benefit the scientific community by allowing discussions within journal clubs to reach the authors and readers. In doing so, early-career scientists can develop peer review skills collaboratively with their mentor while providing constructive feedback that is not anchored in journal reputation. 

Combining formal training with real-world practice, the TAP program offers a practical model for strengthening peer review and advancing more open, collaborative science. This work is made possible by the large and easy to navigate corpus of scientific research available on preprint servers. 

Acknowledgement: We thank Bodo Stern, Elizabeth Walsh, Grant Shreve, and Richard Sever for their thoughtful feedback throughout the writing process and Michele Avissar-Whiting, Bodo Stern, Erik Snapp, and Meredith Schmehl for their helpful contributions to the development of the TAP workshop.