Enabling options for review: from training and transparency to author-centered AI tools

Peer review is widely viewed as a critical aspect of biomedical communication. Ideally, it provides authors with feedback so they can improve manuscripts and gives readers, particularly nonspecialists, some assurance that the work has been vetted by experts in the field. Increasingly articles are being shared and read as preprints, but the fact that more than 80% of preprints posted on bioRxiv and medRxiv go on to be published by peer-reviewed journals indicates the importance the community attaches to the process of review.

As part of our drive to increase interoperability and stimulate evolution of science communication, openRxiv supports review in a number of different ways. Our bioRxiv to Journal (B2J) / medRxiv to Journal (M2J) and J2B/J2M technologies allow authors to easily transfer submitted papers between bioRxiv/medRxiv and peer-reviewed journals, and a number of journals and independent peer review services now display peer reviews alongside bioRxiv and medRxiv preprints as part of the Transparent Review in Preprints (TRiP) project. We also facilitate a number of community review initiatives—for example, PREreview and the ASAPbio Crowd Preprint Review project—by providing mechanisms for authors to request and receive feedback. 

However, the dependence of review on already overburdened academics faced with ever-increasing numbers of papers to review has prompted concern that the process as currently configured is not sustainable. Providing training and expanding the peer review pool are an important part of the solution, and this is what initiatives like PREreview aim to achieve by involving early career researchers and academics from regions currently underrepresented. Meanwhile, the transparency afforded by projects like TRiP should increase confidence in the process overall. 

Automated tools could be another part of the solution. Various AI-based initiatives that analyze manuscripts are now available, and openRxiv will be providing connections to some of these options for authors to try in much the same way we do traditional peer review services. Our first pilot is with q.e.d., an authors-centered AI review platform that was founded by molecular biologist Oded Rechavi and his colleagues. Iteratively developed with feedback from a large number of scientists, q.e.d. uses generative AI to analyse the claims and supporting data presented in manuscripts and identify ‘gaps’ that warrant either further work or a revision of the claims. It offers solutions to mitigate or address these gaps, helping researchers refine their claims and strengthen their conclusions. It also highlights the original aspects of the manuscript through comparisons with other papers as part of an overall report produced for each manuscript. 

The pilot offers bioRxiv authors the opportunity to send their papers directly to q.e.d. from bioRxiv if they choose. The process uses the dedicated B2X pipeline developed for services such as data validation and curation. Authors can only send their own manuscripts; this pilot is an opportunity for researchers to get feedback on their own papers and use that information to improve them. After submitting a paper, authors simply click “Submit Preprint to External Author Service” in their Author Area and select the q.e.d. option. The paper and associated metadata are then transferred to q.e.d., where the analysis is performed and the review report generated for the author—usually within 30 minutes.

We are also working with q.e.d. to enable authors who have already run papers through q.e.d. to submit these automatically from q.e.d. to bioRxiv, and we plan to expand the pilot to other manuscript-assessment tools. Services like q.e.d. will thus offer authors the option to identify strengths and weaknesses in papers before they share preprints for the first time or as they revise them for resubmission or a new version in response to feedback from the wider community. 

Early feedback from scientists on q.e.d.’s performance has been very positive, and new AI-based manuscript-assessment tools are appearing all the time.  It will be interesting to follow how the landscape evolves. One of the promises of preprints has always been that decoupling dissemination from peer review creates opportunities to explore options in the review process. By collaborating with groups like q.e.d., openRxiv hopes to enable experiments that optimize review for the Web and ultimately improve how we strengthen, vet, evaluate and filter the literature.