News
Trust Signals and Accountability
By Richard Sever (Chief Science & Strategy Officer) and Tracy K. Teal (Chief Executive Officer)
Progress in science depends on the trustworthiness of research that is shared. Developing signals that indicate when the content of a manuscript has been verified or evaluated is therefore a key goal for openRxiv, and collaborating with other organizations on indicators that help establish the integrity of information authors supply will increasingly be part of our process. This is particularly important at a time when retaining public trust in science is crucial and generative AI is making it easier to produce seemingly legitimate research that includes false information.
There are several forms of signals that can increase trust in research: critiques and context provided by other scientists, particularly experts in the field; completeness and verifiability of the materials, methods and data; and accountability of the individuals who performed the research and the institutions that fund and oversee them. Such signals do not of course guarantee that the findings reported are correct, but they can help readers assess their merits and the provenance of information presented. These are areas we are exploring for the bioRxiv and medRxiv preprint servers, which are part of a broader ecosystem in which various organizations can provide trust signals.
Among the expert evaluations available for bioRxiv and medRxiv preprints are peer reviews and editorial assessments provided by journals and independent services, community reviews, and reviews by preprint clubs and training programs. Moderated commenting is also available on our platform, so researchers can give feedback and ask questions. In addition, we provide links to versions of articles that subsequently appear in journals, as well as journal retractions. As authors increasingly use automated tools to assess or improve manuscripts, we will also be exploring ways to let them display outputs from such services. Greater transparency and more details of how evaluations are performed further increase confidence in the judgments that are made, and we have been part of the Docmaps project that provides a machine-readable way to document these processes.
Completeness of the materials and methods used and verifiability of the underlying data are arguably more important trust signals. Our partnerships with Sciscore and Dryad are steps in this direction, and we are keen to partner with other organizations that verify content. Sciscore validates the reagents and resources reported in a paper, while Dryad allows authors to link papers to data deposits that have been verified by Dryad curators. We are also exploring partnerships that enable authors to submit more structured, human- and machine-readable content that may provide additional useful signals. As per our terms of service and privacy policy, openRxiv is transparent about partner roles and data handling.
Ultimately our aspiration as a community should be to implement audit trails via which the provenance of data can be traced. Given the ease with which images can now not only be altered but faked entirely, such a goal is probably more realistic than hoping to detect data manipulation post-hoc. This will be easier to accomplish with instruments that can hash information at the point of capture, but increased use of electronic lab notebooks (ELNs) and standardized data-processing pipelines will help. This will also ensure machine readability, which is essential as the consumers of research outputs increasingly become AI agents, as well as human readers.
The extent to which the above signals can ultimately confirm the veracity of research remains to be seen, however, and ultimately none can ascertain whether a researcher has actually done what they claim. Trust in the individuals who perform research therefore lies at the heart of trust in science. First and foremost, we need to know that a researcher is who they say they are. They can then be held accountable for their claims and actions.
Accountability is critical when questions are raised about the veracity of information. Researchers must take responsibility for the data, claims and ideas they present. This is made explicit during submission to bioRxiv/medRxiv, when authors must make a series of formal declarations that they understand they are making a contribution to the scientific record and that providing false information is a serious ethical breach. The declarations are more extensive on medRxiv, including requirements for clinical trial IDs and the names of ethics bodies/IRBs that approved the work.
Identity verification is therefore foundational in research integrity. Achieving this in a way that is proportionate, preserves privacy and does not create barriers for particular groups, career stages or regions is therefore an important challenge for the research ecosystem. bioRxiv and medRxiv display Open Researcher and Contributor IDs (ORCIDs) for authors, but ORCIDs are primarily a researcher disambiguation tool and do not (yet) constitute identity verification for most researchers. This is the goal for new initiatives like Verime and others. More work needs to be done to create universal solutions that are affordable and available for different types of research organizations. Collectively the research community can be trying new approaches and sharing solutions and what we’ve learned together.
Additionally, identifying the institutions that oversee and fund researchers is important. Sadly research misconduct does occur, and openRxiv frequently has to respond in disputes about authorship, data ownership or intellectual property. These must be arbitrated by the organization with which an individual is affiliated. This is one reason openRxiv requires authors to supply an affiliation: we need an organization responsible for ethical oversight to which we can refer allegations of fraud/misconduct. We have also recently introduced standardized funder metadata on bioRxiv and medRxiv, so that readers can see who financed the research. ~80% of papers now include funding information declarations, and these represent another potentially auditable signal.
Ultimately, we envisage a research ecosystem in which trust signals accrue over time to build confidence in the research reported. These will vary depending on the type of research. Meanwhile, the ways in which we scrutinize submissions will evolve. Some checks will be internal and performed upstream when a paper is submitted. Others will occur downstream as part of assessment by the wider community. In extreme cases, concerns raised may result in formal withdrawal of the preprint (similar to a journal retraction) with a public statement detailing the reason. Individuals tempted to submit false information should remember that manuscripts posted on bioRxiv and medRxiv are permanently displayed. A fraudulent contribution, thus, has the potential to be surfaced for years to come. A durable and transparent public record helps preserve accountability and should be a significant disincentive for researchers to breach the trust placed in them by their funders, institutions, and the public.