Availability of research protocols, methods and software

Availability of research protocols, methods and software

/ Wed, 07/31/2024 - 13:26

Abstract

It’s a good open science practice to include requirements on the availability of research protocols and methods in Diamond OA  journal policies. Diamond OA journals and publishers should encourage authors to share them in public repositories and to use persistent identifiers for making the relevant connections between articles and other research outputs. This will make it easier to replicate and build on published work.

To facilitate reproducibility and FAIRification of research, Diamond OA journals and publishers should consider encouraging the use of free/open source software. Diamond OA journals and publishers should define a policy on the availability of research software and ask authors for a statement of software availability.
 

Main Text

A Research Protocol is a detailed document prepared before conducting a study, often written as part of ethics and funding applications, that includes information relating to the background, rationale and aims of the study, as well as the hypotheses which reflect the researchers’ expectations, and a “recipe” for conducting the study, including methodological details and clear analysis plans (FORRT - Framework for Open and Reproducible Research Training Glossary). Research protocols are publicly shared to attract new collaborators or facilitate efficient collaboration across labs (e.g. https://www.protocols.io/). In medical and educational fields, protocols are often a separate article type suitable for publication in journals. 

Availability of software, its acknowledgement and citation 

“Some of the most important scientific breakthroughs of the last decade, such as the solution for the protein structure prediction problem , were made possible because of the availability of rich and comprehensive data sources and powerful software tools for data representation and analysis, numerical computation, and modeling.” (Chan Zuckerberg Initiative Science, 2022). Nevertheless, software is generally not formally cited in scientific publications. At best it is either mentioned in the methods section of an article, or it may be identified through the dependencies of research code deposited by the authors. “Software is a critical part of modern research and yet there is little support across the scholarly ecosystem for its acknowledgement and citation.” (Smith et al, 2016). 

Software availability is essential for scientific reproducibility — the ability to replicate published findings by running the same computational tool on data generated by the study (Brito et al, 2020)). In addition, there is a correlation between the use of free and open-source software, software citation and article retractions. Errors in processing data and computation of results, software errors or misuse are common reasons for scientific self-correction and article retractions. The study found that “retracted articles use less free and open-source software, hampering reproducible research and quality control. Moreover, such differences are also present concerning software citation, where retracted articles less frequently follow software citation guidelines regarding free and open-source software.” (Schindler et al.,2023). 

Measures to ensure the reproducibility of results

Diamond OA journals and publishers can enhance the reproducibility of research outputs by encouraging Open Access to research software, workflows, algorithms, protocols, models, workflows, electronic notebooks and others - needed for the re-use or validation of the article’s conclusions and the validation and reuse of research data. “Reproducibility of some or all results is important as it increases the performance of research and innovation (wider use of research results); it limits waste of resources (less duplication and fewer false baselines); it increases the quality and the reliability of research (stronger methods, controls and reporting); and, as a result, it may increase the trust of citizens in science.” (Horizon Europe Programme Guide)

Tips:

  • Diamond OA journals’ and publishers’ guidelines for authors should ask them to specify how they will ensure robust statistical analysis that can be repeated (power of sample, robust experimental techniques, open software, ...) and make provisions to validate, demonstrate, make interoperable, scale-up and overall make replicable the results of their R&I activities. Authors should be asked to provide information about the research outputs/tools/instruments needed to validate the conclusions of scientific publications or to validate/re-use research data.
  • Diamond OA journals and publishers should consider including the following Reporting standards paragraph in their journal policies: “[Journal title] is committed to serving the research community by ensuring that all articles include enough information to allow others to reproduce the work. A submitted manuscript should contain sufficient detail and references to permit reviewers and, subsequently, readers to verify the claims presented in it - e.g. provide complete details of the methods used, including time frames, etc. Authors are required to review the standards available for many research applications from Equator Network and use those that are relevant for the reported research applications. The deliberate presentation of false claims is a violation of ethical standards.”
  • Where applicable, authors should be asked to deposit step-by-step descriptions of their protocols on protocols.io and to include the persistent DOI in the Methods section of the manuscript. 
  • Where applicable, authors should be asked to upload their scientific code / scripts / models openly in a code repository. 
  • Diamond OA journals and publishers should consider software a legitimate and citable product of research. Software citations should be required in the same way as citations of other research products, such as publications and data: authors should be asked to include them in the reference list of a journal article. 
  • TOP guidelines (Transparency and Openness Promotion) include eight modular standards: 1) Citation standards, 2) Data  transparency, 3) Analytic methods (code) transparency, 4) Research materials  transparency, 5) Design and analysis transparency, 6) Study preregistration, 7) Analysis plan preregistration, and 8) Replication, each with three levels of increasing stringency:  Disclosure – the article must disclose whether or not materials are available; Requirement  – the article must share materials when possible; Verification – third party must verify that  the standard is being met. Diamond OA journals and publishers should select which of the eight transparency standards they wish to implement for transparency and reproducibility in published research and select a level of implementation for each. These features provide flexibility for adoption depending on disciplinary variation, but simultaneously establish community standards. Journal and publisher policies can be evaluated based on the degree to which they comply with the TOP Guidelines - TOP factor, which is a metric that reports the steps that a journal or publisher is taking to implement open science practices.
     

Related toolsuite articles

Related guidelines and training materials

References 


Further reading


Licensing

This article is made available under a Creative Commons Attribution 4.0 International License

Guidelines main menu

Last Updated

Tags
Free and open-source software
Metadata
Open Science
Research data
Transparency