Reproducibility checklist

Reproducibility
Checklist
Author

Esther Plomp

Published

September 4, 2022

This post is a shortened version of the Agile Reproducibility Checklist for articles that you would like to submit (or review).

For all datasets included/produced in the paper, check if data:

  • Is provided in a non-proprietary format

  • Is documented for third parties to reuse

  • Is accessible in a public repository and has an open data licence

For all software tools/libraries/packages and computational workflows included/produced, check if:

  • Reproduction steps are explained in a README (plain text file), flowchart, or script

  • Computational environments (including hardware) are documented or provided

  • Versions of relevant software components (libraries, packages) are provided

  • All parameters and expected execution times for the computational workflow are provided

  • Software developed by the authors is available in a public repository and has an open licence

  • There is a clear connection between tables, figures, maps, and statistical values and the data and code that they are based on, e.g., using file names or documentation in the README

In the Data and Software Availability section:

  • Include data/software availability statements (see The Turing Way for examples)

  • Include the reasons, if any, for not being able to share (parts of) data or code

For all data and software:

  • All datasets and code (used or mentioned) are assigned DOIs

  • Datasets and code are cited throughout the paper

More information