Replication and reproducibility

Scientific verification is based on three main principles: independent evaluation (peer review), independent replication and independent investigation of reproducibility.

Replication and reproducibility are terms often used in a similar meaning. Within a context of scientific verification however it is useful to make a clear distinction between the two.

“…I define “replication” as independent people going out and collecting new data and “reproducibility” as independent people analyzing the same data.” (Roger Peng, 2011)

Replication means repeating a research by independent researchers in order to verify a claim or result from the original research (‘X causes Y’). Another term used for this is ‘repeatability’. A replication research is therefore a new research in which new data are collected. However, the basic planning and methodology of the original research are imitated as much as possible.

Reproducibility research – a better term is reproducibility investigation – focuses mainly on whether data analysis of the research to be reproduced has been performed correctly. Independent researchers reanalyze the original data to check whether results published (figure or table in article) may indeed be deduced from the data. This procedure is sometimes referred to by the terms ‘reanalysis’ or ’duplication’.

Reproducibility proves nothing in the way of correctness or validity of results published. Research that is reproducible is not necessarily replicable.

Replication, reproducibility and RDM
The difference between reproducibility and replication is important for RDM as reproducibility requires original research data plus a list of all processing steps – often in the form of software code developed or used for processing, analysis and presentation of data – to be provided. This requirement does not apply for replication.

The VSNU Code of Conduct for Scientific Practice (p. 8, section 3.3) requires all research data to remain available for reproducibility checks for a minimum of 10 years.