Skip to Main Content

Open Science & Reproducible Research

An overview and resources on Open Science/Reproducibility

How is Open Science Related to reproducibility?

Good, reproducible science is more than just sharing your work, but many Open Science practices promote reproducibility as well as the accessibility and democratization of science. Transparency at every stage of the research process helps other researchers, the public, and external decisionmakers trust your work.

Issues with the modern paradigms of scientific research have been noted for decades - Robert Rosenthal coined the term "file-drawer problem" in 1979 to refer to the tendency to never report null results, leading to wasted duplication of effort and to an over-reporting of false positives. Some trends in Open Science, such as preregistration and registered reports (discussed on the "Open methods" page) are in part attempting to address the file-drawer problem.

Reproducibility Self-Assessment

This short questionnaire from PLOS will ask about a series of Open Science topics for you to check your performance and see areas where you may be able to make your work more open and reproducible.

Terminology - "Replicability" vs "Reproducibility"

Field convention - some use them interchangeably, some use them to mean different things.

From the National Academies of Sciences "Understanding Replicability and Reproducibility", there are three different commonly-used paradigms for what those two words mean:

  • A: The terms are used with no distinction between them.

  • B1: “Reproducibility” refers to instances in which the original researcher's data and computer codes are used to regenerate the results, while “replicability” refers to instances in which a researcher collects new data to arrive at the same scientific findings as a previous study.

  • B2: “Reproducibility” refers to independent researchers arriving at the same results using their own data and methods, while “replicability” refers to a different team arriving at the same results using the original author's artifacts.

This subject guide is using the "B1" definition.

The "reproducibility crisis"

The "reproducibility crisis" across the disciplines

While the term "reproducibility crisis" took off in the early 2010s after a series of impactful publications in the field of psychology, all disciplines have been struggling with irreproducibility for decades.

HARKing, p-hacking, and other abuses of researcher degrees of freedom

One of the ways in which the modern structures of research incentivize poor practices and irreproducible results is through decisions researchers can make when looking at data. Researchers may choose to test a huge set of variables for significant associations and report only those above a significant p-value, or re-work their hypotheses to make it seem as though results were expected and not discovered. Presenting exploratory research as confirmatory misleads readers.

Reproducibility, publishing, and peer review

Reproducibility initiatives across disciplines