Lawrence Tabak voiced concerns on "Enhancing the Transparency and Reproducibility of Science" as a Sanger Series Lecturer on the Virginia Commonwealth University MCV Campus, fall semester, 2014. Students and faculty eager to learn from his experience filled the McGlothlin Medical Education Building to standing capacity with 275 attendees.

"The National Institutes of Health (NIH) is exploring ways to strengthen the rigor and reproducibility of research findings," Tabak says.

A reproducible study is one that scientists can easily set up again and test, which helps to reduce unnecessary funding on repeat studies; however, a study is reproducible only to the extent that it is reported transparently to the scientific community.

Science is often viewed as "self-correcting and largely immune from reproducibility problems," Tabak says, but there is too much emphasis on the "big picture," leading publications to omit necessary details of experiments performed. The experimental-methods and resource sections of published experiments grow smaller and smaller because of inadequate reporting, which makes the experiments more difficult for other scientists to recreate. This makes experimental findings less useful and increases the likelihood that scientists will have needlessly to duplicate past experiments in order to learn from them.

Further complicating the matter, researchers who create "poor experimental designs" tend not to perform or report quality checks within their studies, such as blind assessments, randomization and sample size calculations. These challenges to ensuring transparency are due to "poor training, poor evaluation, difficulty in publishing negative finds and perverse reward incentives," Tabak argues.

Additionally, most of today's academic research publications focus on successful experiments, turning a blind eye toward failed experiments. Tabak points out the importance of publishing failed studies alongside successful ones to reduce funding repetitive research that fails again.

General strategies for addressing the underlying issues are to raise community awareness, to enhance formal training and to protect the quality of funded and published research through the adoption of a more systematic review process. "Potential trans-NIH measures include an integration of checklists for rigorous reporting into all funding opportunity announcements," Tabak says. The checklists will aid in the communication of product and experiment design expectations while providing support to "reviewers in a more systematic review of grant applications."

The NIH also met with representatives from some of the most respected publishers of scientific journals to formulate a set of principles that scientific journals could adopt to help ensure transparency and reproducibility of publishined research. The principles established include commitment to rigorous statistical analysis of studies, to trasparency in reporting studies and to the sharing of all data and materials pertaining to studies, as well as willingness to print refutations of published studies and the development of best-practice guidelines.

 * * * 

Lawrence Tabak is the principal deputy director of the National Institutes of Health (NIH). He previously served as the acting deputy director of NIH in 2009 and as director of the National Institute of Dental and Craniofacial Research from 2000 to 2010.