Redefining Questionable Research Practices and the Role of Registered Reports: A Cross-Disciplinary and Multi-Stakeholder Exploration

Questionable research practices (QRPs) can undermine the integrity of research activities, but the definition of QRPs differs across academic disciplines. A registered report is an approach where peer review of the research and the decision to publish the results occurs before data collection or analysis. Registered reports aim to reduce questionable research practices by incentivizing high quality design and research questions.

The use of registered reports discourages changes to hypotheses or analysis plans after reviewing results; such changes can obscure the research process as implemented. Registered reports have been successfully implemented in some academic disciplines, but their adoption has not been widespread. This project aims to evaluate whether the differing definitions of questionable research practices may be linked to registered report adoption across disciplines.

The project team will identify opportunities to tailor registered reports to address discipline-specific research practices. The project will generate materials for guided discussion of questionable research practices and registered reports, training for journal editors and reviewers, and public outreach to improve trust in science through education about efforts to increase ethical and open research practices.

This project explores factors that underlie discipline-specific variability in use and perceptions of questionable research practices and registered reports. The project will involve three research studies using a combination of meta-scientific, quantitative, and qualitative methods to evaluate existing measures of questionable research practices, develop new measures usable across disciplines, and assess the role of QRPs in registered report adoption.

Study 1 evaluates existing scales for measuring questionable research practices across disciplines in the context of registered reports. Study 2 will develop a new measure which can be used with trainee populations across disciplines. Study 3 uses semi-structured interviews to identify the role registered reports can play in addressing discipline-specific questionable research practices.

These studies take a cross-disciplinary, multi-stakeholder approach and will directly improve research practice as well as develop measurement tools for future research in this area. This project is funded through the ER2 program by the Directorate for Social, Behavioral and Economic Sciences.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.