Measuring Faculty Use of Evidence-Based Classroom Practices and Identifying Perceived Obstacles to Adoption

This project aims to serve the national interest by developing tools for acquiring data from university instructors and students about faculty use of evidence-based teaching practices, and to learn about challenges faculty face in using these best practices in their teaching. This project expands upon previous work that developed and validated the Faculty Inventory of Methods and Practices Associated with Competent Teaching (F-IMPACT).

The F-IMPACT tool helps instructors evaluate their teaching methods. The current project plans to develop a similar survey for students to fill out about what practices they see their instructors utilizing in their classes called S-IMPACT. This new survey tool aims to give fair feedback to teachers that provides actionable items to help improve their teaching.

This project also intends to expand upon identifying faculty-perceived barriers to using evidence-based practices. Towards this end the investigators plan to interview up to 240 teachers at two collaborating universities to create and validate the Observed Barriers to Successful Teaching Questionnaire for Lecturers (OBSTQL). Using better teaching methods has been shown to improve student performance and to increase retention.

The combination of data obtained could drive action/intervention, ultimately improving retention and recruitment into the STEM pipeline in a manner replicable across multiple institutions.

This project plans to utilize a Concerns-Based Adoption Model (CBAM) with three diagnostic dimensions: (1) the Model of Success, (2) Measures of Behaviors, and (3) Measures of Attitudes. The Model of Success is a collection of evidence-based classroom practices developed by a review of the literature on the scholarship of teaching and learning. The Measures of Behaviors used in this project include faculty and student reports via the F- and S-IMPACT surveys, respectively, and validation of reports using the Classroom Observation Protocol for Undergraduate STEM (COPUS).

The Measures of Attitudes include the development and deployment of an attitudinal survey (OBSTQL) based on previous grounded-theory research on faculty-perceived barriers to the implementation of evidence-based practices. Specifically, the project plans to continue development and measure congruent validity of the S-IMPACT through correlation with the F-IMPACT and COPUS, from which both inter-rater and parallel-forms reliability could be determined.

Combined with OBSTQL development, the project plans to answer the following research questions: (1) What differences are there between student-observed and faculty-perceived usage of evidence-based teaching practices? (2) What are the most common barriers to implementation of evidence-based teaching practices? (3) Can student observations of instructor actions reliably measure instructional style? and (4) Does a guided-observation design produce less implicit bias and higher reliability compared to affect-based student evaluations of teaching? The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students.

Through its Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.