Improving the Conceptual Mastery of Engineering Students in High Enrollment Engineering Courses through Oral Exams

This project aims to serve the national interest by improving the conceptual understanding of students in the lower division engineering courses that build the foundation for future success. Mastery of basic concepts is essential for student success and perseverance in engineering programs. To gain this mastery and prepare for life-long learning as engineers, students need to develop effective learning strategies.

Since assessments drive students’ learning strategies, this project will implement and evaluate oral exams in high enrollment introductory engineering courses. The oral exams will be used to assess students’ understanding of fundamental concepts, which is difficult to measure using traditional exams and quizzes. A new training program will be created to prepare faculty and teaching assistants to administer oral exams.

The project will also provide guidelines to help engineering educators implement oral exams in other courses. It is expected that the oral exams will guide students to use learning strategies that enhance conceptual mastery and critical thinking. By strengthening students’ verbal communication skills, oral exams can also help prepare them for the engineering workforce.

This project will add to research on oral exams by contributing new knowledge about the effectiveness of oral exams in undergraduate engineering core courses.

The goal of the project is to improve undergraduate students’ conceptual mastery and critical thinking skills in engineering. Critical thinking has been identified as an essential skill for engineering graduates and conceptual mastery is a prerequisite. It has been shown that students adopt learning strategies based on how they will be assessed.

For example, if an exam is procedural in nature, students will pursue procedural knowledge over deep conceptual understanding. Thus, assessment plays an important role in measuring the level of students' conceptual understanding and in guiding students' selection of learning strategies toward achieving greater concept mastery and critical thinking skills.

The adaptive dialogic nature of oral exams holds promise for improving both assessment of conceptual mastery and students’ learning strategies. However, the issues of the reliability, validity, and scalability to high enrollment courses of oral exams have not been fully addressed. This project will create a framework to integrate oral exams in core undergraduate engineering courses, complementing existing assessment strategies.

Key elements of the project are the involvement of the entire instructional team (instructors and teaching assistants) and the emphasis on training that includes considerations of issues such as bias and students with disabilities. The project will implement an iterative design strategy using an evidence-based approach of evaluation. The effectiveness of the oral exams will be evaluated by tracking student improvements on conceptual questions across consecutive oral exams in a single course, as well as across other courses.

The training program will be made freely available online to faculty at other institutions who are interested in using oral exams for assessment. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.