NSF-BSF: Utilizing Neurophysiological Measures to Better Understand and Improve Engagement and Learning with Intelligent Tutoring Systems

Computer-based intelligent tutoring systems (ITSs) provide students with a personalized learning experience that is tailored to their prior knowledge and learning progression. ITSs have been shown to support student learning and are implemented widely in classrooms, but not all students engage effectively with ITSs, leading to varying learning outcomes.

Prior research primarily relied on data that is automatically collected by tutors (e.g., How many errors a student makes, how fast students answer a question posed by the tutor), but this data cannot provide sufficiently detailed information about learner engagement. For example, students might be slow in responding to a question either because they are distracted or because they are thinking deeply about the problem.

In this proposed project, log-data will be complemented with an array of physiological measures, consisting of eye gaze, Electroencephalography (EEG), and heart rate, to provide a more comprehensive understanding of when and why students get disengaged with ITSs. Neurophysiological data is typically acquired in controlled laboratory environments, but this project will leverage recent technological developments in portable and wearable technologies to study student engagement with ITS in school environments.

Additionally, the investigators will experimentally manipulate the level of tutor assistance (e.g., whether hints are provided automatically or on-demand) and measure its impact on student engagement. The proposed studies will be conducted concurrently in two countries - the U.S. and Israel – which will contribute to the ability to generalize results to a wider range of students.

The results of this project will support the design of more engaging and effective tutors, which could improve the learning experience of tens of thousands of students each year.

The optimal level of assistance provided to students by ITSs is a much-debated topic in learning and instruction since both too much and too little assistance can be detrimental to student engagement and learning (the “assistance dilemma”). Prior research primarily relied on log-data, which cannot capture the multi-dimensional nature of learner engagement.

This project will investigate the mediating role of behavioral, cognitive, and affective components of learner engagement in the relation between tutor assistance and learning outcomes. This goal will be achieved using a multimodal approach to study student engagement with log-data, eye gaze, EEG, heart rate, galvanic skin response, and self-reported measures.

Additionally, the investigators will experimentally manipulate two key features of tutor assistance - the level of information provided by hints (principle-based vs. problem-specific hints) and their mode of presentation (proactive vs. on-demand) - and measure their impact on learner engagement. This research will be conducted in high school-based laboratories in both the U.S. and Israel using a well-tested intelligent tutor for learning chemistry concepts, the StoichTutor.

The project findings will contribute to the Interactive-Constructive- Active-Passive (ICAP) theoretical framework and to the design of more engaging and effective tutors.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.