Integrating Virtual Reality and Large Language Models to Provide Personalized Situated Learning through Adaptive Storytelling

This IUSE Level 1 Engaged Student Learning project aims to serve the national interest by improving training in construction engineering-related courses through VirtuAI-Story - an innovative tool to be created through this project to enable personalized learning in a Virtual Reality (VR) environment enhanced by Artificial Intelligence (AI). By combining the immersive experience of VR with adaptive storytelling from large language models (LLMs), the project intends to provide engaging educational content and create a safe and controlled space for practical training while addressing the need for safer work practices.

VirtuAI-Story is intended to advance learner preparation for real-world jobs, especially in high-risk fields where practical skills are crucial. The project's insights could apply more broadly to various fields where hands-on training is essential. The project also aims to understand how learners interact with an AI-driven virtual agent, providing valuable insights into human-AI dynamics in education.

Importantly, the project will aim to recruit underrepresented groups and low-income students, enhancing their participation in STEM programs.

Through development and evaluation of VirtuAI-Story, this project aims to to address three research questions. Research Question #1: How effectively can situated learning through VR combined with personalization through LLMs be delivered through adaptive storytelling? Research Question #2: How does VirtuAI-Story impact students' engagement (i.e. cognitive, behavioral, affective)?

Research Question #3: How does VirtuAI-Story impact students' learning outcomes (i.e. factual knowledge, conceptual knowledge, procedural knowledge, transfer of knowledge)? Grounded in Situated Cognition and the Cognitive Affective Model of Immersive Learning (CAMIL), the project plans to explore how personalized immersive environments affect learning and engagement.

The research methodology involves content creation, developing VR and LLM components, pilot testing, and comprehensive assessments to evaluate usability, learning outcomes, and student engagement. A goal of the project is to discover how personalized immersive environments impact learning processes, engagement, and outcomes. By applying this technology to construction safety instruction, the project intends to produce empirical evidence on the effectiveness of situated learning environments.

Furthermore, the project endeavors to integrate AI-driven assessments with human evaluations to thoroughly analyze various aspects of student learning and engagement. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.