Using Augmented Reality and Artificial Intelligence to Improve Teaching and Learning Spatial Transformations in STEM Disciplines

The mathematics that are used to describe spatial transformations can be very difficult for undergraduate students. While moving something in the physical world may be easy to understand, describing the same operation with math in the digital world can be daunting. This project will develop new technology using Augmented Reality (AR) and Artificial Intelligence (AI) to improve the teaching and learning of these difficult concepts in STEM disciplines as well as creative endeavors.

The project will test the use of new AR/AI technology to enhance students’ learning of the mathematics behind spatial transformations. This will improve the use of AR/AI-powered precise motion tracking of objects that can collect high resolution in-situ motion and scene data to enhance learning analytics. The project will identify the AR capabilities that can help students conceive, connect, and compare mathematical representations of motion to overcome the well-documented difficulties students face when learning spatial transformations. Strengthening this skill will support their continued development across many STEM disciplines.

An AR/AI-powered innovative learning environment will be developed and evaluated for its effects on teaching and learning major rotation and orientation representations in the Euclidean space. Different levels of abstraction, including Axis-Angle, Euler Angles, Matrices, and Quaternions will be tested. In workshops participants will play with and transform 3D-printed geometry models to evaluate the effectiveness of the AR/AI technology.

The project will assess student learning outcomes by comparing math skills in pre- and post-workshop tests compared to students working through the same tasks without the use of the AR/AI technology. The project will contribute to advancing knowledge across disciplines of spatial and mathematical pedagogies, by exploring: a) the role of novel AR interaction that allows the interplay between physical and virtual manipulatives to engage students in embodied learning; b) the capabilities of AR to make difficult invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation; c) the features of AR that help students see relationships between spatial manipulations and mathematical operations; and d) the potential impact of individual differences in spatial transformations when using AR-assisted mathematical learning.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.