Collaborative Research: Haptic Technology Innovations to Guide Student Gesturing with Dynamic STEM Visualizations

This project aims to determine how sensory feedback delivered to students’ hands helps them understand the meanings of pictures, graphs, and other representations displayed in computer simulations. To accomplish this aim, the project team will design and develop a custom haptic glove that interacts with a computer visualization of a biological cell.

Unlike prior research, the project aims to structure science students’ learning by guiding them to think about the meaning of what their hands feel when they move across a visualization of a biological cell. The context of biology matters because it situates students’ learning in micro- and sub-microscopic disciplines where we know much less about the effects of haptics on learning than we know about learning with large, real-world objects.

Many past studies have demonstrated that using the body and hands to learn can help in similar science contexts, but this project aims to additionally test the claim that the body itself helps students learning by using a novel neuroimaging method (functional Near Infrared Spectroscopy or fNIRS) that allows students to move their bodies while learning and then determine if motor areas of the brain remain active during later problem solving.

This project delivers added value to science education because it will determine when and how specific hand movements (i.e., gestures) combined with feedback from the haptic glove enable science learning above and beyond gestures or sensory feedback (i.e., touch) alone. This knowledge will help address an outstanding problem in the science of learning that involves when and how to use symbolic representations (e.g., equations or numbers) or else concrete representations (e.g., plastic hand-held models).

The project adopts a blended approach that aims to address limitations of computer visualizations that fail to deliver to students a feeling for the small world beyond their senses. Moreover, the technology the project team aims to produce will result in a significant cost reduction to the price of commercial gloves. Last, the project brings together a multi-disciplinary team that will disseminate results to diverse stakeholders that include science instructors, private industry, and learning scientists.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.