CAREER: Mechanisms of the Impact of Gesture Observation on Learning

Observing relevant hand gestures while listening to speech incorporates multiple senses that enhance learning. However, gestures observed when listening to speech vary in the extent to which they enhance learning, and it is currently poorly understood why this is the case. This CAREER award examines how the information conveyed via gesture and its relationship to speech affects the learning of words from a novel second language differing in pronunciation and meaning.

By examining how the brain processes such words learned with gestures differing in these ways, it will provide insight into why various types of gestures affect learning differently.

The research objectives of this CAREER award are to (1) determine how information conveyed via gesture and its congruence with speech at learning affects subsequent processing of learned content conveyed via speech; (2) characterize how sensitivity to information conveyed via gesture and its congruence with speech prior to learning affects the impact of gesture on learning and subsequent processing of learned content conveyed via speech; and (3) identify how brain regions subserving multimodal and language processing affect gesture-speech integration and subsequent processing of learned content conveyed via speech. The research approach achieves these objectives by examining how observing gestures congruent and incongruent with non-native speech sounds and word referents at learning affects subsequent processing of learned second language (L2) words via accuracy and latency, event-related potentials (ERPs), and transcranial direct stimulation (tDCS), which causally affects brain activity.

In doing so, it will contribute to the development of a theory of gesture’s impact on learning applicable across diverse content areas specifying the neural substrates of the cognitive mechanisms of gesture’s impact on learning. The educational objectives of this CAREER award will integrate scaffolded inquiry-based learning utilizing ERP research methods into the educational neuroscience curriculum; engage students from diverse backgrounds students in the proposed research and educational activities; and advance replication and translation of findings in L2 pedagogical contexts.

Together, these activities will help bridge the gap between educational neuroscience research and pedagogical practice in multiple educational contexts. This project is jointly funded by the Science of Learning and Augmented Intelligence Program and the Established Program to Stimulate Competitive Research (EPSCoR).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.