Doctoral Dissertation Research: Articulatory Dynamics and Stability of Multi-gesture Complexes

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).

Spoken language production involves combined and coordinated articulatory actions of the tongue, lips, nasal port, and larynx; yet the natural variation in speech and the difficulty of "seeing" events occurring inside the human vocal tract mean that the elegant choreography of these articulatory events critical to the formation of spoken words remains mysterious. This project is a study of speech production using real-time Magnetic Resonance Imaging (rtMRI)—a technique that can image vocal tract movement during speech.

The project's goal is to characterize the dynamics of articulatory actions as they are organized into coordinated complexes such as speech segments (roughly ‘alphabetic-sized’ units). The investigation of stabilities in coordination will help scientists understand the cognitive representation of the linguistic units that are used to structure and say the words we know.

Ultimately such an understanding has implications for language in breakdown (e.g., due to stoke or disease), for language acquisition, for language and dialect diversity, and for technologies that synthesize and recognize human speech.

This dissertation undertakes real-time MRI experiments on speech production and concomitant computational modeling to examine how the smallest movement 'atoms' of speech segments are systematically coordinated. The project tests the hypothesis that vocal tract movements composing a segmental unit are tightly coordinated with one another. Therefore, their coordination is predicted to be stable across natural contextual variations in speech such as phrase boundaries and emphasis, unlike other movement sequences spanning across segments, which have been shown to exhibit more plastic coordination.

Using data from two under-studied languages, both larynx-oral coordination in glottalic consonants and velum-oral coordination in nasal consonants are dynamically imaged, and vocal tract movements are tracked to analyze articulatory patterning and coordination in space and time. Empirical findings are computationally assessed using a model of dynamical coupling with which degrees of stability can emerge from certain coupling structures.

The project seeks to incorporate coordination relations into linguistic cognitive representations and thereby illuminate the dynamic and emergent nature of speech segments in the encoding system of the phonological units that form words.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.