CRCNS: Building and testing computational models of the neural basis of natural communication

Communication and language processing disorders can significantly impact the lives of millions of people.

The proposed Conversations project will combine an unprecedented ECoG dataset and a cutting-edge MLbased encoding framework to test competing models for neural mechanisms supporting natural language processing and face-to-face communication in natural contexts. Uncovering the neural mechanisms behind

everyday communication can give voice to people with speech and hearing impairments, shed light on communication disorders, and enhance doctor-patient communication. In Aim 1, we will create the "24/7 Conversations" dataset of 750 hrs of continuous electrocorticography (ECoG) data from epilepsy patients

engaging in free daily life conversations. This dataset will be the largest collection of real-life conversations and intracranial neural activity assembled. Aim 2 involves developing an ML-based encoding framework to test the ability of various language models to model our "24/7 Conversations" dataset. ML-based encoding

framework will allow us to explore brain-to-brain communication dynamics during face-to-face conversations. Finally, Aim 3 will leverage the ML-based encoding framework to construct novel computational models that simulate the neural basis of natural language processing, focusing on how the brain predicts and integrates spoken language during spontaneous real-life conversations. Through these

efforts, we aim to map the intricate neural activities associated with language, providing insights into normal and disordered communication.