Tracking the emergence of internal models

Abstract Central to human and animal cognition is the idea of internal models: an internal repository of knowledge about the structure of the world and its affordances that enables prediction and planning. The existence of such models is fundamental to experience. As we move through the world, the raw instantaneous sensory information that we receive is highly impoverished and

dynamic relative to the rich, organized, stable and detailed nature of experience. Learned and perhaps partially innate priors allow the maintenance of a veridical representation of the world around us, and rapid selection and integration of important information relevant to a task. In this proposal, we aim to probe the neural implementation of world models by recording from multiple

brain areas in primates as they navigate naturalistic environments. Through modeling alongside analysis of coordinated recordings across multiple labs during a single sequence of complex tasks, we will develop a holistic understanding of how such models develop, depend on active engagement with the world, and influence perception. In our project, three macaque

neurophysiology labs, with distinct cutting-edge expertise in different brain regions and technologies, will collaborate with computational neuroscientists with different expertise to pursue overlapping aims. We propose a novel collaboration strategy in which all three macaque neurophysiology labs will investigate a common navigation task, and record from different but

overlapping subsets of areas: inferotemporal cortex (IT), motor cortex (MC) and prefrontal cortex (PFC). We focus initially on IT-HC (Tsao), MC-HC (Orsborne), and PFC-HC (Buffalo), on the hypothesis that HC, by virtue of its anatomy, its essential function in episodic memory formation, and its master role in orchestrating subjective experience, constitutes a central hub

for representing the brain’s internal model of the world. To help to interpret this new data, we will develop and probe network models of predictive processing in collaboration with the analysis and theory team. Fairhall, Mihalas, Rao, and Shea-Brown, with complementary expertise in neural coding, predictive coding, and network dynamics, will interact to develop integrated

model frameworks and work with each experimental lab individually on data analysis. Major questions we will address include: How does such a “world model” develop and how is it represented in the brain? What are the consequences of world models for the dynamics of sensory processing and behavior? How is learned structural knowledge integrated with self-

motion during exploration of an environment? How is this information retrieved during memory- guided navigation to a goal?