Adaptive population codes for flexible visually-guided behaviors

Summary/Abstract Active vision requires encoding and remembering relevant information based on current task goals. Classic accounts posit that sensory encoding, attentional selection and working memory are mediated by persistent changes in the firing rates, or the gain, of visually responsive neurons that have a fixed tuning profile

(termed “pure” or “fixed-selectivity” neurons). The focus on gain modulations in fixed-selectivity neurons has revealed a great deal about the basic mechanisms of attention and memory. However, it is becoming increasingly clear that dynamic changes in task demands may require more flexible coding schemes. For example, holding

information in working memory in the same format as the stimulus-evoked response may lead to interference with new sensory inputs. Similarly, flexibly encoding sensory representations to complete one task – say a simple choice between two motor responses – might require a reconfiguration of the representation if another stimulus-

response mapping suddenly becomes relevant. Finally, sensory codes must be flexible in the sense that early in processing they should form high-dimensional representations to represent as much information as possible about the current state of the world. Later in processing, when a decision or motor response needs to be made,

the code should collapse to only represent the smaller subset of relevant choices. All of these computations are more naturally accomplished via the operation of neurons that have flexible tuning for both sensory features and for task demands (termed “mixed-selectivity” neurons). Based on these considerations, we hypothesize that flexible behaviors are supported by mixed-selectivity

neurons that “rotate” high-dimensional neural codes to become robust to interference or to sub-serve other changes in task demands. We will use modelling, psychophysics, and functional magnetic resonance imaging (fMRI) to test predictions about how mixed-selectivity should modulate large-scale activation patterns that are

measured non-invasively in human subjects. Collectively, this work will challenge traditional theories of sensory encoding, attention, and working memory that are based on the notion of fixed-selectivity, and will provide important constraints on models of visual information processing to support more targeted diagnoses and

interventions in clinical settings.