CAREER: The control of learning rate through multi-timescale cholinergic neuromodulation

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).

Determining how well environmental cues predict reward or punishment is critical for adaptive behavior. Past experience is more likely to be useful in stable environments. In humans and other animals, behavioral evidence suggests that learning rates depend on environmental uncertainty.

In constantly changing environments, when uncertainty is high, it would be helpful to learn quickly. In stable environments, learning can be de-prioritized and instead humans and other animals can exploit their learned knowledge. This rate of learning can be formalized as a ‘learning rate’ and the computational theory of reinforcement learning (RL) aims to explain such learning processes.

The proposal will test the role cholinergic neuromodulation, a deep-brain region implicated in a wide array of neurological disorders including Alzheimer’s disease (AD), in setting the learning rate during behavioral tasks. The research within this proposal is complemented with an integrated set of educational goals. A methods workshop on the optical tools that are revolutionizing neuroscience will be developed to augment an ongoing introductory neuroscience course.

This workshop, for twenty students in the research track, will introduce students to optical and molecular tools. In addition, this proposal will build on the Psychological and Brain Sciences department’s goal to promote historically excluded identities through its Early Career Colloquium (ECC). A ‘Neuromodulation of Brain Circuits’ ECC segment will be launched with diverse speakers (4-6 trainees from outside JHU, 2-3 trainees within JHU and 1 keynote faculty talk) and networking events, to build a community of diverse scholars in neuromodulation.

The proposed research will use quantitative behavior in mouse models and theoretical modeling to predict metalearning and then combine two-color, two-photon imaging, chemogenetics, and projection-specific optogenetics to isolate the roles of cholinergic and noradrenergic neuromodulation in setting biological learning rates. The proposal argues that the neural controller of a dynamic learning rate would benefit from three attributes: (1) encode environmental cues, (2) dynamically reflect uncertainty in the environment (i.e., high when uncertain, low when stable), and (3) modulate circuits involved in stimulus-action learning.

Preliminary data show that neuromodulation of auditory cortex meets all three criteria. Cholinergic basal forebrain (CBF) axons in auditory cortex exhibit phasic, stimulus-evoked responses to auditory cues (1) that depend on preceding CBF axon activity, such that early in learning—when uncertainty is high—CBF axons ramp up their ability to discriminate the two auditory cues, and later in learning—when uncertainty is low—this discriminative signal fades (2).

This CBF signal precedes cortical plasticity in a region critical for audiomotor learning (3). These data support a core hypothesis: tonic and phasic CBF signaling dynamically set the rate of cortical plasticity critical for sensorimotor learning. To test this idea, the proposal will isolate phasic, auditory input to the CBF to gain control of this signal (Goal 1), use model-based predictions to test whether CBF axon activity tracks a learning rate parameter in discrimination and reversal learning (Goal 2), and causally manipulate CBF signaling during discrimination and reversal learning and examine learning rate (Goal 3).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.