CRCNS Research Proposal: Computations for spatial-chromatic interactions and their physiological implementation in primary visual cortex

Color and form are often treated as separable features of an image. One can recognize shapes in achromatic photographs and conceptualize the color of an object abstracted from shape. Yet color-specific processing is embedded throughout the visual pathway from the first stage of the visual pathway, where three different types of light sensors (“cones”) with sensitivity to different parts of the visual spectrum initially convert light into electrical impulses.

The color of a given point can in principle be determined by comparing the activation of the three different cone types, but the separate color channels are maintained until the primary visual cortex (V1), where they are finally combined in neurons that concurrently have sensitivity to different spatial patterns. Indeed, while it was initially thought that color and form were processed through separate pathways within V1, recent experiments have highlighted that a surprising fraction of V1 neurons mix them together in a diversity of ways.

Exactly how the mixing occurs, and for what purpose, are critical open questions in understanding human vision, and have been difficult to answer because such mixing is too complicated to characterize using traditional approaches. This project combines large-scale recording of V1 neural activity during tailored “spatio-chromatic” visual stimulation with new computational approaches that offer an unprecedented high-resolution description of color processing within V1 while allowing determination of the underlying function of spatio-chromatic mixing in supporting natural color vision.

The project also provides opportunity for cross-disciplinary training in neurophysiological and machine-learning based statistical modeling of undergraduate and graduate students.

This project is a tight combination of visual neurophysiology, data-driven computational modeling, and simulation. The investigators perform large-scale multi-electrode recordings across cortical lamina to determine the transformations of spatio-chromatic representations from cortical inputs (where color channels are separate) to cortical outputs (where they are mixed).

These recordings are interpreted using nonlinear data-driven models that can provide high-resolution spatio-chromatic maps of the stimuli driving each V1 neuron, and distinguish the underlying computations being performed at each stage. Such characterizations are pushed to achieve cone-resolution by leveraging novel model-based eye-tracking that can account for small eye movements with an order-of-magnitude finer sensitivity than standard approaches.

The first Aim determines the set of principles governing how spatial and chromatic information is combined in V1, which sets the foundation for processing throughout the visual pathway. The second Aim determines whether these rules are the same in the area of cortex processing the center-of-gaze (fovea), which is responsible for high-acuity color vision.

Finally, the last Aim establishes a population decoding framework for linking spatio-chromatic sensitivity of individual V1 cells to the larger systems-wide goals of the visual cortex in processing natural color vision.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.