How does visual long-term memory implement control over visual attention?

In our everyday lives we often apply our prior knowledge while simultaneously interacting with the natural world in all its complexity. Consider a simple trip to the grocery store. First, one might have to find your car keys on a messy desk.

Your desk may have items like a holiday photo, pencil sharpener or a computer that naturally draw attention but are of no use in going to the store, so they must be ignored. Then, having found the keys and starting to drive, one must forget the keys and instead remember to attend to children at play, pedestrians, cyclists, other drivers, traffic signals, street signs and landmarks to safely follow the right route to the store.

Despite all the challenges apparent in even this seemingly simple task, people accomplish it, and far more complex tasks, with apparent ease. This project seeks to gain a better understanding of how prior knowledge guides our current actions and perceptions of the environment. A better handle on this process can help us understand a fundamental element of human ability that still eludes even our most advanced artificial intelligence systems.

A substantial benefit to society is inherent in the research: the potential to understand and thereby improve attentional control for drivers or air traffic controllers. In addition to the broader impact in psychology and neuroscience, scientists will engage in public outreach such as the Annual Brain Blast events, the Brain Bios podcast, and the K-12 Teacher-Neuroscientist partnership to improve understanding and appreciation of neuroscience.

To understand how prior knowledge guides perception and current behavior, this proposal involves non-invasively recording electrical brain activity (with electroencephalography or EEG) from people searching for particular objects in cluttered scenes. These scenes are picture of objects arranged randomly on a computer screen. The researchers will use these recordings to investigate whether prior knowledge (templates in visual long-term memory) needs to be intentionally held in mind to guide visual attention and visuospatial behavior such as visual target detection.

Using this approach, researchers can measure brain activity that tracks what you store in memory, as well as what you pay attention to in the scenes. Analysis of these data will reveal whether brain activity related to object memory is all you need to pay attention to that object. A key question these experiments will resolve is the relative contribution of long-term object memory and short-term working memory to attentional object search.

Competing hypotheses will be tested using EEG combined with noninvasive transcranial brain stimulation. Scientists will also study whether if you have previously found an object in a scene whether it then becomes particularly difficult to ignore in the future. The hypothesis is that previously having searched for an object will lead to more attention being drawn to that object in a future search.

Experiments will test whether this effect can be observed in the response of the brain. Preliminary results suggest that memory shapes attention for specific objects and moreover, that people can simultaneously hold multiple objects in memory, and use these search templates to look for multiple objects at the same time, a feat that is currently impossible for artificial intelligence systems.

These findings will be used to make scientific progress by testing models of human attention and memory, as well as helping us design better artificial intelligence systems that can match human ability and potentially better partner with humans as well.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.