The Effects of Panel Conditioning on Data Quality

This research project will advance scientific understanding of the mechanisms responsible for panel conditioning as well as the effects of panel conditioning on data quality. Panel conditioning refers to changes in reporting, attitudes, and behavior brought about by participating in a wave of a multi-wave panel survey. These changes are realized in responses at subsequent waves of the panel and introduce error in measures of change.

Since panel surveys are primarily designed to measure change, panel conditioning is an important source of measurement error. Prior research largely has been based on observational data that could not clearly identify either the magnitude or the sources of panel conditioning. This project will address this limitation by administering a series of experimental studies to shed light on mechanisms underlying panel conditioning effects.

The project will have broad impacts for social scientists who rely on data from panel surveys such as the Panel Survey of Income Dynamics, the American National Election Studies, and panel surveys administered by the Federal statistical system such as the Current Population Survey and the National Crime Victimization Survey. Results from this project will inform efforts to reduce panel conditioning effects and/or to adjust for them statistically.

The project will promote teaching, training, and learning through courses offered in graduate programs, short courses, publications, and conference presentations.

This project will conduct a series of experiments to address many of the open questions about panel conditioning. The experimental results will improve understanding about the causes of panel conditioning and provide insights as to when such effects should be expected to occur. Six experiments will be conducted, each examining a specific hypothesis about one type of conditioning with an eye to data quality.

Experiment 1 will examine whether motivated misreporting or improved question understanding is the mechanism underlying decreasing reports of victimization over rounds of a survey. Experiment 2 will compare two formats (grouped versus interleafed) for organizing filter questions, examining how effective they are for reducing misreporting driven by a motivation to reduce burden.

Experiment 3 will investigate whether attitudes become more consistent over time and what cognitive mechanisms might explain changes in consistency. Experiment 4 will examine how repeated questioning affect reports on potentially sensitive topics and what mechanisms underlie any observed changes in reporting. Experiment 5 will explore how different types of questions affect voting behavior and reports about voting, and whether asking respondents if they intend to vote increases the likelihood of voting.

Experiment 6 will examine whether there are practice effects on reports about knowledge and personality with repeated interviews. This award is supported by the MMS Program and a consortium of Federal statistical agencies.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.