Taking Large-Scale Surveys into the Future

Society has come to rely on large-scale surveys of human populations. Until recently, in-person surveys have been seen as the ‘gold standard,’ primarily because of their high response rates. However, cooperation with such surveys has been declining dramatically.

At the same time, an increasing share of the U.S. population has gained access to the internet, which has given rise to web surveys. Such surveys are much less expensive than in-person ones, are not vulnerable to interruptions related to events like pandemics, and are not subject to errors that interviewers may introduce. At present, however, our understanding of the trade-offs between in-person and web administration for lengthy surveys is not well developed.

The project assesses the impact of converting the in-person General Social Survey -- an essential part of the data infrastructure for social science in the United States -- to a state-of-the-art web survey. The results will be of significant value to survey researchers.

The project conducts an experimental study of the effects of survey mode and delivery format using the 2022 General Social Survey (GSS) questionnaire. A nationally representative sample of households will be randomly assigned to one of two different versions of the web survey. One version will be a single long instrument.

The other version will break up the 90-minute GSS into three parts, asking respondents to complete each module at three separate times. The goal is to reduce respondent burden by breaking up the respondent’s task into more manageable pieces. The web surveys will be administered at approximately the same time as the 2022 in-person GSS.

Comparison of the in-person GSS to the long web survey will measure the effect of mode of administration. Comparison of the long versus the modular web surveys will provide a measure of the effects of breaking up the survey into shorter tasks. Analysis will compare the three surveys by: 1) response rates and indicators of non-response bias; 2) indicators of the response process and data quality; 3) GSS demographic, behavioral, and opinion measures; and 4) costs.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.