Collaborative Research: Aggregating Evidence for the Analysis of Experiments

This award funds research to develop a new methodology to distinguish between research findings that can be generalized across populations, places, and time, and those that cannot be generalized. This research addresses a fundamental challenge in empirical research: determining which experimental or observational findings are generalizable across different environments and populations.

Existing methods do this by using restrictive assumptions, potentially leading to false generalization. By introducing a new methodology to detect generalizability, this research helps identify features of the environment, population characteristics, and treatment conditions that systematically contribute to generalizable results and those that exhibit context-specific or unpredictable results, and therefore not generalizable.

The research results improve the reliability of evidence-based decision recommendations and the quality of decision design. By offering a rigorous approach to identify generalizability, this research makes significant contributions to economics science and beneficially informs decision makers and practitioners. The results of this research aid improved decision making, speed up economic growth, and hence improve living standards.

This award funds a research agenda that develops a new methodology to distinguish between research results that are generalizable and those that are not. Methodologically, the research advances statistical meta-analysis by developing estimators and classification tools that distinguish between predictable (generalizable) and unpredictable (environment-specific) treatment effects.

Unlike standard approaches, this framework allows researchers to pinpoint critical environmental or demographic factors driving effects heterogeneity. The resulting methodology can be integrated into various fields---including economics, public health, and education---enabling more nuanced insights into whether, when, and why certain policies or interventions are particularly effective. Empirically, the project applies these techniques to varied datasets in economics.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.