EAGER: Harnessing Accurate Bias in Large-Scale Language Models

Machine learning models reflect patterns in the data they are trained on, and can, unfortunately, exhibit negative social biases such as prejudice, sexism, or racism. Most research seeks to mitigate this bias, but this work flips the paradigm and explores an alternative by asking: can the bias in machine learning models be harnessed for good? There is strong evidence that some language models exhibit a property called "accurate bias": the patterns captured by the models correlate strongly with human values, judgements, and opinions in ways that are accurately intertwined with time, geography, personal identity, and cultural milieu.

In fact, the correlations are so strong and fine-grained that models exhibiting accurate bias can be studied as a surrogate for human subjects, implying researchers can derive actionable insight by experimenting on models in ways that are not possible with humans. By developing a robust methodology and best practices for extracting and analyzing the accurate bias in language models, it is possible to develop new tools for the social sciences, and could revolutionize any field that studies humans, such as psychology, cognitive science, or political science.

To accomplish these goals, this EArly Grant for Exploratory Research (EAGER) will systematically study language models to determine the possibilities and limitations of accurate bias. As an EAGER, these research activities will be highly exploratory, designed to amass preliminary results and develop technical proofs of concept to support future research.

The work will blend methods from machine learning and social sciences to develop a preliminary theory of accurate bias, and a suite of accompanying methodological and technical best practices. By studying the feasibility of leveraging accurate bias in large-scale language models, this work could deliver fundamental insights into the values, opinions and thought processes of humans.

This work could also deliver insights into how to improve language models, including improving their ability to reason symbolically, and a deeper understanding of the relationship between prompt engineering, data curation, fine-tuning, and the informativity of the final model. Technical elements of our proposal, such as work on prompt engineering and controllable text generation, could have significant applicability outside the context of social science research, and stand on their own right as advances of interest to the machine learning community.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.