CAREER: Robust, Fair, and Culturally Aware Commonsense Reasoning in Natural Language

Recent advances in artificial intelligence have led to the proliferation of Large Language Models (LLMs). LLMs are models that cane be used for interactions with human users through written language; for example, a user inputs an instruction or question in English to the LLM-based program, and the LLM outputs a response in fluent English. With these linguistic capabilities, LLMs are being developed for use in applications that are both ubiquitous (e.g., internet search, customer support, writing tools) and high-stakes (e.g., mental health care, classroom education, assistive technology for people with disabilities).

Despite their growing adoption, many fundamental properties of LLMs aren’t yet well understood, and pressing questions remain about when and whether LLMs can be entrusted with such important tasks. For example, when instructed to make simple predictions about every-day situations, like cooking a meal or riding in a vehicle, LLMs can make strange and surprising errors, exhibiting concerning lapses in basic common sense judgment and reasoning abilities.

Additionally, these predictions made by LLMs can reflect social stereotypes and cultural assumptions which, at best, limit the usefulness of the technology for certain populations and, at worst, cause active harm. This project seeks to address unfairness and bias due to stereotyping and cultural context by proposing a generalized framework for defeasible commonsense inference in natural language in which a system compares two similar situations with respect to their support for a given inference.

The proposed work aims at developing scientific methods to measure and improve the abilities of LLMs to (1) reason correctly about every-day situations, (2) do so in a manner that is fair and unprejudiced, and (3) adapt these reasoning abilities across specific cultural contexts. By measuring these fundamental capabilities of LLMs, we can better understand and mitigate the risks of applying this technology in high-stakes settings.

The three phases of the project focus on the (1) robustness, (2) social fairness, and (3) cultural awareness dimensions of reasoning in LLMs. The project assumes a basic task formulation in which a situation description is provided to an LLM (e.g., “Someone drops a glass”), and the LLM must either evaluate a possible inference, or generate an inference from scratch (“The glass breaks”).

In phase 1, methods will be developed to automatically manipulate situation descriptions in order to train and evaluate an LLM’s ability to make nuanced inferences, with the goal of learning to distinguish which factors influence a particular inference and which ones do not (e.g., when trying to predict if a dropped glass is going to break, the thickness of the glass matters but the color of the glass does not.) In phase 2, methods will be developed to automatically test whether LLMs make socially fair inferences, for example via name substitution tests, and to intervene when a proposed output is detected as unfair. In phase 3, survey participants from the U.S. and Ghana will answer multiple stages of questions about every-day situations; the collected data will be used to develop evaluation questions for a case study on the adaptability of LLMs across these two cultural settings.

For each phase of the project, the resulting datasets, methods, and scientific findings will be made available to the public.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.