CRII: III: A Bias-Aware Approach to Modeling Users in Interactive Information Retrieval

People often act intuitively and are subject to systematic biases when making decisions under uncertainty due to their inability to calculate all the possible consequences of their choices - a fundamental cognitive phenomenon called "bounded rationality". Without proactive information supports, these decisions could be driven by misleading information, cognitive biases and heuristics and may result in significant deviations from desired outcomes: Health information seekers may easily trust medical misinformation that confirms their existing expectations.

Students often heavily rely on top ranked results and stop at short satisficing answers, rather than exploring more credible and informative Web pages. Online shoppers tend to quickly accept immediate mediocre recommendations after encountering several bad quality products (with low reference levels), without examining all available options. By investigating users’ systematic biases, this project aims to break new grounds for information retrieval (IR) research and address fundamental bottlenecks in the development of bias-aware search systems.

The outcomes of this project can help people better leverage the power of information through 1) incorporating the knowledge about their biases into search algorithms, and 2) proactively capturing bias-related search problems and promoting informed, unbiased decision-making.

The project seeks to study users’ systematic biases and leverage the learned knowledge in improving the explanatory and predicative power of IR models. The technical aims of the project include: (1) understanding the relationships between search interactions and users’ systematic biases; (2) building bias-aware prediction models of search interactions; (3) developing a scalable and potentially transformative approach to modeling users and their decision-making processes under biases in interactive IR.

To achieve these goals, the investigator will conduct a series of user studies and experiments. First, the research team will carry out controlled lab studies to examine the associations between users’ search interactions and several major systematic biases that have been widely confirmed by behavioral experiments, including reference dependence, framing effect, and loss aversion.

Then, the team will extract new features and create bias-aware models for predicting users’ search behavior, experience, and problems. Finally, this project will apply deep neural networks in developing more fine-grained bias-aware models based on large scale test collections and search logs, and evaluate the performance of modified models in a wider range of search scenarios.

The proposed models can provide a more solid behavioral and psychological basis for supporting the simulations of search interactions. Such simulations, properly constructed, could address major challenges in the design of boundedly-rational formal models and bias-aware intelligent systems.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.