RI: Small: Using and Gathering Data for Efficient Batch Reinforcement Learning

Imagine if we could provide each child with the right support, at the right time, for helping them learn best, or to ensure a diabetes patient is being given the best interventions to help them manage their chronic condition over time at home. Unfortunately such personalization is expensive. More scalable computerized approaches can lack the real-time information needed to provide effective personalization, or the ability to specialize interventions.

However, the huge rise in more user-friendly software tools means that it is now possible to do such targeted personalization in a broad array of settings. This research will develop new methods for leveraging existing data, and create algorithms to acquire new data in a way that is compatible with the limitations of common systems. This work could help enable personalized interventions across a much broader array of applications than is currently benefiting from such approaches.

The research will be particularly focused on the technical challenges arising from areas like education and healthcare.

More specifically, this research will create data efficient algorithms and statistical estimators for leveraging past datasets about decisions made and their outcomes, and for acquiring new batch data that might lead to better results to create decision policies-- mappings from features describing the current context to a particular decision or intervention. In particular, the project will center on developing new algorithms that optimize policies with data efficient, minimal assumption lower statistical bounds on their future performance; bound the benefit of gathering a budget of additional data; and, inspired by insights from optimal experimental design, create algorithms for constructing non-adaptive policies that can be used to gather data that then can be leveraged to identify a near-optimal decision policy.

The research will focus on both settings where a single decision is made for a particular context, and where a sequence of decisions are made and the decisions made impact the next context observed (common in sequential decision making under uncertainty processes).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.