CAREER: Structural Estimation and Optimization for Partially Observable Markov Decision Processes and Markov Games

This Faculty Early Career Development Program (CAREER) grant will contribute to the Nation's economic prosperity by developing analytical methods to enhance security and risk management of supply chains. As a particular use case, the project will focus on sustainable seafood supply chain operations. The US is the second largest consumer of seafood in the world, and fisheries agencies are seeking substantial reforms in management practices to better manage fishery population dynamics.

The effectiveness of management strategies hinges upon fish stock assessment which is subject to many sources of uncertainties and noisy information, and is further compounded by illegal, unreported and unregulated (IUU) fishing, which steals natural resources, threatens ocean ecosystems and seafood supply, and undermines port and maritime security. This award supports development of a new framework, analytics, and algorithms that can learn preferences and behavior of fishermen and fishing adversaries from imperfect data, identify ways to modulate their behavior, and search for effective strategies to promote sustainable operations and to combat IUU fishing.

The educational plan will utilize similar methods to elicit students' needs and preferences from observable data and to design effective education strategies that promote inclusion, equity, diversity, and accessibility. The research will be informed through collaboration with the US Coast Guard Academy and the North Carolina Division of Marine Fisheries.

In addition, tutorials and workshops on data analytics involving both students and supply chain practitioners will provide important recruitment and outreach opportunities.

This research will investigate a methodological framework comprising structural estimation, optimization, and integrated analysis for dynamic decision making under imperfect information. The current literature on learning and optimizing dynamic decisions mainly assumes that the system is perfectly observable. While there is an extensive literature on the analysis and optimization for partially observable Markov decision processes (POMDPs), this project will focus on the inverse estimation of the primitives of a POMDP model based upon observable histories, an understudied area.

Both optimization and inverse estimation for multi-party decision processes are also considered through partially observable Markov games (POMGs). This research address a knowledge gap by developing (i) new estimation methods to learn model parameters of POMDPs and POMGs from their corresponding data trajectories; (ii) efficient solution procedures for leader-follower POMGs with imprecise reward; and (iii) an integrated methodology of Estimation-Optimization-Analysis based on POMDPs and POMGs to improve an agent's performance by learning, targeted modulating, and adapting to other agents' decision behaviors.

The methodologies will be applied to improve fish stock rebuilding efforts, support defense agencies in combating IUU fishing, and identify best practices of course delivery strategies in education.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.