CAREER: Novel Neural Network Architectures Inspired by Optimization Algorithms

Machine learning, particularly through deep neural networks, has revolutionized society, transforming fields such as computer vision and natural language processing and profoundly impacting daily life. In chemical engineering and process systems engineering (PSE), neural networks have made significant contributions across various scales—from designing new molecules for drug discovery to simulating chemical plant operations.

However, a key limitation remains: the “black-box” nature of these models makes them difficult to interpret, often resulting in outputs that do not adhere to essential physical laws. This lack of reliability is especially concerning in safety-critical applications like process design and control, where compliance with strict physical constraints is crucial.

To address this, a promising solution called physics-informed neural networks (PINNs) has emerged, embedding physical laws within neural networks to improve their accuracy and reliability. Still, PINNs have limitations, particularly in enforcing nonlinear and logical constraints that are common in PSE. This project proposes developing optimization-inspired neural networks (OINNs), a new class of neural architectures that integrate optimization principles to rigorously enforce physical and logical constraints.

This approach not only enhances model reliability but also aims to broaden the utility of machine learning in engineering and beyond. In addition to the scientific advancements, the project includes educational initiatives to equip the next generation of chemical engineers with foundational AI and optimization skills. The Principal Investigator (PI) will redesign an existing data science course at Purdue University, emphasizing AI's role in engineering and integrating machine learning with core chemical engineering principles.

The PI will recruit undergraduate researchers through the Research Experiences for Undergraduates (REU) program. Moreover, the PI will engage K-12 educators and students through outreach programs, including the development of a video game and working with K-12 teachers. These activities aim to broaden public understanding of engineering and inspire diverse young learners to explore STEM fields.

This project proposes the development of optimization-inspired neural networks (OINNs) that incorporate strict, physically meaningful constraints directly within their architecture. These networks will be designed to address limitations of physics-informed neural networks (PINNs), which often rely on “soft constraints” that do not rigorously enforce physical laws and may not be suitable for safety-critical applications.

The OINN architecture integrates optimization theory, enabling strict adherence to both linear and nonlinear constraints, as well as embedding domain-specific knowledge and accommodating uncertainties in parameters. To achieve these goals, the research will leverage optimization techniques from linear programming, conic programming, and mixed-integer programming to build layers that represent various physical and logical constraints.

The OINN framework will be benchmarked against PINNs and other models in process design, process control, and chemical structure prediction, assessing improvements in prediction accuracy, interpretability, and constraint satisfaction. Through this approach, the project aims to establish a new approach for reliable and explainable machine learning models in process systems engineering.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.