CAREER: Explainability for Integrated Cyber-Physical Systems

Integrated Cyber-Physical Systems (i-CPS) have transformed industries like manufacturing, transportation, healthcare, and energy management. I-CPS is characterized by multiple CPS services functioning concurrently within the same environment, high involvement of domain experts and everyday users in decision-making processes, real-time interactions with other systems, and operation within uncertain and constantly changing physical environments.

Examples of such systems include smart cities, smart agriculture, and intelligent transportation, among others. However, stakeholders often hesitate to trust these systems due to a lack of explainability, which raises concerns about reliability, especially in safety-critical areas. Additionally, while advancements in deep learning have enabled powerful CPS capabilities, understanding of these systems is declining.

Despite the significance and urgency of enabling explainability for i-CPS, research in explainable CPS with AI-driven components is still in its infancy. This research aims to develop explainable and trustworthy i-CPS capable of justifying their decisions and incorporating field operators’ queries and domain knowledge to adapt to evolving environments.

The technical contributions to CPS research and education encompass four thrusts. Thrust I develops a new explainable architecture with formal explainers that interpret individual components of i-CPS and their interactions. Innovations include the first explainable CPS architecture, novel formal methods-based approaches addressing five layers of i-CPS, and explainable online decision support for “what if” scenarios.

Thrust II formalizes and incorporates user domain knowledge, specifications, and feedback to enhance the learning-enabled components of CPS. Key advances include developing new formal knowledge distillation and unlearning approaches with uncertainty quantification that quickly adapt the system to shifting distributions in a lifelong evolving deployment.

Thrust III evaluates the proposed methods through a real-world emergency response deployment in Nashville, TN, involving multiple CPS-based city services and stakeholders from different departments. Innovations include a logic-based LLM-enabled system that connects our proposed explainers to field operators, real-world deployment principles, and qualitative and quantitative metrics to assess explainability and impact.

Thrust IV develops an interdisciplinary education plan. The key innovation is to bring rigorousness and explainability to CPS classes in Computer Science and create a new course, CPS in the field, for multi-disciplinary students.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.