SaTC: CORE: Small: Advancing Model Forensics with Systematic Parsing, Injection Detection, and Model Provenance Attribution

This project will improve the ability to analyze attacks on formats for sharing machine learning models, as part of a larger research program to advance digital forensics techniques around artificial intelligence (AI). Digital forensics involves the scientific acquisition, authentication, and analysis of digital evidence as it applies to the law. It is used to gather evidence in court cases, aiding in the conviction of criminals, saving lives, and promoting equity and justice.

The widespread use of AI in our daily lives and critical systems necessitates an understanding of how to investigate AI failures when they occur. However, little work currently exists in forensics for AI systems in general or shared machine learning models in particular. These shared model formats are important to developing AI-based systems that cross organizational boundaries, providing a practically important starting point for advancing science, practice, and education around digital forensics for AI-based systems.

In particular, the work addresses a legal forensic necessity for the admissibility of digital evidence in courts of law that will have a substantial impact on the cybersecurity industry. The success of this initiative will have far-reaching implications for industries that rely heavily on ML models, such as government, finance, healthcare, and transportation, by improving their ability to detect and mitigate potential threats and vulnerabilities.

To achieve this, the project focuses on the field of model forensics, particularly on HDF5 model file forensic parsing, data injection, and provenance attribution. The primary objective is to enhance the current state-of-the-art and develop novel tools and techniques to aid practitioners in analyzing and connecting model files to their respective training systems.

The project has three specific aims: (1) Develop and deploy forensic tools for reconstructing HDF5 model files, and assess their precision in data recovery from extensive datasets using benchmark performance indicators; (2) Evaluate data injection and detection methods in HDF5 by implementing a large-scale experiment and assessing performance using established detection and retrieval metrics; and (3) Design and appraise a systematic approach for AI model ballistics (associating a model with its training hardware, software, and data) using standardized testing protocols, reference datasets, and performance criteria to ensure accurate, consistent, and replicable outcomes.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.