SaTC: CORE: Small: Enhancing Security and Mitigating Harm in AI-Generated Vision Language Models

The extraordinary benefits of large generative AI models also come with a substantial risk of misuse and potential for harm. Given that roughly 3.2 billion images are uploaded daily on social networks and a rapidly growing percentage of these are AI generated, the need for robust multimodal harm prevention is more pressing now than ever. This project takes significant strides toward these needs by detecting toxicity and bias in AI-generated vision language model content.

The techniques to be developed will also support automatic detoxification of harmful images. These techniques will be valuable in many domains, and can help stakeholders in government, regulatory bodies, and policy making. This project will engage journalism and other students in the project.

The efforts to promote broadening participation in computing and improve diversity include undergraduate research internships and an annual AI summer camp for high school students.

This project pursues three technical objectives. The first is a prompting framework for harmful content provenance in AI-generated vision language models with the use of a novel prompting method utilizing multimodal knowledge graphs, organized by who, what, when, where, and why semantic schema, and stored and optimized utilizing techniques such as joint embedding, contrastive learning, and negative sampling methods.

A second objective is machine unlearning as a proactive measure to mitigate biases within AI-generated vision language models. The third objective is to detoxify harmful images through selective blurring of harmful segments. The project’s focus is on harm reduction particularly for vulnerable populations.

The project’s evaluation framework includes automated metrics and human evaluations. The project will share open-source web codebase, datasets and demos that can be tested live.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.