CAREER: Declarative AI Data Cleaning with Guarantees

The rise of powerful and data-hungry artificial intelligence (AI) models has shifted the focus of research from model-centric AI to data-centric AI, since the quality of machine learning often reflects the quality of the underlying data. For example, one challenge in measuring health equity is the availability and quality of demographic data in decision making applications -- data with attributes of race, ethnicity, language and gender, respectively.

The primary focus of data-centric AI has been acquiring, cleaning, and improving data quality, but not the downstream performance of models. Learning from massive data requires not only high-quality data but also large and expensive computational resources that consume substantial energy that produce a massive carbon footprint. The proposed work involves a model-driven view of data quality that will consider not only the completeness and correctness of data but also the performance of downstream models.

Summarizing data so that machine learning can be done efficiently, without significant loss in other aspects of model quality, goes hand in hand with model-driven data cleaning. This co-optimization of data and model involves treating all important dimensions of model quality (e.g., accuracy, fairness, and robustness) as first-class citizens during data acquisition.

With the rise of large-scale models, model training on summarized datasets that are free of bias will open avenues to addressing societal concerns around the cost of model training and fine-tuning, the impact of training and inference on the environment, and the social debacles of biased outputs of these models.

This project will advocate a novel paradigm for model-driven data cleaning, focusing on model-aware data acquisition, coreset (weighted subset) construction, and the eventual declarativity of the cleaning process. The project will introduce a set of theoretically rigorous model-aware data quality constraints based on distribution representation, group fairness, coverage, diversity, and reconstructability.

This project will address four fundamental research challenges: (1) the synergies of distribution-aware and model-aware data quality constraints -- jointly studying the theory of data quality, model performance, and model sustainability; (2) model-driven data repairing -- developing an extensible suite of data acquisition and coreset construction techniques that holistically solve data quality issues while keeping the process efficient; (3) data cleaning for various modalities—contextualizing model and data quality for tabular and multi-dimensional data; and (4) declarative AI data cleaning --developing an optimizer that efficiently generates the most effective data cleaning plans with guarantees on model performance.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.