I-Corps: Probabilistic artificial intelligence (AI)-based software for proactive data quality assessment

The broader impact/commercial potential of this I-Corps project is the development of a solution that identifies and corrects data data entry errors. The accelerating adoption of e-commerce and digital systems has increased the use of digital data entry systems. This may create challenges for providing equitable service to people with low income and education levels.

The proposed data quality software finds and eliminates user errors at the point of data entry, providing proactive, secure and convenient support for users while filling in forms. The solution may also provide accurate data for companies. Compared to existing data quality solutions, this product eliminates the need for time-consuming and costly post data collection cleaning.

The solution may benefit data-centric domains such as health care, finance, e-commerce, and tax and government benefit application systems where data errors can be costly. The goal is to provide users with proactive help that may reduce incorrect processing and potential issues with timely access. This proposed real-time, error alert and correction technology may benefit data entry specialists and consumers through corrective capability, while the companies that process the digital documents and online forms/web submissions may benefit from the cost-savings in terms of both post-submission corrective costs and potential damage control costs.

This I-Corps project is based on the development of an artificial intelligence-driven analytical process and smart software that helps users submit mistake-free information at the time of data entry. The algorithms analyze the data to reveal unusual entries through the use of basic format checks, data matching, and probabilistic machine learning algorithms.

Context based and personalized probabilistic analytical methods enable both variable and data accuracy assessment. The Bayesian nature of these algorithms allows incorporation of expert opinion and user feedback. The proposed software provides an interface that uses the analytical output and works with the user to verify their data entry before submission.

This interface helps users fix mistakes at the time of data entry. Use of probabilistic algorithms allow the use of relative occurrence weights instead of computing over the whole data base. This makes the process faster with diminished privacy and security concerns over raw data transfer.

The proposed technology involves multiple research areas including Bayesian statistics, data analytics, and software development. This project aims to verify a secure, trustworthy, and validated approach based on a proactive data quality paradigm and a suite of algorithms including probabilistic methods.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.