I-Corps: Deep Forgery Detection Technology

The broader impact/commercial potential of this I-Corps project is the potential development of innovations to authenticate multimedia objects to be used as evidence in legal proceedings. Digital media admitted as legal evidence must meet court-defined values for authenticity, integrity, and veracity. Moreover, deep fakes or multimedia content that has been partially or completely computer-generated with the intent to deceive has progressed to a level where detection in some cases is beyond the ability of humans to achieve unassisted.

In the legal public sector, this technology may be leveraged by a judge or court or used to gather evidence by the police, prosecutor, FBI, or other government entity. On the private side, its use is not only in courtrooms but in depositions, or during eDiscovery, a process already used for vetting electronic documents like email.

This I-Corps project is based on the development of new technology to possibly detect audio-visual forgeries, including various types of deep fakes used in the manipulation and/or falsification of digital multimedia. Existing Forensic Examiners typically do not satisfy the requirements of criminal justice and social media platforms; they are limited in scope and are unable to answer complex questions regarding the nature of the forgery.

This is particularly true when the media is recorded using low-powered devices and/or has been subjected to an anti-forensic attack. Unlike the state-of-the-art tools which rely on deep learning algorithms for pattern matching to detect single forgeries (e.g., video or voice deep fakes), this innovation uses a multimodal and neuro-symbolic artificial intelligence (AI) approach, and could detect multiple complex audiovisual forgeries, including deep fakes by performing the deep inspection at the file- and frame- levels.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.