Collaborative Research: SaTC: CORE: Medium: Novel Algorithms and Tools for Empowering People Who Are Blind to Safeguard Private Visual Content

People who are blind regularly use personal devices to take pictures and videos and share them with others. Primary reasons are to seek assistance with everyday visual tasks (e.g., recognizing objects, reading mail) and to socialize online. Regardless of the reason, they have no easy or independent means of assessing whether an image or video they are about to share inadvertently contains private information.

For this project, an interdisciplinary team will design and evaluate an automated assistant that alerts users of privacy disclosures in their visual media and edits their pictures and videos to obfuscate any potential private content. If successful, these contributions will enable blind individuals for the first time to independently avoid accidental visual privacy leaks, thereby empowering them to live more independent and connected lives.

This project can also be helpful to the broader population as they sometimes overlook private information in their visual content. The project team anticipates this work would generalize, with minor adaptations, to benefit other populations such as people with low vision, people with cognitive impairments, aging adults, and children.

This project will involve designing novel computer vision algorithms and end-user mechanisms that empower people who are blind to independently safeguard private information in their pictures and videos. Specifically, it will consist of three key tasks: (1) creating back-end computer vision algorithms that learn to locate private content in images and videos by observing only a few examples of it (i.e., few shot semantic segmentation), (2) creating back-end computer vision algorithms that learn to locate the foreground object in images and videos (i.e., foreground object segmentation) in order to support retaining only that content, and (3) establishing effective design choices for an accessible front-end interface that engenders a level of trust that is appropriate given the algorithms' performance.

The team will facilitate future extensions of this work by sharing the generated artifacts including the code for privacy-preserving algorithms, characterization of use with design recommendations for privacy-preserving technology, and a working prototype.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.