Collaborative Research: SaTC: CORE: Medium: Broad-Spectrum Facial Image Protection with Provable Privacy Guarantees

With the growing ubiquity of smartphones and other mobile devices, image sharing is gaining increasing popularity in social networks. Privacy protection has now become a crucial issue to be addressed because sharing images can reveal personal and social environments and details of private lives. While many social media allow users to set privacy preferences, these are rarely sufficient due to the limitations of the options, complexity of the problem, and the tedious nature of privacy configuration.

The objective of this project is to design an intelligent and automatic broad-spectrum image protection system that offers provable privacy guarantees for multiple parties involved in social image sharing while maintaining image quality. Specifically, the researchers on this project aim to overcome the following challenges: (i) Privacy protection for human subjects and sensitive objects in the background of the images; (ii) Consideration of location-dependent image sensitivities whereby images taken at certain places (e.g., pubs, hospitals) may impact privacy, such as people in the images who do not want their occurrences or co-occurrences at those locations to be known; (iii) Strong enforcement of the privacy protection that conforms with different privacy needs of multiple people in the same image.

The success of the proposed research will address the rising privacy concerns of image sharing on social sites and benefit billions of social network users. A range of educational activities will be also carried out including curriculum development, professional training for college students and outreach to K-12 teachers and students, with emphasis on under-represented groups.

This project will greatly advance the state-of-the-art facial privacy protection during online image sharing with the following innovative research ideas. First, a new image privacy policy language and an efficient policy management system will be designed for managing broad-spectrum privacy concerns of multiple parties. Second, formal privacy models will be defined to quantify privacy risks and provide provable privacy guarantees during policy enforcement.

Third, new deep-learning-based image modification approaches such as facial modification/replacement and image cropping will be investigated to simultaneously address different users' privacy needs regarding the same image while preserving the aesthetics nature. Finally, a combination of interface and incentive design will be conducted to obtain more accurate user feedback and evaluate the effectiveness and practicality of the proposed system.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.