EAGER: Visual Representation Learning Using Mixed Labeled and Unlabeled Data

Recent advances in deep learning has led to great results in visual recognition and object detection. These deep learning models have various applications from self-driving cars to early disease diagnosis and household robots. However, most such models are supervised, meaning that they need large scale manually annotated datasets to tune the parameters, and obtaining the annotation may be expensive in many applications.

This project explores a family of self-supervised learning algorithms where the learning is based on unlabeled data only. The new models can learn visual features that can be used for various visual recognition tasks including object detection and action recognition. This project provides research opportunities for under-represented groups and integrates research outcomes into the course curriculum.

This project studies a family of self-supervised learning algorithms that can learn rich features from unlabeled images and videos. Self-supervised learning algorithms harvest the knowledge from unlabeled data by modeling some regularity in the space of natural images or videos. This project studies novel self-supervised learning algorithms based on constraining the learning by relating transformations of images to transformations of their representations.

Moreover, this project studies a novel multi-task learning framework for aggregating the knowledge learned from multiple supervised and self-supervised learning algorithms. This algorithm uses quantization methods to ignore the task specific details of the representation in transferring the knowledge. This algorithm results in a rich set of representations that generalize well across various visual recognition tasks.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.