CNS Core: Small: Not All Cameras are Created Equal: Systems Support for Highly Adaptive Video Analytics Pipelines

The ubiquity of video camera deployments, coupled with steady improvements in computer vision algorithms, has given rise to a diverse range of video analytics applications. Use cases include surveillance, traffic scheduling, disaster response, and more. Yet despite their promise, video analytics deployments are far from widespread.

A key reason is that video analysis is often prohibitively expensive: video is data-intensive, stressing the network, and analysis typically involves Deep Neural Networks (DNNs) to query video, requiring substantial compute resources. This project aims to design and implement practical video analytics systems that can adapt their execution to most efficiently utilize end-to-end compute and network resources, i.e., across cameras, servers, and the networks between them.

The key insight underlying the proposed work is to adaptively place analytics tasks by leveraging frame-transforming techniques that are diverse in terms of resource requirements and accuracy, e.g., lightweight frame differencing versus expensive object detection DNNs. Along these lines, the project involves three synergistic directions. First, it rigorously classifies existing frame transforming techniques, investigating the correlation between their computation costs, potential data reduction, and impact on response accuracy.

Second, it develops end-to-end systems that can automatically select the appropriate frame transforming technique to run on a camera with the goal of optimizing for response latency and accuracy given the available resources. Third, it develops techniques to extend adaptive video analytics to emerging camera settings, e.g., multi-camera, steerable, energy-harvesting; these systems rely on the extraction of spatial and temporal relationships between camera feeds to guide resource allocation decisions.

The proposed research targets a large slice of the population (given the breadth of video analytics applications), and improves both the accessibility and potential of video analytics deployments. The developed systems enable affordable (but effective) video analytics for organizations of different scale, allowing them to make the most of their available resources.

Furthermore, the work motivates novel applications that were previously deemed impractical, e.g., real-time monitoring of rural areas via energy-harvesting cameras. The project also involves outreach efforts to attract students from populations currently under-represented in computer science. Key to these efforts is magnifying the interdisciplinary nature of video analytics pipelines which span systems, networks, machine learning, and computer vision.

The software and research artifacts designed as part of this project are released on a public website: http://web.cs.ucla.edu/~ravi/adaptive_video_analytics/. The site is regularly maintained and includes replication instructions and packages. Project data are kept on the site for at least 5-years after publication, with extensions based on public interest.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.