CNS Core: Small: NSF-MeitY: A Unified Framework for Video Analytics Optimization and Adaptation

Edge-cloud video analytics systems, also known as video analytics pipelines (VAPs), are being deployed in major cities around the world to support diverse applications, spanning public safety, transportation, healthcare, retail, and more. Unfortunately, the status quo of developing and deploying a VAP for a new application is largely manual and labor-intensive. (1) A VAP developer must implement end-to-end pipelines on heterogeneous hardware by writing low-level software for each component. (2) The developer must pick and choose the right set of machine learning models and their placements pre-deployment to minimize the cloud computing bill while providing acceptable latency and accuracy. (3) The developer must adapt the pipeline post-deployment in response to changes in the environment (e.g., network bandwidth, light conditions, traffic density).

Each step is challenging to perform, presenting significant hurdles to the development and deployment of new VAP applications. This project aims to develop a unified framework to simplify and automate video analytics pipeline development, optimization, and adaptation by streamlining all three steps in developing and deploying a new VAP application. Under such a framework, application domain experts specify high-level analytics tasks (logical operators) to be performed on the camera frames and all candidate physical implementations for each logical operator (physical operators).

Pipeline authors describe the pipelines via graphs, and the framework will automatically generate an optimal physical implementation for initial deployment and deploy an adaptation engine that monitors changes in environmental conditions and automates adaptation to new physical plans that satisfy application latency and accuracy constraints.

The project will have direct, practical implications to the video analytics industry and is poised for substantial societal impacts. (1) Impact on industry: The proposed VAP framework will advance the state-of-the-art by providing a much-needed solution that significantly eases the development effort of VAP vendors and shortens the time-to-deployment of new and increasingly diverse VAP applications. (2) Impact on society: The technologies developed for enabling the framework will foster wide adoption of important societal VAP applications, spanning transportation, healthcare, retail, public safety, and more. (3) Impact on other research fields: The work will have far-reaching impacts outside the area of video analytics systems by developing general query optimization techniques, which will also be applicable to traditional database management. The technologies developed in the project will be disseminated and transferred to the broader research community and the IT industry.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.