Collaborative Research: CNS Core: Medium: Dynamic Data-driven Systems - Theory and Applications

Modern computer systems must be continually optimized in a data-driven manner to maintain performance, even as their deployment and workload environments change. This holds for traditional systems like content delivery networks and emerging architectures such as edge/cloud systems. The design of dynamic data-driven systems requires both theoretical advancements and new systems architectures.

A key challenge is a tradeoff between optimality, i.e., choosing an optimal deployment for the current environment in terms of performance and/or cost, and smoothness, i.e., ensuring that the deployment changes are not too costly at any point. This project seeks to develop tools at the intersection of machine learning and optimization that enable systems to balance between optimality and smoothness.

Further, this project deploys and empirically evaluates these tools in the context of 360 video streaming as a representative case study.

Smoothness is not a traditional system performance measure, and so it is typically enforced only in ad hoc ways by existing systems. However, it is a crucial consideration for systems that seek to continuously optimize their configuration since the switching costs associated with changing configurations can be significant. Managing the tradeoff between optimality and smoothness in a rigorous fashion can lead to dramatic improvements; however, it is challenging since it requires a robust data-driven design that can determine whether it is worth incurring a switching cost in the present, without knowledge of the future environment.

This project develops analytic tools that enable the design of algorithms for dynamic systems that balance optimality and smoothness through the integration of data-driven and optimization approaches. There are also planned test-bed deployment activities for 360 video streaming.

The project will provide new foundational tools for the design of dynamic systems across multiple application areas. While we choose video streaming as our target application, the proposed fundamental research is applicable much more broadly. Notably, this project broadens the participation of underrepresented groups in STEM areas through programs at both K-12 and undergraduate levels.

Planned activities include developing accelerated mathematics programs for middle-school students, summer programs for middle-school and high-school students, and summer research programs for undergraduate students.

This is a collaborative project with investigators from the University of Massachusetts Amherst, California Institute of Technology, and the State University of New York at Stony Brook. The results of this project will be maintained on the project website at https://groups.cs.umass.edu/hajiesmaili/soco/. These will include technical reports of the research findings, software prototypes of the algorithms designed, datasets, and experimental results collected for the 360 video streaming experiments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.