CAREER: Automated and Efficient Machine Learning as a Service

Machine-Learning-as-a-Service (MLaaS) is an emerging computing paradigm that provides optimized execution of machine learning tasks, such as model design, model training, and model serving, on cloud infrastructure. Explosive growth in model complexity and data size along with the surging demands of MLaaS is already resulting in substantial increases in computational resource and energy requirements.

Unfortunately, existing MLaaS systems have poor resource management and limited support for user specified performance and cost requirements, exacerbating waste in computing resources and energy. This project aims to utilize the unique features of MLaaS to design efficient, automated, and user-centric MLaaS systems. This approach will significantly reduce resource waste and shorten the model design cycles through a variety of novel optimization approaches and by eliminating candidate models that fail to meet model serving latency and target accuracy.

To support complete MLaaS workflow, this project will also develop MLaaS model serving methodologies that can meet service level latency requirements with minimum resource consumption using intelligent autoscaling.

This project has the potential to tremendously reduce the resource and energy consumptions as well as the carbon footprint associated with the fast-growing societal demands in machine learning and cloud computing. Important insights and technologies will be produced targeting resource management and energy saving of the next-generation machine learning systems and cloud infrastructure.

The findings of this project will also contribute to related fields of parallel and distributed systems, performance evaluation and optimization, and green computing. This project will carry out substantial integrated education activities including new course and online education development, integration of industry feedback in education. Additionally, the work will impact undergraduate and graduate students by training them in the art of system optimization combined with the latest machine learning domain knowledge while combining outreach and engagement of students from underrepresented groups and especially women.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.