Collaborative Research: PPoSS: Planning: Model-Driven Compiler Optimization and Algorithm-Architecture Co-Design for Scalable Machine Learning

There is an inexorable need for increased computational performance and improved energy efficiency in the development and use of machine-learning (ML) models. Currently available frameworks for ML have limitations in developing non-traditional models, e.g., using tensor networks, as well as in developing and using models that are too large to fit in the physical memory of processors.

The design of efficient hardware accelerators and the mapping of ML algorithms to them is another challenge. This planning project presents a plan of action to address these needs via advances to model-driven compiler optimization. The research conducted in this project is enhancing productivity, performance, and portability in developing software for ML.

It is enabling new ML applications to be developed with high productivity, with high achieved performance, and performance-portability over a diverse set of hardware platforms. It is enabling greater "democratization of ML", permitting researchers who only have access to low-end hardware platforms to be able to run the largest models -- infeasible today due to limitations of existing ML frameworks.

The project involves training activities tailored for K-12 students, undergraduate students, and graduate students.

In this planning project, the following primary technical directions are explored: (1) ML Algorithms: flexible new ML models, offering trade-offs between model size, model execution time, model accuracy, and energy efficiency; (2) Optimizing Compilers: advances in polyhedral compiler optimization to enable parametric tilesize optimization and code generation for diverse target platforms, including CPUs, GPUs, and accelerators; (3) ML Accelerators: new ML accelerator designs for sparse and dense operators, optimized for multiple criteria via comprehensive design space exploration. Broader impact aims of the project include the "Democratization of AI”, to enable state-of-the-art ML models to be used by all, on widely available non-state-of-the-art hardware.

To achieve these goals, the project integrates expertise in computer architecture, optimizing compilers, ML algorithms, and high-performance computing.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.