CNS Core: Small: Ultra Low Power Hardware AI Accelerator for Training at the Edge

Advances in the Artificial Intelligence (AI) domain have enabled a plethora of applications in recent years, many of which were well beyond imagination even a decade ago. A typical use-case scenario involves training a Deep Neural Network (DNN) on a server farm involving powerful processor systems, and then using a trained model for inference at the edge of the network.

To enable efficient and widespread use of inference, there is an emergence of domain-specific architectures (e.g. Google Tensor Processing Unit). The current training at the cloud---inference at the edge model can expose the data to privacy breaches, suffer large data transfer bottlenecks, and scale poorly with our growing smart ecosystem.

With an expected growth to trillions of connected devices by 2035, the cost of data transfer, as well as, server energy expenditures, are set to explode. This research project will realize retraining capabilities at the edge, in a hardware accelerator with limited power and resource constraints. The project will allow AI edge devices to largely operate as their own standalone engines, breaking down their dependency on the cloud.

Specifically, the project will explore problems and solutions in the following directions: (1) managing timing related errors at ultra-low power operation while preserving training convergence; (2) exploiting diverse utilization of hardware components during training for energy efficiency; and (3) developing an open sourced AI hardware edge simulation environment that will spawn further research on low power training at the edge. This research project will establish a foundation for incremental training at the edge, thereby reshaping the ecosystem of AI computation.

The proposed AI edge platform, if successfully developed, can facilitate newcapabilities in our ubiquitous interconnected world. For example, the proposed framework will allow a hand-held fitness device to retrain itself using the personalized traits and behavior of a user, incrementally updating its base model that was trained over huge datasets. The project will engage in an extensive outreach program through three closely-related activities: (i) high school girls will be introduced to hands-on engineering exercises through active participation in the Engineering Extravaganza event at Utah State University; (ii) AI learning modules will be disseminated to K-12 classrooms in Utah; and (iii) the edge AI simulation platform developed in this project will be shared in an open source Github repository, allowing academic and industrial researchers to explore AI hardware design techniques beyond those in this project.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.