SHF: Small: Enabling On-Device Bayesian Neural Network Training via An Integrated Architecture-System Approach

Deep-learning-based AI technologies, such as deep convolutional neural networks (DNNs), have recently achieved amazing success in numerous applications. However, DNN models can become unreliable due to the uncertainty in data, hence giving a false judgement and possibly incurring a disaster. To address this issue, Bayesian Neural Networks (BNNs), which possess a property of uncertainty estimation, have been increasingly employed in a wide range of real-world AI applications that demand reliable and robust decisions (e.g., self-driving, rescue robots, medical image diagnosis).

Recently, training models in a distributed manner at mobile devices (e.g., federated learning) has become a popular and efficient training paradigm that achieves stronger data privacy, reduced data traffic and less response time compared to the cloud-centric training. Especially, federate learning on BNN models has received extensive attention. Since on-device learning is the essential step towards distributed BNN training, this project aims to enable fast and energy-efficient BNN training locally on resource-limited mobile devices.

The project will have transformative impact on adopting AI technologies in many emergent domains that require swift, low-power, and most importantly, robust and reliable model training on edge and mobile devices (such as automation, medical, transportation, oil and gas, business, Internet-of-Things, and so on), helping to better explore the world and making everyday living and working more convenient and efficient. This project will also contribute to society through engaging under-represented groups, outreach to high-school students, curriculum development on machine learning, and disseminating research infrastructure for education and training.

BNNs can be viewed as a probabilistic model where each model parameter (i.e., weight) is a probability distribution. There are two major BNN training approaches, which train the model via weight sampling (called Gaussian-based BNNs (GBNNs)) on the one hand and output feature map sampling (called Dropout-based Bayesian Convolutional NNs (DBCNNs)) on the other.

Existing DNN accelerators are oblivious to the BNN stochastic training processes and achieving low training efficiency, and a uniform BNN accelerator is impractical due to the different sampling methods applied in GBNNs and DBCNNs. The investigators are developing a synergetic architecture-system research program that exploits and leverages the unique features of GBNNs and DBCNNs to achieve an order-of-magnitude reduction on training cost while still maintaining the model robustness, thus enabling on-device BNN training.

The program comprises three objectives. (1) To enable on-device GBNN training, the investigators are dynamically eliminating/reducing the Gaussian random variables and intermediate data induced data movements, and furthermore, boosting the GBNN training speed by resorting to multi-chip-module-based DNN accelerators. (2) To enable on-device DBCNN training, the investigators are dynamically exploiting and eliminating the computation and data movement redundancy buried in the stochastic training processes. (3) The investigators are studying the efficient on-device training for mobile AI devices that involved in the distributed training for both global GBNN and DBCNN models and evaluating the overall system.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.