ACED Fab: Runtime Reconfigurable Array (RTRA) Technology for AI/ML

Fast-developing artificial intelligence (AI) and machine learning (ML) models, which double in complexity every 3-4 months, outpace the development of underlying hardware accelerators for AI/ML. Presently, deployment of AI/ML algorithms is limited by hardware that lacks agility and energy efficiency to support new models, limiting progress in science and engineering applications (e.g., next-generation wireless systems).

Hardware adaptation without sacrificing energy efficiency is needed to embed AI/ML models into mobile, edge, and cloud devices. Researchers from the University of California, Los Angeles (UCLA) along with National Taiwan University (NTU) and Taiwan Semiconductor Research Institute (TSRI) are joining together to develop and demonstrate proof-of-concept runtime reconfigurable array (RTRA) technology that addresses the issues of energy efficiency and agility in existing devices.

RTRA is a forward-looking architecture, where agile AI/ML hardware pipelines are dynamically reconfigured to embed new models or respond to dynamic environments. Such technology requires finely balanced hardware and software. Simple hardware leads to complex software (e.g., Field Programmable Gate Array, or FPGA) and simple software leads to complex hardware (e.g., Central Processing Unit, or CPU).

RTRA balances hardware and software to enable spatial and temporal flexibility. Spatio-temporal randomization of RTRA is more immune against physical attacks, reverse engineering, supply chain and hardware Trojans, due to its unique software/hardware approach. The project team will use system-based cross-disciplinary approach to address several key challenges of RTRA: 1) efficient programming paradigm, 2) 2D scheduling of hardware resources, 3) high-level abstraction that can fulfill low-level hardware potential, 4) multi-program tenancy, and 5) reconfiguration speed at the pipeline level (tens of clock cycles).

The project aims to develop a runtime reconfigurable array technology for AI/ML that provides program switch decisions at sub-microsecond scale, supports multiple active programs, multi-size compile, and priority handling. RTRA should be programmable from high-level languages such as C or Python. Online hardware scheduling enables rapid runtime reconfiguration.

The hardware agility is enabled by online dynamic multi-program hardware scheduling, domain-specific reconfigurable array and area-efficient interconnect for multi-program tenancy and flexible data interfaces. The hardware includes embedded processor for control, system memory, and data interfaces for heterogeneous system integration. The energy efficient processing (>10x better than FPGA, with another ~10x higher resource utilization) is a significant improvement over existing FPGA AI accelerators, while adding spatiotemporal dynamics for advanced AI/ML models.

An internally developed software toolchain (from C or Python to soft binary) will be made available to compile and test various AI/ML use cases. The project team brings unique capability to develop, test, and utilize an integrated system. Broadly, the technology enables rapid deployment of newly developed algorithms, accelerating the deployment of innovative applications.

Further, the ability to quickly repurpose hardware provides an opportunity to utilize “dark silicon” and potentially displace today’s fixed-function hardware accelerators with energy-efficient runtime reconfigurable fabric. RTRA technology is envisioned to meet the needs of future communications and AI/ML workloads, with readily accessible C or Python programming.

The project leverages the complementary academic talent in the U.S. and Taiwan to impact semiconductor engineering and education.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.