NSF-SNSF: Scalable accelerated learning in continuous neuronal systems

Computer simulations are an essential part of modern scientific inquiry. They allow detailed investigations of phenomena at time and

length scales difficult to observe directly. In particular for complex systems such as the human brain, they have become

indispensable for testing hypotheses and making predictions for experiments. Cognitive phenomena such as learning and memory evolve

on timescales from minutes to years, while the underlying processes on the level of individual nerve cells involve milliseconds timing.

Bridging these timescales in computer simulations requires significant computational resources. This project develops a new hardware system

specifically designed to meet the increasing computational demands arising for increasingly sophisticated multi-scale brain models. This

new tool will both support fundamental neuroscience research and accelerate the translation of insights into biological computation

into novel computing devices for commercially relevant applications. While the project focuses on computational neuroscience models, the

implementation challenges and proposed solutions are applicable to the simulation of a broad class of dynamical systems such as power grids,

financial markets, and epidemics. Finally, this project will support the continuous development of open-source workflows for the design of asynchronous digital hardware.

Despite tremendous progress in recent decades, the biophysical mechanisms and computational principles underlying memory and learning

in the brain remain poorly understood. Due to the complexity of biological systems and the difficulty in reproducibility (in the

engineering sense) of results, researchers rely on computational models to test hypotheses and to make experimental predictions.

Modern models of neuronal computation and learning are formulated in continuous-time with continuous interactions, i.e., as systems of

ordinary differential equations. Yet, these are a poor fit for existing accelerators which are optimized for either continuous time

with discrete (spiking) interactions or discrete time with continuous interactions. A significant implementation challenge on the path to a

scalable accelerator lies in efficient parallelization of the spatially discrete, non-locally and non-uniformly coupled systems of

equations, in particular due to the communication requirements arising from continuous interactions in continuous time. To overcome these

challenges, we propose a convergent simulator architecture that can accelerate continuous-time models of learning using neuromorphic

hardware principles. On the algorithmic end, we exploit the insight that continuously interacting systems do not necessarily require dense

communication, i.e., that communication in implementations of these dynamics can be both sparse and adaptive to computational demand. The

resulting algorithms are ideally suited for implementation in asynchronous digital hardware with fine-grained parallelism and share

similarities with existing neuromorphic computing approaches. To validate our approach and ensure its usefulness to the scientific

community, we create both an FPGA implementation and silicon prototype for a recently proposed cortical learning model and develop novel model variants for processing and learning across long time scales.

This collaborative U.S.-Swiss project is supported by the U.S. National Science Foundation (NSF) and the Swiss National Science Foundation (SNSF), where NSF funds the U.S. investigator and SNSF funds the partners in Switzerland.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.