FuSe: Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials (COSMIC)

The proposal aims at developing a new technology for artificial intelligence (AI) and machine learning (ML). Using environmentally sustainable materials, a system for processing information – specifically for image processing – will be developed. The current AI models, such as OpenAI’s ChatGPT, require a massive amount of power and resources to run.

More companies are rushing to create their ChatGPT clones, which is not sustainable and will only exacerbate climate change. Specialized hardware for vision sensing that is smaller in footprint and more energy-efficient to run AI models on conventional devices like laptops, without relying on cloud servers, is proposed. Artificial vision sensors can transform the future by enabling vision beyond the human eye’s capabilities, leading to impact across the entire landscape of modern life.

By detecting changes in motion, it will lead to a new generation of self-driving cars for future “smarter” cities. The proposed vision sensors will enable improved monitoring of processes in the industry, and improved understanding of body movements of athletes, all with low energy consumption and on a small area. Future products enabled by the scientific discoveries and advancements of this work will augment the semiconductor industry.

For instance, improved synthesis processes for electronic nanomaterials that are fully recyclable will supplement the industry’s growth, creating new job opportunities. Semiconductor workforce development will be strongly emphasized and enabled by this grant, and students trained through this project will be equipped to join the multitudes of industrial efforts accompanied by the CHIPS for America act and other initiatives.

Local college students will be exposed to the developments and trained as a workforce for the semiconductor industry. High school teachers will be exposed to cutting-edge research in semiconductors to convey the enthusiasm of the field to their students.

A materials-devices-system co-design for in-sensor computing with pixel arrays heterogeneously integrated with crossbar arrays to form a convolutional neural network (CNN) for image processing, with a special emphasis on sustainable manufacturing, is proposed by the Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials (COSMIC) team. The team will pursue co-design and heterogeneous integration for bio-inspired sensing-to-action to optimize power consumption, performance, and hardware footprint using machine learning architectures.

Pursuit of these goals will be undertaken by experts in neuromorphic devices (Roy), nanomaterials for sustainable electronics (Franklin), and neuromorphic circuits and algorithms (Li). The objectives are to: A. Develop in-pixel computing circuits with the highest possible fill factor of each pixel; B.

Integrate the pixel arrays with a crossbar-based in-memory computing circuitry to realize a CNN for image classification and segmentation; and C. Develop the entire process with recyclable materials to reduce electronic waste. For in-pixel computing, novel optoelectronic synaptic devices using two-dimensional materials as channel and gate electrode, with multi-wavelength sensing capabilities will be designed and optimized.

The in-pixel computing circuit forms the first layer of the CNN. Subsequent layers of the CNN will be realized with crossbar arrays of memristive synapses. Materials-device co-design will ensure the best characteristics of the synaptic devices for in-pixel computing and crossbar implementation, while device-system co-design will mitigate the remaining non-idealities while maximizing performance through a hybrid analog/digital computing scheme.

Peripheral circuits with nanomaterials will be co-designed based on device performance and precision requirements of the crossbar circuits. The recyclability of all circuits will be studied and ensured for sustainable manufacturing. Finally, the optoelectronic synapse-based pixel arrays will be heterogeneously integrated with the crossbar arrays and peripheral circuitry, and the performance of this hardware for image processing tasks, such as classification and segmentation, will be evaluated.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.