AIMMI: Audio and Image Multi-Modal Intelligence via a Low-Power SoC With 2-MByte On-Chip MRAM for IoT Devices

In this article, we present an ultra-low-power multi-modal signal processing system on chip (SoC) [audio and image multi-modal intelligence (AIMMI)] that integrates a versatile deep neural network (DNN) engine with audio and image signal processing accelerators for multi-modal Internet-of-Things (Io...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE journal of solid-state circuits 2024-10, Vol.59 (10), p.3488-3501
Hauptverfasser: Fan, Zichen, Zhang, Qirui, An, Hyochan, Xu, Boxun, Xu, Li, Tseng, Chien-Wei, Peng, Yimai, Bejarano-Carbo, Andrea, Abillama, Pierre, Cao, Ang, Liu, Bowen, Lee, Changwoo, Wang, Zhehong, Kim, Hun-Seok, Blaauw, David, Sylvester, Dennis
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this article, we present an ultra-low-power multi-modal signal processing system on chip (SoC) [audio and image multi-modal intelligence (AIMMI)] that integrates a versatile deep neural network (DNN) engine with audio and image signal processing accelerators for multi-modal Internet-of-Things (IoT) intelligence. In order to get high energy efficiency under resource-constrained IoT scenarios, AIMMI features three efficiency-boosting techniques: 1) 2-MB on-chip non-volatile magnetoresistive RAM (MRAM) to store all DNN weights with MRAM-cache microarchitecture that incorporates dynamic power gating to reduce both leakage and dynamic power consumption; 2) a deliberate power management scheme that enables optimized power modes under different operating situations; and 3) a novel reconfigurable neural engine (NE) with energy-efficient dataflow for comprehensive DNN instructions. Fabricated in TSMC 22-nm ultra-low leakage (ULL) technology with MRAM, AIMMI achieves up to 3-10-TOPS/W peak energy efficiency and consumes only 0.25-3.84 mW. It demonstrates convolutional neural network (CNN), generative adversarial network (GAN), and back-propagation (BP) operations on a single accelerator SoC for multi-modal fusion, outperforming state-of-the-art DNN processors by 1.4 \times -4.5 \times in energy efficiency.
ISSN:0018-9200
1558-173X
DOI:10.1109/JSSC.2024.3410306