SOFA: A Compute-Memory Optimized Sparsity Accelerator via Cross-Stage Coordinated Tiling

Benefiting from the self-attention mechanism, Transformer models have attained impressive contextual comprehension capabilities for lengthy texts. The requirements of high-throughput inference arise as the large language models (LLMs) become increasingly prevalent, which calls for large-scale token...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Wang, Huizheng, Fang, Jiahao, Tang, Xinru, Yue, Zhiheng, Li, Jinxi, Qin, Yubin, Guan, Sihan, Yang, Qize, Wang, Yang, Li, Chao, Hu, Yang, Yin, Shouyi
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Hardware Architecture
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!