A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing

Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implement...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Meyer, Svea Marie, Weidel, Philipp, Plank, Philipp, Campos-Macias, Leobardo, Shrestha, Sumit Bam, Stratmann, Philipp, Richter, Mathis
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Meyer, Svea Marie
Weidel, Philipp
Plank, Philipp
Campos-Macias, Leobardo
Shrestha, Sumit Bam
Stratmann, Philipp
Richter, Mathis
description Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.
doi_str_mv 10.48550/arxiv.2409.15022
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2409_15022</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2409_15022</sourcerecordid><originalsourceid>FETCH-arxiv_primary_2409_150223</originalsourceid><addsrcrecordid>eNqFjrEKwjAURbM4iPoBTr4fsKaxBR1FKw4KQp0tIX2pD9qkpqno39sWd6d7udwDh7F5yINoE8d8Jd2bXoGI-DYIYy7EmN13cCBZWCNLSL1rlW8d5l2VHiGtpUK42BxLsAbOlh4EArR1kGhNitD4nkJZkSkgxWeLpiOuzipsmm6bspGWZYOzX07Y4pjc9qflYJLVjirpPllvlA1G6_-PLzgUQMg</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing</title><source>arXiv.org</source><creator>Meyer, Svea Marie ; Weidel, Philipp ; Plank, Philipp ; Campos-Macias, Leobardo ; Shrestha, Sumit Bam ; Stratmann, Philipp ; Richter, Mathis</creator><creatorcontrib>Meyer, Svea Marie ; Weidel, Philipp ; Plank, Philipp ; Campos-Macias, Leobardo ; Shrestha, Sumit Bam ; Stratmann, Philipp ; Richter, Mathis</creatorcontrib><description>Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.</description><identifier>DOI: 10.48550/arxiv.2409.15022</identifier><language>eng</language><subject>Computer Science - Artificial Intelligence ; Computer Science - Emerging Technologies ; Computer Science - Learning ; Computer Science - Neural and Evolutionary Computing</subject><creationdate>2024-09</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2409.15022$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2409.15022$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Meyer, Svea Marie</creatorcontrib><creatorcontrib>Weidel, Philipp</creatorcontrib><creatorcontrib>Plank, Philipp</creatorcontrib><creatorcontrib>Campos-Macias, Leobardo</creatorcontrib><creatorcontrib>Shrestha, Sumit Bam</creatorcontrib><creatorcontrib>Stratmann, Philipp</creatorcontrib><creatorcontrib>Richter, Mathis</creatorcontrib><title>A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing</title><description>Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.</description><subject>Computer Science - Artificial Intelligence</subject><subject>Computer Science - Emerging Technologies</subject><subject>Computer Science - Learning</subject><subject>Computer Science - Neural and Evolutionary Computing</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNqFjrEKwjAURbM4iPoBTr4fsKaxBR1FKw4KQp0tIX2pD9qkpqno39sWd6d7udwDh7F5yINoE8d8Jd2bXoGI-DYIYy7EmN13cCBZWCNLSL1rlW8d5l2VHiGtpUK42BxLsAbOlh4EArR1kGhNitD4nkJZkSkgxWeLpiOuzipsmm6bspGWZYOzX07Y4pjc9qflYJLVjirpPllvlA1G6_-PLzgUQMg</recordid><startdate>20240923</startdate><enddate>20240923</enddate><creator>Meyer, Svea Marie</creator><creator>Weidel, Philipp</creator><creator>Plank, Philipp</creator><creator>Campos-Macias, Leobardo</creator><creator>Shrestha, Sumit Bam</creator><creator>Stratmann, Philipp</creator><creator>Richter, Mathis</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240923</creationdate><title>A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing</title><author>Meyer, Svea Marie ; Weidel, Philipp ; Plank, Philipp ; Campos-Macias, Leobardo ; Shrestha, Sumit Bam ; Stratmann, Philipp ; Richter, Mathis</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-arxiv_primary_2409_150223</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Artificial Intelligence</topic><topic>Computer Science - Emerging Technologies</topic><topic>Computer Science - Learning</topic><topic>Computer Science - Neural and Evolutionary Computing</topic><toplevel>online_resources</toplevel><creatorcontrib>Meyer, Svea Marie</creatorcontrib><creatorcontrib>Weidel, Philipp</creatorcontrib><creatorcontrib>Plank, Philipp</creatorcontrib><creatorcontrib>Campos-Macias, Leobardo</creatorcontrib><creatorcontrib>Shrestha, Sumit Bam</creatorcontrib><creatorcontrib>Stratmann, Philipp</creatorcontrib><creatorcontrib>Richter, Mathis</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Meyer, Svea Marie</au><au>Weidel, Philipp</au><au>Plank, Philipp</au><au>Campos-Macias, Leobardo</au><au>Shrestha, Sumit Bam</au><au>Stratmann, Philipp</au><au>Richter, Mathis</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing</atitle><date>2024-09-23</date><risdate>2024</risdate><abstract>Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.</abstract><doi>10.48550/arxiv.2409.15022</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2409.15022
ispartof
issn
language eng
recordid cdi_arxiv_primary_2409_15022
source arXiv.org
subjects Computer Science - Artificial Intelligence
Computer Science - Emerging Technologies
Computer Science - Learning
Computer Science - Neural and Evolutionary Computing
title A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-14T02%3A08%3A32IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Diagonal%20Structured%20State%20Space%20Model%20on%20Loihi%202%20for%20Efficient%20Streaming%20Sequence%20Processing&rft.au=Meyer,%20Svea%20Marie&rft.date=2024-09-23&rft_id=info:doi/10.48550/arxiv.2409.15022&rft_dat=%3Carxiv_GOX%3E2409_15022%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true