Pyramid Vector Quantization for Deep Learning

Pyramid Vector Quantization for Deep Learning

This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: Liguori, Vincenzo
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Learning
Computer Science - Neural and Evolutionary Computing
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Liguori, Vincenzo
description This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers and an N dimensional PVQ vector can be calculated with only additions and subtractions and one multiplication. This is advantageous since tensor products, commonly used in NNs, can be re-conduced to a dot product or a set of dot products. Finally, it is stressed that any NN architecture that is based on an operation that can be re-conduced to a dot product can benefit from the techniques described here.
doi_str_mv 10.48550/arxiv.1704.02681
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_1704_02681</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1704_02681</sourcerecordid><originalsourceid>FETCH-LOGICAL-a671-ef835f07286487e1fda2dffab9883ddfc48eafb533441216484e2b1d104dec203</originalsourceid><addsrcrecordid>eNotzs1qwkAUBeDZuCjaB3BlXiDp3JlJMl2KP7UQUEG6DTeZe8tAjWEapenTm8auDhwOh0-IOcjE2DSVLxh-_C2BXJpEqszCk4gPfcCzd9EH1d0lRMcrNp3_xc5fmoiHYk3URgVhaHzzORMTxq9vev7PqThtN6fVLi72b--rZRFjlkNMbHXKMlc2MzYnYIfKMWP1aq12jmtjCblKtTYGFAwjQ6oCB9I4qpXUU7F43I7esg3-jKEv_9zl6NZ3YI889A</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Pyramid Vector Quantization for Deep Learning</title><source>arXiv.org</source><creator>Liguori, Vincenzo</creator><creatorcontrib>Liguori, Vincenzo</creatorcontrib><description>This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers and an N dimensional PVQ vector can be calculated with only additions and subtractions and one multiplication. This is advantageous since tensor products, commonly used in NNs, can be re-conduced to a dot product or a set of dot products. Finally, it is stressed that any NN architecture that is based on an operation that can be re-conduced to a dot product can benefit from the techniques described here.</description><identifier>DOI: 10.48550/arxiv.1704.02681</identifier><language>eng</language><subject>Computer Science - Learning ; Computer Science - Neural and Evolutionary Computing</subject><creationdate>2017-04</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,777,882</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/1704.02681$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.1704.02681$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Liguori, Vincenzo</creatorcontrib><title>Pyramid Vector Quantization for Deep Learning</title><description>This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers and an N dimensional PVQ vector can be calculated with only additions and subtractions and one multiplication. This is advantageous since tensor products, commonly used in NNs, can be re-conduced to a dot product or a set of dot products. Finally, it is stressed that any NN architecture that is based on an operation that can be re-conduced to a dot product can benefit from the techniques described here.</description><subject>Computer Science - Learning</subject><subject>Computer Science - Neural and Evolutionary Computing</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotzs1qwkAUBeDZuCjaB3BlXiDp3JlJMl2KP7UQUEG6DTeZe8tAjWEapenTm8auDhwOh0-IOcjE2DSVLxh-_C2BXJpEqszCk4gPfcCzd9EH1d0lRMcrNp3_xc5fmoiHYk3URgVhaHzzORMTxq9vev7PqThtN6fVLi72b--rZRFjlkNMbHXKMlc2MzYnYIfKMWP1aq12jmtjCblKtTYGFAwjQ6oCB9I4qpXUU7F43I7esg3-jKEv_9zl6NZ3YI889A</recordid><startdate>20170409</startdate><enddate>20170409</enddate><creator>Liguori, Vincenzo</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20170409</creationdate><title>Pyramid Vector Quantization for Deep Learning</title><author>Liguori, Vincenzo</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a671-ef835f07286487e1fda2dffab9883ddfc48eafb533441216484e2b1d104dec203</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Computer Science - Learning</topic><topic>Computer Science - Neural and Evolutionary Computing</topic><toplevel>online_resources</toplevel><creatorcontrib>Liguori, Vincenzo</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Liguori, Vincenzo</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Pyramid Vector Quantization for Deep Learning</atitle><date>2017-04-09</date><risdate>2017</risdate><abstract>This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers and an N dimensional PVQ vector can be calculated with only additions and subtractions and one multiplication. This is advantageous since tensor products, commonly used in NNs, can be re-conduced to a dot product or a set of dot products. Finally, it is stressed that any NN architecture that is based on an operation that can be re-conduced to a dot product can benefit from the techniques described here.</abstract><doi>10.48550/arxiv.1704.02681</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.1704.02681
ispartof
issn
language eng
recordid cdi_arxiv_primary_1704_02681
source arXiv.org
subjects Computer Science - Learning
Computer Science - Neural and Evolutionary Computing
title Pyramid Vector Quantization for Deep Learning
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-18T18%3A24%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Pyramid%20Vector%20Quantization%20for%20Deep%20Learning&rft.au=Liguori,%20Vincenzo&rft.date=2017-04-09&rft_id=info:doi/10.48550/arxiv.1704.02681&rft_dat=%3Carxiv_GOX%3E1704_02681%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true