QGen: On the Ability to Generalize in Quantization Aware Training
Quantization lowers memory usage, computational requirements, and latency by utilizing fewer bits to represent model weights and activations. In this work, we investigate the generalization properties of quantized neural networks, a characteristic that has received little attention despite its impli...
Gespeichert in:
Hauptverfasser: | , , , , , , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Quantization lowers memory usage, computational requirements, and latency by
utilizing fewer bits to represent model weights and activations. In this work,
we investigate the generalization properties of quantized neural networks, a
characteristic that has received little attention despite its implications on
model performance. In particular, first, we develop a theoretical model for
quantization in neural networks and demonstrate how quantization functions as a
form of regularization. Second, motivated by recent work connecting the
sharpness of the loss landscape and generalization, we derive an approximate
bound for the generalization of quantized models conditioned on the amount of
quantization noise. We then validate our hypothesis by experimenting with over
2000 models trained on CIFAR-10, CIFAR-100, and ImageNet datasets on
convolutional and transformer-based models. |
---|---|
DOI: | 10.48550/arxiv.2404.11769 |