Bayesian Low-rank Adaptation for Large Language Models

Low-rank adaptation (LoRA) has emerged as a new paradigm for cost-efficient fine-tuning of large language models (LLMs). However, fine-tuned LLMs often become overconfident especially when fine-tuned on small datasets. Bayesian methods, with their inherent ability to estimate uncertainty, serve as p...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-02
Hauptverfasser:	Yang, Adam X, Robeyns, Maxime, Wang, Xi, Aitchison, Laurence
Format:	Artikel
Sprache:	eng
Schlagworte:	Adaptation Bayesian analysis Calibration Large language models Mathematical models Parameters
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!