modeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models

We introduce modeLing, a novel benchmark of Linguistics Olympiad-style puzzles which tests few-shot reasoning in AI systems. Solving these puzzles necessitates inferring aspects of a language's grammatical structure from a small number of examples. Such puzzles provide a natural testbed for lan...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-06
Hauptverfasser:	Chi, Nathan A, Malchev, Teodor, Riley, Kong, Chi, Ryan A, Huang, Lucas, Chi, Ethan A, McCoy, R Thomas, Radev, Dragomir
Format:	Artikel
Sprache:	eng
Schlagworte:	Benchmarks Linguistics Modelling Reasoning
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!