Information-Theoretic Properties of Languages and Their Grammars

Information-Theoretic Properties of Languages and Their Grammars

This document describes means for computing a number of information-theoretic properties of languages and their grammars. For example, the entropy of a system of symbols is widely recognized as a measure of that system's complexity and organization. It is shown how the entropy of a language can...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: MacLennan,B J
Format: Report
Sprache:eng
Schlagworte:
COMPUTATIONS
CONTEXT FREE GRAMMARS
Cybernetics
ENTROPY
ESTIMATES
GRAMMARS
INFORMATION THEORY
LANGUAGE
Linguistics
NAVAL RESEARCH
STATISTICAL ANALYSIS
Statistics and Probability
Strings
SYMBOLS
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This document describes means for computing a number of information-theoretic properties of languages and their grammars. For example, the entropy of a system of symbols is widely recognized as a measure of that system's complexity and organization. It is shown how the entropy of a language can be computed in a simple way from a grammar annotated with production probabilities. The author then develops means for statistically estimating these production probabilities from measureable properties of strings in the language. He also considers the computation of other information theoretic properties of language and grammars, such as the average information born by a symbol in a language and the average information used by the productions of a grammar. (Author)