Organizing a System for Continuous Speech Understanding

In spectrogram reading experiments [Klatt and Stevens, Boston Speech Conference (1972)], the performance obtained by human experts for phonetic segmentation and labeling was approximately 75% partially or completely labeled but correctly specified, 15% mislabeled, and 10% segments missed. When syntactic, semantic, and vocabulary constraints were applied, the success rate in identifying words was 96%. How does one design a computer system to match this performance at word identification? This paper describes the BBN speech understanding system, which is being designed to extract the meaning from continuous speech inputs in a limited semantic environment by using syntactic, semantic, and vocabulary information to overcome vagueness and errors in phonetic segmentation and labeling. The system contains components for acoustic feature extraction, phonetic segmentation and labeling, word retrieval from partial phonetic information, evaluation of word-match quality, and syntactic and semantic evaluation of partial utterances and proposal of words from context. A control component embodies the strategy for applying different components at appropriate times and places depending on conditions which will vary from point to point in an analysis.