Using Natural Language Conventions in the User Interface Design of Automatic Speech Recognition Systems

Four experiments were performed to examine how natural language conventions might be used to improve the user interface for systems using automatic speech recognition (ASR). Two new technological developments offered us this opportunity: word spotting permits recognition of key words embedded in extraneous speech; barge in permits the user to speak to the system while it plays instructional prompts. Our goal was to take advantage of natural language conventions to design an optimal prompting arrangement that would accommodate both novice and expert users. The conversational conventions we focused on included people's readiness to speak in response to a direct question and during an appropriately timed conversational pause. We studied these conventions in the context of an automated operator service. Our results indicated that a prompt-arranged as a leading question, followed by a brief pause and then a list of key words-met our goals for both the initial prompt and, if ASR failed. a subsequent reprompt. This approach resulted in fast, accurate responding, a user interface that received high user acceptance ratings, and an interface that was usable by both novice and expert users.