Speech recognition application or server using iterative recognition constraints

A speech recognition application including a recognition module configured to receive input utterances and an application module configured to select a recognition from the speech recognition module using output from a first iteration to select a recognition result for a second iteration. In one embodiment, the application module eliminates a previous rejected recognition result or results from the N-Best list for recognition. In another embodiment, the application module rescores N-Best entries based upon N-Best lists or information from another iteration. In another illustrated embodiment, the application module uses a limited grammar from a current N-Best list for subsequent recognition, for example for rerecognition using a recorded input from a previous iteration.