Vocoder intelligibility and quality test methods

The U.S. Department of Defense's Digital Voice Processing Consortium (DDVPC) is in the process of choosing a new 2400 bps vocoder standard. As part of the selection process, a study was conducted to determine the test measures to be used for the evaluation of the intelligibility and quality of competing vocoder algorithms. The DDVPC requirements for a vocoder differ substantially from commercial interests. Severe ambient acoustic noise backgrounds coupled with hostile transmission channel conditions must be considered. The commercially popular mean opinion score (MOS) test method was evaluated as a replacement or companion test of quality to the diagnostic acceptability measure (DAM), traditionally used by the DDVPC. The study showed that a properly structured MOS test can achieve equal resolution, reliability, and validity to that of the DAM at equivalent costs. Any MOS test series must be structured to minimize contextual effects. Certain severe conditions must use the degraded mean opinion score (DMOS) test method to achieve usable resolution.