Choice of reference conditions for speech preference tests

Preference tests used in the evaluation of speech-communication systems typically employ as reference conditions different levels of one type of distortion, produced by a single reference system. A new preference test has been developed to explore the possible advantages of using several reference systems that represent fundamentally different types of distortions. The new test was experimentally compared with the more conventional test, and it was found that the preference judgments obtained with the new test exhibited a significantly smaller variance among the listeners. This finding suggests that, by using several reference systems that represent an appropriate variety of distortions, preference tests can be constructed that are more efficient than those developed to date.