Average Token Delay: A Duration-aware Latency Metric for Simultaneous Translation

In simultaneous translation, translation begins before the end of an input speech segment. Its evaluation should be conducted based on latency and quality. For users, the smallest possible latency is preferable. Most existing metrics measure latency based on the start timings of partial translations and ignore their duration. This implies that such metrics do not penalize the latency caused by a long translation output, which delays user comprehension and subsequent translations. In this paper, we propose a novel latency evaluation metric for simultaneous translation called the Average Token Delay (ATD), which focuses on the duration of partial translations. We demonstrate its effectiveness through analyses that simulate user-side latency based on the Ear-Voice Span (EVS). In our experiments, ATD had the highest correlation with EVS among the baseline latency metrics under most conditions. These results suggest that ATD provides a more accurate evaluation of latency.