How Well Can We Measure Visual Attention? Psychometric Properties of Manual Response Times and First Fixation Latencies in a Visual Search Paradigm

Attentional biases are considered to be important for the development and maintenance of anxiety disorders, but despite widespread research in this area, the psychometric properties of measures targeting attentional allocation processes have rarely been evaluated. The current study assessed the reliability and validity of manual response times and first fixation latencies within a visual search paradigm, including data from one hundred twenty-two participants ( n = sixty-two participants with social anxiety disorder). We found raw manual response times to be highly reliable, while raw first fixation latencies, in most cases, were not. Bias scores were neither reliable for manual responses nor for first fixation latencies. However, we found indicators of convergent validity, as raw values and also bias scores of both measures were significantly correlated. These results raise doubts about our ability to accurately measure visual attention, especially via eye-tracking procedures.