Evaluating the reliability and construct validity of the Eckardt symptom score as a measure of achalasia severity

Background Achalasia is a disease of mechanical esophageal dysfunction characterized by dysphagia, chest pain, regurgitation, and malnutrition. The Eckardt symptom score (ESS) is the gold standard self‐report assessment tool. Current guidelines outline a three‐step approach to patient reported outcomes measure design. Developed prior to these policies, the ESS has not undergone rigorous testing of its reliability and validity. Methods Adult achalasia patients retrospectively identified via a patient registry were grouped based on treatment history. Patients were grouped PREPOST (completed ESS, GERDQ, brief esophageal dysphagia questionnaire, NIH PROMIS Global Health, high resolution manometry, timed barium esophagram prior to treatment and after) and POST (completed measures only after treatment). Clinical characteristics, treatment type and date were obtained via medical record. Standardized psychometric analyses for reliability and construct validity were performed. Key Results 107 patients identified; 83 POST and 24 PREPOST. The ESS has fair internal consistency and split‐half reliability with a single factor structure. Dysphagia accounts for half the variance in ESS, while chest pain and weight loss account for 10% each. Pre‐post‐surgical assessment demonstrates improvements in ESS, except for weight loss. Effect sizes range from 0.24 to 2.53, with greatest change in regurgitation. Validity of the ESS is supported by modest correlations with GERDQ, HRQOL, and physiological data. Conclusions & Inferences The ESS demonstrates fair reliability and validity, with a single factor structure mostly explained by dysphagia. Based on psychometric findings, weight loss and chest pain items may be decreasing ESS reliability and validity. Further assessment of the ESS under FDA guidelines is warranted.