Outlier-response pattern checks to improve measurement with the Positive and Negative Syndrome Scale (PANSS)

•A set of consistency flags to help identify careless PANSS ratings based on aberrant response patterns.•Flags elaborated in paper can be applied.•Flags represent actual patterns of inconsistencies found.•Application of flags to ratings may improve reliability of ratings and validity of trials. We derived outlier-response pattern checks to flag possible careless PANSS (Positive and Negative Syndrome Scale) administrations based on analysis of 122,000 administrations from 29 registration trials of antipsychotics from NEWMEDS data repository. Flags identify outlier administrations based on frequency of endorsing a given response value, use of even or odd values, consecutive use of same value, variability of values, responses per specific item, and values on multiple items. Outlier flags were compared to published expert derived scoring inconsistency flags and tested in Monte Carlo simulated data, with known inconsistency, and appear to be useful at identifying administrations that require review.