A Bayesian Item Response Theory Approach to Symptom Validity Detection: Evaluating Psychological Screening Inventory-2 Response Profile Likelihoods

Invalid data in forensic assessment are most often indicated by excessive endorsement of psychiatric symptoms. Although this method of identifying invalid profiles is generally effective, it does not make use of all conditional dependencies in data. Modern psychometric methodologies can be used to identify aberrant response profiles through model-based indices known as person fit statistics. Specifically, the likelihoods of examinees' response profiles can be compared against observed or simulated likelihoods that are derived from empirical models of emotional and psychiatric functioning. This study demonstrates how person fit indices based on item response theory models can be used to detect misfitting response profiles in forensic assessment. Archival data from the Psychological Screening Inventory-2 (R. I. Lanyon, 2010a ) were evaluated with Bayesian estimation and posterior predictive model checking to compare the response profile log-likelihoods of 74 forensic participants with 1,046 normative participants. Results suggest 61 % of forensic examinees but only 5 % of normative examinees had misfitting data. Misfitting “fake bad” forensic profiles appeared to be associated with overly discrepant endorsement of symptoms, and misfitting “fake good” forensic profiles appeared to be associated with overly narrow endorsement of symptoms. The high rate of misfit among forensic examinees challenges the appropriateness of basing interpretations of forensic data on reliability and validity coefficients from normative samples. However, because aspects of the methodology are still untested in forensic and clinical assessment (e.g., the use of priors in this study), future research is needed to evaluate its appropriateness for clinical practice.