Reliability, Validity, and Accuracy of the Intervention Selection Profile--Function: a Brief Functional Assessment Tool

The purpose this study was to evaluate the reliability, validity, and accuracy of scores from the "Intervention Selection Profile--Function (ISP-Function)": a brief functional assessment tool founded upon Direct Behavior Rating (DBR) methodology. Participants included 34 teacher-student dyads. Using the ISP-Function, teachers rated the extent to which students exhibited disruptive behavior, as well as the frequency with which disruptions were met with four consequences. Ratings were completed across three 10-minute sessions, during which a research assistant also collected systematic direct observation (SDO) data regarding the same behavior and consequences. Results indicated adequate temporal reliability (=0.70) was attained for the Adult Attention and Peer Attention target across the three ratings; in contrast, up to 8-18 data points would be needed to achieve adequate reliability across the remaining targets. Findings further suggested that while ISP-Function ratings of Disruptive Behavior, Adult Attention, and Peer Attention were moderately to highly correlated with SDO data, correlations were in the low range for the Access to Items/Activities and Escape/Avoidance targets. Finally, analysis of difference scores showed that on average, mean ISP-Function scores fell within only 0.33 to 1.81 points of mean SDO scores (on the 0-10 DBR scale). Agreement coefficients indicative of exact score agreement were less consistent, suggesting accuracy ranged from poor to substantial. Results are promising, but future research is necessary to support applied ISP-Function use. "Impact Statement": Study results support the continued validation of the "Intervention Selection Profile--Function," with findings speaking to the reliability, validity, and accuracy of three of five scales within the tool. The availability of such tools enhances the likelihood of schools collecting functional assessment data to inform the implementation of Tier 2 interventions. [This paper was published in "School Psychology" (EJ1227784).]