Development and preliminary reliability testing of an assessment of patient independence in performing a treatment program: standardized scenarios

Physical therapists often assess patient independence through observation; however, it is not known if therapists make these judgments reliably. We have developed a standardized method to assess a patient's ability to perform his or her treatment program independently. To develop a standardized assessment of patient independence in performance of a treatment program and examine the intra- and inter-rater reliability decisions made by two physical therapists. Test-retest. An assessment of patient independence in performance was developed. Standardized patient scenarios were used to assess the intra- and inter-tester reliability of two physical therapists. Percentage of agreement (%) and kappa's coefficient (k and k(w)) indexed rater reliability. Intra-rater reliability of therapist 1 was as follows: knowledge: % = 95, k = 0.90; performance: % = 95, k(w) = 0.82. Intra-rater reliability of therapist 2 was as follows: knowledge: % = 85, k = 0.68; performance: % = 94, k(w) = 0.80. Inter-rater reliability for knowledge was % = 91 and k = 0.79 and for performance was % = 91 and k(w) = 0.72. Trained therapists displayed substantial to excellent intra-rater reliability and substantial inter-rater reliability in assessing a patient's independence in a treatment program. :