The effect of some environmental factors on interobserver agreement

Because there is no truth criterion to measure the accuracy of behavioral recording, behavior analysts rely on interobserver-agreement scores to increase the believability of their data. This study investigated the effects of the presence and absence of a subject's peers on within-session interobserver-agreement scores. Ten variables that were components of both of the main conditions and thought to potentially affect agreement scores also were studied. The results show that interobserver agreement was significantly better in the presence of certain stimuli (i.e., when the subject's peers were present than when they were not). Of the 10 additional variables analyzed, one — Can't See-Can't Hear — correlated significantly with the differences in interobserver-agreement scores. These results suggest that experimenters need to be aware of the variations in their observers' behavior and the factors affecting it. The importance of acceptable levels of inter-observer agreement for data used in making treatment decisions also is discussed.