Agreement on Child Maltreatment Decisions: A Nonrandomized Study on the Effects of Structured Decision-Making

Background Practitioners investigating cases of suspected child maltreatment often disagree whether a child is subject to or at risk of abuse or neglect in the family and, if so, what to do about such abuse or neglect. Structured decision-making is considered to be a solution to the problem of subjective judgments and decisions. Objective This study investigates the effects of ORBA, a method for structured decision-making in Advice and Reporting Centres for Child Abuse and Neglect (ARCCAN), on interrater agreement of judgments and decisions. Methods Two groups of ARCCAN practitioners, one trained in using ORBA and one untrained, used a questionnaire to make judgments and decisions on the same case vignettes. Interrater agreement on the judgments was obtained by calculating the percentage of agreement, intra class correlation, and the Kappa coefficient. Results Both ORBA trained and untrained practitioners showed little agreement on judgments and decisions, except for the judgment on child maltreatment substantiation, for which trained practitioners showed fair agreement. Agreement among trained and untrained practitioners only differed for some judgments and decisions, and differences were not always in the same direction. Conclusions This result indicates no convincing evidence that structured decision-making leads to better agreement on decisions concerning child abuse and neglect. Recommendations for improvements in uniform decision-making and further research are given.