A note on the standard error in the contingency matching technique

"The contingency coefficient which expresses the validity of a matching experiment possesses two standard or probable errors, one referring to variations in the judges who match the material, the other to variations in the subjects or sets of material which are matched. The SE of n judges who match one set of material, or of m sets of material matched by one judge, can be predicted by the same formula; and this formula is empirically confirmed by the results of large-scale experiments. But the SE of judges who match more than one set, or of material matched by more than one judge, can only be determined on the basis of observed variations within the available experimental results (except in the hypothetical case where there is no inter-correlation between the sets, or between the judges, respectively) . . . . In order to obtain a reliable matching contingency coefficient, not only should the group of judges, but also the number of sets of material, be fairly large. Both nt and mt should, like N in a correlational experiment, amount to one hundred or more."