A comparison of current measures of the accuracy of feeling-of-knowing predictions

Discusses 8 quantitative measures of feeling-of-knowing accuracy that have been used in the literature. The 3 measures considered at length are J. T. Hart's difference score, L. A. Goodman and W. H. Kruskal's (1954) gamma correlation, and the phi correlation. Quantitative relations between these measures are reported, as are connections with some basic axioms and a probabilistic conception of feeling-of-knowing accuracy. The currently most popular measure, the Hart difference score, has serious shortcomings. The Goodman-Kruskal gamma seems to be best. The remaining measures are inappropriate for the available feeling-of-knowing data for a variety of reasons. Also discussed are the implications of these results for other situations in which ordered 2 × 2 tables are examined to determine the relationship between predictions and criterion performance. (60 ref)