DOES "NATIONALLY" NORMED REALLY MEAN NATIONALLY?

Major test publishers design their "national" norming samples carefully and use random selection to fill the various cells in their sample designs. However, only a relatively small fraction of school districts so selected agree to actually participate in these norming studies. The paper investigates the hypothesis that this phenomenon introduces systematic "self-selection" biases into the ultimate samples, which tend to overrepresent districts already using a specific publisher's (1) tests, or (2) instructional materials. A survey of "accepting" and "declining" districts in three recent major national norming studies tends to confirm these hypotheses. Several serious negative implications of this situation are presented. Finally, possible remedies for the problem are suggested.