The Item-Number Distortion Effect in Rank Order Testing: An Example Using the Clifton Strengthsfinder Inventory

A possible distortion effect in tests that use (a) forced-choice ipsative scaling and (b) different numbers of items to determine categorical representation was investigated. It was proposed that the greater the number of items used to determine representation in a category, the lower the likelihood that category will be selected. The Clifton StrengthsFinder questionnaire, in which responses to 177 items are used to determine which of 34 talent themes are respondents' top five “Signature Themes,” was examined to ascertain whether or not the aforementioned distortion effect affected test results. Analysis of 1,198,645 completions of the questionnaire provided confirmatory evidence for the distortion effect. These findings call into question the trustworthiness of the Clifton StrengthsFinder questionnaire and any test that uses a similar design format.