Survey error in an international context: an empirical assessment of cross-cultural differences regarding scale effects

Likert-type of questions are prone to differences in response styles. The aim of the present study is to examine possible cross-cultural differences regarding scale effects resulting from different answer categories. This study aims to extend the literature on scale effects to a foreign language. An English questionnaire containing two experimental questions was presented to students enrolled in international MSc programs. In addition to the experimental questions, the questionnaire contained filler questions, demographic questions, and questions on English proficiency. The questionnaires differed only from each other with respect to answer categories and scale presented with the experimental questions. The questionnaire was presented during class, to 526 students coming from over 70 different countries. These countries were classified according to continent and language into 10 groups of which nine were large enough to be included in the analysis. After considering possible confounding factors, like age, gender, English language proficiency, and disciplinary background, scale effects were shown to be present across groups of countries. These differences seriously affect rankings. Concludingly, scale effects persist in non-native language questionnaires, and the effect sizes differ across cultures.