Non-response bias in estimates of prevalence of club-based sport participation from an Australian national physical activity, recreation and sport survey

An estimate of the prevalence of an activity derived from a sample survey is potentially subject to non-response bias, whereby people not involved in the activity are less likely to respond than those involved. Quantifying the extent of non-response bias is generally difficult, since it involves estimating differences between respondents for whom data is directly available from the survey, and non-respondents, for whom data is generally not directly or readily available. However, in the case of the Australian Exercise Recreation and Sport Survey (ERASS), comparative "gold standard" benchmarks exist for some aspects of the survey, in the form of state sporting association (SSA) registration databases, each of which purports to constitute a complete enumeration of club-based players of a particular sport. ERASS estimates of the prevalence of participation in four major club-based team sports in the Australian state of Victoria in the year 2010 were compared with prevalences based on numbers of registered participants in the corresponding SSA databases. Comparisons were made for the adult population as a whole (ERASS scope being 15+ years of age), and for strata defined by age and geographical region. Because three of the four sports investigated are strongly sex-specific, no sex breakdowns were conducted. In each case the proportion of ERASS respondents reporting participation, with associated confidence limits, was compared with the corresponding SSA count expressed as a proportion of the population, to form an ERASS/SSA prevalence ratio with associated confidence limits. The 24 ERASS/SSA ratios ranged from 1.72 to 7.80. Most ratios lay in the range 2 to 3. The lower 95% confidence bound for the ratio was greater than 1.0 in 23 out of 24 cases. ERASS estimates of prevalence of these particular aspects of sport participation were higher than SSA estimates, to statistically significant degrees. The effect sizes (i.e. the discrepancies represented by the ratios) were large enough to be of great practical importance. It is conjectured that non-response bias is the most likely explanation for the discrepancies.