Do educational reforms increase or decrease health inequalities: A matter of methods?

Evaluating whether social policies reduce health inequalities is complicated by the fact that these upstream determinants may also change the socioeconomic distribution. Failure to account for these compositional changes may severely bias the effect estimation procedure. In this article, we illustrate how a health inequality impact assessment of a policy that (also) changes the socioeconomic distribution may produce biased results. First, we show why analyses that do not account for compositional changes fail to estimate the correct counterfactual outcome. This problem most notably occurs when using repeated cross-sectional data, often the only available option to evaluate the health effect of large-scale policies. Second, we conducted a microsimulation study to estimate the magnitude of the bias under various conditions. The results showed that the actual impact of the policy on health inequalities is often underestimated and may even produce results that are in the opposite direction of the actual causal effect of the policy. Future studies should explore new strategies, such as simulation methods, to assess the impact of policies that (also) cause changes in the socioeconomic composition of the population, to enable researchers to accurately estimate their effect on health inequalities. •Evaluating the effect of social policies on health inequalities is important.•Effect evaluations will be biased without correction for compositional changes.•Results from microsimulations show this bias may be substantial.