Using Path Analysis to Test Theory of Change: A Quantitative Process Evaluation of the MapSan Trial

Abstract BackgroundAlthough theory-driven evaluations should have empirical components, few evaluations of public health interventions quantitatively test the causal model made explicit in the theory of change (ToC). In the context of a shared sanitation trial (MapSan) in Maputo, Mozambique, we report findings of a quantitative process evaluation assessing intervention implementation, participant response and impacts on hypothesised intermediary outcomes on the pathway to trial health outcomes. We examine the utility of path analysis in testing intervention theory using process indicators from the intervention’s ToC. MethodsProcess data were collected through a cross-sectional survey of intervention and control compounds of the MapSan trial >24-months post-intervention, sampling adult residents and compound leaders. Indicators of implementation fidelity (dose received, reach) and participant response (participant behaviours, intermediary outcomes) were compared between trial arms. The intervention’s ToC (formalised post-intervention) was converted to an initial structural model with multiple alternative pathways. Path analysis was conducted through linear structural equation modelling (SEM) and generalised SEM (probit model), using a model trimming process and grouped analysis to identify the most parsimonious models that explained variation in outcomes, and incorporating contextual factors of respondents and compounds. ResultsAmong study compounds, the MapSan intervention was implemented with high fidelity, and with a strong participant response in intervention compounds: improvements were made to intermediary outcomes related to sanitation ‘quality’ – latrine cleanliness, maintenance and privacy – but not to handwashing (presence of soap or soap residue). These outcomes varied by type of intervention: single-cabin latrines or multiple-cabin blocks (designed for >20 users). Path analysis suggested that changes in intermediary outcomes were likely driven by direct effects of intervention facilities, with little contribution from hygiene promotion activities nor core ToC elements expected to mediate change: a compound sanitation committee and maintenance fund. A distinct structural model for two compound size subgroups (≤20 members vs. >20 members) explained differences by intervention type, and other contextual factors influenced specific model parameters. ConclusionsWhile process evaluation found that the MapSan intervention achieved sufficient fidelity a