Analysis of longitudinal data from outcome‐dependent visit processes: Failure of proposed methods in realistic settings and potential improvements

The timing and frequency of the measurement of longitudinal outcomes in databases may be associated with the value of the outcome. Such visit processes are termed outcome dependent, and previous work showed that conducting standard analyses that ignore outcome‐dependent visit times can produce highly biased estimates of the associations of covariates with outcomes. The literature contains several classes of approaches to analyze longitudinal data subject to outcome‐dependent visit times, and all of these are based on simplifying assumptions about the visit process. Based on extensive discussions with subject matter investigators, we identified common characteristics of outcome‐dependent visit processes that allowed us to evaluate the performance of existing methods in settings with more realistic visit processes than have been previously investigated. This paper uses the analysis of data from a study of kidney function, theory, and simulation studies to examine a range of settings that vary from those where all visits have a low degree of missingness and outcome dependence (which we call “regular” visits) to those where all visits have a high degree of missingness and outcome dependence (which we call “irregular” visits). Our results show that while all the approaches we studied can yield biased estimates of some covariate effects, other covariate effects can be estimated with little bias. In particular, mixed effects models fit by maximum likelihood yielded little bias in estimates of the effects of covariates not associated with the random effects and small bias in estimates of the effects of covariates associated with the random effects. Other approaches produced estimates with greater bias. Our results also show that the presence of some regular visits in the data set protects mixed model analyses from bias but not other methods.