Model assessment and model building in fMRI

Model quality is rarely assessed in fMRI data analyses and less often reported. This may have contributed to several shortcomings in the current fMRI data analyses, including: (1) Model mis‐specification, leading to incorrect inference about the activation‐maps, SPM{t} and SPM{F}; (2) Improper model selection based on the number of activated voxels, rather than on model quality; (3) Under‐utilization of systematic model building, resulting in the common but suboptimal practice of using only a single, pre‐specified, usually over‐simplified model; (4) Spatially homogenous modeling, neglecting the spatial heterogeneity of fMRI signal fluctuations; and (5) Lack of standards for formal model comparison, contributing to the high variability of fMRI results across studies and centers. To overcome these shortcomings, it is essential to assess and report the quality of the models used in the analysis. In this study, we applied images of the Durbin‐Watson statistic (DW‐map) and the coefficient of multiple determination (R2‐map) as complementary tools to assess the validity as well as goodness of fit, i.e., quality, of models in fMRI data analysis. Higher quality models were built upon reduced models using classic model building. While inclusion of an appropriate variable in the model improved the quality of the model, inclusion of an inappropriate variable, i.e., model mis‐specification, adversely affected it. Higher quality models, however, occasionally decreased the number of activated voxels, whereas lower quality or inappropriate models occasionally increased the number of activated voxels, indicating that the conventional approach to fMRI data analysis may yield sub‐optimal or incorrect results. We propose that model quality maps become part of a broader package of maps for quality assessment in fMRI, facilitating validation, optimization, and standardization of fMRI result across studies and centers. Hum. Brain Mapping 20:227–238, 2003. © 2003 Wiley‐Liss, Inc.