Resampling and cross-validation techniques: a tool to reduce bias caused by model building?

The process of model building involved in the analysis of many medical studies may lead to a considerable amount of over‐optimism with respect to the predictive ability of the ‘final’ regression model. In this paper we illustrate this phenomenon in a simple cutpoint model and explore to what extent bias can be reduced by using cross‐validation and bootstrap resampling. These computer intensive methods are compared to an ad hoc approach and to a heuristic method. Besides illustrating all proposals with the data from a breast cancer study we perform a simulation study in order to assess the quality of the methods. © 1997 John Wiley & Sons, Ltd.