The effects of system failure error on predictions

While Brunswik's conception of ecological validity has been widely discussed, his conception of ecological unreliability, or error in the data on which inferences or predictions are based, has been largely ignored. Further, when psychologists discuss error in continuous data, that error has, perhaps because of the impact of classical psychometric theory, been thought of as Gaussian, and labeled measurement error. The present paper reports a two-cue MCPL experiment in which there are multiple observations of each cue on each trial; these observations make the data error, or ecological unreliability, highly salient. In one condition both cues were degraded only by measurement error, i.e., the five observations varied randomly around the true cue value. In the second condition both cues were degraded by measurement error, while one of the two cues was also degraded by a form of error we call System Failure (SF) error on 30% of the trials. SF error refers to the form of error which we associate with technological rather than biological systems, in that any given observation may be irrelevant to what is supposed to be measured. The SF condition was significantly more predictable than the measurement error only condition, yet subjects had significantly lower achievement.