Human-Artificial Intelligence Collaboration in Prediction: A Field Experiment in the Retail Industry

This study investigates the role of human intervention in artificial intelligence/machine learning (AIML)-driven predictions. By doing so, we distinguish between three different types of human-AIML collaboration: automation, adjustable automation, and augmentation. We theorize that prediction uncertainty and time horizon represent two critical determinants of forecast accuracy. Based on a field experiment involving AIML-driven demand forecasts approximately 1,888 stock-keeping units in the retail industry, we rely on a multivalued treatment effect methodology to measure the effects of human-AIML collaboration on forecast accuracy. Our findings show that human intervention complements AIML-driven forecasts most effectively (augmentation) in predictions with long time horizons and low uncertainty. However human intervention is least likely to contribute to the effectiveness of AIML predictions (automation) in environments with short time horizons and high uncertainty. We discuss implications for extant theory and propose a framework outlining the conditions in which human intervention is most likely to add predictive value to human-AIML collaborations.