Deployment of Causal Effect Estimation in Live Games of Dota 2

In this article, we provide an application that produces consistent in-game estimates of win probabilities in Dota 2 . Previous work shows that common methods of identifying the effect of in-game features are strongly inconsistent, which we corroborate here with a large dataset. We further provide an in-game application for players to see these estimates during the game as a training tool, along with displaying the estimated marginal impact of the primary actions (kills, last hits, and tower damage), which are previously known only by intuition. In a double-blind setting, we are the first to identify that users observe a difference between estimates produced by an inconsistent and consistent approach. Users show a significant preference for the consistent approach along several dimensions. Participants specifically identified the consistent approaches as having better quality advice by a large and significant margin, about four points on a ten-point scale.