Deciding when to intervene: a Markov decision process approach

The aim of this paper is to point out the difference between static and dynamic approaches to choosing the optimal time for intervention. The paper demonstrates that classical approaches, such as decision trees and influence diagrams, hardly cope with dynamic problems: they cannot simulate all the real-world strategies and consequently can only calculate suboptimal solutions. A dynamic formalism based on Markov decision processes (MPPs) is then proposed and applied to a medical problem: the prophylactic surgery in mild hereditary spherocytosis. The paper compares the proposed approach with a static approach on the same medical problem. The policy provided by the dynamic approach achieved significant gain over the static policy by delaying the intervention time in some categories of patients. The calculations are carried out with DT-Planner, a graphical decision aid specifically built for dealing with dynamic decision processes.