The Use of Continuous Action Representations to Scale Deep Reinforcement Learning for Inventory Control

Deep reinforcement learning (DRL) can solve complex inventory problems with a multi-dimensional state space. However, most approaches use a discrete action representation and do not scale well to problems with multi-dimensional action spaces. We use DRL with a continuous action representation for inventory problems with a large (multi-dimensional) discrete action space. To obtain feasible discrete actions from a continuous action representation, we add a tailored mapping function to the policy network that maps the continuous outputs of the policy network to a feasible integer solution. We demonstrate our approach to multi-product inventory control. We show how a continuous action representation solves larger problem instances and requires much less training time than a discrete action representation. Moreover, we show its performance matches state-of-the-art heuristic replenishment policies. This promising research avenue might pave the way for applying DRL in inventory control at scale and in practice.