Gantry Work Cell Scheduling through Reinforcement Learning with Knowledge-guided Reward Setting

In this paper, a manufacturing work cell utilizing gantries to move between machines for loading and unloading materials/parts is considered. The production performance of the gantry work cell highly depends on the gantry movements in real operation. This paper formulates the gantry scheduling problem as a reinforcement learning problem, in which an optimal gantry moving policy is solved to maximize the system output. The problem is carried out by the Q-learning algorithm. The gantry system is analyzed and its real-time performance is evaluated by permanent production loss and production loss risk, which provide a theoretical base for defining reward function in the Q-learning algorithm. A numerical study is performed to demonstrate the effectiveness of the proposed policy by comparing with the first-come-first-served policy.