Design and implementation of a model-following controller for autonomous surface ships based on actor–critic reinforcement learning

The integration of the shipping industry with artificial intelligence is gradually transforming ship navigation from automated to remote-controlled, with the ultimate goal of achieving full autonomy. Compared with conventional control algorithms that require parameterized models of ship dynamics, deep reinforcement learning does not rely on parameterized models, nor does it require a complex parameter tuning procedure during deployment. Instead, it is based on the interaction between the agent and the environment, and continuously optimizes the decision-making process to obtain the maximum reward and the optimal actions. This enables it to effectively ensure the accuracy and adaptability of motion control. In this paper, a model-following control strategy is proposed and implemented for autonomous surface ships based on DDPG and SAC algorithms, with the control objective of tracking the state of three degrees of freedom (3 DoF) of any given ship. A pre-training procedure has been carried out in an approximate simulation environment to improve its applicability in the real environment. To validate the proposed method, simulations and physical experiments have been conducted on a scaled Azimuth Stern Drive (ASD) tug. •A deep reinforcement learning-based model-following control method is proposed for autonomous surface ships.•A model-following control system is designed and implemented on a model-scaled ship.•A pre-training strategy is adopted to transfer the control policy learned from simulation to practice.