Towards practical reinforcement learning for tokamak magnetic control

Reinforcement learning (RL) has shown promising results for real-time control systems, including the domain of plasma magnetic control. However, there are still significant drawbacks compared to traditional feedback control approaches for magnetic confinement. In this work, we address key drawbacks of the RL method; achieving higher control accuracy for desired plasma properties, reducing the steady-state error, and decreasing the required time to learn new tasks. We build on top of Degrave et al. (2022), and present algorithmic improvements to the agent architecture and training procedure. We present simulation results that show up to 65% improvement in shape accuracy, achieve substantial reduction in the long-term bias of the plasma current, and additionally reduce the training time required to learn new tasks by a factor of 3 or more. We present new experiments using the upgraded RL-based controllers on the TCV tokamak, which validate the simulation results achieved, and point the way towards routinely achieving accurate discharges using the RL approach. •Develop and expand techniques for creating tokamak magnetic controllers through reinforcement learning.•Improve shape accuracy by up to 65% and reduce steady-state offsets in simulation.•Reduce training time for controller generation by a factor of 3 or more through episode chunking and agent transfer.•Comparison of new controllers with experimental results on the Tokamak à Configuration Variable (TCV).