Curriculum-based Asymmetric Multi-task Reinforcement Learning
We introduce CAMRL, the first curriculum-based asymmetric multi-task learning (AMTL) algorithm for dealing with multiple reinforcement learning (RL) tasks altogether. To mitigate the negative influence of customizing the one-off training order in curriculum-based AMTL, CAMRL switches its training mo...
Gespeichert in:
Hauptverfasser: | , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | We introduce CAMRL, the first curriculum-based asymmetric multi-task learning
(AMTL) algorithm for dealing with multiple reinforcement learning (RL) tasks
altogether. To mitigate the negative influence of customizing the one-off
training order in curriculum-based AMTL, CAMRL switches its training mode
between parallel single-task RL and asymmetric multi-task RL (MTRL), according
to an indicator regarding the training time, the overall performance, and the
performance gap among tasks. To leverage the multi-sourced prior knowledge
flexibly and to reduce negative transfer in AMTL, we customize a composite loss
with multiple differentiable ranking functions and optimize the loss through
alternating optimization and the Frank-Wolfe algorithm. The uncertainty-based
automatic adjustment of hyper-parameters is also applied to eliminate the need
of laborious hyper-parameter analysis during optimization. By optimizing the
composite loss, CAMRL predicts the next training task and continuously revisits
the transfer matrix and network weights. We have conducted experiments on a
wide range of benchmarks in multi-task RL, covering Gym-minigrid, Meta-world,
Atari video games, vision-based PyBullet tasks, and RLBench, to show the
improvements of CAMRL over the corresponding single-task RL algorithm and
state-of-the-art MTRL algorithms. The code is available at:
https://github.com/huanghanchi/CAMRL |
---|---|
DOI: | 10.48550/arxiv.2211.03352 |