Online state space generation by a growing self-organizing map and differential learning for reinforcement learning

In this research, we develop a method integrating a growing self-organizing map and differential learning system for online reinforcement learning which adaptively builds the state structure. In the conventional method, models and information on the environment are required beforehand, whereas the proposed method automatically estimates the state transitions from differentials of input signals and from these builds the state space without reference to prior information on the environment. Also, since it is an online learning method, the proposed method requires less computation and no batch memory. Through numerical experiments, we show that the proposed method has the same performance as the conventional method with the information given and that the learning time is shortened by abstraction of the state space. •Development of online state space generation system for reinforcement learnings.•Improvement of search efficiency of reinforcement learnings.•Realization of learning from only the sensor information (or feature vector), the reward, and the number of actions.