Automatic Discovery of Subgoals for Sequential Decision Problems Using Potential Fields

This paper presents a new method by which a sequential decision agent can automatically discover subgoals online. The agent discovers subgoals using potential field. The method uses a reward function to generate a potential field, and then abstracts some features from the potential field as candidates of subgoals. Based on the candidates, the agent can determine its behaviors online through some heuristics in unknown environment. The best-known and most often-cited problem with the potential field method is local minima. But our method does not have this limitation because the local minima are used to form subgoals. The disadvantage of the local minima in the previous approaches of potential field turns out to be an advantage in our method. We illustrate the method using a simple gridworld task.