Q-Tree Search: An Information-Theoretic Approach Toward Hierarchical Abstractions for Agents With Computational Limitations

In this article, we develop a framework to obtain graph abstractions for decision-making where the abstractions emerge as a function of the agent's available resources. We discuss the connection of the proposed approach with information-theoretic signal compression and formulate a novel optimization problem to obtain tree-based abstractions that are a function of the agent's computational resources. The structural properties of the new problem are discussed in detail and two algorithmic approaches are proposed. We discuss the quality of, and prove relationships between, the solutions obtained by the two proposed algorithms. The framework is applied to a variety of environments to obtain hierarchical abstractions.