Methods and systems for performing reinforcement learning in hierarchical and temporally extended environments

A system implementing reinforcement learning the system comprising a computer processor and a computer readable medium having computer executable instructions executed by said processor; said computer readable medium including instructions for providing: an action values module that receives environmental state as input, containing at least one adaptive element that learns state and/or action values based on an error signal; an action selection module coupled to the action values module; and an error calculation module coupled to both the action values and action selection module, which computes an error signal used to update state and/or action values in the action values module.