Environment generation using one or more neural networks

A simulated environment is generated based at least in part on stored information associated with objects within the simulated environment generated by one or more neural networks. The neural networks may include a generative network to receive action input or more prior images for an agent in the simulated environment, the generative network including a rending engine to distinguish between static objects and dynamic objects. Static objects may be mapped in a memory module, and a dynamics engine may be used to generate state information from action input and prior images. Embodiments include the use of the simulation for testing a robot, or as a virtual environment for a game.