TASK PLANNING ACCOUNTING FOR OCCLUSION OF SENSOR OBSERVATIONS

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for planning robotic movements to capture desired sensor measurements. One of the methods includes generating a three-dimensional representation of a robotic operating environment, wherein the robotic operating environment comprises a robot and a sensor, including: generating a first three-dimensional representation of a field of view of the sensor in the robotic operating environment, and generating a second three-dimensional representation of a desired observation of an object in the robotic operating environment; generating a plurality of candidate plans for the robot; selecting, from the candidate plans, a particular candidate plan that intersects the first three-dimensional representation of the field of view of the sensor and the second three-dimensional representation of the desired observation of the object; and causing the robot to execute the particular candidate plan to make the desired observation of the object in the robotic operating environment.