Using a low-detail representation of surfaces to influence a high-detail representation of the surfaces

A computer-implemented technique is described herein for generating a digital representation of real-world surfaces in an environment. The technique involves receiving sensor data captured by one or more depth-sensing devices. The technique then provides a low-detail (LD) representation of surfaces in the physical environment based on the sensor data; that LD representation describes one or more planes in the physical environment. The technique then generates a high-detail (HD) representation of the surfaces based on the sensor data and the LD representation. Finally, the technique provides an output result based on the LD representation and/or the HD representation. According to one illustrative aspect, the technique produces the HD representation such that it is non-duplicative of information that is already present in the LD representation. The technique performs this task by identifying and excluding HD information that is already adequately represented by the LD representation.