Noise-free differentiable ray casting

A differentiable ray casting technique may be applied to a model of a three-dimensional (3D) scene (scene includes lighting configuration) or object to optimize one or more parameters of the model. The one or more parameters define geometry (topology and shape), materials, and lighting configuration (e.g., environment map, a high-resolution texture that represents the light coming from all directions in a sphere) for the model. Visibility is computed in 3D space by casting at least two rays from each ray origin (where the two rays define a ray cone). The model is rendered to produce a model image that may be compared with a reference image (or photograph) of a reference 3D scene to compute image space differences. Visibility gradients in 3D space are computed and backpropagated through the computations to reduce differences between the model image and the reference image.