IMAGE-BASED ENVIRONMENT RECONSTRUCTION WITH VIEW-DEPENDENT COLOUR

Disclosed is a method and system for: obtaining 3D data structure comprising nodes, each node representing voxel of 3D grid of voxels, wherein node stores viewpoint information, with any of: (i) colour tile that captures colour information of voxel and depth tile, (ii) reference information indicative of unique identification of colour and depth tiles; utilising 3D data structure for training neural network(s), wherein input of neural network(s) comprises 3D position of point in real-world environment and output of neural network(s) comprises colour and opacity of point; and for new viewpoint, determining visible nodes whose voxels are visible from new viewpoint; for visible node, selecting depth tile(s) whose viewpoint(s) matches new viewpoint most closely; reconstructing 2D geometry of objects from depth tiles; and utilising neural network(s) to render colours for pixels of output colour image.