Video transmission based on independently encoded background updates

Systems and methods are provided for alleviating bandwidth limitations of video transmission, enhancing the quality of videos at a receiver, and improving the VR/AR experience. In particular, an improved video transmission and rendering system is provided for generating high-resolution videos. The systems have therein a transmitter and a VR/AR receiver; the transmitter includes an outer encoder and a core encoder, while the receiver includes a core decoder and an outer decoder. The outer encoder is adapted to receive the video from a source and separately output a salient video and an encoded three-dimensional background, and the outer decoder is adapted to merge the background with the salient video thereby producing an augmented video. Also provided is a system that simulates pan-tilt-zoom (PTZ) operations without PTZ hardware. Further provided are methods for video transmission whereby a three-dimensional background model is generated, a background independently encoded, updated incrementally, and the background and the updates transmitted independently from the video.