SYSTEM AND METHOD FOR EXTRACTING AND TRANSPLANTING LIVE VIDEO AVATAR IMAGES

A system for extracting and transplanting live video avatar images including a depth sensor for creating a depth map based first live video avatar of a user or object disposed in a heterogeneous first environment with an arbitrary background; a processor coupled to the depth sensor; code fixed in a tangible medium for execution by the processor for extracting the depth map from the first environment to provide an extracted depth map based live video avatar; and a display system coupled to the processor for showing the extracted depth map based live video avatar in a second environment diverse from the first environment. In a second embodiment, the system includes a camera coupled to the processor to provide live video images of the user in the first environment and code for spatially filtering the images to provide a spatially filtered extracted second live video avatar. This embodiment further includes code for combining the first live video avatar with the second live video avatar to provide an enhanced extracted depth map based third live video avatar. Images from multiple cameras and or depth sensors are combined simultaneously to provide the third live video avatar using the spatially enhanced extracted depth map. A routing server is included for receiving streams from multiple users and sending to each user the live video avatar images from other users based on their locations in a shared space or for use in a local user's AR environment.