VIRTUAL EYEGLASS SET FOR VIEWING ACTUAL SCENE THAT CORRECTS FOR DIFFERENT LOCATION OF LENSES THAN EYES

A virtual eyeglass set, comprising a right lens and a left lens, wherein each of the right lens and the left lens comprising a light field camera, having multiple viewpoints, on a first side, and a light field display and a direction camera on a second side; and a processor configured to display, at each of the right lens and the left lens, a light field image based on data received via the light field camera, the light field image presenting an image of objects beyond the virtual eyeglass set from a perspective of an eye of a user wearing the virtual eyeglass set. The processor is configured to determine, based on a signal of the direction camera, a direction in which the user's eyes are pointing, based on the determined direction, determine a single viewpoint, from the multiple viewpoints in each camera, which is along a ray extending from the user's eyes to the object, and generate images for display to the user in the light field displays on the lenses based only on the image data captured from the viewpoints along the rays, wherein the images are based on the image data captured from the viewpoints along the rays that extend from the user's eyes to the object, enabling the user to view the object in the light field display of the lenses as if the lenses were transparent.