AR GESTURE USER INTERFACE SYSTEM FOR PORTABLE TERMINAL

PROBLEM TO BE SOLVED: To provide a user interface system capable of performing interlocking to control of a portable terminal or operation of an application within the terminal while feeling an actual touch of a virtual object by displaying the virtual object while overlapping with a real image photographed by a compact camera mounted on the back of the portable terminal, and operating the virtual object with a finger of the hand placed in a space in the depth of the terminal.SOLUTION: A space in the depth of a terminal T where a hand (h) is placed is photographed by a compact camera 1 mounted on the back of the terminal T, motions of a finger (f) are recognized by processing an obtained image sequence, and a virtual keyboard (k) and a recognition result are superimposed on the image sequence and displayed on a display 2 of the terminal T. Thus, an operation region becomes a wide space in the depth of the terminal T and terminal operation with higher convenience can be implemented.