The virtual interaction panel: an easy control tool in augmented reality systems

In this paper, we propose and develop an easy control tool called Virtual Interaction Panel (VirIP) for Augmented Reality (AR) systems, which can be used to control AR systems. This tool is composed of two parts: the design of the VirIPs and the tracking of an interaction pen using a Restricted Coulomb Energy (RCE) neural network. The VirIP is composed of some virtual buttons, which have meaningful information that can be activated by an interaction pen during the augmentation process. The interaction pen is a general pen‐like object with a certain color distribution. It is tracked using a RCE network in real‐time and used to trigger the VirIPs for AR systems. In our system, only one camera is used for capturing the real world. Therefore, 2D information is used to trigger the virtual buttons to control the AR systems. The proposed method is real‐time because the RCE‐based image segmentation for a small region is fast. It can be used to control AR systems quite easily without any annoying sensors attached to entangling cables. This proposed method has good potential in many AR applications in manufacturing, such as assembly without the need for object recognition, collaborative product design, system control, etc. Copyright © 2004 John Wiley & Sons, Ltd.