A Robotic Perceptual System Utilizing Passive Vision and Active Touch

This paper presents a robotic perceptual system which utilizes passive vision and active touch. The task is one-fingered exploration of a single unmodeled object for apprehension— the determination of the features of the object and the rela tions among them. A two-stage exploration is utilized. Vision is first used in a feedforward manner to segment the object and to obtain its position. Touch is then used in a feedback mode to further explore the object. In designing this system, we have addressed several issues. The first concerns the way in which the robotic perceptual system should be structured. The model which we propose here is based upon theories of human perception. It consists of a highly modularized set of knowledge-based modules, each of which is domain specific and informationally encapsulated. Within the framework of this model, we have designed both vision and touch subsys tems. In each case, we have defined the primitives, features, and representations extracted and created by the system. The visual system is passive and relatively simple. The touch system is active and is an attempt to systematically structure a robotic haptic perception system. Finally, we have ad dressed the issues of how these two subsystems interact dur ing active exploration and how the information from each is integrated. The work described has been implemented and tested on a robot system consisting of a combination tactile array/force-torque sensor, a PUMA robot arm, a pair of CCD cameras, and a Vax 750.