Human Intention Modeling and Interactive Computer Vision

For many years, computer vision and robotics researchers have worked hard chasing the illusive goals such as "can the robot find a boy in the scene" or "can your vision system automatically segment the cat from the background". These tasks require a lot of prior knowledge and contextual information, and perhaps more importantly, understanding of human intention. How to model human intention into vision and robotic systems is, however, very challenging and can only be solved through human-computer interaction. In this talk, we propose that many difficult vision tasks can be solved with interactive vision systems, by combining powerful and real-time vision techniques with intuitive and clever user interfaces. We will show two interactive vision systems we developed recently, Lazy Snapping (Siggraph 2004) and Image Completion (Siggraph 2005). Lazy Snapping cuts out an object from a picture using graph cut, while Image Completion recovers unknown region in a picture with belief propagation. A key element in designing such interactive systems is how we model the user's intention using conditional probability (context) and likelihood associated with user interactions. Given how ill-posed most image understanding problems are, it is proposed that interactive computer vision is the paradigm we should focus today's vision research on where the key is the understanding and modeling of human intention.