E-Partner: A Photo-Realistic Conversation Agent

An E-Partner is a photo-realistic conversation agent, which has a talking head that not only look photo-realistic but also can have a conversation with the user about a given topic. The conversation is multimedia-enriched in that the E-Partner presents relevant multimedia materials throughout the conversation. To address the challenges presented by the complex conversation domain and task, and to achieve adaptive behaviors, we have derived a novel dialogue manager design consisting of five parts: a domain model, a dialogue model, a discourse model, a task model, and a user model. We also extended existing facial animation techniques to create photo-realistic talking heads that facilitate conversational interactions. Some practical issues like how to handle the uncertainty from speech level are also discussed.