Use of multimedia data for emoticons in instant messaging

The present invention provides a method, and corresponding apparatus, for use of emoticons in IM applications by using sensory information captured by a device. Such information can include video, still image, and/or audio information. In one embodiment, based on a trigger to the system, multimedia input is captured, and relevant features are extracted from it. The extracted information is interpreted, and the interpreted information is mapped onto one or more specific pre-existing emoticons. These specific emoticons are then inserted into the IM communication via an IM API. In another aspect of the present invention, new emoticons are created based on the multimedia information captured. This can include generation of realistic emoticons based on the expressions on the user's face. Animated emoticons can also be created.