Video summarization based on the psychological unfolding of a drama

This paper proposes a method of composing digests of TV videos based on their psychological unfolding. Most of the studies of video summarization to date have been based on the video structure, such as cuts or objects in the video. It is generally considered, however, that the observer of a TV drama emphasizes instead the psychological unfolding of the video content (such as the climax). However, no method of summarizing a drama based on the psychological aspect of the video has been proposed. Track structures [such as cutting, lines spoken by the actors, BGM (background music), and sound effects] constitute physical components, and the dramatist skillfully manipulates the time structure based on his empirical knowledge in order to present a particular psychological impression. This paper proposes a method in which the psychologically important part is detected on the basis of empirical knowledge as a corresponding time‐series pattern of the track structure, and a summarized video is generated. As an experiment, the proposed method was applied to a TV drama, and the time length was reduced to approximately 1/14. Subjective evaluation experiments show that the proposed method preserves the content of the main plot, with naturalness of extraction, thus indicating the usefulness of the proposed method. © 2002 Wiley Periodicals, Inc. Syst Comp Jpn, 33(11): 39–49, 2002; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10066