Unsupervised video summarization via clustering validity index

Although lots of the prior works have been proposed to solve the representative selection problem of video summarization, the main difficulty is still left for determining the optimal representatives’ number of the raw videos that are not annotated. In this paper, we propose an unsupervised video summarization method by motion-based frame selection and a novel clustering validity indexes to determine the optimal representatives of the original video. The proposed framework segments shots and selects candidate frames by evaluating their forward and backward motion and can automatically select representatives to highlight all the significant visual properties. Shots are segmented uniformly and the frame with the largest motion is extracted in each segmentation to form the video candidate frame subset. Then Affinity Propagation combined with the validity index is used to automatically select the optimal representatives from the candidate frame subset. Our experimental result on several benchmark datasets demonstrates the robustness and effectiveness of our proposed method.