VideoForest: Interactive Visual Summarization of Video Streams Based on Danmu Data

Zhida Sun1, Mingfei Sun1, Nan Cao2, Xiaojuan Ma1.
1The Hong Kong University of Science and Technology, 2TongJi University.

Paper Slide Video

VideoForest is a visualization system designed to convert an input video stream augmented with danmu data into a visual summary of its content with highlights along the storyline under users’ supervision.

Abstract

Emerging online video-sharing websites such as YouTube allow users to access a huge number of videos and generate feedback via reviews and/or live comments (a.k.a. danmu), making it possible to summarize videos based on media and user responses collectively. A video summary produced by existing techniques may not fully capture the audience's perception of and reaction to the source video, and thus may be less reflective. In this paper, we introduce VideoForest, a visualization system designed to convert an input video augmented with danmu commentary data into a tree-like visual summary of content highlights under users' supervision. The proposed visualization design employs a forest metaphor. The overall summary of different video sessions is illustrated as scene trees on top of the session timeline ground, with the roots depicting the corresponding danmu messages. VideoForest can also generate a detailed synopsis of the user-selected video segment(s) as a compact storyline in the form of circle packing. We evaluate our system via case studies with real video data based on experts' feedback. The results suggest the power and potential of the system.

Forest-Metaphor Visual Design

siggraphasia16_teaser

In this design, session-level key-frames of the video are clustered and visualized in scene trees (a) planted in a video timeline ground (b,c) with roots (d) showing the corresponding danmu posts. The video timeline ground consists of (b) a heatmap view that presents the contextual information extracted from danmu posts such as sentiments and volume and, (c) an interactive timeline view that enables users to specify the video sessions of interest.

System User Interface

siggraphasia16_ui

The VideoForest system’s user interface consists of (a) the primary timeline summary view and (b) the session summary view.

BibTeX


                            @inproceedings{10.1145/3002151.3002159,
                            author = {Sun, Zhida and Sun, Mingfei and Cao, Nan and Ma, Xiaojuan},
                            title = {VideoForest: interactive visual summarization of video streams based on danmu data},
                            year = {2016},
                            isbn = {9781450345477},
                            publisher = {Association for Computing Machinery},
                            address = {New York, NY, USA},
                            url = {https://doi.org/10.1145/3002151.3002159},
                            doi = {10.1145/3002151.3002159},
                            booktitle = {SIGGRAPH ASIA 2016 Symposium on Visualization},
                            articleno = {10},
                            numpages = {8},
                            keywords = {multimedia (image/video/music) visualization},
                            location = {Macau},
                            series = {SA '16}
                            }