Tamires Tessarolli de Souza Barbieri
ABSTRACT
Currently, videos play an important role in people's every day lives and there is a continuous increase in their production and consumption. However, to make video information access a feasible task, smarter and more efficient strategies are needed in order to deal with this complex and huge data volume. In this context, automatic video summarization becomes a significant tool. This research area aims to produce a smaller version of original video, keeping its most meaningful information. Particularly for some relavant domains, like news and education, it is important a summary to be generated from a set of related videos - different videos about the same subject. In the automatic multi-video summarization field a common and essential step is to define content selection criteria. A promising approach, yet unexploited by related work, is to apply inclusion and exclusion criteria derived from human strategies. In this work, we investigate how exclusion criteria can help to keep multi-video summaries short and informative. We introduce an approach to detect subjectivity into video segments from news domain, allowing those segments to be not included into the summary. Our approach is evaluated by the means of objective measures of effectiveness. We also analyze the impact that removing such subjective segments has on multi-video summarization outputs by conducting a study with final users. Results indicate that the proposed approach is able of producing more compact summaries, keeping the relevant content for the users.
- T. T. S. Barbieri and R. Goularte. 2020. Content selection criteria for news multi-video summarization based on human strategies. International Journal on Digital Libraries (2020), 1--14. https://doi.org/10.1007/s00799-020-00281-9Google Scholar
- R. T. Camargo, V. Agostini, A. Di-Felippo, and T.A.S. Pardo. 2013. Manual Typification of Source Texts and Multi-document Summaries Alignments. Procedia - Social and Behavioral Sciences 95 (2013), 498--506. https://doi.org/10.1016/j.sbspro.2013.10.674Google ScholarCross Ref
- R. T. Camargo, A. Di-Felippo, and T. A. S. Pardo. 2015. On Strategies of Human Multi-Document Summarization. In Proceedings of the 10th Brazilian Symposium in Information and Human Language Technology (Natal, Brazil) (STIL, 15). Sociedade Brasileira de Computação, 141--150. https://www.aclweb.org/anthology/ W15-5618Google Scholar
- Gabriella Csurka, Christopher R. Dance, Lixin Fan, Jutta Willamowski, and Cédric Bray. 2004. Visual categorization with bags of keypoints. In In Workshop on Statistical Learning in Computer Vision, ECCV. 1--22.Google Scholar
- K. Dale, E. Shechtman, S. Avidan, and H. Pfister. 2012. Multi-video browsing and summarization. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 1--8. https://doi.org/10.1109/CVPRW.2012.6239253Google Scholar
- L. Hasler. 2007. From Extracts to Abstracts: Human Summary Production Operations for Computer-Aided Summarisation. In Proceedings of the RANLP Workshop on Computer-aided Language Processing (Borovest, Bulgary) (CALP'07). 11--18. https://acl-bg.org/proceedings/2007/RANLP_W7%202007.pdf#page=18Google Scholar
- Richang Hong, Jinhui Tang, Hung-Khoon Tan, Shuicheng Yan, Chongwah Ngo, and Tat-Seng Chua. 2009. Event Driven Summarization for Web Videos. In Proceedings of the First SIGMM Workshop on Social Media (Beijing, China) (WSM '09). ACM, New York, NY, USA, 43--48. https://doi.org/10.1145/1631144.1631154Google ScholarDigital Library
- I. Ide, Y. Zhang, R. Tanishige, K. Doman, Y. Kawanishi, D. Deguchi, and H. Murase. 2017. Summarization of News Videos Considering the Consistency of Auditory and Visual Contents. In 2017 IEEE International Symposium on Multimedia (ISM). 193--199. https://doi.org/10.1109/ISM.2017.33Google Scholar
- Zhong Ji, Yuanyuan Zhang, Yanwei Pang, and Xuelong Li. 2018. Hypergraph Dominant Set Based Multi-video Summarization. Signal Processing 148, C (Jul 2018), 114--123. https://doi.org/10.1016/j.sigpro.2018.01.028Google Scholar
- A. Kanehira, L. Van Gool, Y. Ushiku, and T. Harada. 2018. Viewpoint-Aware Video Summarization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7435--7444. https://doi.org/10.1109/CVPR.2018.00776Google Scholar
- R Kannan, G Ghinea, S Swaminathan, and S Kannaiyan. 2013. Improving video summarization based on user preferences. In Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG). 1--4. https://doi.org/10.1109/NCVPRIPG.2013.6776187Google ScholarCross Ref
- Y. Li and B. Merialdo. 2010. Multi-video summarization based on AV-MMR. In International Workshop on Content-Based Multimedia Indexing (CBMI). 1--6. https://doi.org/10.1109/CBMI.2010.5529899Google ScholarCross Ref
- Yingbo Li and B. Merialdo. 2010. Multi-video summarization based on Video-MMR. In 11th International Workshop on Image Analysis for Multimedia Interactive Services WIAMIS 10. 1--4.Google Scholar
- Y. Li and B. Merialdo. 2011. Multi-video summarization based on OB-MMR. In 9th International Workshop on Content-Based Multimedia Indexing (CBMI). 163--168. https://doi.org/10.1109/CBMI.2011.5972539Google Scholar
- Yingbo Li and Bernard Merialdo. 2012. Video Summarization Based on Balanced AV-MMR. Springer Berlin Heidelberg, 370--382. https://doi.org/10.1007/978-3-642-27355-1_35Google Scholar
- Yingbo Li and Bernard Merialdo. 2016. Multimedia maximal marginal relevance for multi-video summarization. Multimedia Tools and Applications 75, 1 (2016), 199--220. https://doi.org/10.1007/s11042-014-2287-5Google ScholarDigital Library
- Yingbo Li, Bernard Merialdo, Mickael Rouvier, and Georges Linares. 2011. Static and Dynamic Video Summaries. In Proceedings of the 19th ACM International Conference on Multimedia (Scottsdale, Arizona, USA) (MM '11). ACM, New York, NY, USA, 1573--1576. https://doi.org/10.1145/2072298.2072068Google ScholarDigital Library
- G.D. Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60 (2004), 91--110. https://doi.org/10.1023/B:VISI.0000029664.99615.94Google ScholarDigital Library
- C. D. Manning, P. Raghavan, and H. Schütze. 2009. An Introduction to Information Retrieval. Cambridge University Press.Google Scholar
- Market.US. 2019. Amazon Prime Video Statistics and Facts. https://market.us/statistics/online-video-and-streaming-sites/amazon-prime-video/ Access on: 02/19/2020.Google Scholar
- Netflix. 2017. 2017 on Netflix - A Year in Bingeing. https://media.netflix.com/en/press-releases/2017-on-netflix-a-year-in-bingeing Access on: 02/19/2020.Google Scholar
- Netflix. 2020. About Netflix - Netflix has been leading the way for digital content since 1997. https://media.netflix.com/en/about-netflix Access on: 02/19/2020.Google Scholar
- L. Nie, R. Hong, L. Zhang, Y. Xia, D. Tao, and N. Sebe. 2016. Perceptual Attributes Optimization for Multivideo Summarization. IEEE Transactions on Cybernetics 46, 12 (Dec 2016), 2991--3003. https://doi.org/10.1109/TCYB.2015.2493558Google ScholarCross Ref
- S. H. Ou, C. H. Lee, V. S. Somayazulu, Y. K. Chen, and S. Y. Chien. 2015. On-Line Multi-View Video Summarization for Wireless Video Sensor Network. IEEE Journal of Selected Topics in Signal Processing 9, 1 (Feb 2015), 165--179. https://doi.org/10.1109/JSTSP.2014.2331916Google Scholar
- S. H. Ou, Y. C. Lu, J. P. Wang, S. Y. Chien, S. D. Lin, M. Y. Yeti, C. H. Lee, P. B. Gibbons, V. S. Somayazulu, and Y. K. Chen. 2014. Communication-efficient multi-view keyframe extraction in distributed video sensors. In IEEE Visual Communications and Image Processing Conference. 13--16. https://doi.org/10.1109/VCIP.2014.7051492Google Scholar
- R. Panda, N. C. Mithun, and A. K. Roy-Chowdhury. 2017. Diversity-Aware Multi-Video Summarization. IEEE Transactions on Image Processing 26, 10 (Oct 2017), 4712--4724. https://doi.org/10.1109/TIP.2017.2708902Google ScholarDigital Library
- R. Panda and A. K. Roy-Chowdhury. 2017. Collaborative Summarization of Topic-Related Videos. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4274--4283. https://doi.org/10.1109/CVPR.2017.455Google Scholar
- Anand Rajaraman and Jeffrey David Ullman. 2011. Data Mining. Cambridge University Press, 1--17. https://doi.org/10.1017/CBO9781139058452.002Google Scholar
- M.J. Swain and D.H. Ballard. 1991. Color Indexing. International Journal of Computer Vision 7, 1 (Nov 1991), 11--32. https://doi.org/10.1007/BF00130487Google ScholarDigital Library
- Alvin Toffler. 1984. Future Shock (1 ed.). Bantam. 576 pages.Google Scholar
- F. Wang and B. Merialdo. 2009. Multi-document video summarization. In IEEE International Conference on Multimedia and Expo. 1326--1329. https://doi.org/10.1109/ICME.2009.5202747Google Scholar
- M. Wang, R. Hong, G. Li, Z. J. Zha, S. Yan, and T. S. Chua. 2012. Event Driven Web Video Summarization by Tag Localization and Key-Shot Identification. IEEE Transactions on Multimedia 14, 4 (Aug 2012), 975--985. https://doi.org/10.1109/TMM.2012.2185041Google ScholarDigital Library
- L. Zhang, Y. Gao, R. Hong, Y. Hu, R. Ji, and Q. Dai. 2015. Probabilistic Skimlets Fusion for Summarizing Multiple Consumer Landmark Videos. IEEE Transactions on Multimedia 17, 1 (Jan 2015), 40--49. https://doi.org/10.1109/TMM.2014.2370257Google ScholarDigital Library
- L. Zhang, Y. Xia, K. Mao, H. Ma, and Z. Shan. 2015. An Effective Video Summarization Framework Toward Handheld Devices. IEEE Transactions on Industrial Electronics 62, 2 (Feb 2015), 1309--1316. https://doi.org/10.1109/TIE.2014.2336639Google Scholar
- Yanqiu Zhang, Cuixia Ma, Jinkai Zhang, Dongliang Zhang, and Yongjin Liu. 2013. An Interactive Personalized Video Summarization Based on Sketches. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI '13). ACM, New York, NY, USA, 249--258. https://doi.org/10.1145/2534329.2534343Google ScholarDigital Library
- Ying Zhang, He Ma, and Roger Zimmermann. 2013. Dynamic Multi-video Summarization of Sensor-Rich Videos in Geo-Space. Springer Berlin Heidelberg, 380--390. https://doi.org/10.1007/978-3-642-35725-1_35Google Scholar
- Ying Zhang, Guanfeng Wang, Beomjoo Seo, and Roger Zimmermann. 2012. Multivideo Summary and Skim Generation of Sensor-rich Videos in Geo-space. In Proceedings of the 3rd Multimedia Systems Conference (Chapel Hill, North Carolina) (MMSys '12). ACM, New York, NY, USA, 53--64. https://doi.org/10.1145/2155555.2155565Google ScholarDigital Library
- Ying Zhang, Luming Zhang, and Roger Zimmermann. 2015. Aesthetics-Guided Summarization from Multiple User Generated Videos. ACM Trans. Multimedia Comput. Commun. Appl. 11, 2, Article 24 (jan 2015), 24:1-24:23 pages. https://doi.org/10.1145/2659520Google ScholarDigital Library
- Y. Zhang and R. Zimmermann. 2016. Efficient Summarization From Multiple Georeferenced User-Generated Videos. IEEE Transactions on Multimedia 18, 3 (March 2016), 418--431. https://doi.org/10.1109/TMM.2016.2520827Google ScholarDigital Library
- W. Zhao, C. Ngo, H. Tan, and X. Wu. 2007. Near-Duplicate Keyframe Identification With Interest Point Matching and Pattern Learning. IEEE Transactions on Multimedia 9, 5 (2007), 1037--1048. https://doi.org/10.1109/TMM.2007.898928Google ScholarDigital Library
- J. Zhu, S. Liao, and S. Z. Li. 2016. Multicamera Joint Video Synopsis. IEEE Transactions on Circuits and Systems for Video Technology 26, 6 (June 2016), 1058--1069. https://doi.org/10.1109/TCSVT.2015.2430692Google ScholarCross Ref
Index Terms
- Investigating Subjectivity Criterion for Multi-video Summarization
Recommendations
Video Summarization using Text Subjectivity Classification
WebMedia '22: Proceedings of the Brazilian Symposium on Multimedia and the WebVideo summarization has attracted researchers’ attention because it provides a compact and informative video version, supporting users and systems to save efforts in searching and understanding content of interest. Current techniques employ different ...
Content selection criteria for news multi-video summarization based on human strategies
AbstractIn the recent years, the multimedia data volume produced and available for access has increased continuously and quickly, notably video content. This context has also increased the overload information problem: finding content of interest in the ...
Latent dirichlet allocation based multi-document summarization
AND '08: Proceedings of the second workshop on Analytics for noisy unstructured text dataExtraction based Multi-Document Summarization Algorithms consist of choosing sentences from the documents using some weighting mechanism and combining them into a summary. In this article we use Latent Dirichlet Allocation to capture the events being ...
Comments