Novosibirsk State University Journal of Information Technologies
Scientic Journal

ISSN 2410-0420 (Online), ISSN 1818-7900 (Print)

Switch to

All Issues >> Contents: Volume 12, Issue No 4 (2014)

An overview of complex content-based video retrieval methods
I. K. Nikitin

Moscow Aviation Institute

UDC code: 004.932.4

The paper focuses on an overview of the different existing methods in content-based video retrieval. During the last decade there was a rapid growth of video posted on the Internet. This imposes urgent demands on video retrieval. Video has a complex structure and can express the same idea in different ways. This makes the task of searching for video more complicated. Video titles and text descriptions cannot give the hole information about objects and events in the video. This creates a need for content-based video retrieval. There is a semantic gap between low-level video features, that can be extracted, and the users' perception. Complex content-based video retrieval can be regarded as the bridge between traditional retrieval and semantic-based video retrieval.

Key Words
video retrieval, video reranking, video mining, video classification, video annotation, shots, scenes, near-duplicates video, frames

How to cite:
Nikitin I. K. An overview of complex content-based video retrieval methods // Vestnik NSU Series: Information Technologies. - 2014. - Volume 12, Issue No 4. - P. 71-82. - ISSN 1818-7900. (in Russian).

Full Text in Russian

Available in PDF

1. Nevenka Dimitrova, Hong-Jiang Zhang, Behzad Shahraray, Ibrahim Sezan, Thomas Huang, and Avideh Zakhor. Applications of video-content analysis and retrieval // IEEE MultiMedia. 2002. Vol. 9 (3). P. 42–55.
2. Yuk Ying Chung, Wai Kwok Jess Chin, Xiaoming Chen, David Yu Shi, Eric Choi, and Fang Chen. Performance analysis of using wavelet transform in content based video retrieval system // Proceedings of the 2007 Annual Conference on International Conference on Computer Engineering and Applications, CEA’07. Stevens Point, Wisconsin, USA, 2007. P. 277–282.
3. Smeaton A. F. Techniques used and open challenges to the analysis, indexing and retrieval of digital video // Information Systems. 2006. Vol. 32 (4). P. 545–559.
4. Laurence Nigay, Joëlle Coutaz. A design space for multimodal systems: Concurrent processing and data fusion // In Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems. New York, 1993. P. 172–178.
5. Haase B., Marc Eliot Davis, Davis M. Media streams: Representing video for retrieval and repurposing. Technical report. 1995.
6. Vijaya Kumar Kamabathula and Sridhar Iyer. Automated tagging to enable fine-grained browsing of lecture videos // 2012 IEEE Fourth International Conference on Technology for Education. 2011. P. 96–102.
7. Yanwei Fu, Yanwen Guo, Yanshu Zhu, Feng Liu, Chuanming Song, and ZhiHua Zhou. Multiview video summarization // Multimedia, IEEE Transactions on. 2010. Vol. 12 (7). P. 717–729.
8. Meng Wang, R. Hong, Guangda Li, Zheng-Jun Zha, Shuicheng Yan, and TatSeng Chua. Event driven web video summarization by tag localization and key-shot identification // Multimedia, IEEE Transactions on. 2012. Vol. 14 (4). P. 975–985.
9. Xu Chen, AO. Hero, and S. Savarese. Multimodal video indexing and retrieval using directed information // Multimedia, IEEE Transactions on. 2012. Vol. 14 (1). P. 3–16.
10. Zheng-Jun Zha, Meng Wang, Yan-Tao Zheng, Yi Yang, Richang Hong, Chua T.-S. Interactive video indexing with statistical active learning. Multimedia, IEEE Transactions on. 2012. № 14 (1). P. 17–27.
11. Jun Wu, Marcel Worring. Efficient genre-specific semantic video indexing // IEEE Transactions on Multimedia. 2012. № 14 (2). P. 291–302.
12. Muhammad Nabeel Asghar, Fiaz Hussain, Rob Manton. Video indexing: A survey // International Journal of Computer and Information Technology. 2014. Vol. 3. P. 148–169.
13. Huurnink B., Snoek C. G. M., Rijke M. de, Smeulders A. W. M. Contentbased analysis improves audiovisual archive retrieval // Multimedia. IEEE Transactions on. 2012. Vol. 14 (4). P. 1166–1178.
14. Tamizharasan C., Chandrakala S. A survey on multimodal content based video retrieval // International Journal of Emerging Technology and Advanced Engineering. Chennai, INDIA, 2013. Vol. 3.
15. Karpenko A., Aarabi P. Tiny videos: A large data set for nonparametric video retrieval and frame classification // Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2011. Vol. 33 (3). P. 618–630.
16. Xiangang Cheng, Liang-Tien Chia. Stratification-based keyframe cliques for effective and efficient video representation // IEEE Transactions on Multimedia. 2011. Vol. 13 (6). P. 1333–1342.
17. Yu-Gang Jiang, Qi Dai, Jun Wang, Chong-Wah Ngo, Xiangyang Xue, Shih-Fu Chang. Fast semantic diffusion for large-scale context-based image and video annotation // Image Processing, IEEE Transactions on. 2012. Vol. 21 (6). P. 3080–3091.
18. Hong Qing Yu, Pedrinaci C., Dietze S., Domingue J. Using linked data to annotate and search educational video resources for supporting distance learning // Learning Technologies, IEEE Transactions on. 2012. Vol. 5 (2). P. 130–142.
19. Andre B., Vercauteren T., Buchner A. M., Wallace M. B., Ayache N. Learning semantic and visual similarity for endomicroscopy video retrieval // Medical Imaging, IEEE Transactions. 2012. Vol. 31 (6). P. 1276–1288.
20. Tianzhu Zhang, Changsheng Xu, Guangyu Zhu, Si Liu, Hanqing Lu. A generic framework for video annotation via semi-supervised learning // IEEE Transactions on Multimedia. 2012. P. 1206–1219.
21. Wei-Ta Chu, Shang-Yin Tsai. Rhythm of motion extraction and rhythm-based cross-media alignment for dance videos // Multimedia, IEEE Transactions on. 2012. Vol. 14 (1). P. 129–141.
22. Xinmie Tian, Linjun Yang, Jingdong Wang, Xiuqing Wu, Xian-Sheng Hua. Bayesian visual reranking // Trans. Multi. 2011. Vol. 13 (4). P. 639–652.
23. Bashar Tahayna, Mohammed Belkhatir, M. Saadat Alhashmi, O’Daniel Th. Optimizing support vector machine based classification and retrieval of semantic video events with genetic algorithms // Image Processing (ICIP). 2010 17th IEEE International Conference on. 2010. P. 1485–1488.
24. Mehmet Emre Sargin, Hrishikesh Aradhye. Boosting video classification using cross-video signals // Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference. 2011. P. 1805–1808.
25. JaeDeok Lim, ByeongCheol Choi, SeungWan Han, ChoelHoon Lee. Adult movie classification system based on multimodal approach with visual and auditory features // In Information Science and Digital Content Technology (ICIDT), 2012 8th International Conference on. 2012. Vol. 3. P. 745–748.
26. Ionescu B., Seyerlehner K., Rasche Ch., Vertan C., Lambert P. Video Genre Categorization and Representation using Audio-Visual Information. 2012. Vol. 21 (2).
27. Ba Tu Truong, Svetha Venkatesh. Video abstraction, A systematic review and classification // ACM Trans. Multimedia Comput. Commun. 2007. Vol. 3 (1).
28. Xu-Dong Zhang, Tie-Yan Liu, Kwok-Tung Lo, Jian Feng. Dynamic selection and effective compression of key frames for video abstraction // Pattern Recogn. Lett. 2003. Vol. 24 (9–10). P. 1523–1532.
29. Kazunori Matsumoto, Masaki Naito, Keiichiro Hoashi, Fumiaki Sugaya. Svm-based shot boundary detection with a novel feature // IEEE International Conference on Multimedia and Expo. 2006. P. 1837–1840.
30. Ba Tu Truong, S. Venkatesh, C. Dorai. Scene extraction in motion pictures // IEEE Trans. Cir. and Sys. for Video Technol. 2003. Vol. 13 (1). P. 5–15.
31. H. Sundaram and Shih-Fu Chang. Video scene segmentation using video and audio features // Multimedia and Expo, 2000. IEEE International Conference on. 2000. Vol. 2. P. 1145–1148.
32. Liang-Hua Chen, Yu-Chun Lai, Hong-Yuan Mark Liao. Movie scene segmentation using background information // Pattern Recogn. 2008. Vol. 41 (3). P. 1056–1065.
33. Stephan Repp, Andreas Grob, Christoph Meinel. Browsing within lecture videos based on the chain index of speech transcription // IEEE Transactions on Learning Technologies. 2008. Vol. 1 (3). P. 145–156.
34. Rong Yan, Alexander G. Hauptmann. A review of text and image retrieval approaches for broadcast news video // Inf. Retr. 2007. Vol. 10 (4–5). P. 445–484.
35. John Adcock, Andreas Girgensohn, Matthew Cooper, Ting Liu, Lynn Wilcox, Eleanor Rieffel. Fxpal experiments for trecvid 2004 // Proceedings of the TREC Video Retrieval Evaluation (TRECVID). 2004. P. 70–81.
36. Hauptmann A. G., Baron R. V., Chen M. Y., Christel M., Duygulu P., Huang C., Jin R., Lin W. H., Ng D., Moraveji N., Papernick N., Snoek C. G. M., Tzanetakis G., Yang J., Yan R., Wactlar H. D. Informedia at trecvid 2003: Analyzing and searching broadcast news video // Proceedings of the TRECVID Workshop. 2003.
37. Sivic J., Everingham M., Zisserman A.. Person spotting: Video shot retrieval for face sets // In ACM International Conference on Image and Video Retrieval. 2005.
38. Huiping Li, Doermann D. Video indexing and retrieval based on recognized text // Multimedia Signal Processing, 2002 IEEE Workshop on. 2002. P. 245–248.
39. Seyerlehner K., Schedl M., Pohle T., Knees P. Using blocklevel features for genre classification, tag classification and music similarity estimation. 2010.
40. Chih-Wen Su, H.-Y.M. Liao, Hsiao-Rong Tyan, Chia-Wen Lin, Duan-Yu Chen, Kuo-Chin Fan. Motion flow-based video retrieval // Multimedia, IEEE Transactions on. 2007. Vol. 9 (6). P. 1193–1201.
41. Anjulan A., Canagarajah C. N. A unified framework for object retrieval and mining // IEEE Transactions on Circuits and Systems for Video Technology. 2009. Vol. 19 (1). P. 63–76.
42. Quack T., Ferrari V., Gool L. Video mining with frequent item set configurations // Int. Conf. Image Video Retrieval. 2006. P. 360–369.
43. Yu-Gang Jiang, Chong-Wah Ngo, Jun Yang. Towards optimal bag-offeatures for object categorization and semantic video retrieval // In Proceedings of the 6th ACM International Conference on Image and Video Retrieval, CIVR ’07. New York, NY, USA, 2007. P. 494–501.
44. Peng Chang, Mei Han, Yihong Gong. Extract highlights from baseball game video with hidden markov models. 2002. P. 609–612.
45. Hong G. Y., Fong B., Fong A. C. M. An intelligent video categorization engine // Kybernetes. 2005. Vol. 34 (6). P. 784–802.
46. Linjun Yang, Jiemin Liu, Xiaokang Yang, Xian-Sheng Hua. Multimodality web video categorization // Proceedings of the International Workshop on Workshop on Multimedia Information Retrieval, MIR ’07. New York, NY, USA, 2007. P. 265–274.
47. Weal M. J., Michaelides D. T., Page K., D. Roure C. De, Monger E., Gobbi M. Semantic annotation of ubiquitous learning environments // IEEE Transactions on Learning Technologies. 2012. Vol. 5 (2). P. 143–156.
48. Yusuf Aytar, Mubarak Shah, Jiebo Luo. Utilizing semantic word similarity measures for video retrieval // 2013 IEEE Conference on Computer Vision and Pattern Recognition. 2008. P. 1–8.
49. Sivic J., Zisserman A. Video google. Efficient visual search of videos // Toward CategoryLevel Object Recognition. 2006. P. 127–144.
50. Browne P., Smeaton A. F. Video retrieval using dialogue, keyframe similarity and video objects // ICIP. 2005. Vol. (3). P. 1208–1211.
51. Cees G. M. Snoek, Bouke Huurnink, Laura Hollink, Maarten de Rijke, Guus Schreiber, Marcel Worring. Adding semantics to detectors for video retrieval // IEEE Transactions on Multimedia. 2007. Vol. 9 (5). P. 975–986.
52. Liang-Hua Chen, Kuo-Hao Chin, Hong-Yuan Liao. An integrated approach to video retrieval // In Alan Fekete and Xuemin Lin, editors, Nineteenth Australasian Database Conference (ADC 2008). Wollongong, NSW, Australia, 2008. Vol. 75. P. 49–55.
53. Kulesh V., Petrushin V. A., Sethi I. K. The perseus project: Creating personalized multimedia news portal. O. R. Zaiane, S. J. Simoff (eds.). MDM/KDD. University of Alberta, 2001. P. 31–37.
54. Kexue Dai, Jun Zhang, Guohui Li. Video mining: concepts, approaches and applications // Multi-Media Modelling Conference Proceedings, 2006 12th International. 2006.

Publication information
Main title Vestnik NSU Series: Information Technologies, Volume 12, Issue No 4 (2014).
Parallel title: Novosibirsk State University Journal of Information Technologies Volume 12, Issue No 4 (2014).

Key title: Vestnik Novosibirskogo gosudarstvennogo universiteta. Seriâ: Informacionnye tehnologii
Abbreviated key title: Vestn. Novosib. Gos. Univ., Ser.: Inf. Tehnol.
Variant title: Vestnik NGU. Seriâ: Informacionnye tehnologii

Year of Publication: 2014
ISSN: 1818-7900 (Print), ISSN 2410-0420 (Online)
Publisher: Novosibirsk State University Press
DSpace handle

|Home Page| |All Issues| |Information for Authors| |Journal Boards| |Ethical principles| |Editorial Policy| |Contact Information| |Old Site in Russian|
© 2006-2017, Novosibirsk State University.