Efficient large scale near-duplicate video detection base on spark

Lv, Jinna; Wu, Bin; Yang, Shuai; Jia, Bingjing; Qiu, Peigang

doi:10.1109/bigdata.2016.7840693

Cited by 9 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They reported Marlin achieved 25X and 23X speedup against the sequential feature extraction algorithm and similarity search, respectively. The challenge of extracting distinctive features is addressed by Lv et al [173] for the efficiency of extraction closely related videos from the large scale data based on local and global features utilizing Spark. To balance precision and efficiency, they introduced a multi-feature based distributed system, including local and global features.…”

Section: A Content-based Video Retrievalmentioning

confidence: 99%

Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues

2020

View full text Add to dashboard Cite

The proliferation of multimedia devices over the Internet of Things (IoT) generates an unprecedented amount of data. Consequently, the world has stepped into the era of big data. Recently, on the rise of distributed computing technologies, video big data analytics in the cloud has attracted the attention of researchers and practitioners. The current technology and market trends demand an efficient framework for video big data analytics. However, the current work is too limited to provide a complete survey of recent research work on video big data analytics in the cloud, including the management and analysis of a large amount of video data, the challenges, opportunities, and promising research directions. To serve this purpose, we present this study, which conducts a broad overview of the state-of-the-art literature on video big data analytics in the cloud. It also aims to bridge the gap among large-scale video analytics challenges, big data solutions, and cloud computing. In this study, we clarify the basic nomenclatures that govern the video analytics domain and the characteristics of video big data while establishing its relationship with cloud computing. We propose a service-oriented layered reference architecture for intelligent video big data analytics in the cloud. Then, a comprehensive and keen review has been conducted to examine cutting-edge research trends in video big data analytics. Finally, we identify and articulate several open research issues and challenges, which have been raised by the deployment of big data technologies in the cloud for video big data analytics. To the best of our knowledge, this is the first study that presents the generalized view of the video big data analytics in the cloud. This paper provides the research studies and technologies advancing the video analyses in the era of big data and cloud computing. INDEX TERMS big data, intelligent video analytics, cloud-based video analytics system, video analytics survey, deep learning, distributed computing, intermediate results orchestration, cloud computing.

show abstract

Section: A Content-based Video Retrievalmentioning

confidence: 99%

Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues

2020

View full text Add to dashboard Cite

show abstract

“…However, their work is also limited to image processing and does not provide any video processing APIs. Lv et al [22] introduced a Spark based solution for near-duplicate video detection while, in Reference [23], the authors proposed a distributed solution for face search and classification. In contrast, Huang et al [24] developed convolutional neural networks based method for recognizing objects in traffic video data.…”

Section: Related Workmentioning

confidence: 99%

SIAT: A Distributed Video Analytics Framework for Intelligent Video Surveillance

Uddin

Alam

et al. 2019

Symmetry

View full text Add to dashboard Cite

In recent years, the amount of intelligent CCTV cameras installed in public places for surveillance has increased enormously and as a result, a large amount of video data is produced every moment. Due to this situation, there is an increasing request for the distributed processing of large-scale video data. In an intelligent video analytics platform, a submitted unstructured video undergoes through several multidisciplinary algorithms with the aim of extracting insights and making them searchable and understandable for both human and machine. Video analytics have applications ranging from surveillance to video content management. In this context, various industrial and scholarly solutions exist. However, most of the existing solutions rely on a traditional client/server framework to perform face and object recognition while lacking the support for more complex application scenarios. Furthermore, these frameworks are rarely handled in a scalable manner using distributed computing. Besides, existing works do not provide any support for low-level distributed video processing APIs (Application Programming Interfaces). They also failed to address a complete service-oriented ecosystem to meet the growing demands of consumers, researchers and developers. In order to overcome these issues, in this paper, we propose a distributed video analytics framework for intelligent video surveillance known as SIAT. The proposed framework is able to process both the real-time video streams and batch video analytics. Each real-time stream also corresponds to batch processing data. Hence, this work correlates with the symmetry concept. Furthermore, we introduce a distributed video processing library on top of Spark. SIAT exploits state-of-the-art distributed computing technologies with the aim to ensure scalability, effectiveness and fault-tolerance. Lastly, we implant and evaluate our proposed framework with the goal to authenticate our claims.

show abstract

“…Recently, in the literature, several works adopted the Apache Spark framework to efficiently process large-scale video data. For instance, Lv et al [13] extracted the local and global features with Spark for near-duplicate video detection. In [14] and [15], the authors performed dynamic feature extraction on top of Spark for action recognition.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Facial Expression Understanding Using Deep Spatiotemporal LDSP On Spark

2021

View full text Add to dashboard Cite

Facial expressions are the most common medium for expressing human emotions. Due to the wide range of real-world applications, facial expression understanding has received extensive attention from researchers. One of the most vital issues of facial expression recognition is the extraction and modeling of the temporal dynamics of facial emotions from videos. Additionally, the rapid growth of video data from various multimedia sources is becoming a serious concern. Therefore, to address these issues, in this paper, we introduce a novel approach on top of Spark for facial expression understanding from videos. First, we propose a new dynamic feature descriptor, namely, the local directional structural pattern from three orthogonal planes (LDSP-TOP), which analyzes the structural aspects of the local dynamic texture. Second, we design a 1-D convolutional neural network (CNN) to capture additional discriminative features. Third, a long short-term memory (LSTM) autoencoder is employed to learn the spatiotemporal features. Finally, an extensive experimental investigation is carried out to demonstrate the performance and scalability of the proposed framework.

show abstract

Efficient large scale near-duplicate video detection base on spark

Cited by 9 publications

References 18 publications

Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues

Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues

SIAT: A Distributed Video Analytics Framework for Intelligent Video Surveillance

Dynamic Facial Expression Understanding Using Deep Spatiotemporal LDSP On Spark

Contact Info

Product

Resources

About