Video Index Point Detection and Extraction Framework Using Custom YoloV4 Darknet Object Detection Model

Abstract

The trend of learning from videos instead of documents has increased. There could be hundreds and thousands of videos on a single topic, with varying degrees of context, content, and depth of the topic. The literature claims that learners are nowadays less interested in viewing a complete video but prefers the topic of their interests. This develops the need for indexing of video lectures. Manual annotation or topic-wise indexing is not new in the case of videos. However, manual indexing is time-consuming due to the length of a typical video lecture and intricate storylines. Automatic indexing and annotation is, therefore, a better and efficient solution. This research aims to identify the need for automatic video indexing for better information retrieval and ease users navigating topics inside a video. The automatically identified topics are referred to as "Index Points." 137-layer YoloV4 Darknet Neural Network creates a custom object detector model. The model is trained on approximately 6000 video frames and then tested on a suite of 50 videos of around 20 hours of run time. Shot Boundary detection is performed using Structural Similarity fused with a Binary Search Pattern algorithm which outperformed the state-of-the-art SSIM technique by reducing the processing time to approximately 21% and providing around 96% accuracy. Generation of accurate index points in terms of true positives and false negatives is detected through precision, recall, and F1 score, which varies between 60-80% for every video. The results show that the proposed algorithm successfully generates a digital index with reasonable accuracy in topic detection.

Image Aesthetic Assessment: A Comparative Study of Hand-Crafted & Deep Learning Models

“…Deep learning uses artificial neural networks to automatically learn complex low and high-level features useful for computer vision tasks [99], [100]. In many cases, deep learning has produced results comparable to human accuracy or even surpassed humans in many areas.…”

Stacked Deep Dense Neural Network Model to Predict Alzheimer’s Dementia Using Audio Transcript Data

“…The first model used is one dimensional (1D) CNN. The fundamental architecture of CNN enables network layers to learn numerous complex characteristics that a simple neural network cannot [35], [36]. Every input is passed through the embedding layer and then through 1D convolution layer followed by max-pooling layer.…”

A Novel Framework for Arabic Dialect Chatbot Using Machine Learning

“…Computational Intelligence and Neuroscience e bot uses AI keyword matching to analyze user input and match it with the most relevant problem/solution. AI keyword matching is a Python package deal [31,32] with Arabic letters intelligently, needlessly to afford subscription fees to third-party platforms for creating an Arabic chatbot. Figures 5 and 6 show the solved cases in terms of evaluating the performance of the proposed chatbot.…”