A Framework for Lecture Video Segmentation from Extracted Speech Content

Chand, Dipesh; Ogul, Hasan

doi:10.1109/sami50585.2021.9378632

Cited by 4 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Through the utilization of neural networks and boosted margin maximization, this method exhibits promise in accurately identifying and organizing crucial segments within lecture videos, thereby enhancing content retrieval and accessibility. [22] focuses on the development of a segmentation method for lecture videos based on speech patterns, including pitch, volume, pause rates, and the initial time of each audio chunk, as well as content cues. Leveraging speech recognition techniques, the proposed approach aims to accurately identify and segment lecture videos into meaningful segments.…”

Section: Video Segmentation Based On Audio/image/text Algorithmsmentioning

confidence: 99%

Semantic Segmentation of Educational Videos for Micro Learning Objects in Adaptive E-Learning

Halawa,

Gamalel-Din,

Nasr

2023

Journal of Al-Azhar University Engineering Sector

View full text Add to dashboard Cite

E-Learning is gaining prominence, especially in lifelong learning, primarily through lecture videos. However, these videos often encompass multiple topics or serve various instructional roles within a single subject. In adaptive e-Learning, the smaller and granular the units, the more versatile presentations and personalized lectures are composed. Such units are known as Micro Learning Objects (MLOs). Consequently, the necessity emerges to segment these lecture videos into multiple MLOs, each fulfilling a distinct instructional role in a lecture. This article presents an automatic model leveraging advanced language models to segment lecture videos semantically into Micro Learning Objects (MLOs). Additionally, a new wellsegmented dataset of educational videos (YT-EV) was introduced, in which the video is segmented according to a pre-defined timestamped agenda. The model is trained on general text datasets to understand LO segments and subsequently fine-tuned using transfer learning on video datasets to achieve better segmentation results. The experimental results showed an F1-score of value 0.657, which is considered promising and emphasizes the significance of text transcript-based video segmentation for enhancing adaptive e-Learning.

show abstract

Section: Video Segmentation Based On Audio/image/text Algorithmsmentioning

confidence: 99%

Semantic Segmentation of Educational Videos for Micro Learning Objects in Adaptive E-Learning

Halawa,

Gamalel-Din,

Nasr

2023

Journal of Al-Azhar University Engineering Sector

View full text Add to dashboard Cite

show abstract

“…The video is usually divided into small segments (e.g by using voice activity detection [38]) or temporal windows. Then, existing transcripts or automatic speech recognition (ASR) are used to create an embedding for each segment based on methods like bags of words [39], word2vec [38]- [40], and TF-IDF [41]. Some methods also consider additional acoustic features [38], [40].…”

Section: ) Audio-based Analysismentioning

confidence: 99%

“…Then, existing transcripts or automatic speech recognition (ASR) are used to create an embedding for each segment based on methods like bags of words [39], word2vec [38]- [40], and TF-IDF [41]. Some methods also consider additional acoustic features [38], [40]. The embeddings of contiguous audio segments are then compared and if the distance is too large then both segments are assumed to belong to different topics and therefore the video is split.…”

Section: ) Audio-based Analysismentioning

confidence: 99%

“…A few methods rely on local search (e.g. using genetic algorithms) to identify the temporal splits that optimize one or multiple objective functions [38], [40]. Other usage of audio transcripts of lecture videos include summaries based on word clouds with support for advanced key-word based navigation [42], abstractive textual lecture video summaries [43], generation of mappings between lecture video segments and text book sub-sections [41] and semantic annotation of lecture videos for recommender systems [44].…”

Section: ) Audio-based Analysismentioning

confidence: 99%

See 1 more Smart Citation

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

Davila

Setlur

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Recording and sharing of educational or lecture videos has increased in recent years. Within these recordings, we find a large number of math-oriented lectures and tutorials which attract students of all levels. Many of the topics covered by these recordings are better explained using handwritten content on whiteboards or chalkboards. Hence, we find large numbers of lecture videos that feature the instructor writing on a surface. In this work, we propose a novel method for extraction and summarization of the handwritten content found in such videos. Our method is based on a fully convolutional network, FCN-LectureNet, which can extract the handwritten content from the video as binary images. These are further analyzed to identify the unique and stable units of content to produce a spatial-temporal index of handwritten content. A signal which approximates content deletion events is then built using information from the spatial-temporal index. The peaks of this signal are used to create temporal segments of the lecture based on the notion that sub-topics change when large portions of content are deleted. Finally, we use these segments to create an extractive summary of the handwritten content based on key-frames. This will facilitate content-based search and retrieval of these lecture videos. In this work, we also extend the AccessMath dataset to create a novel dataset for benchmarking of lecture video summarization called LectureMath. Our experiments on both datasets show that our novel method can outperform existing methods especially on the larger and more challenging dataset. Our code and data will be made publicly available.INDEX TERMS fully convolutional networks, handwritten text detection, image binarization, lecture videos, video summarization.

show abstract