Automatic Lecture Video Content Summarizationwith Attention-based Recurrent Neural Network

Andra, Muhammad Bagus; Usagawa, Tsuyoshi

doi:10.1109/icaiit.2019.8834514

Cited by 8 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…using genetic algorithms) to identify the temporal splits that optimize one or multiple objective functions [38], [40]. Other usage of audio transcripts of lecture videos include summaries based on word clouds with support for advanced key-word based navigation [42], abstractive textual lecture video summaries [43], generation of mappings between lecture video segments and text book sub-sections [41] and semantic annotation of lecture videos for recommender systems [44]. None of the lecture video summarization methods evaluated in this work use audio signals or transcripts.…”

Section: ) Audio-based Analysismentioning

confidence: 99%

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

Davila

Setlur

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Recording and sharing of educational or lecture videos has increased in recent years. Within these recordings, we find a large number of math-oriented lectures and tutorials which attract students of all levels. Many of the topics covered by these recordings are better explained using handwritten content on whiteboards or chalkboards. Hence, we find large numbers of lecture videos that feature the instructor writing on a surface. In this work, we propose a novel method for extraction and summarization of the handwritten content found in such videos. Our method is based on a fully convolutional network, FCN-LectureNet, which can extract the handwritten content from the video as binary images. These are further analyzed to identify the unique and stable units of content to produce a spatial-temporal index of handwritten content. A signal which approximates content deletion events is then built using information from the spatial-temporal index. The peaks of this signal are used to create temporal segments of the lecture based on the notion that sub-topics change when large portions of content are deleted. Finally, we use these segments to create an extractive summary of the handwritten content based on key-frames. This will facilitate content-based search and retrieval of these lecture videos. In this work, we also extend the AccessMath dataset to create a novel dataset for benchmarking of lecture video summarization called LectureMath. Our experiments on both datasets show that our novel method can outperform existing methods especially on the larger and more challenging dataset. Our code and data will be made publicly available.INDEX TERMS fully convolutional networks, handwritten text detection, image binarization, lecture videos, video summarization.

show abstract

Section: ) Audio-based Analysismentioning

confidence: 99%

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

Davila

Setlur

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Andra and Usagawa [40] have proposed summarizing the transcript of a lecture video by using attention with a recurrent neural network. The suggested method consists of the following steps: First, the preprocessing step.…”

Section: Introductionmentioning

confidence: 99%

Multimodal video abstraction into a static document using deep learning

Abdulsahib

Abdulmunim

2023

IJECE

View full text Add to dashboard Cite

<p><span lang="EN-US">Abstraction is a strategy that gives the essential points of a document in a short period of time. The video abstraction approach proposed in this research is based on multi-modal video data, which comprises both audio and visual data. Segmenting the input video into scenes and obtaining a textual and visual summary for each scene are the major video abstraction procedures to summarize the video events into a static document. To recognize the shot and scene boundary from a video sequence, a hybrid features method was employed, which improves detection shot performance by selecting strong and flexible features. The most informative keyframes from each scene are then incorporated into the visual summary. A hybrid deep learning model was used for abstractive text summarization. The BBC archive provided the testing videos, which comprised BBC Learning English and BBC News. In addition, a news summary dataset was used to train a deep model. The performance of the proposed approaches was assessed using metrics like Rouge for textual summary, which achieved a 40.49% accuracy rate. While precision, recall, and F-score used for visual summary have achieved (94.9%) accuracy, which performed better than the other methods, according to the findings of the experiments.</span></p>

show abstract