Expanding the point

Video is becoming a prevalent medium for e-learning. Lecture videos contain text information in both the visual and aural channels: the presentation slides and lecturer's speech. This paper examines the relative utility of automatically recovered text from these sources for lecture video retrieval. To extract the visual information, we apply video content analysis to detect slides and optical character recognition to obtain their text. Automatic speech recognition is used similarly to extract spoken text from the recorded audio. We perform controlled experiments with manually created ground truth for both the slide and spoken text from more than 60 hours of lecture video. We compare the automatically extracted slide and spoken text in terms of accuracy relative to ground truth, overlap with one another, and utility for video retrieval. Results reveal that automatically recovered slide text and spoken text contain different content with varying error profiles. Experiments demonstrate that automatically extracted slide text enables higher precision video retrieval than automatically recovered spoken text.

show abstract

Client-side backprojection of presentation slides into educational video

Kharitonova

Tung

Danehy

et al. 2012

Proceedings of the 20th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

A significant part of many videos of lectures is presentation slides that occupy much of the field of view. Further, for a student studying the lecture, having the slides sharply displayed is especially important, compared with the speaker, background, and audience. However, even if the original capture supports it, the bandwidth required for real time viewing is substantive, especially in the context of mobile devices. Here we propose reconstructing the video on the client side by backprojecting high resolution slide images into the video stream with the slide area blacked out. The high resolution slide deck can be sent once, and inserted into the video on the client side based on the transformation (a homography) computed in advance. We further introduce the idea that needed homography transformations can be approximated using affine transformations, which allows it to be done using built-in capabilities of HTML 5. We find that it is possible to significantly reduce bandwidth by compressing the modified video, while improving the slide area quality, but leaving the non-slide area roughly the same.

show abstract

Expanding the point

Cited by 6 publications

References 1 publication

MobiSLIC: Content-Aware Energy Saving for Educational Videos on Mobile Devices

MobiSLIC: Content-Aware Energy Saving for Educational Videos on Mobile Devices

Presentation video retrieval using automatically recovered slide and spoken text

Client-side backprojection of presentation slides into educational video

Contact Info

Product

Resources

About