Real-time annotation of video objects on tablet computers

Abstract

When using a tablet computer, sketching is a natural approach for users to annotate video scenes. However, when these annotations are done in real-time and overlaid in the video, their context can be lost due to changes in the scene being annotated. We propose an approach towards maintaining the annotations' context, by using object tracking to create anchors onto which further annotations can be attached. To this end, the annotator is capable of using different tracking methods, including a Kinect sensor and/or the TLD object tracking algorithm.The challenges involved in designing an interface to support the association of video annotations with tracked objects in real-time are also discussed. In particular, we discuss our alternative approaches to handle moving object selection on live video, which we have called "Hold and Overlay" and "Hold and Speed Up". In addition, the results of a set of preliminary tests are reported.

Sketches vs skeletons

“…Our work builds particularly on the work of Goldman et al [9], who use automated tracking to support annotations that follow objects in a video or trace their paths. Silva et al [17] use tracking to provide dynamic annotations for live video, this throws up a number of interaction design challenges that we do not need to consider in our work on recorded video.…”

An Interactive Device for Exploring Thematically Sorted Artworks

“…For instance Diogo Cabrals doctorate studies including the TKB/Creation-Tool 7 for the realtime annotation of dancers recordings [12] could be coupled with contentbased analysis of such recordings [13] so as to benefit from both sides, human and machine, of semi-automated analysis. Alternative tools for intra-media audiovisual annotation that focus on the user experience are ChronoViz 8 [2] and VCode/VData 9 [6].…”

Laces

“…Catching up: Bezerianos et al [4] discuss the difficulty of interacting with changing content, and techniques to "catch up" when the user did not observe a visual change. Silva et al [30] present the Hold and Speed Up technique so users may apply annotations to the current frame of a live video. For a user to be able to edit while also paying attention to capture, the system must allow these two activities to share focus effectively-neither one can consume the full attention of the user.…”