Context-Aware Computer Aided Inbetweening

Abstract

This paper presents a context-aware computer aided inbetweening (CACAI) technique that interpolates planar strokes to generate inbetween frames from a given set of key frames. The inbetweening is context-aware in the sense that not only the stroke's shape but also the context (i.e., the neighborhood of a stroke) in which a stroke appears are taken into account for the stroke correspondence and interpolation. Given a pair of successive key frames, the CACAI automatically constructs the stroke correspondence between them by exploiting the context coherence between the corresponding strokes. Meanwhile, the construction algorithm is able to incorporate the user's interaction with ease and allows the user more effective control over the correspondence process than existing stroke matching techniques. With a one-to-one stroke correspondence, the CACAI interpolates the shape and context between the corresponding strokes for the generation of intermediate frames. In the interpolation sequence, both the shape of individual strokes and the spatial layout between them are well retained such that the feature characteristics and visual appearance of the objects in the key frames can be fully preserved even when complex motions are involved in these objects. We have developed a prototype system to demonstrate the ease of use and effectiveness of the CACAI.

Context‐Aware Mixed Reality: A Learning‐Based Framework for Semantic‐Level Interaction

“…Prior approaches have studied context and semantic understandings in 3D virtual environments, for example semantic inferring in interactive visual data exploration [NEF12]; enhancing software quality for multi‐modal virtual reality (VR) systems [FWL17]; visual text analytics [EFN12]; and interactive urban visualization [DZMQ16]. Context awareness is also introduced in computer‐aided graphic design, such as inbetweening of animation[Yan18]; 3D particle clouds selection[YEII16]; and illustrative volume rendering [RBG07]. Virtual object classifications are proposed in VR applications using semantic associations to describe virtual object behaviours [CTB*12].…”

FTP‐SC: Fuzzy Topology Preserving Stroke Correspondence

“…Most of the existing computer aided inbetweening techniques are mainly concerned with the stroke interpolation problem, and several practical solutions to the problem have been proposed, such as the logarithmic spiral trajectory in [WNS*10] and the context‐aware interpolation of [Yan18]. For the stroke correspondence construction problem, topology ambiguities, i.e., a feature may be drawn with different number of strokes between successive key frames or a part in some keyframe may disappear in the next keyframe due to occlusion, make it infeasible to construct a full stroke correspondence between the successive key frames by using the fully automatic local or global optimization strategies [NSC*11, BZBM*16].…”

“…However, such algorithm often fails in practice, since it is akin to a difficult artificial intelligence problem to infer the occluded parts from the key drawings or to associate the stroke(s) that correspond to a common feature between the successive key frames. Experiments show that a feasible solution to the stroke correspondence construction problem would be to keep the artist into the loop [WNS*10, Yan18, Lim18], e.g., drawing occluded lines or merging/spliting the strokes, ultimately making possible the one‐to‐one correspondence between the strokes on two successive key frames. Furthermore, to make this solution practical and efficient, an effective matching technique for one‐to‐one stroke correspondence is necessary, such that the loop could be automated as much as possible.…”

“…Furthermore, BetweenIT is also sensitive to the spatial variation between the strokes, since such variation may destroy the compatibility in the strokes’ intersection and connectivity; thus, it is more suitable for tight pairs of key frames, i.e., the involved motions are small or simple. Instead of the connectivity‐compatibility constraints, the CACAI [Yan18] uses geometric shape of the strokes and their neighborhood information to establish the one‐to‐one stroke correspondence between the key frames. Obviously, this algorithm tends to generate mismatches when large geometric variations occur between the corresponding strokes, as shown in Figure .…”

“…A simple case is used to show the problems of the one‐to‐one stroke correspondence using the approaches proposed in [WNS*10] and [Yan18]. The correspondence result between the left and right key frames is automatically constructed using [Yan18], where the same color indicates a correspondence. On the other hand, the gap and imprecise intersection in the key frames make [WNS*10] unsuitable for this case.…”