Controllable Animation of Fluid Elements in Still Images

Abstract

Figure 1. Our approach takes in input image along with the user-provided motion hints (red arrows in (a) and the user-provided mask (white in (b)) indicating the regions of fluid elements to be animated and outputs the sequence of frames of the animated videos.

Simulating Fluids in Real-World Still Images

“…However, the seamless alignment of animated results with source input image can not be well guaranteed and requiring lots of manual efforts. With the development of deep learning, recent works [11,29] propose to automatically predict each frame of motion and appearance via learning from reconstruction loss with real world videos for training. [11] first simplifies the motion estimation part as a single frame motion prediction via motion Eulerian integration and then proposes a deep feature warping technique to narrow down the size of blank areas caused by warping.…”

“…Finally, it adaptively blends forward features and backward features through a learnable parameter Z to generate animated fluid video. Based on [11,29] proposes to regress fluid motion conditioned to user's sparse guidance and generate paired training data through motion speed clustering. They further proposes to use multi-scale representation to capture different fluid speed in different resolution.…”

“…The direction of movements is divided into eight discrete directions, and for each, an absolute value shows fast or slow, and the probability is predicted to simplify the training. There are also some interactive motion prediction systems [29,43,21] designed to generate a more accurate motion with as little user guidance as possible.…”

“…Given an image of a scene containing liquid regions such as streams, waterfalls and oceans, humans can imagine how the liquid in the still image would move. The problem of animating fluid from a single image has gradually become a blooming topic recently with a series of works [6,11,29,9,21]. Their general frameworks can be summarized as two parts: (1) predicting/calculating the optical flow that indicates the movement of the liquid and (2) synthesizing the future frame image based on the motion prediction/calculation.…”

“…They require sparse hints (ie, labelling the velocity direction and relative amplitude) on random pixels of the liquid region as additional input. The feature clustering model [29] or convolutional network [11] is then used to form a dense motion field. As the precision depends on the level of motion details a user can provide, such methods are time-costing in complex fluid scenes.…”