Smash

Monszpart, Áron; Thuerey, Nils; Mitra, Niloy J.

doi:10.1145/2980179.2982421

Cited by 10 publications

(1 citation statement)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, motion parameters, including positions and poses, for fast moving objects were estimated in a study [36]. A method was proposed to infer the collision parameters of objects and reconstruct the 3D scene, including collisions, after watching a video in which the collision between two objects was obscured by a curtain [37]. These methods can be applied to real scenes without a large number of data, because they do not use machine-learning techniques.…”

Section: Learning For Real Scenesmentioning

confidence: 99%

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Dohae

Lee

2020

Sensors

View full text Add to dashboard Cite

We propose a deep neural network model that recognizes the position and velocity of a fast-moving object in a video sequence and predicts the object’s future motion. When filming a fast-moving subject using a regular camera rather than a super-high-speed camera, there is often severe motion blur, making it difficult to recognize the exact location and speed of the object in the video. Additionally, because the fast moving object usually moves rapidly out of the camera’s field of view, the number of captured frames used as input for future-motion predictions should be minimized. Our model can capture a short video sequence of two frames with a high-speed moving object as input, use motion blur as additional information to recognize the position and velocity of the object, and predict the video frame containing the future motion of the object. Experiments show that our model has significantly better performance than existing future-frame prediction models in determining the future position and velocity of an object in two physical scenarios where a fast-moving two-dimensional object appears.

show abstract

Section: Learning For Real Scenesmentioning

confidence: 99%

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Dohae

Lee

2020

Sensors

View full text Add to dashboard Cite

show abstract

Unsupervised Intuitive Physics from Visual Observations

Ehrhardt

Monszpart

Mitra

et al. 2019

Computer Vision – ACCV 2018

Self Cite

View full text Add to dashboard Cite

We are interested in learning models of intuitive physics similar to the ones that animals use for navigation, manipulation and planning. In addition to learning general physical principles, however, we are also interested in learning "on the fly", from a few experiences, physical properties specific to new environments. We do all this in an unsupervised manner, using a meta-learning formulation where the goal is to predict videos containing demonstrations of physical phenomena, such as objects moving and colliding with a complex background. We introduce the idea of summarizing past experiences in a very compact manner, in our case using dynamic images, and show that this can be used to solve the problem well and efficiently. Empirically, we show via extensive experiments and ablation studies, that our model learns to perform physical predictions that generalize well in time and space, as well as to a variable number of interacting physical objects.Preprint. Under review.

show abstract

Calipso: physics-based image and video editing through CAD model proxies

Haouchine¹,

Roy²,

Courtecuisse

et al. 2018

Vis Comput

View full text Add to dashboard Cite

Figure 1: Our method allows a user to interact in a 3D manner with objects in images and videos by producing rigid and deformable transforms, topological changes and physical attribute editing (e.g., mass, stiffness, gravity). Using an intuitive mesh refinement and 3D/2D alignment approach and estimating dynamics from image flow, our system produces a final composition without cumbersome user input, while preserving visual consistency. Left: editing Dali's painting by interactively pulling the watch or decreasing its stiffness leading to melting it. Right: bending a car bridge by pulling it from a side or by progressively increasing its mass. AbstractWe present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.

show abstract

Smash

Cited by 10 publications

References 50 publications

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Future-Frame Prediction for Fast-Moving Objects with Motion Blur

Unsupervised Intuitive Physics from Visual Observations

Calipso: physics-based image and video editing through CAD model proxies

Contact Info

Product

Resources

About