Learning the Depths of Moving People by Watching Frozen People

Li, Zhengqi; Dekel, Tali; Cole, Forrester; Tucker, Richard; Snavely, Noah; Liu, Ce; Freeman, William T.

doi:10.1109/cvpr.2019.00465

Cited by 234 publications

(187 citation statements)

References 52 publications

Supporting

Mentioning

184

Contrasting

Unclassified

Order By: Relevance

“…Generally, monocular depth estimation includes techniques that use a temporal sequence of images captured from a single camera[9]-[11]. However, in this paper, we focus on methods that use only a single image from a single viewpoint for depth estimation.…”

mentioning

confidence: 99%

Adversarial Patch Attacks on Monocular Depth Estimation Networks

et al. 2020

View full text Add to dashboard Cite

Thanks to the excellent learning capability of deep convolutional neural networks (CNN), monocular depth estimation using CNNs has achieved great success in recent years. However, depth estimation from a monocular image alone is essentially an ill-posed problem, and thus, it seems that this approach would have inherent vulnerabilities. To reveal this limitation, we propose a method of adversarial patch attack on monocular depth estimation. More specifically, we generate artificial patterns (adversarial patches) that can fool the target methods into estimating an incorrect depth for the regions where the patterns are placed. Our method can be implemented in the real world by physically placing the printed patterns in real scenes. We also analyze the behavior of monocular depth estimation under attacks by visualizing the activation levels of the intermediate layers and the regions potentially affected by the adversarial attack.

show abstract

mentioning

confidence: 99%

Adversarial Patch Attacks on Monocular Depth Estimation Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…This does not mean, however, that these approaches are not suitable in general. Recent work that explicitly dealt with monocular depth estimation, for instance, with focus on videos with humans [ 61 ] or for obstacle detection in autonomous cars [ 62 ], showed that monocular depth estimation from neural networks could be a promising technique in the future.…”

Section: Crime Scene Analysis Framework: Processing Pipelinementioning

confidence: 99%

Toward Mass Video Data Analysis: Interactive and Immersive 4D Scene Reconstruction

Kraus

Pollok

Miller

et al. 2020

Sensors

View full text Add to dashboard Cite

The technical progress in the last decades makes photo and video recording devices omnipresent. This change has a significant impact, among others, on police work. It is no longer unusual that a myriad of digital data accumulates after a criminal act, which must be reviewed by criminal investigators to collect evidence or solve the crime. This paper presents the VICTORIA Interactive 4D Scene Reconstruction and Analysis Framework (“ISRA-4D” 1.0), an approach for the visual consolidation of heterogeneous video and image data in a 3D reconstruction of the corresponding environment. First, by reconstructing the environment in which the materials were created, a shared spatial context of all available materials is established. Second, all footage is spatially and temporally registered within this 3D reconstruction. Third, a visualization of the hereby created 4D reconstruction (3D scene + time) is provided, which can be analyzed interactively. Additional information on video and image content is also extracted and displayed and can be analyzed with supporting visualizations. The presented approach facilitates the process of filtering, annotating, analyzing, and getting an overview of large amounts of multimedia material. The framework is evaluated using four case studies which demonstrate its broad applicability. Furthermore, the framework allows the user to immerse themselves in the analysis by entering the scenario in virtual reality. This feature is qualitatively evaluated by means of interviews of criminal investigators and outlines potential benefits such as improved spatial understanding and the initiation of new fields of application.

show abstract

“…YouTube is used as a data source for many different applications. For video analytics, [4] used~2000 videos obtained from YouTube of people doing "The Mannequin Challenge" [5], to train a model to predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving. Reference [6] uses video clips collected from YouTube depicting human actors performing various acrobatic stunts (e.g.…”

Section: Related Workmentioning

confidence: 99%

Automated Techniques for Creating Speech Corpora from Public Data Sources for ML Training

Drabeck¹,

Ramanan²,

Woo³

et al. 2020

IJMLC

View full text Add to dashboard Cite

For machine learning (ML) to work well, there is a need for large amounts of good quality training data. Obtaining such data is often the key bottleneck for the entire ML development process. Using humans to do explicit collection has been the main approach, but this tends to be expensive and time-consuming. Therefore, there is significant interest in creating alternative data collection techniques. We explore these alternative data collection techniques in the context of speech data in this paper. We were initially motivated by the problem of wake word engine training, where we need a large number of utterances for specific wake words. Given that there are already large public repositories of media data (e.g., YouTube, DailyMotion), we were curious as to how feasible it is to find the utterances that we need. Our results are encouraging as we found many different types of words can readily be found and downloaded in the quantity and quality needed to create training corpora for DL training. Usually > 30% of the found words are suitable for corpus creation. Greater than 80% of the top 10,000 ranks words and > 50% of the top 20,000 words we selected easily produced > 5000 found words, which is sufficient to train a high quality Wake Word Engine. Besides general words, we specifically looked for words used in wake word engine construction such as Name/Place/Product Name. Here, again, we find most common names/places/products return more than a sufficient number of words for corpus creation. Only uncommon names and places (like Atticus or Maximus) are difficult to find in sufficient quantities for corpus creation. We demonstrate a wake word engine trained from words we found in YouTube has the equivalent performance to one trained with traditional human collected words. Even though we were focused on wake words, our approach is general. It can be applied to create speech corpus for various purposes.

show abstract

Learning the Depths of Moving People by Watching Frozen People

Abstract: Train Static scene, moving camera MannequinChallenge (MC) Dataset Predicted depth Human Mask Initial depth from flow RGB Image MVS Depth (supervison) Inference Our depth predictions Moving people, moving camera

Cited by 234 publications

References 52 publications

Adversarial Patch Attacks on Monocular Depth Estimation Networks

Adversarial Patch Attacks on Monocular Depth Estimation Networks

Toward Mass Video Data Analysis: Interactive and Immersive 4D Scene Reconstruction

Automated Techniques for Creating Speech Corpora from Public Data Sources for ML Training

Contact Info

Product

Resources

About