Pan, zoom, scan &amp;#x2014; Time-coherent, trained automatic video cropping

Deselaers, Thomas; Dreuw, Philippe; Ney, Hermann

doi:10.1109/cvpr.2008.4587729

Cited by 58 publications

(57 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intelligent zoom exists for conventional cameras using panzoom-tilt transformations [6] and light-fields [2]. Here we demonstrate, for the first time, intelligent LIDAR zoom.…”

Section: A Smart Lidar Zoommentioning

confidence: 97%

Directionally Controlled Time-of-Flight Ranging for Mobile Sensing Platforms

Tasneem

Wang

Xie

et al. 2018

Robotics: Science and Systems XIV

View full text Add to dashboard Cite

Abstract-Scanning time-of-flight (TOF) sensors obtain depth measurements by directing modulated light beams across a scene. We demonstrate that control of the directional scanning patterns can enable novel algorithms and applications. Our analysis occurs entirely in the angular domain and consists of two ideas. First, we show how to exploit the angular support of the light beam to improve reconstruction results. Second, we describe how to control the light beam direction in a way that maximizes a well-known information theoretic measure. Using these two ideas, we demonstrate novel applications such as adaptive TOF sensing, LIDAR zoom, LIDAR edge sensing for gradient-based reconstruction and energy efficient LIDAR scanning. Our contributions can apply equally to sensors using mechanical, optoelectronic or MEMS-based approaches to modulate the light beam, and we show results here on a MEMS mirror-based LIDAR system. In short, we describe new adaptive directionally controlled TOF sensing algorithms which can impact mobile sensing platforms such as robots, wearable devices and IoT nodes. Creating TOF sensors for personal drones, VR/AR glasses, IoT nodes and other miniature platforms would require transcending the energy constraints due to limited battery capacity. Recent work has addressed some aspects of TOF energy efficiency with novel illumination encodings. For example, by synchronizing illumination patterns to match sensor exposures [1], low-power reconstruction can occur for scenes with significant ambient light. Additionally, spatio-temporal encodings have been shown to be efficient for both structured light illumination [30] and TOF illumination as well [29]. I. INTRODUCTIONIn this paper, we demonstrate new efficiencies that are possible with angular control of a TOF sensor. We demonstrate this with a single LIDAR beam reflected off a microelectromechanical (MEMS) mirror. The voltages that control the MEMS actuators allow analog (continuous) TOF sensing angles. As a modulator, MEMS mirrors have well-known advantages of high-speed and fast response to control [32].

show abstract

“…Intelligent zoom exists for conventional cameras using panzoom-tilt transformations [6] and light-fields [2]. Here we demonstrate, for the first time, intelligent LIDAR zoom.…”

Section: A Smart Lidar Zoommentioning

confidence: 97%

Directionally Controlled Time-of-Flight Ranging for Mobile Sensing Platforms

Tasneem

Wang

Xie

et al. 2018

Robotics: Science and Systems XIV

View full text Add to dashboard Cite

show abstract

“…From the original image, on the left, a saliency map is computed (in the middle) from which an area with higher intensity is extracted using some algorithm and its bounding-box will represent the zoom. A technique to determine automatically the "right" viewing area for spatio-temporal images is proposed in Deselaers et al (2008). Images are first analyzed to determine relevant regions by using three strategies: the visual saliency of spatial images, optical flow for movements and the appearance of the image.…”

Section: General Retargeting Methodsmentioning

confidence: 99%

Human Attention Modelization and Data Reduction

Mancaş¹,

Beul²,

Riche³

et al. 2012

Video Compression

View full text Add to dashboard Cite

“…The technique favors the original aspect ratio leading to no distortion, but the selection is performed per shot which may be inadequate if scene content is changing over time. When facing a sparse content, Deselaers et al [13] allows the alteration of the image by enlarging the original aspect ratio to potentially enclose more columns in the pan-scan window; one additional strategy is to zoom out by adding black stripes/pixels when RoI is spatially sparse. Some approaches [10,12] intentionally prefer preserving the aspect ratio without distortion of the original frame, although others [11,14,13] have based their algorithm on introducing local distortion of the frame for a better rendering of original content.…”

Section: Introductionmentioning

confidence: 99%

“…However, most of the time, a subjective comparison is performed between a new algorithm and a baseline algorithm (the seam-carving algorithm is the most used as the baseline). Another approach [13] goes further in the validation by using annotated ground truth. They have created a database of relevant pixels by manually annotating the relevant regions of the considered sequences.…”

Section: Introductionmentioning

confidence: 99%

How to Measure the Relevance of a Retargeting Approach?

Chamaret

Meur

Guillotel

et al. 2012

Trends and Topics in Computer Vision

View full text Add to dashboard Cite

Abstract. Most cell phones today can receive and display video content. Nonetheless, we are still signicantly behind the point where premium made for mobile content is mainstream, largely available, and aordable. Signicant issues must be overcome. The small screen size is one of them. Indeed, the direct transfer of conventional contents (i.e. not specically shot for mobile devices) will provide a video in which the main characters or objects of interest may become indistinguishable from the rest of the scene. Therefore, it is required to retarget the content. Dierent solutions exist, either based on distortion of the image, on removal of redundant areas, or cropping. The most ecient ones are based on dynamic adaptation of the cropping window. They signicantly improve the viewing experience by zooming in the regions of interest. Currently, there is no common agreement on how to compare dierent solutions. A retargeting metric is proposed in order to gauge its quality. Eye-tracking experiments, zooming eect through coverage ratio and temporal consistency are introduced and discussed.

show abstract

Pan, zoom, scan — Time-coherent, trained automatic video cropping

Cited by 58 publications

References 12 publications

Directionally Controlled Time-of-Flight Ranging for Mobile Sensing Platforms

Directionally Controlled Time-of-Flight Ranging for Mobile Sensing Platforms

Human Attention Modelization and Data Reduction

How to Measure the Relevance of a Retargeting Approach?

Contact Info

Product

Resources

About