X-World: Accessibility, Vision, and Autonomy Meet

Zhang, Jimuyang; Zheng, Minglan; Boyd, Matthew; Ohn-Bar, Eshed

doi:10.1109/iccv48922.2021.00962

Cited by 9 publications

(7 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the data were recorded in New York City, many demographics are captured. This is particularly important since some of these groups, such as wheelchair users and people with varying levels and types of disabilities, are absent from large-scale datasets in computer vision and robotics, creating a steep barrier to developing accessibility-aware autonomous systems [ 52 ]. Identifying pedestrians with disabilities, the qualities of their behavior, and ease at traversing the sensed urban environment is an area of possible exploration with datasets such as this one.…”

Section: Discussionmentioning

confidence: 99%

StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

Piadyk

Rulff

Brewer

et al. 2023

Sensors

View full text Add to dashboard Cite

Access to high-quality data is an important barrier in the digital analysis of urban settings, including applications within computer vision and urban design. Diverse forms of data collected from sensors in areas of high activity in the urban environment, particularly at street intersections, are valuable resources for researchers interpreting the dynamics between vehicles, pedestrians, and the built environment. In this paper, we present a high-resolution audio, video, and LiDAR dataset of three urban intersections in Brooklyn, New York, totaling almost 8 unique hours. The data were collected with custom Reconfigurable Environmental Intelligence Platform (REIP) sensors that were designed with the ability to accurately synchronize multiple video and audio inputs. The resulting data are novel in that they are inclusively multimodal, multi-angular, high-resolution, and synchronized. We demonstrate four ways the data could be utilized — (1) to discover and locate occluded objects using multiple sensors and modalities, (2) to associate audio events with their respective visual representations using both video and audio modes, (3) to track the amount of each type of object in a scene over time, and (4) to measure pedestrian speed using multiple synchronized camera views. In addition to these use cases, our data are available for other researchers to carry out analyses related to applying machine learning to understanding the urban environment (in which existing datasets may be inadequate), such as pedestrian-vehicle interaction modeling and pedestrian attribute recognition. Such analyses can help inform decisions made in the context of urban sensing and smart cities, including accessibility-aware urban design and Vision Zero initiatives.

show abstract

Section: Discussionmentioning

confidence: 99%

StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

Piadyk

Rulff

Brewer

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…[GRL*19] use a GCNN for mesh generation; Zhang et al . [ZLM*19] regress camera and mesh parameters with an iterative regression module; finally, Zhou et al . [ZHX*20] apply an inverse kinematics network, for the first time in the context of hands.…”

Section: State‐of‐the‐art Methodsmentioning

confidence: 99%

“…All of them train on synthetic or mixed datasets with ground‐truth 3D hand meshes and poses and some fine‐tune on in‐the‐wild images using either 2D annotations only [BBT19] or rendered depth maps [GRL*19]. Moreover, MANO differentiability enables end‐to‐end trainable architectures [BBT19, ZLM*19]. Further characteristics of these methods are: Boukhayma et al .…”

Section: State‐of‐the‐art Methodsmentioning

confidence: 99%

See 1 more Smart Citation

State of the Art in Dense Monocular Non‐Rigid 3D Reconstruction

Tretschk

Navami

Mallikarjun

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

3D reconstruction of deformable (or non‐rigid) scenes from a set of monocular 2D image observations is a long‐standing and actively researched area of computer vision and graphics. It is an ill‐posed inverse problem, since—without additional prior assumptions—it permits infinitely many solutions leading to accurate projection to the input 2D images. Non‐rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set‐ups such as stereo or multi‐view systems. This survey focuses on state‐of‐the‐art methods for dense non‐rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods—that handle arbitrary scenes and make only a few prior assumptions—and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high‐level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.

show abstract

“…While humans can effortlessly transfer general navigation knowledge across settings and platforms [37,48,54,55], current real-world development of navigation agents generally deploys within a fixed pre-assumed setting (e.g., ge-ographical location, use-case) and carefully calibrated sensor configurations. Consequently, each autonomy use-case generally requires its own prohibitive data collection and platform-specific annotation efforts [4,6,19,58,81]. Due to such development bottlenecks, brittle navigation models trained in-house by various developers (e.g., Tesla's Autopilot [24], Waymo's Driver [4], Amazon's Astro [71], Fedex's Roxo [26], etc.)…”

Section: Introductionmentioning

confidence: 99%