Urban Mosaic

Miranda, Fábio; Hosseini, Maryam; Lage, Marcos; Doraiswamy, Harish; Dove, Graham; Silva, Cláudio T.

doi:10.1145/3313831.3376399

Cited by 14 publications

(4 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Again, these data are not video or synchronized and do not include audio. The next dataset is Urban Mosaic [ 8 ], which is a tool for exploring the urban environment through a spatially and temporally dense data set of 7.7 million street-level images of New York City captured over the period of one year. Similarly, these data are image-only and unsynchronized across views.…”

Section: Related Workmentioning

confidence: 99%

StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

Piadyk

Rulff

Brewer

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Access to high-quality data is an important barrier in the digital analysis of urban settings, including applications within computer vision and urban design. Diverse forms of data collected from sensors in areas of high activity in the urban environment, particularly at street intersections, are valuable resources for researchers interpreting the dynamics between vehicles, pedestrians, and the built environment. In this paper, we present a high-resolution audio, video, and LiDAR dataset of three urban intersections in Brooklyn, New York, totaling almost 8 unique hours. The data were collected with custom Reconfigurable Environmental Intelligence Platform (REIP) sensors that were designed with the ability to accurately synchronize multiple video and audio inputs. The resulting data are novel in that they are inclusively multimodal, multi-angular, high-resolution, and synchronized. We demonstrate four ways the data could be utilized — (1) to discover and locate occluded objects using multiple sensors and modalities, (2) to associate audio events with their respective visual representations using both video and audio modes, (3) to track the amount of each type of object in a scene over time, and (4) to measure pedestrian speed using multiple synchronized camera views. In addition to these use cases, our data are available for other researchers to carry out analyses related to applying machine learning to understanding the urban environment (in which existing datasets may be inadequate), such as pedestrian-vehicle interaction modeling and pedestrian attribute recognition. Such analyses can help inform decisions made in the context of urban sensing and smart cities, including accessibility-aware urban design and Vision Zero initiatives.

show abstract

Section: Related Workmentioning

confidence: 99%

StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

Piadyk

Rulff

Brewer

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The research reported in this paper was undertaken in conjunction with audio and machine listening experts from the SONYC project [BSN∗19], and utilizes data generated by the project's sensors. Our collaborators have background in urban science and machine listening [DTZM∗18,CCSB19,MHL∗20,WBS∗21]. In addition, the project communicates their findings to the media [BB20] and works closely with the NYC Dept.…”

Section: Sounds Of New York Citymentioning

confidence: 99%

“…Machine learning has opened a new horizon in data exploration across various fields, with numerous systems making use of the powerful capabilities it provides. For instance, Urban Mosaic [MHL∗20] uses deep learning representations to search for patterns in a large collection of street‐level images. II‐20 [ZWVW20] allows users to generate image classifiers using novel interactions.…”

Section: Related Workmentioning

confidence: 99%

Urban Rhapsody: Large‐scale exploration of urban soundscapes

Rulff

Miranda

Hosseini

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Noise is one of the primary quality‐of‐life issues in urban environments. In addition to annoyance, noise negatively impacts public health and educational performance. While low‐cost sensors can be deployed to monitor ambient noise levels at high temporal resolutions, the amount of data they produce and the complexity of these data pose significant analytical challenges. One way to address these challenges is through machine listening techniques, which are used to extract features in attempts to classify the source of noise and understand temporal patterns of a city's noise situation. However, the overwhelming number of noise sources in the urban environment and the scarcity of labeled data makes it nearly impossible to create classification models with large enough vocabularies that capture the true dynamism of urban soundscapes. In this paper, we first identify a set of requirements in the yet unexplored domain of urban soundscape exploration. To satisfy the requirements and tackle the identified challenges, we propose Urban Rhapsody, a framework that combines state‐of‐the‐art audio representation, machine learning and visual analytics to allow users to interactively create classification models, understand noise patterns of a city, and quickly retrieve and label audio excerpts in order to create a large high‐precision annotated database of urban sound recordings. We demonstrate the tool's utility through case studies performed by domain experts using data generated over the five‐year deployment of a one‐of‐a‐kind sensor network in New York City.

show abstract

“…Various embedding approaches have been developed based on a target application domain and data types, such as numerical values [BCV13], word/text [MSC * 13,LM14,AX19,FYC * 22], graphs/networks [HYL17,GF18,ZYZZ20], media data [AGH * 23], or a combination of those [BCV13, WMWG17]. Learning effective embeddings is crucial for performing downstream tasks in various fields, such as information retrieval [JSR * 19], natural language processing [EAKC * 20], social network analysis [AAM * 21], and urban planning [MHL * 20]. To further enhance the accuracy, explainability, and credibility of either the embedding process or the higher‐level objectives, many studies incorporate a human‐in‐the‐loop process in addition to optimizing the computational components [EHR * 14].…”

Section: Introductionmentioning

confidence: 99%

VA + Embeddings STAR: A State‐of‐the‐Art Report on the Use of Embeddings in Visual Analytics

Huang

Witschard

Kucher

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term “embedding” when describing the computational approach. Within this context, embeddings are usually (relatively) low‐dimensional, distributed representations of various data types (such as texts or graphs), and since they have proven to be extremely useful for a variety of data analysis tasks across various disciplines and fields, they have become widely used. Existing visualization approaches aim to either support exploration and interpretation of the embedding space through visual representation and interaction, or aim to use embeddings as part of the computational pipeline for addressing downstream analytical tasks. To the best of our knowledge, this is the first survey that takes a detailed look at embedding methods through the lens of visual analytics, and the purpose of our survey article is to provide a systematic overview of the state of the art within the emerging field of embedding visualization. We design a categorization scheme for our approach, analyze the current research frontier based on peer‐reviewed publications, and discuss existing trends, challenges, and potential research directions for using embeddings in the context of visual analytics. Furthermore, we provide an interactive survey browser for the collected and categorized survey data, which currently includes 122 entries that appeared between 2007 and 2023.

show abstract

Urban Mosaic

Cited by 14 publications

References 81 publications

StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

Urban Rhapsody: Large‐scale exploration of urban soundscapes

VA + Embeddings STAR: A State‐of‐the‐Art Report on the Use of Embeddings in Visual Analytics

Contact Info

Product

Resources

About