Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding

Abstract

Understanding intrinsic patterns and predicting spatiotemporal characteristics of cities require a comprehensive representation of urban neighborhoods. Existing works relied on either inter- or intra-region connectivities to generate neighborhood representations but failed to fully utilize the informative yet heterogeneous data within neighborhoods. In this work, we propose Urban2Vec, an unsupervised multi-modal framework which incorporates both street view imagery and point-of-interest (POI) data to learn neighborhood embeddings. Specifically, we use a convolutional neural network to extract visual features from street view images while preserving geospatial similarity. Furthermore, we model each POI as a bag-of-words containing its category, rating, and review information. Analog to document embedding in natural language processing, we establish the semantic similarity between neighborhood (“document”) and the words from its surrounding POIs in the vector space. By jointly encoding visual, textual, and geospatial information into the neighborhood representation, Urban2Vec can achieve performances better than baseline models and comparable to fully-supervised methods in downstream prediction tasks. Extensive experiments on three U.S. metropolitan areas also demonstrate the model interpretability, generalization capability, and its value in neighborhood similarity analysis.

Detecting Neighborhood Gentrification at Scale via Street-level Visual Data

“…Street-level visual data have been utilized in a variety of application scenarios in cities, such as indicating regional functions [11], examining environmental associations with chronic health outcomes [12] and measuring populace's well-being [13]. They have also been adopted in spatial-temporal representation learning [14] [15] [16] to produce region embedding for urban neighborhoods. Recent research have shown the possibility to infer socioeconomic attributes such as income, race, education, and voting patterns from street view data [17].…”

Effective Urban Region Representation Learning Using Heterogeneous Urban Graph Attention Network (HUGAT)

“…Many studies explored region embedding for predictions of region specific features, including sociodemographic feature prediction [12,31], crime prediction [28,37], economic growth prediction [11], and land-usage classification [34,37]. There exist studies utilizing mobility data to model the association of regions based on people movement based on pointwise mutual information [34] or skip-gram objectives [35].…”

Geographic Ratemaking With Spatial Embeddings

“…More recent approaches aim to create general geographic embeddings. See Blier-Wong et al (2020), Hui et al (2020), Wang et al (2020) and references therein for alternative model architectures.…”