FaceLift: a transparent deep learning framework to beautify urban scenes

Abstract

In the area of computer vision, deep learning techniques have recently been used to predict whether urban scenes are likely to be considered beautiful: it turns out that these techniques are able to make accurate predictions. Yet they fall short when it comes to generating actionable insights for urban design. To support urban interventions, one needs to go beyond predicting beauty, and tackle the challenge of recreating beauty. Unfortunately, deep learning techniques have not been designed with that challenge in mind. Given their "black-box nature", these models cannot be directly used to explain why a particular urban scene is deemed to be beautiful. To partly fix that, we propose a deep learning framework (which we name FaceLift 1 ) that is able to both beautify existing urban scenes (Google Street views) and explain which urban elements make those transformed scenes beautiful. To quantitatively evaluate our framework, we cannot resort to any existing metric (as the research problem at hand has never been tackled before) and need to formulate new ones. These new metrics should ideally capture the presence (or absence) of elements that make urban spaces great. Upon a review of the urban planning literature, we identify five main metrics: walkability, green spaces, openness, landmarks and visual complexity. We find that, across all the five metrics, the beautified scenes meet the expectations set by the literature on what great spaces tend to be made of. This result is further confirmed by a 20-participant expert survey in which FaceLift have been found to be effective in promoting citizen participation. All this suggests that, in the future, as our framework's components are further researched and become better and more sophisticated, it is not hard to imagine technologies that will be able to accurately and efficiently support architects and planners in the design of the spaces we intuitively love.

Imagine a Walkable City: Physical activity and urban imageability across 19 major cities

“…Figure 3 depicts the street network orientation (min = 1.98, max = 3.57, μ = 3.21, σ = 0.46) of high-entropic cities (e.g., Helsinki, London) and low-entropic ones (e.g., Hamburg, New York). • Walkability and Landmarks: Specific urban elements (e.g., small streets, absence of cars) have been shown to contribute to physical activity [22][23][24]. To capture the presence of these elements, we processed street scenes from the crowd-sourced mapping platform of Mapillary.…”

“…This classifies a scene into 205 outdoor scene types. We arranged these types into two categories based on a previously developed taxonomy [24] grounded on the urban design literature [9,26,27]. The first category is the walkability category, and contains types contributing to physical activity (e.g., pavilion, plaza, boardwalk, alley).…”

The Soundscape Indices (SSID) Protocol: A Method for Urban Soundscape Surveys—Questionnaires with Acoustical and Contextual Information

“…To select the appropriate measurement point, it should be ensured that the following contextual factors representative of the site are present in the spatial recording: openness, greenness, presence of landmarks, dominant use (walking, staying), and social presence (related to the dominant use). These are identified as objective metrics often used in urban and landscape research [36][37][38][39][40], possibly contributing to soundscape assessment [23,41]. This relies on researcher's opinion-driven assessment-it is advised to observe the location for a moment and then choose the point representative of the context and the first-person user experience.…”

Façade Design Support System with Control of Image Generation using GAN

“…Existing methods that use GAN to generate buildings include models that use segment labels as input to generate façade images [7][8][9] [10] and models that generate cityscapes [11][12] [13]. Many of these models generate images appropriate for their inputs.…”