What dictates income in New York City? SHAP analysis of income estimation based on Socio-economic and Spatial Information Gaussian Processes (SSIG)

Abstract

Income inequality presents a key challenge to urban sustainability across the developed economies. Traditionally, accurate high granularity income data are generally obtained from field surveys. However, due to privacy considerations, field subjects are hesitant to provide accurate personal income data. A Socio-economic & Spatial-Information-GP (SSIG) model is thereby developed to estimate district-based high granularity income for New York City (NYC). As compared to the state-of-the-art Gaussian Processes (GP) income estimation model based entirely on spatial information, SSIG incorporates socio-economic domain-specific knowledge into a GP model. For SSIG to be explainable, SHapley Additive exPlanations (SHAP) analysis is undertaken to evaluate the relative contribution of various key individual socio-economic variables to district-based per-capita and median household income in NYC. Differentiating from traditional income inequality studies based predominantly on linear or log-linear regression model, SSIG presents a novel income-based model architecture, capable of modelling complex non-linear relationships. In parallel, SHAP analysis serves an effective analytical tool for identifying the key attributes to income inequality. Results have shown that SSIG surpasses other state-of-the-art baselines in estimation accuracy, as far as per-capita and median household income estimation at the Tract-level and the ZIP-level in NYC are concerned. SHAP results have indicated that having a bachelor or a postgraduate degree can accurately predict income in NYC, despite that between-district income inequality due to Sex/Race remains prevalent. SHAP has further confirmed that between-district income gap is more associated with Race than Sex. Furthermore, ablation study shows that socio-economic information is more predictive of income at the ZIP-level, relative to the spatial information. This study carries significant implications for policy-making in a developed context. To promote urban economic sustainability in NYC, policymakers can attend to the growing income disparity (income inequality) contributed by Sex and Race, while giving more higher education opportunities to residents in the lower-income districts, as the estimated per-capita income is more sensitive to the proportion of adults ≥25 holding a bachelor’s degree. Finally, interpretative SHAP analysis is useful for investigating the relative contribution of socio-economic inputs to any predicted outputs in future machine-learning-driven socio-economic analyses.

A New Graph-Based Deep Learning Model to Predict Flooding with Validation on a Case Study on the Humber River

“…The Shapley additive explanations (SHAP) (documentation available at https: //shap.readthedocs.io/en/latest/, accessed on 5 April 2023) uses game theory to provide further insight into the ML results, relating input variables and the achieved output value in terms of importance, correlation, and influence of each input variable over the final prediction [53,71]. The SHAP analysis was found to be a reliable tool for authors to fully understand their results in multidisciplinary fields, such as pharmaceuticals [72], material engineering [73], Earth system modeling [74], and social factors driving income [75].…”

“…understand their results in multidisciplinary fields, such as pharmaceuticals [72], material engineering [73], Earth system modeling [74], and social factors driving income [75].…”

Graph-Based Deep Learning Model for Forecasting Chloride Concentration in Urban Streams to Protect Salt-Vulnerable Areas

“…The determination of the influence of each variable provides deeper insight into how the model provides its results, being a viable option to explain the analyzed ML paradigm locally. The employment of SHAP analysis by those with expertise in different knowledge areas, such as pharmaceutical [56], engineering [57], and social sciences [58], renders it a valuable tool for researchers. In Figure 7, we present a flow chart outlining the tasks performed during our study.…”