Reliability of streetscape audits comparing on‐street and online observations: MAPS-Global in 5 countries

Abstract

Background Microscale environmental features are usually evaluated using direct on-street observations. This study assessed inter-rater reliability of the Microscale Audit of Pedestrian Streetscapes, Global version (MAPS-Global), in an international context, comparing on-street with more efficient online observation methods in five countries with varying levels of walkability. Methods Data were collected along likely walking routes of study participants, from residential starting points toward commercial clusters in Melbourne (Australia), Ghent (Belgium), Curitiba (Brazil), Hong Kong (China), and Valencia (Spain). In-person on the street and online using Google Street View audits were carried out by two independent trained raters in each city. The final sample included 349 routes, 1228 street segments, 799 crossings, and 16 cul-de-sacs. Inter-rater reliability analyses were performed using Kappa statistics or Intraclass Correlation Coefficients (ICC). Results Overall mean assessment times were the same for on-street and online evaluations (22 ± 12 min). Only a few subscales had Kappa or ICC values < 0.70, with aesthetic and social environment variables having the lowest overall reliability values, though still in the “good to excellent” category. Overall scores for each section (route, segment, crossing) showed good to excellent reliability (ICCs: 0.813, 0.929 and 0.885, respectively), and the MAPS-Global grand score had excellent reliability (ICC: 0.861) between the two methods. Conclusions MAPS-Global is a feasible and reliable instrument that can be used both on-street and online to analyze microscale environmental characteristics in diverse international urban settings.

Análise da Microescala da Caminhabilidade

“…Ainda diferentes organizações referem-se também para as ruas-sem-saídas. Devido a pouca presença das ruas-sem-saída e seu uso como ambiente de recreação no Brasil (Dos Santos, Hino e Höfelmann, 2019), ao ponto da secção ser desconsiderada da auditoria em outros estudos (Queralt et al, 2021) devido à quantidade amostral baixa, assim como rara concordância nos coeficientes ICC e Kappa (Cain et al, 2018), no estudo empírico realizado, as ruas-sem-saída (Figura 13), apresentaram escores com baixa pontuação. Itens vinculados ao uso da bicicleta, incluídos nas subseções de Paisagem Urbana, Segmentos e Cruzamentos, foram auditadas e os resultados apontam ausência de infraestrutura de suporte ao modal, como ciclovias nas vias e cruzamentos, bicicletários, sinalização exclusiva e bike-box.…”

“…Estudos sobre a relação entre o ambiente construído e a mobilidade ativa na população de países com economias em desenvolvimento parecem ser limitados (Arellana et al, 2019;Elshahat, O'Rorke e Adlakha, 2020), principalmente nos bairros de baixa renda das metrópoles brasileiras (Dos Santos, Hino e Höfelmann, 2019), onde indivíduos de baixo nível socioeconômico estão sujeitos ao deslocamento utilitário por restrição de escolhas modais (Adkins et al, 2017). A própria aplicação do MAPS-Global em países com este perfil apresenta lacunas para moderar a confiabilidade e a validade das auditorias virtuais (Queralt et al, 2021), além da limitação de aplicar a ferramenta em bairros inteiros, de maneira a não captar as características do ambiente construído em sua totalidade (Thornton et al, 2016).…”

Inter-rater reliability of streetscape audits using online observations: Microscale Audit of Pedestrian Streetscapes (MAPS) global in Japan

“…( Cain et al, 2018 ) In a study of five cities worldwide (Melbourne, Ghent, Curitiba, Hong Kong, and Valencia) using MAPS-Global, the inter-rater reliability of on-street, live and online, and local and remote online observations was tested and found to have moderate to excellent reliability for all data collection modes. ( Cain et al, 2018 , Fox et al, 2021 , Queralt et al, 2021 , Vanwolleghem et al, 2016 ) Queralt et al conducted a study using MAPS-Global in the above five cities and confirmed the high reliability of almost all the items for both on-street and online observations. However, aesthetic and social environment variables had the lowest overall reliability values, although they were still in the “good to excellent” category.…”

“…However, aesthetic and social environment variables had the lowest overall reliability values, although they were still in the “good to excellent” category. ( Queralt et al, 2021 ) Fox et al reported a significant association between measurements of local on-street observations conducted in the same five cities and remote online observations conducted in the United States. ( Fox et al, 2021 ) Thus, online observations using MAPS-Global have been confirmed to be highly reliable.…”

Correspondence Between Feelings Towards Neighbors and Appearance of Neighborhood: Analysis by Combining a Mail Survey and Google Street View

“…In terms of research practice, several studies examined the validity of virtual auditing objects on GSV, which is sometimes termed as virtual systematic social observation. Queralt et al (2021) compared virtual audit on GSV with physical observation in cities of five different countries. They showed that levels of concordance for streetscape elements seemed to be high enough between those observations, though physical disorders (e.g., trash) and aesthetics (e.g., trees) tended to be at relatively lower levels of concordance, compared with buildings and roads.…”