Measuring Disorder: Observer Bias in Systematic Social Observations at Streets and Neighborhoods

Abstract

Objectives The present study focuses on Systematic Social Observation (SSO) as a method to investigate physical and social disorder at different units of analysis. The study contributes to the aggregation bias debate and to the 'social science of ecological assessment' in two ways: first, by presenting a new model that directly controls for observer bias in ecological constructs and second, by attempting to identify systematic sources of bias in SSO that affect the valid and reliable measurement of physical and social disorder at both street segments and neighborhoods.Methods Data on physical disorder (e.g., litter, cigarette butts) and social disorder (e.g., loitering adults) from 1422 street segments in 253 different neighborhoods in a conurbation of the greater The Hague area (the Netherlands) are analyzed using cross-classified multilevel models.Results Neighborhood differences in disorder are overestimated when scholars fail to recognize the cross-classified data structure of an SSO study that is due to allocation of street segments to observers and neighborhoods. Not correcting for observer bias and Disclaimer This research was funded out of independent resources and has no actual or potential conflict of interest including any financial personal or other relationships with other people of organizations. -016-9333-6 observational conditions underestimates the disorder-crime association at street segment/grid cell level, but overestimates this association at the neighborhood level.Conclusion Findings indicate that SSO can be used for measuring disorder at both street segment level and neighborhood level. Future studies should pay attention to observer bias prior to their data collection by selecting a minimum number of observers, offering extensive training, and collecting information on the urban background of the observers.

Using Systematic Observation and Polar Coordinates Analysis to Assess Gender-Based Differences in Park Use in Barcelona

“…After the first observational session, the observer moved to the next POS of the route and started a new 45-min observational session until complete the assigned route. With this procedure, short observation sessions were ensured reducing the risk of observer fatigue and reactance (Hoeben et al, 2018 ). At the end of the study, every POS was observed at 8 different observational periods, a median of 5 different days, by at least 3 different observers and 3 different weekdays, to diminish some bias.…”

Neighborhood and community factors related to youth conduct disorder among adolescents

“…and if left unchecked, disorganization will continue to increase and lead to further degradation of the neighborhood. Additionally, the loss of community control will start to occur, hinting at "no one cares" (Hoeben, Steenbeek, & Pauwels, 2018;Wilson & Kelling, 1982). Indeed, numerous empirical studies confirm a negative relationship between neighborhood disorder health problems including gonorrhea, substance use, and higher levels of anxiety and stress (Cohen et al, 2000;Gary, Stark, & LaVeist, 2007;Jang & Johnson, 2006).…”

Using Multitrait-Multimethod (MTMM) Techniques to Examine the Convergent and Discriminant Validity of Social Disorder

“…Compared to survey methods, SSO collects observational assessments of public spaces through video recording (later transcribed by researchers) or field researchers’ direct observations following systematic procedures (Sampson and Raudenbush 1999). In the case of disorder, SSO refers to trained observers systematically tally all signs of disorder present on a street or face block during the observation periods (Hoeben, Steenbeek, and Pauwels, 2018). SSO methods have numerous features that make them attractive to researchers studying disorder.…”

“…For one, they can be cheaper than administering resident surveys as researchers can simply collect the data themselves through visits to study sites (Weisburd et al 2004, 2006), coding of videotapes or photographs (Sampson and Raudenbush 1999; Wallace and Schalliol 2015), or even through coding of images using Google Street View or other online sources (Hsu and Miller 2017; Odgers et al 2012). This method also allows researchers to clearly define phenomenon they wish to observe and record directly rather than relying upon residents to act as informants to report community issues through their own filtering or incomplete memories (see Hoeben et al 2018; Skogan 1990; Ward, Link, and Taylor 2017). Moreover, researchers who use SSO data do not have to deal with nonresponse, memory loss/distortion, or socially desirable response issues that can introduce bias to data collected through surveys (Hoeben et al 2018).…”