Reliability and One-Year Stability of the PIN3 Neighborhood Environmental Audit in Urban and Rural Neighborhoods

Abstract

Reliable and stable environmental audit instruments are needed to successfully identify the physical and social attributes that may influence physical activity. This study described the reliability and stability of the PIN3 environmental audit instrument in both urban and rural neighborhoods. Four randomly sampled road segments in and around a one-quarter mile buffer of participants' residences from the Pregnancy, Infection, and Nutrition (PIN3) study were rated twice, approximately 2 weeks apart. One year later, 253 of the year 1 sampled roads were re-audited. The instrument included 43 measures that resulted in 73 item scores for calculation of percent overall agreement, kappa statistics, and log-linear models. For same-day reliability, 81% of items had moderate to outstanding kappa statistics (kappas ≥ 0.4). Two-week reliability was slightly lower, with 77% of items having moderate to outstanding agreement using kappa statistics. One-year stability had 68% of items showing moderate to outstanding agreement using kappa statistics. The reliability of the audit measures was largely consistent when comparing urban to rural locations, with only 8% of items exhibiting significant differences (α < 0.05) by urbanicity. The PIN3 instrument is a reliable and stable audit tool for studies assessing neighborhood attributes in urban and rural environments.

Environmental Predictors of a Drug Offender Crime Script: A Systematic Social Observation of Google Street View Images and CCTV Footage

“…The original coding was completed in September 2018 and the retest took place in November 2019. This time lapse of more than 1 year substantially minimizes any possible memory effects and is consistent with time lapses seen in other environmental audits (see Porter et al, 2018). Results of the test–retest indicate an average agreement of 94% across the 20 separate indicators, indicating a high level of intrarater reliability.…”

Measuring Procedural Justice Policy Adherence During Use of Force Events: The Body-Worn Camera as a Performance Monitoring Tool

“…Observations are not independent of the observer and, in Finlay's (2002) discussion of reflexivity as intersubjective reflection, she notes, "Psychodynamic researchers remind us to explore how conversation or text affects us and to reflect on what we bring to it ourselves" (p. 217). Thus while prior SSOs have included a single coder when appropriate (e.g., Porter et al, 2018;Sytsma et al, 2020), we acknowledge that the results presented here owe to the positionality of a single coder.…”

Understanding the Predictors of Street Robbery Hot Spots: A Matched Pairs Analysis and Systematic Social Observation

“…18 The original coding was completed in June 2019 with the test–retest coding completed in February 2020. This substantial lapse in time minimizes the possibility of memory effects influencing the results (Porter et al, 2018). Results of the test–retest analyses indicate there was an average agreement of 95% across the 12 disorder indicators, suggestive of a high level of intrarater reliability and GSV navigability.…”