Tohme

Abstract

Building on recent prior work that combines Google Street View (GSV) and crowdsourcing to remotely collect information on physical world accessibility, we present the first "smart" system, Tohme, that combines machine learning, computer vision (CV), and custom crowd interfaces to find curb ramps remotely in GSV scenes. Tohme consists of two workflows, a human labeling pipeline and a CV pipeline with human verification, which are scheduled dynamically based on predicted performance. Using 1,086 GSV scenes (street intersections) from four North American cities and data from 403 crowd workers, we show that Tohme performs similarly in detecting curb ramps compared to a manual labeling approach alone (Fmeasure: 84% vs. 86% baseline) but at a 13% reduction in time cost. Our work contributes the first CV-based curb ramp detection system, a custom machine-learning based workflow controller, a validation of GSV as a viable curb ramp data source, and a detailed examination of why curb ramp detection is a hard problem along with steps forward.

User Contribution Patterns and Completeness Evaluation of Mapillary, a Crowdsourced Street Level Photo Service

“…However, on‐site assessments should be conducted when characterizing micro‐environments for special activities, such as cycling (Vanwolleghem et al ). Several studies applied computer vision to Street View imagery, for example, to identify sidewalk curbs (Hara et al ), or to determine the geographic location of photographs based on reference imagery extracted from Street View (Zamir and Shah ). Yin et al () provide technical details on downloading and assembling Street View images and their use for pedestrian detection using machine vision and learning technology.…”

“…Geolocated street‐level photographs are an important data source for a variety of transportation analysis tasks, including the identification of road features, such as traffic signs (Gonzalez et al ), the evaluation of wheelchair accessibility of sidewalks (Hara et al ), or the deployment of navigational tools for visually impaired citizens (Guy and Truong ). Mapillary is the first platform to provide detailed street photos based on crowdsourcing, adding to the list of Web 2.0 applications that administer and facilitate the collection of Volunteered Geographic Information (VGI) (Goodchild ).…”

Crowd-Mapping Urban Objects from Street-Level Imagery

“…In an early feasibility study [9] manually created a database consisting of 100 images from Google Street View. Researches from the same group afterwards studied the combination of crowdsourcing and Google Street View to identify street-level accessibility problems [10] and developed an online curb ramp data collection system Tohme [11]. The system enables workers to remotely label curb ramps through Google Street View with the assistance of computer vision and machine learning techniques.…”

“…Recent studies [8][9][10][11]31] have shown that the combination of street-level imagery with micro-task crowdsourcing can successfully engage a larger number of contributors -possibly unfamiliar with the targeted urban area -through object annotation tasks. While demonstrating the feasibility of a micro-task crowdsourcing approach, these works do not fully address the important quality and scalability issues that generally characterize large scale object annotation campaigns.…”

“…The proposed method is novel with regard to the combination of an object annotation Web interface for micro-task crowdsourcing with object density and geo-location inference methods. Task scheduling and assignment strategies provide increased performance in terms of cost/quality11 https://www.gps.gov/systems/gps/performance/accuracy/…”