Recognizing Textures with Mobile Cameras for Pedestrian Safety Applications

Abstract

As smartphone rooted distractions become commonplace, the lack of compelling safety measures has led to a rise in the number of injuries to distracted walkers. Various solutions address this problem by sensing a pedestrian's walking environment. Existing camera-based approaches have been largely limited to obstacle detection and other forms of object detection. Instead, we present TerraFirma, an approach that performs material recognition on the pedestrian's walking surface. We explore, first, how well commercial off-the-shelf smartphone cameras can learn texture to distinguish among paving materials in uncontrolled outdoor urban settings. Second, we aim at identifying when a distracted user is about to enter the street, which can be used to support safety functions such as warning the user to be cautious. To this end, we gather a unique dataset of street/sidewalk imagery from a pedestrian's perspective, that spans major cities like New York, Paris, and London. We demonstrate that modern phone cameras can be enabled to distinguish materials of walking surfaces in urban areas with more than 90% accuracy, and accurately identify when pedestrians transition from sidewalk to street.

Multi-city street-sidewalk imagery from pedestrian mobile cameras

“…TerraFirma has been used for designing and validating a pedestrian safety application [4]. The high-level pipeline for the system is shown in Figure 3.…”

Wide-View Sidewalk Dataset Based Pedestrian Safety Application

“…According to the parameters the lens and camera used, the number of pixels occupied by the object in the captured image can be calculated. For example, the pixel number P of an object in the captured image for a common monocular camera can be defined by (6).…”

“…In (6), the H × W is the image resolution. The L object × W object is the area of the object, and the L FOV × W FOV is the area of camera field.…”

“…For a determined network architecture or detection algorithm, there is a minimum input requirement. That is the minimum value of P. Then the maximum value of L FOV can be obtained by (6), which is the most suitable L FOV needed. Finally, the θ FOV can be calculated by (2).…”

“…For example, Inspector can only determine whether a pedestrian enters a street by identifying special boundaries between sidewalks and streets [5], and such boundaries are unusual. TerraFirma can use smartphones to do texture detection on sidewalks, however, TerraFirma based applications can only detect the transition from the sidewalk to the street [6] of pedestrians.…”