Contextual Detection of Pedestrians and Vehicles in Orthophotography by Fusion of Deep Learning Algorithms

Abstract

In the context of smart cities, monitoring pedestrian and vehicle movements is essential to recognize abnormal events and prevent accidents. The proposed method in this work focuses on analyzing video streams captured from a vertically installed camera, and performing contextual road user detection. The final detection is based on the fusion of the outputs of three different convolutional neural networks. We are simultaneously interested in detecting road users, their motion, and their location respecting the static environment. We use YOLOv4 for object detection, FC-HarDNet for background semantic segmentation, and FlowNet 2.0 for motion detection. FC-HarDNet and YOLOv4 were retrained with our orthophotographs dataset. The last step involves a data fusion module. The presented results show that the method allows one to detect road users, identify the surfaces on which they move, quantify their apparent velocity, and estimate their actual velocity.

Leveraging Marine Predators Algorithm with Deep Learning Object Detection for Accurate and Efficient Detection of Pedestrians

“…Certain limitations of contemporary schemes can be concluded based on several improvement approaches modelled: 1) a probation system for robust identity matching, 2) including high-level pedestrian features to boost pedestrian tracking, and 3) a similarity measure compiling multiple cues for identity matching. Ansarnia et al [13] concentrate on examining video streams captured in vertically fixed camera and effectuating contextual road user recognition. Kolluri and Das [14] developed an IPDC-HMODL method (intellectual multimodal pedestrian detection and classification utilizing hybrid meta-heuristic optimizer with DL).…”

Convolutional Neural Networks Adapted for Regression Tasks: Predicting the Orientation of Straight Arrows on Marked Road Pavement Using Deep Learning and Rectified Orthophotography

“…For example, Soilan et al [9] use ortho-imagery to identify arrow signs that were manually segmented as ground truth for an application system using mobile laser scanning (MLS). Ansarnia et al [10] use orthophotography from vertically installed cameras for pedestrian and vehicle detection, and their approach involves the use of DL for different tasks including image classification (where the YOLO algorithm [11] was used). Both papers discuss the potential for DL-based approaches to accurately detect the position of elements on the road transport network.…”

Bioinspired Garra Rufa Optimization-Assisted Deep Learning Model for Object Classification on Pedestrian Walkways

“…The DL technique is an advanced domain in the machine learning (ML) field that aims to determine the complex models of modest representations. The DL algorithms commonly depend on artificial neural networks (ANNs) that contain numerous hidden layers with nonlinear processing components [8]. The term "deep" corresponds to the presence of several hidden layers that are employed to modify the representation of the data.…”