Fall Detection Using Multi-Property Spatiotemporal Autoencoders in Maritime Environments

Abstract

Man overboard is an emergency in which fast and efficient detection of the critical event is the key factor for the recovery of the victim. Its severity urges the utilization of intelligent video surveillance systems that monitor the ship’s perimeter in real time and trigger the relative alarms that initiate the rescue mission. In terms of deep learning analysis, since man overboard incidents occur rarely, they present a severe class imbalance problem, and thus, supervised classification methods are not suitable. To tackle this obstacle, we follow an alternative philosophy and present a novel deep learning framework that formulates man overboard identification as an anomaly detection task. The proposed system, in the absence of training data, utilizes a multi-property spatiotemporal convolutional autoencoder that is trained only on the normal situation. We explore the use of RGB video sequences to extract specific properties of the scene, such as gradient and saliency, and utilize the autoencoders to detect anomalies. To the best of our knowledge, this is the first time that man overboard detection is made in a fully unsupervised manner while jointly learning the spatiotemporal features from RGB video streams. The algorithm achieved 97.30% accuracy and a 96.01% F1-score, surpassing the other state-of-the-art approaches significantly.

Small-Scale and Occluded Pedestrian Detection Using Multi Mapping Feature Extraction Function and Modified Soft-NMS

“…Annotating the visible part of a body [54][55][56][57][58][59] is the other method; this technique consumes resources for annotation. NMS must remove duplicate bounding boxes when detecting pedestrians.…”

Robotic Maintenance of Road Infrastructures: The HERON Project

“…The HERON system consists of: i) autonomous ground robotic vehicle that will be supported by autonomous drones to coordinate maintenance works and the pre-/post-intervention phase [3]; ii) various robotic equipment (see Fig. 2), including sensors and actuators (e.g., tools for cut and fill, surface material placement and compaction, modular components installation, laser scanners for 3D mapping) placed on the main vehicle [4]; iii) sensing interface installed both to the robotic platform and to the RI to allow improved monitoring (situational awareness) of the structural, functional and RI's and markings' conditions [5]; iv) the control software that interconnects the sensing interface with the actuating robotic equipment [6]; v) Augmented Reality (AR) visualization tools that enable the robotic system to see in detail surface defects and markings under survey [7]; vi) Artificial Intelligence/AI-based toolkits that will act as the middleware of a twofold role for: a) optimally coordinating the road maintenance/upgrading workflows and b) intelligent processing of distributed data coming from the vehicle and the infrastructure sensors for safe operations and not disruption of other routine operations or traffic flows [8][9][10][11]; vii) integrate all data in an enhanced visualisation user interface supporting decisions [12,13]; and viii) communication modules to allow for Vehicle-to-Infrastructure/Everything data exchange for predictive maintenance and increase users' safety [14]. Consequently, HERON will allow for a modular design of the system operation, maximizing its capabilities and adaptability for various transport infrastructures, while reducing fatal accidents, maintenance costs, and traffic disruptions, thus increasing the network capacity and efficiency.…”