Unsupervised Man Overboard Detection Using Thermal Imagery and Spatiotemporal Autoencoders

Bakalos, Nikolaos; Katsamenis, Iason; Karolou, Eleni Eirini; Doulamis, Nikolaos

doi:10.3233/faia210103

Cited by 3 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many researchers have studied the assessment of fall risk [11][12][13][14], but most of them examined older adults, and little research has been conducted on fall risk identification for seafarers on a ship. In addition, existing studies related to classifying fall risks have been undertaken via different devices like radars and cameras [15][16][17][18][19]. However, wearable sensor-based fall risk evaluation studies have been relatively insufficient.…”

Section: Discussionmentioning

confidence: 99%

“…There are fewer studies on the prediction or detection of seafarers' falls in the maritime field. Only a few previous studies have attempted to detect MOBs through ML technologies [15][16][17][18][19]. Tsekenis et al [15] implemented an ML-based system to detect MOBs using radar sensors.…”

Section: Introductionmentioning

confidence: 99%

“…They achieved higher accuracy (97.24%) with a random forest algorithm. Bakalos et al investigated identifying MOB events using simple RGB streams and thermal imagery through convolutional spatiotemporal autoencoders, which can detect a fall as an anomaly [16,17]. Armeniakos et al [18] built a human fall detection system using multiple long-range millimeter-wave band radar sensors.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning-Based Approach to Identifying Fall Risk in Seafarers Using Wearable Sensors

Choi,

Knarr,

Youn

et al. 2024

JMSE

View full text Add to dashboard Cite

Falls on a ship cause severe injuries, and an accident falling off board, referred to as “man overboard” (MOB), can lead to death. Thus, it is crucial to accurately and timely detect the risk of falling. Wearable sensors, unlike camera and radar sensors, are affordable and easily accessible regardless of the weather conditions. This study aimed to identify the fall risk level (i.e., high and low risk) among individuals on board using wearable sensors. We collected walking data from accelerometers during the experiment by simulating the ship’s rolling motions using a computer-assisted rehabilitation environment (CAREN). With the best features selected by LASSO, eight machine learning (ML) models were implemented with a synthetic minority oversampling technique (SMOTE) and the best-tuned hyperparameters. In all ML models, the performance in classifying fall risk showed overall a good accuracy (0.7778 to 0.8519), sensitivity (0.7556 to 0.8667), specificity (0.7778 to 0.8889), and AUC (0.7673 to 0.9204). Logistic regression showed the best performance in terms of the AUC for both training (0.9483) and testing (0.9204). We anticipate that this study will effectively help identify the risk of falls on ships and aid in developing a monitoring system capable of averting falls and detecting MOB situations.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning-Based Approach to Identifying Fall Risk in Seafarers Using Wearable Sensors

Choi,

Knarr,

Youn

et al. 2024

JMSE

View full text Add to dashboard Cite

show abstract

“…First, HERON aims to develop an integrated robotic platform with increased navigation and positioning capabilities for maintenance and upgrading tasks and concurrent assessment of the RIs. HERON improves Real-Time (RT) perception and cognition of the robotic vehicles, through Simultaneous Localisation and Mapping (SLAM) and novel Machine Learning techniques [20,21]. SLAM will provide the concurrent mapping generation of the robotic vehicles' space (complementary to GPS).…”

Section: The Heron Projectmentioning

confidence: 99%

Robotic Maintenance of Road Infrastructures: The HERON Project

Katsamenis¹,

Bimpas²,

Protopapadakis³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Of all public assets, road infrastructure tops the list. Roads are crucial for economic development and growth, providing access to education, health, and employment. The maintenance, repair, and upgrade of roads are therefore vital to road users' health and safety as well as to a well-functioning and prosperous modern economy. The EU-funded HERON project will develop an integrated automated system to adequately maintain road infrastructure. In turn, this will reduce accidents, lower maintenance costs, and increase road network capacity and efficiency. To coordinate maintenance works, the project will design an autonomous ground robotic vehicle that will be supported by autonomous drones. Sensors and scanners for 3D mapping will be used in addition to artificial intelligence toolkits to help coordinate road maintenance and upgrade workflows.

show abstract

Fall Detection Using Multi-Property Spatiotemporal Autoencoders in Maritime Environments

et al. 2022

View full text Add to dashboard Cite

Man overboard is an emergency in which fast and efficient detection of the critical event is the key factor for the recovery of the victim. Its severity urges the utilization of intelligent video surveillance systems that monitor the ship’s perimeter in real time and trigger the relative alarms that initiate the rescue mission. In terms of deep learning analysis, since man overboard incidents occur rarely, they present a severe class imbalance problem, and thus, supervised classification methods are not suitable. To tackle this obstacle, we follow an alternative philosophy and present a novel deep learning framework that formulates man overboard identification as an anomaly detection task. The proposed system, in the absence of training data, utilizes a multi-property spatiotemporal convolutional autoencoder that is trained only on the normal situation. We explore the use of RGB video sequences to extract specific properties of the scene, such as gradient and saliency, and utilize the autoencoders to detect anomalies. To the best of our knowledge, this is the first time that man overboard detection is made in a fully unsupervised manner while jointly learning the spatiotemporal features from RGB video streams. The algorithm achieved 97.30% accuracy and a 96.01% F1-score, surpassing the other state-of-the-art approaches significantly.

show abstract

Unsupervised Man Overboard Detection Using Thermal Imagery and Spatiotemporal Autoencoders

Cited by 3 publications

References 22 publications

Machine Learning-Based Approach to Identifying Fall Risk in Seafarers Using Wearable Sensors

Machine Learning-Based Approach to Identifying Fall Risk in Seafarers Using Wearable Sensors

Robotic Maintenance of Road Infrastructures: The HERON Project

Fall Detection Using Multi-Property Spatiotemporal Autoencoders in Maritime Environments

Contact Info

Product

Resources

About