ToF/Radar early feature-based fusion system for human detection and tracking

Zoghlami, Feryel; Sen, Okan Kamil; Heinrich, Harald; Schneider, Germar; Erçelik, Emeç; Knoll, Alois; Villmann, Thomas

doi:10.1109/icit46573.2021.9453703

Cited by 10 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, it improves the accuracy of detection with the CV algorithms. In this proposed system, accuracy of 99% is achieved, and 100% for humans in the scene, this improves on the work of Zoghlami et al . (2021) who report an accuracy of 98%.…”

Section: Discussionsupporting

confidence: 78%

“…In Zoghlami et al (2021) a multi-sensor data fusion approach is proposed, using data from a three-dimension Time of Flight camera (ToF) and a radar sensor. The authors claim the system has the ability to detect humans walking around with a confidence up to 98%.…”

Section: Multi-sensor Approaches To Speed and Separation Monitoringmentioning

confidence: 99%

“…In Zoghlami et al . (2021) a multi-sensor data fusion approach is proposed, using data from a three-dimension Time of Flight camera (ToF) and a radar sensor.…”

Section: Literature Reviewmentioning

confidence: 99%

See 2 more Smart Citations

Managing safety of the human on the factory floor: a computer vision fusion approach

Humphries,

Van de Ven,

Amer

et al. 2024

TECHS

View full text Add to dashboard Cite

PurposeMaintaining the safety of the human is a major concern in factories where humans co-exist with robots and other physical tools. Typically, the area around the robots is monitored using lasers. However, lasers cannot distinguish between human and non-human objects in the robot’s path. Stopping or slowing down the robot when non-human objects approach is unproductive. This research contribution addresses that inefficiency by showing how computer-vision techniques can be used instead of lasers which improve up-time of the robot.Design/methodology/approachA computer-vision safety system is presented. Image segmentation, 3D point clouds, face recognition, hand gesture recognition, speed and trajectory tracking and a digital twin are used. Using speed and separation, the robot’s speed is controlled based on the nearest location of humans accurate to their body shape. The computer-vision safety system is compared to a traditional laser measure. The system is evaluated in a controlled test, and in the field.FindingsComputer-vision and lasers are shown to be equivalent by a measure of relationship and measure of agreement. R2 is given as 0.999983. The two methods are systematically producing similar results, as the bias is close to zero, at 0.060 mm. Using Bland–Altman analysis, 95% of the differences lie within the limits of maximum acceptable differences.Originality/valueIn this paper an original model for future computer-vision safety systems is described which is equivalent to existing laser systems, identifies and adapts to particular humans and reduces the need to slow and stop systems thereby improving efficiency. The implication is that computer-vision can be used to substitute lasers and permit adaptive robotic control in human–robot collaboration systems.

show abstract

Section: Discussionsupporting

confidence: 78%

Section: Multi-sensor Approaches To Speed and Separation Monitoringmentioning

confidence: 99%

See 1 more Smart Citation

Managing safety of the human on the factory floor: a computer vision fusion approach

Humphries,

Van de Ven,

Amer

et al. 2024

TECHS

View full text Add to dashboard Cite

show abstract

“…We showed the architecture for such a use case with two different sensors delivering two output signals. Figure 7 below shows an overview of how the signals from two sensor systems, ToF and radar sensors, can be combined and create value out of the fused data for obstacles detection (more general details about the ToF/radar fusion system itself was already described in the papers [22] and [23]). An MQTT broker acts as the main information node.…”

Section: Resultsmentioning

confidence: 99%

Micro Service based Sensor Integration Efficiency and Feasibility in the Semiconductor Industry

Schneider¹,

Patolla²,

Fehr³

et al. 2022

Infocommunications journal

View full text Add to dashboard Cite

The semiconductor industry is strongly increasing the production capacities and the product portfolio for a wide range of applications that are needed in the worldwide supply chains e.g. the automotive, computer and security industry. The complex manufacturing processes require more automation, dig- italisation and IoT frameworks, especially for highly automated semiconductor manufacturing plants. Over the last years, this industry spent much effort to control highly sensitive materials in production by product monitoring using advanced process control by various sensors in production. Nevertheless, until today, sensor integration, especially for such sensors that are not supported by the equipment vendors, is time-consuming and complicated. This article aims to use a micro-service-based approach by Eclipse Arrowhead as an open-source microservice architecture and implementation platform [1]. This architecture is an easy and powerful framework that can be used for multiple sensor applications to control the manufacturing material flow in a modern semiconductor plant with a high product mix. The article describes how the engineering process was designed, the architecture of the use case and the main benefits in the operational business are shown.

show abstract

“…• Scene perception and object avoidance: Seminal work in object detection using image processing [83,132,142], Radar [3,6,14,140], lidar [12,113,129], and ultrasound [8,42,47,67,81,136] have signicantly advanced the eld of study. Since the dominant physical eects change as scale reduces, conventional object avoidance techniques such as electromagnetic motors and robot navigation algorithms cannot be applied on microrobots.…”

Section: Related Workmentioning

confidence: 99%

SPiDR

Bai

Garg

Roy

2022

Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services

View full text Add to dashboard Cite

This paper presents the design and implementation of SPiDR, an ultra-low-power spatial sensing system for miniature mobile robots. This acoustic sensor produces a cross-sectional map of the eld-of-view using only one speaker/microphone pair. While it is challenging to have enough spatial diversity of signal with a single omnidirectional source, we leverage sound's interaction with small structures to create a 3D-printed passive lter, called a stencil, that can project spatially coded signals on a region at a ne granularity. The system receives a linear combination of the reections from nearby objects and applies a novel power-aware depth-map reconstruction algorithm. The algorithm rst estimates the approximate locations of the objects in the scene and then iteratively applies fractional multi-resolution inversion. SPiDR consumes only 10<, of power to generate a depth-map in real-world scenario with over 80% structural similarity score with the scene. CCS CONCEPTS• Computer systems organization ! Embedded and cyberphysical systems; Sensors and actuators.

show abstract

ToF/Radar early feature-based fusion system for human detection and tracking

Cited by 10 publications

References 13 publications

Managing safety of the human on the factory floor: a computer vision fusion approach

Managing safety of the human on the factory floor: a computer vision fusion approach

Micro Service based Sensor Integration Efficiency and Feasibility in the Semiconductor Industry

SPiDR

Contact Info

Product

Resources

About