Smart Visual Beacons with Asynchronous Optical Communications using Event Cameras

Wang, Ziwei; Ng, Yonhon; Henderson, Jack; Mahony, Robert

doi:10.1109/iros47612.2022.9982016

Cited by 9 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While batch methods have achieved high accuracy, they incur additional latency depending on the time interval of the batch (e.g., 50ms). Asynchronous methods, if implemented on appropriate hardware and software, have the potential to run on a timescale closer to that of events < 1ms, which has recently shown great potential in high-speed optical communication [34], tracking [35] and high-frequency signal processing [36], [37], [38]. A further distinction may be made between pure event reconstruction methods and hybrid event-frame methods that use a mix of (registered) events and image frames.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Asynchronous Kalman Filter for Hybrid Event Cameras

Wang

Scheerlinck

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Event cameras are ideally suited to capture High Dynamic Range (HDR) visual information without blur but provide poor imaging capability for static or slowly varying scenes. Conversely, conventional image sensors measure absolute intensity of slowly changing scenes effectively but do poorly on HDR or quickly changing scenes. In this paper, we present an asynchronous linear filter architecture, fusing event and frame camera data, for HDR video reconstruction and spatial convolution that exploits the advantages of both sensor modalities. The key idea is the introduction of a state that directly encodes the integrated or convolved image information and that is updated asynchronously as each event or each frame arrives from the camera. The state can be read-off as-often-as and whenever required to feed into subsequent vision modules for real-time robotic systems. Our experimental results are evaluated on both publicly available datasets with challenging lighting conditions and fast motions, along with a new dataset with HDR reference that we provide. The proposed AKF pipeline outperforms other state-of-the-art methods in both absolute intensity error (69.4% reduction) and image similarity indexes (average 35.5% improvement). We also demonstrate the integration of image convolution with linear spatial kernels Gaussian, Sobel, and Laplacian as an application of our architecture.

show abstract

Section: Related Workmentioning

confidence: 99%

“…To compute the spatial convolution of an image by fusing events and frames we follow the steps outlined above. The event stream is replaced by the convolved event stream (35), and the image frame is replaced by the convolved version of the log intensity frame (34) as shown in Figure 2.…”

Section: Continuous-time Filter For Convolved Eventsmentioning

confidence: 99%

An Asynchronous Kalman Filter for Hybrid Event Cameras

Wang

Scheerlinck

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“… Static OCC performance: MAR for increasing beacon frequencies in comparison with the state-of-the-art baseline (Wang et al, 2022 ). Results were obtained at a 50 cm distance.…”

Section: Resultsmentioning

confidence: 99%

“…Early work combined the fine temporal and spatial resolution of an event camera with blinking LEDs at different frequencies to perform visual odometry (Censi et al, 2013 ). Recent work makes use of these cameras to implement OCC with smart beacons and transmit a message with the UART protocol (Wang et al, 2022 ), delivering error-free messages of static beacons at up to 4 kbps indoors and up to 500 bps at 100 m distance outdoors with brighter beacons, but without tracking. This paper, combined with the tracking approach presented in von Arnim et al ( 2007 ), are the baseline of our work.…”

Section: Introductionmentioning

confidence: 99%

Dynamic event-based optical identification and communication

von Arnim,

Lecomte,

Borras

et al. 2024

Front. Neurorobot.

View full text Add to dashboard Cite

Optical identification is often done with spatial or temporal visual pattern recognition and localization. Temporal pattern recognition, depending on the technology, involves a trade-off between communication frequency, range, and accurate tracking. We propose a solution with light-emitting beacons that improves this trade-off by exploiting fast event-based cameras and, for tracking, sparse neuromorphic optical flow computed with spiking neurons. The system is embedded in a simulated drone and evaluated in an asset monitoring use case. It is robust to relative movements and enables simultaneous communication with, and tracking of, multiple moving beacons. Finally, in a hardware lab prototype, we demonstrate for the first time beacon tracking performed simultaneously with state-of-the-art frequency communication in the kHz range.

show abstract

“…Early work combined the fine temporal and spatial resolution of an event camera with blinking LEDs at different frequencies to perform visual odometry [7]. Recent work also relies on smart beacons to transmit a message with the UART protocol [8], delivering error-free messages at up to 4 kbps indoors and up to 500 bps at 100 m distance outdoors with brighter beacons.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Event-based Optical Identification and Communication

Arnim¹,

Lecomte²,

Woźniak³

et al. 2023

Preprint

View full text Add to dashboard Cite

Optical identification is often done with spatial or temporal visual pattern recognition and localization. Temporal pattern recognition, depending on the technology, involves a tradeoff between communication frequency, range and accurate tracking. We propose a solution with light-emitting beacons that improves this trade-off by exploiting fast event-based cameras and, for tracking, sparse neuromorphic optical flow computed with spiking neurons. In an asset monitoring use case, we demonstrate that the system, embedded in a simulated drone, is robust to relative movements and enables simultaneous communication with, and tracking of, multiple moving beacons. Finally, in a hardware lab prototype, we achieve state-of-the-art optical camera communication frequencies in the kHz magnitude.

show abstract

Smart Visual Beacons with Asynchronous Optical Communications using Event Cameras

Cited by 9 publications

References 30 publications

An Asynchronous Kalman Filter for Hybrid Event Cameras

An Asynchronous Kalman Filter for Hybrid Event Cameras

Dynamic event-based optical identification and communication

Dynamic Event-based Optical Identification and Communication

Contact Info

Product

Resources

About