Super high-speed vision platform for processing 1024×1024 images in real time at 12500 fps

Sharma, Atul; Shimasaki, Kohei; Gu, Qingyi; Aoyama, Tadayoshi; Takaki, Takeshi; Ishii, Idaku; Tamura, Koichi; Tajima, Kiyohiro

doi:10.1109/sii.2016.7844055

Cited by 12 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, an HFR vision system to sense temporal alterations in highspeed projection data is required. With millisecond-level accuracy, HFR vision systems operated at hundreds or thousands of fps have been used for various industrial applications [45][46][47][48]. A saccade mirror and HFR cameras have been used to add visual information in real-time for the projection-based mixed reality of dynamic objects [49].…”

Section: Related Workmentioning

confidence: 99%

Projection-mapping-based object pointing using a high-frame-rate camera-projector system

et al. 2021

Self Cite

View full text Add to dashboard Cite

The novel approach to physical security based on visible light communication (VLC) using an informative object-pointing and simultaneous recognition by high-framerate (HFR) vision systems is presented in this study. In the proposed approach, a convolutional neural network (CNN) based object detection method is used to detect the environmental objects that assist a spatiotemporal-modulated-pattern (SMP) based imperceptible projection mapping for pointing the desired objects. The distantly located HFR vision systems that operate at hundreds of frames per second (fps) can recognize and localize the pointed objects in real-time. The prototype of an artificial intelligence-enabled camera-projector (AiCP) system is used as a transmitter that detects the multiple objects in real-time at 30 fps and simultaneously projects the detection results by means of the encoded-480-Hz-SMP masks on to the objects. The multiple 480-fps HFR vision systems as receivers can recognize the pointed objects by decoding pixel-brightness variations in HFR sequences without any camera calibration or complex recognition methods. Several experiments were conducted to demonstrate our proposed method’s usefulness using miniature and real-world objects under various conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

Projection-mapping-based object pointing using a high-frame-rate camera-projector system

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, to monitor high-speed phenomena continuously, we need HFR cameras to improve the shooting speed and performance. In this study, the proposed system consists of a monochrome HFR camera system that is an extension of Fastcam SA-X2 developed by Co. Photron and Hiroshima University; it provides a complementary metal oxide semiconductor (CMOS) sensor-based super high-speed vision platform that enables real-time image processing more than 10,000 fps against a megapixel image and global electronic shutter with excellent light sensitivity [ 4 ]. This camera is used as a receiver with an embedded external board that has an onboard FPGA for image processing; it produces output images with a resolution of 512 × 512 with a 12-bit dynamic range at 3125 fps in real time.…”

Section: Hfr Projector-camera-based Vlc Systemmentioning

confidence: 99%

“…With the recent rapid advances in computer and image sensor technologies, many high frame rate (HFR) vision systems that can capture and process images simultaneously at thousands of frames per second have been developed [ 1 , 2 , 3 , 4 , 5 ]; many tracking algorithms, such as optical flow estimation [ 6 , 7 ], cam-shift tracking [ 8 ], and feature-point tracking [ 9 ], have been accelerated by the parallel implementation of these algorithms on field programmable gate array (FPGA) and graphics processing units. These HFR vision systems can have a large bandwidth that can simultaneously recognize high-speed phenomena, which are too fast to be recognized by the naked human eyes and standard video cameras operating at dozens of frames per second.…”

Section: Introductionmentioning

confidence: 99%

HFR Projector Camera Based Visible Light Communication System for Real-Time Video Streaming

Sharma

Raut

Shimasaki

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

This study develops a projector–camera-based visible light communication (VLC) system for real-time broadband video streaming, in which a high frame rate (HFR) projector can encode and project a color input video sequence into binary image patterns modulated at thousands of frames per second and an HFR vision system can capture and decode these binary patterns into the input color video sequence with real-time video processing. For maximum utilization of the high-throughput transmission ability of the HFR projector, we introduce a projector–camera VLC protocol, wherein a multi-level color video sequence is binary-modulated with a gray code for encoding and decoding instead of pure-code-based binary modulation. Gray code encoding is introduced to address the ambiguity with mismatched pixel alignments along the gradients between the projector and vision system. Our proposed VLC system consists of an HFR projector, which can project 590 × 1060 binary images at 1041 fps via HDMI streaming and a monochrome HFR camera system, which can capture and process 12-bit 512 × 512 images in real time at 3125 fps; it can simultaneously decode and reconstruct 24-bit RGB video sequences at 31 fps, including an error correction process. The effectiveness of the proposed VLC system was verified via several experiments by streaming offline and live video sequences.

show abstract

“…HFR vision systems with millisecond-level accuracy operate at hundreds or thousands of frames per second and have been developed for various industrial applications [44,45,46,47]. For example, a saccade mirror and HFR cameras have been used to add visual information in real time for projection-based mixed reality of dynamic objects [48].…”

Section: Introductionmentioning

confidence: 99%

Projection-Mapping-Based Object Pointing Using a High-Frame-Rate Camera-Projector System

Ishii

Kumar

Raut

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

An informative object pointing method using a spatiotemporal-modulated pattern projection is proposed to recognize and localize pointed objects by using a distantly located high-frame-rate vision system. We developed a prototype for projection-mapping-based object pointing that consists of an AI-camera-enabled projection (AiCP) system used as a transmitter, for informative projection mapping, and an HFR vision system operated as a receiver. The AiCP system detects multiple objects in real time at 30 fps with a CNN-based object detector, and simultaneously encodes and projects the recognition results of the detector as 480-Hz-modulated light patterns on to the objects to be pointed. The multiple 480-fps cameras can directly recognize and track the objects pointed at by the AiCP system without camera calibration or complex recognition methods by decoding the brightness signals of pixels in the images. To demonstrate the eectiveness of our proposed method, several desktop experiments using miniature objects and scenes were conducted under various conditions.

show abstract

Super high-speed vision platform for processing 1024×1024 images in real time at 12500 fps

Cited by 12 publications

References 10 publications

Projection-mapping-based object pointing using a high-frame-rate camera-projector system

Projection-mapping-based object pointing using a high-frame-rate camera-projector system

HFR Projector Camera Based Visible Light Communication System for Real-Time Video Streaming

Projection-Mapping-Based Object Pointing Using a High-Frame-Rate Camera-Projector System

Contact Info

Product

Resources

About