4.9 A 1ms high-speed vision chip with 3D-stacked 140GOPS column-parallel PEs for spatio-temporal image processing

Yamazaki, Toshiaki; Katayama, Hironobu; Uehara, Shuji; Nose, Atsushi; Kobayashi, Masatsugu; Shida, Sayaka; Odahara, Masaki; Takamiya, Kenichi; Hisamatsu, Yasuaki; Matsumoto, Shimpei; Miyashita, Leo; Watanabe, Yoshihiro; Izawa, Takashi; Muramatsu, Y.; Ishikawa, Masatoshi

doi:10.1109/isscc.2017.7870271

Cited by 86 publications

(39 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The different layers can be implemented by different semiconductor processes, respectively, and then be integrated into one 3D stacked chip. Recently the examples of the 3D neuromorphic vision chips have been reported [22,56]. This will become a trend of the research on the neuromorphic vision chip.…”

Section: Future Neuromorphic Vision Chipsmentioning

confidence: 97%

“…This chip achieves 1000 fps object tracking. On the other hand, a high resolution 3D-stacked vision chip was reported [22]. The vision chip architecture consists of a 1296 × 976 (1.27 M) pixel array, column-parallel Analog-to-Digital Converters (ADCs) and SIMD column-parallel processing elements (PEs).…”

Section: Fd Vision Chipsmentioning

confidence: 99%

“…On the other hand, the FD chip has the advantages in image resolution, static object detection, time-multiplex image processing, and chip area. In FD vision chip, the photo pixel circuit is compact such that the image sensor may have higher fill factor [17,22]. By contrast, the AER imager pixel in ED vision chip integrates one pixel along with one processing circuit unit [25,41].…”

Section: Omparison Of Frame-and Event-driven Vision Chipsmentioning

confidence: 99%

See 2 more Smart Citations

Neuromorphic vision chips

2018

Sci. China Inf. Sci.

View full text Add to dashboard Cite

The paper reviews the progress of neuromorphic vision chip research in decades. It focuses on two kinds of the neuromorphic vision chips: frame-driven (FD) and event-driven (ED) vision chips. The FD and ED vision chips are very different from each other in system architecture, image sensing, image information coding, image processing algorithm, design methodology. The vision chips can overcome serial data transmission and processing bottlenecks in traditional image processing systems. They can perform the high speed image capture and real-time image processing operations. This paper selects two typical chips from the two kinds of vision chips, respectively, and introduces their architectures, image sensing schemes, image processing processors and system operation. The FD neuromorphic reconfigurable vision chip comprises a high speed image sensor, a processing element array and self-organizing map neural network. The FD vision chip has the advantages in image resolution, static object detection, time-multiplex image processing, and chip area. The ED neuromorphic vision chip system is based on address-event-representation image sensor and event-driven multi-kernel convolution network. The ED vision chip has the advantages in fast sensing, low communication bandwidth, brain-like processing, and high energy efficiency. Finally, this paper discusses the architecture and the challenges of the future neuromorphic vision chip and indicates that the reconfigurable vision chip with left-and right-brain functions integrated in the three dimensional (3D) large-scale integrated circuit (LSI) technology becomes a trend of the research on the vision chip.

show abstract

Section: Future Neuromorphic Vision Chipsmentioning

confidence: 97%

Section: Fd Vision Chipsmentioning

confidence: 99%

Section: Omparison Of Frame-and Event-driven Vision Chipsmentioning

confidence: 99%

See 1 more Smart Citation

Neuromorphic vision chips

2018

Sci. China Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Besides the method of realizing a high-speed vision system with traditional configuration as shown in Figure 5(a), new high-speed vision system ( Figure 5 with newly developed vision chip is introduced. The new vision chip combines high-frame-rate imaging and highly parallel signal processing with high-resolution, high-sensitivity, low-power consumption [26]. The 1/3.2-inch 1.27 Mpixel 500 fps (0.31 Mpixel 1000 fps 2 Â 2 binning) vision chip is fabricated with 3D-stacked column-parallel analog-to-digital converters (ADCs) and 140 giga-operations per second (GOPS) programmable single instruction multiple data (SIMD) columnparallel processing elements (PEs) for high-speed spatiotemporal image processing.…”

Section: High-speed Vision System With Newly Developed Vision Chipmentioning

confidence: 99%

Dynamic Compensation Framework to Improve the Autonomy of Industrial Robots

Huang¹,

Yamakawa²,

Ishikawa³

2020

Industrial Robotics - New Paradigms

Self Cite

View full text Add to dashboard Cite

It is challenging to realize the autonomy of industrial robots under external and internal uncertainties. A majority of industrial robots are supposed to be programmed by teaching-playback method, which is not able to handle with uncertain working conditions. Although many studies have been conducted to improve the autonomy of industrial robots by utilizing external sensors with model-based approaches as well as adaptive approaches, it is still difficult to obtain good performance. In this chapter, we present a dynamic compensation framework based on a coarse-to-fine strategy to improve the autonomy of industrial robots while at the same time keeping good accuracy under many uncertainties. The proposed framework for industrial robot is designed along with a general intelligence architecture that is aiming to address the big issues such as smart manufacturing, industrial 4.0.

show abstract

“…With rapid advancements in computer vision technologies, various real-time high-frame-rate (HFR) vision systems operating at 1000 fps or more have been developed [70][71][72][73], and their effectiveness has been demonstrated in tracking applications such as robot manipulations [74][75][76][77], multi-copter tracking [78,79], optical flow [80], camshift tracking [81], multi-object tracking [82], feature point tracking [83], and face tracking [84]. These systems were computationally accelerated by parallel-implementation on field-programmable gate arrays (FPGAs) and graphics processing units (GPUs) to obtain real-time HFR video processing.…”

Section: Introductionmentioning

confidence: 99%

Real-time high-resolution video stabilization using high-frame-rate jitter sensing

et al. 2019

View full text Add to dashboard Cite

In this study, the novel approach of real-time video stabilization system using a high-frame-rate (HFR) jitter sensing device is demonstrated to realize the computationally efficient technique of digital video stabilization for high-resolution image sequences. This system consists of a high-speed camera to extract and track feature points in graylevel 512 × 496 image sequences at 1000 fps and a high-resolution CMOS camera to capture 2048 × 2048 image sequences considering their hybridization to achieve real-time stabilization. The high-speed camera functions as a real-time HFR jitter sensing device to measure an apparent jitter movement of the system by considering two ways of computational acceleration; (1) feature point extraction with a parallel processing circuit module of the Harris corner detection and (2) corresponding hundreds of feature points at the current frame to those in the neighbor ranges at the previous frame on the assumption of small frame-to-frame displacement in high-speed vision. The proposed hybrid-camera system can digitally stabilize the 2048 × 2048 images captured with the high-resolution CMOS camera by compensating the sensed jitter-displacement in real time for displaying to human eyes on a computer display. The experiments were conducted to demonstrate the effectiveness of hybrid-camera-based digital video stabilization such as (a) verification when the hybrid-camera system in the pan direction in front of a checkered pattern, (b) stabilization in video shooting a photographic pattern when the system moved with a mixed-displacement motion of jitter and constant low-velocity in the pan direction, and (c) stabilization in video shooting a real-world outdoor scene when an operator holding hand-held hybrid-camera module while walking on the stairs.

show abstract

4.9 A 1ms high-speed vision chip with 3D-stacked 140GOPS column-parallel PEs for spatio-temporal image processing

Cited by 86 publications

References 3 publications

Neuromorphic vision chips

Neuromorphic vision chips

Dynamic Compensation Framework to Improve the Autonomy of Industrial Robots

Real-time high-resolution video stabilization using high-frame-rate jitter sensing

Contact Info

Product

Resources

About