Bio-inspired motion detection in an FPGA-based smart camera module

Köhler, Tim; Röchter, Frank; Lindemann, Jens Peter; Möller, Ralf

doi:10.1088/1748-3182/4/1/015008

Cited by 13 publications

(9 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This allows for the extraction of depth information and detection of nearby contours from the optic flow generated during self-motion under a wide range of light levels. These findings may generalize to other biological and technical visual systems based on correlation-type movement detection, because these are also limited in the sensor coding range and resolution (Harrison and Koch, 1999 ; Köhler et al, 2009 ; Plett et al, 2012 ; Meyer et al, 2016 ).…”

Section: Introductionmentioning

confidence: 65%

Peripheral Processing Facilitates Optic Flow-Based Depth Perception

Lindemann

Egelhaaf

2016

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Flying insects, such as flies or bees, rely on consistent information regarding the depth structure of the environment when performing their flight maneuvers in cluttered natural environments. These behaviors include avoiding collisions, approaching targets or spatial navigation. Insects are thought to obtain depth information visually from the retinal image displacements (“optic flow”) during translational ego-motion. Optic flow in the insect visual system is processed by a mechanism that can be modeled by correlation-type elementary motion detectors (EMDs). However, it is still an open question how spatial information can be extracted reliably from the responses of the highly contrast- and pattern-dependent EMD responses, especially if the vast range of light intensities encountered in natural environments is taken into account. This question will be addressed here by systematically modeling the peripheral visual system of flies, including various adaptive mechanisms. Different model variants of the peripheral visual system were stimulated with image sequences that mimic the panoramic visual input during translational ego-motion in various natural environments, and the resulting peripheral signals were fed into an array of EMDs. We characterized the influence of each peripheral computational unit on the representation of spatial information in the EMD responses. Our model simulations reveal that information about the overall light level needs to be eliminated from the EMD input as is accomplished under light-adapted conditions in the insect peripheral visual system. The response characteristics of large monopolar cells (LMCs) resemble that of a band-pass filter, which reduces the contrast dependency of EMDs strongly, effectively enhancing the representation of the nearness of objects and, especially, of their contours. We furthermore show that local brightness adaptation of photoreceptors allows for spatial vision under a wide range of dynamic light conditions.

show abstract

Section: Introductionmentioning

confidence: 65%

Peripheral Processing Facilitates Optic Flow-Based Depth Perception

Lindemann

Egelhaaf

2016

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…In recent years, advances in embedded vision systems such as progress in microprocessor power and FPGA technology allowed the creation of compact smart cameras with low cost and, this increased the smart camera applications performance, as shown in [11,6,7,8]. In current embedded vision applications, smart cameras represent a promising on-board solution under different application domains: motion detection [25], object detection/tracking [32,31], inspection and surveillance [16], human behavior recognition [20], etc. In any case, flexibility of application domain relies on the large variety of image processing algorithms that can be implemented inside the camera.…”

Section: Introductionmentioning

confidence: 99%

Robust feature extraction algorithm suitable for real-time embedded applications

Aguilar-González

Arias-Estrada

Berry

2017

J Real-Time Image Proc

View full text Add to dashboard Cite

International audienceSmart cameras integrate processing close to the image sensor, so they can deliver high-level information to a host computer or high-level decision process. One of the most common processing is the visual features extraction since many vision-based use-cases are based on such algorithm. Unfortunately, in most of cases, features detection algorithms are not robust or do not reach real-time processing. Based on these limitations, a feature detection algorithm that is robust enough to deliver robust features under any type of indoor / outdoor scenarios is proposed. This was achieved by applying a non-textured corner filter combined to a subpixel refinement. Furthermore, an FPGA architecture is proposed. This architecture allows compact system design, real-time processing for Full HD images (it can process up to 44 frames/91.238.400 pixels per second for Full HD images), and high efficiency for smart camera implementations (similar hardware resources than previous formulations without subpixel refinement and without non-textured corner filter). For accuracy/robustness, experimental results for several real world scenes are encouraging and show the feasibility of our algorithmic approach

show abstract

“…Aubepart et al (2004) used a Field Programmable Gate Array (FPGA) based solution with a linear 12-photodiode array, theoretically capable of handling up to 245 input elements. Köhler et al (2009) proposed a solution of higher spatial resolution at 120 × 100 input pixels over a 40 • horizontal field of view and a temporal resolution of 100 frames per second (fps), expandable up to 200 fps in bright outdoor conditions. But despite promising results in artificially structured environments, the system did not work in naturalistic settings.…”

Section: Introductionmentioning

confidence: 99%

Bio-inspired visual ego-rotation sensor for MAVs

et al. 2012

View full text Add to dashboard Cite

Flies are capable of extraordinary flight maneuvers at very high speeds largely due to their highly elaborate visual system. In this work we present a fly-inspired FPGA based sensor system able to visually sense rotations around different body axes, for use on board micro aerial vehicles (MAVs). Rotation sensing is performed analogously to the fly's VS cell network using zero-crossing detection. An additional key feature of our system is the ease of adding new functionalities akin to the different tasks attributed to the fly's lobula plate tangential cell network, such as object avoidance or collision detection. Our implementation consists of a modified eneo SC-MVC01 SmartCam module and a custom built circuit board, weighing less than 200 g and consuming less than 4 W while featuring 57,600 individual two-dimensional elementary motion detectors, a 185 • field of view and a frame rate of 350 frames per second. This makes our sensor system compact in terms of size, weight and power requirements for easy incorporation into MAV platforms, while autonomously performing all sensing and processing on-board and in real time.

show abstract

Bio-inspired motion detection in an FPGA-based smart camera module

Cited by 13 publications

References 17 publications

Peripheral Processing Facilitates Optic Flow-Based Depth Perception

Peripheral Processing Facilitates Optic Flow-Based Depth Perception

Robust feature extraction algorithm suitable for real-time embedded applications

Bio-inspired visual ego-rotation sensor for MAVs

Contact Info

Product

Resources

About