Despite extensive research conducted in machine vision for harvesting robots, practical success in this field of agrobotics is still limited. This article presents a comprehensive review of classical and state-of-the-art machine vision solutions employed in such systems, with special emphasis on the visual cues and machine vision algorithms used. We discuss the advantages and limitations of each approach and we examine these capacities in light of the challenges ahead. We conclude with suggested directions from the general computer vision literature which could assist our research community meet these challenges and bring us closer to the goal of practical selective fruit harvesting robots.

No abstract

Visual context is used in different forms for saliency computation. While its use in saliency models for fixations prediction is often reasoned, this is less so the case for approaches that aim to compute saliency at the object level. We argue that the types of context employed by these methods lack clear justification and may in fact interfere with the purpose of capturing the saliency of whole visual objects. In this paper we discuss the constraints that different types of context impose and suggest a new interpretation of visual context that allows the emergence of saliency for more complex, abstract, or multiple visual objects. Despite shying away from an explicit attempt to capture "objectness" (e.g., via segmentation), our results are qualitatively superior and quantitatively better than the state-of-the-art.

When walking on an uneven surface or complex terrain, humans tend to gaze downward. This behaviour is usually interpreted as an attempt to acquire useful information to guide locomotion. Visual information, however, is not used exclusively for guiding locomotion; it is also useful for postural control. Both locomotive and postural control have been shown to be sensitive to the visual flow arising from the respective motion of the individual and the three-dimensional environment. This flow changes when a person gazes downward and may present information that is more appropriate for postural control. To investigate whether downward gazing can be used for postural control, rather than exclusively for guiding locomotion, we quantified the dynamics of standing and walking posture in healthy adults, under several visual conditions. Through these experiments we were able to demonstrate that gazing downward, just a few steps ahead, resulted in a steadier standing and walking posture. These experiments indicate that gazing downward may serve more than one purpose and provide sufficient evidence of the possible interplay between the visual information used for guiding locomotion and that used for postural control. These findings contribute to our understanding of the control mechanism/s underlying gait and posture and have possible clinical implications.

Downward gazing is often observed when walking requires guidance. This gaze behavior is thought to promote walking stability through anticipatory stepping control. This study is part of an ongoing effort to investigate whether downward gazing also serves to enhance postural control, which can promote walking stability through a feedback/reactive mechanism. Since gaze behavior alone gives no indication as to what information is gathered and the functions it serves, we aimed to investigate the cognitive demands associated with downward gazing, as they are likely to differ between anticipatory and feedback use of visual input. To do so, we used a novel methodology to compromise walking stability in a manner that could not be resolved through modulation of stepping. Then, using interference methodology and neuroimaging, we tested for (1) interference related to dual tasking, and (2) changes in prefrontal activity. The novel methodology resulted in an increase in the time spent looking at the walking surface. Further, while some dual-task interference was observed, indicating that this gaze behavior is cognitively demanding, several gaze parameters pertaining to downward gazing and prefrontal activity correlated. These correlations revealed that a greater tendency to gaze onto the walking surface was associated with lower PFC activity, as is expected when sensory information is used through highly automatic, and useful, neural circuitry. These results, while not conclusive, do suggest that gazing onto the walking surface can be used for purposes other than anticipatory stepping control, bearing important motor-control and clinical implications.

Segregation of objects from the background is a basic and essential property of the visual system. We studied the neural detection of objects defined by orientation difference from background in barn owls ( Tyto alba). We presented wide-field displays of densely packed stripes with a dominant orientation. Visual objects were created by orienting a circular patch differently from the background. In head-fixed conditions, neurons in both tecto- and thalamofugal visual pathways (optic tectum and visual Wulst) were weakly responsive to these objects in their receptive fields. However, notably, in freely viewing conditions, barn owls occasionally perform peculiar side-to-side head motions (peering) when scanning the environment. In the second part of the study we thus recorded the neural response from head-fixed owls while the visual displays replicated the peering conditions; i.e., the displays (objects and backgrounds) were shifted along trajectories that induced a retinal motion identical to sampled peering motions during viewing of a static object. These conditions induced dramatic neural responses to the objects, in the very same neurons that where unresponsive to the objects in static displays. By reverting to circular motions of the display, we show that the pattern of the neural response is mostly shaped by the orientation of the background relative to motion and not the orientation of the object. Thus our findings provide evidence that peering and/or other self-motions can facilitate orientation-based figure-ground segregation through interaction with inhibition from the surround. NEW & NOTEWORTHY Animals frequently move their sensory organs and thereby create motion cues that can enhance object segregation from background. We address a special example of such active sensing, in barn owls. When scanning the environment, barn owls occasionally perform small-amplitude side-to-side head movements called peering. We show that the visual outcome of such peering movements elicit neural detection of objects that are rotated from the dominant orientation of the background scene and which are otherwise mostly undetected. These results suggest a novel role for self-motions in sensing objects that break the regular orientation of elements in the scene.

When walking on an uneven surface or complex terrain, humans tend to gaze downward.Previous investigations indicate that visual information can be used for online control of stepping. Behavioral investigations suggest that, during walking, the availability of visual information increases stepping accuracy, but probably through a feedforward control mechanism. Consequently, downward gazing (DWG) is usually interpreted as a strategy used to acquire useful information for online and/or feedforward control of stepping.Visual information is not exclusively used for guiding locomotion; a wealth of literature has been published on the usefulness of visual information for feedback postural control. Critically, postural control has been shown to be sensitive to the visual flow arising from the respective motion of the individual and the 3D environment.To investigate whether DWG can be used to enhance feedback control of posture, rather than feedforward/online control of gait, we conducted a series of experiments that explore this possible interplay. Through these experiments we were able to show that DWG, just a few steps ahead, results in a steadier standing and walking posture, without the need for accuracy. Moreover, we were able to demonstrate that humans resort to DWG when walking stability is compromised, even when destabilizing features were visually unpredictable.This series of experiments provides sufficient evidence of the possible interplay between visual information used for guiding locomotion and that used for postural control. Moreover, this evidence raises concerns regarding the way we interpret gaze behavior without the knowledge of the type and use of the information gathered.

No abstract