Structure-based color learning on a mobile robot under changing illumination

Abstract

A central goal of robotics and AI is to be able to deploy an agent to act autonomously in the real world over an extended period of time. To operate in the real world, autonomous robots rely on sensory information. Despite the potential richness of visual information from on-board cameras, many mobile robots continue to rely on non-visual sensors such as tactile sensors, sonar, and laser. This preference for relatively low-fidelity sensors can be attributed to, among other things, the characteristic requirement of realtime operation under limited computational resources. Illumination changes pose another big challenge. For true extended autonomy, an agent must be able to recognize for itself when to abandon its current model in favor of learning a new one; and how to learn in its current situation. We describe a self-contained vision system that works on-board a vision-based autonomous robot under varying illumination conditions. First, we present a baseline system capable of color segmentation and object recognition within the computational and memory constraints of the robot. This relies on manually labeled data and operates under constant and reasonably uniform illumination conditions. We then relax these limitations by introducing algorithms for (i) Autonomous planned color learning, where the robot uses the knowledge of its environment (position, size and shape of objects) to automatically generate a suitable motion sequence and learn the desired colors, and (ii) Illumination change detection and adaptation, where the robot recognizes for itself when the illumination conditions have changed sufficiently to warrant revising its knowledge of colors. Our al-M. Sridharan ( ) · P. Stone gorithms are fully implemented and tested on the Sony ERS-7 Aibo robots.

Multi-agent versus multi-robot and other Byzantine discussions

“…This encouraged us to establish the journal in English and give it an international scope. For instance, Dr. P. Stone, one of the authors of the articles that make up this issue, perfectly matches such a profile, having made significant contributions to the area of software agents, such as [12], but also being very active in traditional robotics, for instance [11].…”

Improved Unsupervised Color Segmentation Using a Modified HSV Color Model and a Bagging Procedure in K-Means++ Algorithm

“…We can find some recent works using neural networks [44][45][46], Gaussian mixture model [47,48], support vector machine [49][50][51], and support vector machine with -means family based training [52,53]. Even though -means algorithm is old, it is still used in image segmentation due to its ease of implementation [39,[52][53][54].…”

Practical color-based motion capture

“…Dynamic color models have been proposed to cope with illumination changes, e.g. [McKenna et al 1999;Kakumanu et al 2007;Sridharan and Stone 2007]. The strong occlusions that appear with our shirt would be challenging for these models because one or several color patches can disappear for long periods of time.…”