Developmental Bayesian Optimization of Black-Box with Visual Similarity-Based Transfer Learning

Abstract

We present a developmental framework based on a long-term memory and reasoning mechanisms (Vision Similarity and Bayesian Optimisation). This architecture allows a robot to optimize autonomously hyper-parameters that need to be tuned from any action and/or vision module, treated as a black-box. The learning can take advantage of past experiences (stored in the episodic and procedural memories) in order to warmstart the exploration using a set of hyper-parameters previously optimized from objects similar to the new unknown one (stored in a semantic memory). As example, the system has been used to optimized 9 continuous hyper-parameters of a professional software (Kamido) both in simulation and with a real robot (industrial robotic arm Fanuc) with a total of 13 different objects. The robot is able to find a good object-specific optimization in 68 (simulation) or 40 (real) trials. In simulation, we demonstrate the benefit of the transfer learning based on visual similarity, as opposed to an amnesic learning (i.e. learning from scratch all the time). Moreover, with the real robot, we show that the method consistently outperforms the manual optimization from an expert with less than 2 hours of training time to achieve more than 88% of success.

Bayesian Optimization for Developmental Robotics with Meta-Learning by Parameters Bounds Reduction

“…The architecture of the cognitive robotics framework (see Fig. 2) is based upon the work of Petit et al [7]. It consists of the construction and exploitation with different reasoning capacities of a Long-Term Memory storing information in 3 sub-memories as described by Tulving [14]: 1) the episodic memory storing data from personal experiences and events, then linked to specific place and time, 2) the procedural memory containing motor skills and action strategies learnt during the lifetime and 3) the semantic memory filled with facts and knowledge about the world.…”

“…The experiment setup is similar to the describe in [7] allowing to compare some of their results with ours. We are indeed aiming at optimizing some parameters of a professional software called Kamido 1 (from Sileane) that we are treating as a black-box.…”

“…We also introduce a partial To be able to compare each learning run with the same learning condition, we authorize a finite budget for the BO process of 35 iterations, decomposed as follows: 10 for the "initial design", 20 for the core BO process and 5 as repetitions of the optimized set of parameters in order to provide a more precise estimation of the performance. As opposed to the experiment done in [7], we decided to constrain more the learning setup, providing only 30 (10+20) iterations instead of 68 (18+50). Indeed, the learning curve seemed to flattened around this number of iterations in their work, so we wanted to compare the quality of the optimization at an earlier stage.…”

“…For the experiments presented in this work, we used some objects from [7], namely the reference A, C1, C2, D and D' in order to compare the performance of the method with a smaller learning budget as explained earlier. We also introduce new objects, some from a CAD database of real industrial reference (P1 and P2), and some from other common databases, such as hammer t and hammer j from turbosquid, m782 and m784 from Princeton Shape Benchmark [25], and bathDetergent and cokeSmallGrasp from KIT [26].…”

“…Our contribution consists of an enhanced version of the work from Petit et al [7]: a developmental cognitive architecture providing a robot with a long-term memory and reasoning modules. It is allowing the robot to store optimization runs for bin-picking tasks using a professional grasping software, and utilize such experiences to increase the performance of new optimizations.…”