Toward Creative Problem Solving Agents: Action Discovery through Behavior Babbling

Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework

“…Suarez-Hernandez et al (Suárez-Hernández, Segovia-Aguas, Torras, & Alenyà, 2020) introduce a novel approach for the unsupervised synthesis of new action primitives. Gizzi et al (Gizzi, Castro, & Sinapov, 2019;Gizzi, Hassan, Lin, Rhea, & Sinapov, 2021) discover novel actions through action segmentation and behavior babbling, respectively, which they use as a method for knowledge expansion. In this way, the agent can then re-plan toward a goal in a novel scenario with the new knowledge.…”

“…The CPS agent, upon incurring a plan execution failure, attempts to discover new actions (represented both symbolically and non-symbolically, Figure 9) through segmenting formerly known actions. In similar follow-up work, Gizzi et al (Gizzi et al, 2021) utilize a framework for action discovery that applies low-level parameter variations to discover new actions. Low-level parameter variations change symbolic level predicates by either, a) generating a novel effect, or b) generating a set of effects equivalent to the original action, which are then added back into the knowledge base.…”

“…Their cost function encodes the number of editions required for the new action, such as the number of additions and deletions in effects, as well as changes in the number of preconditions for the actions. Similarly, Gizzi et al (Gizzi et al, 2021) apply behavior babbling to discover new actions, by varying action controller parameters by partitioning their values evenly along a range of permissible values. For example, given a min and max value of a parameters, of 0 and 100 respectively, a user selected partition value of 5 would cause babbling experimentation for 1, 25, 50, 75 and 100.…”

“…In proof-of-concept cases, it is typical to use a 2D or 3D physics-based environments, so the end-to-end behavior of the agent can be visually and algorithmically validated. Common examples of such environments include "Gazebo" (Gizzi et al, 2019;Levihn & Christensen, 2015;Toussaint et al, 2018;Gizzi et al, 2021;Oddi et al, 2020), "Unity" (Bapst et al, 2019;Pathak et al, 2019), "OpenAIGym" (Wang, Lehman, Clune, & Stanley, 2019;Ha, 2019;Schaff et al, 2019), "Mujoco" (Baker et al, 2019), "DeepMind Lab" (Leibo, Hughes, Lanctot, & Graepel, 2019), "PyBullet" (Qin et al, 2020;Chitnis et al, 2021;Silver et al, 2021), and "Bullet Physics Library" (Zhao et al, 2020). While all of these environments provide similar support, each cater well to different CPS cases.…”

“…Behavior validation can happen through the use of perceptual information, or through intrinsic motivation. In the case of perceptual validation, Gizzi et al (Gizzi et al, 2021) demonstrate babbling through systematic variations in continuous parameters of known actions, in order to discover new actions to be used in CPS tasks. New actions are validated through novelty/usefulness in predicate end effects.…”