Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning

Ganapathi, Aditya; Florence, Pete; Varley, Jake; Burns, Kaylee; Goldberg, Ken; Zeng, Andy

doi:10.48550/arxiv.2203.01983

Cited by 4 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We now evaluate procedure cloning in larger scale settings on tasks including simulated robotic navigation [82] and manipulation [83,14], and learning to play MinAtar [84] (a miniature version of Atari [85]). Procedure cloning exhibits significant generalization to previously unseen maze layouts, positions of objects being manipulated, and environment configurations such as transition stochasticity and game difficulty in each of the tasks, respectively.…”

Section: Methodsmentioning

confidence: 99%

“…Task description. The bimanual sweeping task [83,14] requires two 7-DoF robot arms equipped with spatula-like end-effectors to sweep a pile of particles evenly into two bowls while avoiding dropping particles between the tips of the spatulas. The scripted oracle for collecting expert trajectories uses access to privileged information including object poses and contact points, which are not accessible at test time.…”

Section: Evaluating Image-based Robot Manipulationmentioning

confidence: 99%

“…Rather, only high-resolution (96 × 96) images in conjunction with end-effector positions and orientations are given to the imitation-learned agent during inference. Actions are end-effector positions for each robot arm (6 dimensions for each arm for a total of 12 dimensions for the action); for how to incorporate kinematic actions into this environment, we refer the reader to [83]. Random seeds determining the initial position of the particles are partitioned so that test-time configurations are not seen in training.…”

Section: Evaluating Image-based Robot Manipulationmentioning

confidence: 99%

See 2 more Smart Citations

Chain of Thought Imitation with Procedure Cloning

Yang¹,

Schuurmans²,

Abbeel³

et al. 2022

Preprint

View full text Add to dashboard Cite

Imitation learning aims to extract high-performance policies from logged demonstrations of expert behavior. It is common to frame imitation learning as a supervised learning problem in which one fits a function approximator to the input-output mapping exhibited by the logged demonstrations (input observations to output actions). While the framing of imitation learning as a supervised input-output learning problem allows for applicability in a wide variety of settings, it is also an overly simplistic view of the problem in situations where the expert demonstrations provide much richer insight into expert behavior. For example, applications such as path navigation, robot manipulation, and strategy games acquire expert demonstrations via planning, search, or some other multi-step algorithm, revealing not just the output action to be imitated but also the procedure for how to determine this action. While these intermediate computations may use tools not available to the agent during inference (e.g., environment simulators), they are nevertheless informative as a way to explain an expert's mapping of state to actions. To properly leverage expert procedure information without relying on the privileged tools the expert may have used to perform the procedure, we propose procedure cloning, which applies supervised sequence prediction to imitate the series of expert computations. This way, procedure cloning learns not only what to do (i.e., the output action), but how and why to do it (i.e., the procedure). Through empirical analysis on navigation, simulated robotic manipulation, and game-playing environments, we show that imitating the intermediate computations of an expert's behavior enables procedure cloning to learn policies exhibiting significant generalization to unseen environment configurations, including those configurations for which running the expert's procedure directly is infeasible. 1

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Evaluating Image-based Robot Manipulationmentioning

confidence: 99%

Section: Evaluating Image-based Robot Manipulationmentioning

confidence: 99%

See 1 more Smart Citation

Chain of Thought Imitation with Procedure Cloning

Yang¹,

Schuurmans²,

Abbeel³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…CaP exhibit a degree of cross-embodiment support [59], [60] by performing the same task differently depending on the action APIs. In the example below, we give Hints of the action APIs, and the resultant plan changes depending on the whether or not the robot is omnidirectional or unidirectional.…”

Section: Cross Embodiment Examplementioning

confidence: 99%

Code as Policies: Language Model Programs for Embodied Control

Liang¹,

Huang²,

Xia³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Large language models (LLMs) trained on codecompletion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these codewriting LLMs can be re-purposed to write robot policy code, given natural language commands. Specifically, policy code can express functions or feedback loops that process perception outputs (e.g., from object detectors [2], [3]) and parameterize control primitive APIs. When provided as input several example language commands (formatted as comments) followed by corresponding policy code (via few-shot prompting), LLMs can take in new commands and autonomously re-compose API calls to generate new policy code respectively. By chaining classic logic structures and referencing third-party libraries (e.g., NumPy, Shapely) to perform arithmetic, LLMs used in this way can write robot policies that (i) exhibit spatial-geometric reasoning, (ii) generalize to new instructions, and (iii) prescribe precise values (e.g., velocities) to ambiguous descriptions ("faster") depending on context (i.e., behavioral commonsense). This paper presents code as policies: a robot-centric formalization of language model generated programs (LMPs) that can represent reactive policies (e.g., impedance controllers), as well as waypoint-based policies (vision-based pick and place, trajectory-based control), demonstrated across multiple real robot platforms. Central to our approach is prompting hierarchical code-gen (recursively defining undefined functions), which can write more complex code and also improves state-of-the-art to solve 39.8% of problems on the HumanEval [1] benchmark. Code and videos are available at https://code-as-policies.github.io # do it again but faster, to the left, and with a banana. while not detect_object("banana"):robot.set_velocity(x=0, y=-0.2, z=0)Representing code as policies inherits a number of benefits from LLMs: not only the capacity to interpret natural language, but also

show abstract

“…Specifically, given a list of obstacle categories described with natural language, we can localize those obstacles at runtime to generate a binary map for collision avoidance and/or shortest path planning. A prominent use case for this is sharing a VLMap of the same environment between different robots with different embodiments (i.e., cross-embodiment problem [36], [37]), which may be useful for multi-agent coordination [38]. For example, a large mobile robot may need to navigate around a table (or other large furniture), while a drone can directly fly over it.…”

Section: Generating Open-vocabulary Obstacle Mapsmentioning

confidence: 99%

Visual Language Maps for Robot Navigation

Huang¹,

Mees²,

Zeng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visuallanguage models pretrained on Internet-scale data (e.g., image captions). While this is useful for matching images to natural language descriptions of object goals, it remains disjoint from the process of mapping the environment, so that it lacks the spatial precision of classic geometric maps. To address this problem, we propose VLMaps, a spatial map representation that directly fuses pretrained visual-language features with a 3D reconstruction of the physical world. VLMaps can be autonomously built from video feed on robots using standard exploration approaches and enables natural language indexing of the map without additional labeled data. Specifically, when combined with large language models (LLMs), VLMaps can be used to (i) translate natural language commands into a sequence of open-vocabulary navigation goals (which, beyond prior work, can be spatial by construction, e.g., "in between the sofa and the TV" or "three meters to the right of the chair") directly localized in the map, and (ii) can be shared among multiple robots with different embodiments to generate new obstacle maps on-the-fly (by using a list of obstacle categories). Extensive experiments carried out in simulated and real-world environments show that VLMaps enable navigation according to more complex language instructions than existing methods. Videos are available at https://vlmaps.github.io.

show abstract

Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning

Cited by 4 publications

References 0 publications

Chain of Thought Imitation with Procedure Cloning

Chain of Thought Imitation with Procedure Cloning

Code as Policies: Language Model Programs for Embodied Control

Visual Language Maps for Robot Navigation

Contact Info

Product

Resources

About