Towards robots conducting chemical experiments

Lisca, Gheorghe; Nyga, Daniel; Balint-Benczedi, Ferenc; Langer, Hagen; Beetz, Michael

doi:10.1109/iros.2015.7354110

Cited by 21 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach is related to a group of studies attempting to bridge the gap between natural language and robotic action (Bollini et al, 2013; Lisca et al, 2015; Misra et al, 2016; Tellex et al, 2011). We, in fact, address a question that is slightly narrower: programming robots by instruction, which puts limitations on the language provided to the robots.…”

Section: Discussionmentioning

confidence: 99%

“…A group of approaches exist for handling strongly taskspecific natural language instructions. One example is presented by Lisca et al (2015), who analyze how to transform natural language instructions for chemical experiments into robot control programs. These authors address much more complicated instructions than those in our study (an example being ''Neutralize 75 ml of hydrochloric acid''.).…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

“…(Note that the SEC in the table is not hand-designed but denotes an objective sequence of touches and un-touches of objects throughout manipulation.) Plans for actual robotic action in Lisca et al (2015) (e.g., aspirating a pipette and pipetting in the ''neutralization'' action) are manually linked during the design of the core structure. Automated inference processes, which are emphasized in this study, are possible only after hand design and learning steps have been accomplished.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

“…To reduce the language-analysis effort, we constrain the instruction language to some degree, specifically requiring instructions to be phrased with an appropriate level of granularity. Language processing is not central to our study and, as a consequence, we differentiate ourselves from that group of existing systems that emphasizes the translation of fully complex natural language, ubiquitous and incomplete, into robotic execution (albeit usually in rather limited domains) (Bollini et al, 2013; Lisca et al, 2015; Misra et al, 2016; Tellex et al, 2011). We use simpler language than is used in these studies, more related to the way one would give an instruction to a child or a “newbie” in a workshop.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cut & recombine: reuse of robot action components based on simple language instructions

Tamošiūnaitė

Aein

Braun

et al. 2019

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

Human beings can generalize from one action to similar ones. Robots cannot do this and progress concerning information transfer between robotic actions is slow. We have designed a system that performs action generalization for manipulation actions in different scenarios. It relies on an action representation for which we perform code-snippet replacement, combining information from different actions to form new ones. The system interprets human instructions via a parser using simplified language. It uses action and object names to index action data tables (ADTs), where execution-relevant information is stored. We have created an ADT database from three different sources (KUKA LWR, UR5, and simulation) and show how a new ADT is generated by cutting and recombining data from existing ADTs. To achieve this, a small set of action templates is used. After parsing a new instruction, index-based searching finds similar ADTs in the database. Then the action template of the new action is matched against the information in the similar ADTs. Code snippets are extracted and ranked according to matching quality. The new ADT is created by concatenating code snippets from best matches. For execution, only coordinate transforms are needed to account for the poses of the objects in the new scene. The system was evaluated, without additional error correction, using 45 unknown objects in 81 new action executions, with 80% success. We then extended the method including more detailed shape information, which further reduced errors. This demonstrates that cut & recombine is a viable approach for action generalization in service robotic applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Comparison With the State Of The Artmentioning

confidence: 99%

Section: Comparison With the State Of The Artmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cut & recombine: reuse of robot action components based on simple language instructions

Tamošiūnaitė

Aein

Braun

et al. 2019

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…Robots are already capable of performing complex manipulations tasks, demonstrated through numerous everyday activities such as towel folding [1], pancake making [2] or pipetting in a chemistry lab [3]. None of these tasks would be possible without a capable perception framework that is able to guide the individual manipulation actions of the robot.…”

Section: Introductionmentioning

confidence: 99%

Amortized Object and Scene Perception for Long-term Robot Manipulation

Balint-Benczedi,

Beetz

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Mobile robots, performing long-term manipulation activities in human environments, have to perceive a wide variety of objects possessing very different visual characteristics and need to reliably keep track of these throughout the execution of a task. In order to be efficient, robot perception capabilities need to go beyond what is currently perceivable and should be able to answer queries about both current and past scenes. In this paper we investigate a perception system for long-term robot manipulation that keeps track of the changing environment and builds a representation of the perceived world. Specifically we introduce an amortized component that spreads perception tasks throughout the execution cycle. The resulting query driven perception system asynchronously integrates results from logged images into a symbolic and numeric (what we call sub-symbolic) representation that forms the perceptual belief state of the robot.

show abstract