Crowdsourced Action-Model Acquisition for Planning

Zhuo, Hankz Hankui

doi:10.1609/aaai.v29i1.9667

Cited by 13 publications

(4 citation statements)

References 21 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the action model acquisition part has been evaluated in the context of CAMA already (Zhuo 2015), here we focus on evaluating the initial state acquisition part. We evaluated our approach in three planning domains, i.e., blocks 1 , depots 2 and driverlog 4 .…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Acquiring Planning Knowledge via Crowdsourcing

Gao

Zhuo

Kambhampati

et al. 2015

HCOMP

View full text Add to dashboard Cite

Plan synthesis often requires complete domain models and initial states as input. In many real world applications, it is difficult to build domain models and provide complete initial state beforehand. In this paper we propose to turn to the crowd for help before planning. We assume there are annotators available to provide information needed for building domain models and initial states. However, there might be a substantial amount of discrepancy within the inputs from the crowd. It is thus challenging to address the planning problem with possibly noisy information provided by the crowd. We address the problem by two phases. We first build a set of Human Intelligence Tasks (HITs), and collect values from the crowd. We then estimate the actual values of variables and feed the values to a planner to solve the problem.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Building HITs for Action Models: For this part, we build on our work with the CAMA system (Zhuo 2015). We enumerate all possible preconditions and effects for each action.…”

Section: The Pan-crowd Approachmentioning

confidence: 99%

Acquiring Planning Knowledge via Crowdsourcing

Gao

Zhuo

Kambhampati

et al. 2015

HCOMP

View full text Add to dashboard Cite

show abstract

“…It requires the same observability of ARMS. Other approaches like AMAN [34] and RIM [35] include noisy observations and incomplete models, respectively, which means that observations of plans may be captured by imperfect sensors, thus introducing uncertainty under a full degree of observability. LOCM [36,37] initiated a family of inductive learning systems that use Finite State Machines to learn even from null state information, i.e., without the need of initial, goal or intermediate states.…”

Section: Related Workmentioning

confidence: 99%

Learning temporal action models from multiple plans: A constraint satisfaction approach

Garrido

2022

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

“…Most approaches that learn complete models handle deterministic tasks, and although they can tackle partial observability (Mourão et al, 2012;Zhuo and Kambhampati, 2013) or apply transfer learning (Zhuo and Yang, 2014), they do not consider uncertain effects. In this work we focus on models with uncertain effects.The most similar approaches to ours are those that learn relational action models with uncertain effects (Pasula et al, 2007;Deshpande et al, 2007;Mourão, 2014).…”

Section: Previous Workmentioning

confidence: 99%

Learning relational models with human interaction for planning in robotics

Martínez Martínez

View full text Add to dashboard Cite

Automated planning has proven to be useful to solve problems where an agent has to maximize a reward function by executing actions. As planners have been improved to salve more expressive and difficult problems, there is an increasing interest in using planning to improve efficiency in robotic tasks. However, planners rely on a domain model, which has to be either handcrafted or learned. Although learning domain models can be very costly, recent approaches provide generalization capabilities and integrate human feedback to reduce the amount of experiences required to learn. In this thesis we propase new methods that allow an agent with no previous knowledge to solve certain problems more efficiently by using task planning. First, we show how to apply probabilistic planning to improve robot performance in manipulation tasks (such as cleaning the dirt or clearing the tableware on a table). Planners obtain sequences of actions that get the best result in the long term, beating reactive strategies. Second, we introduce new reinforcement learning algorithms where the agent can actively request demonstrations from a teacher to learn new actions and speed up the learning process. In particular, we propase an algorithm that allows the user to set the mínimum quality to be achieved, where a better quality also implies that a larger number of demonstrations will be requested . Moreover, the learned model is analyzed to extract the unlearned or problematic parts of the model. This information allow the agent to provide guidance to the teacher when a demonstration is requested, and to avoid irrecoverable errors. Finally, a new domain model learner is introduced that, in addition to relational probabilistic action models, can also learn exogenous effects. This learner can be integrated with existing planners and reinforcement learning algorithms to salve a wide range of problems. In summary, we improve the use of learning and task planning to salve unknown tasks. The improvements allow an agent to obtain a larger benefit from planners, learn faster, balance the number of action executions and teacher demonstrations, avoid irrecoverable errors, interact with a teacher to solve difficult problems, and adapt to the behavior of other agents by learning their dynamics. All the proposed methods were compared with state-of-the-art approaches, and were also demonstrated in different scenarios, including challenging robotic tasks. La planificación automática ha probado ser de gran utilidad para resolver problemas en los que un agente tiene que ejecutar acciones para maximizar una función de recompensa. A medida que los planificadores han sido capaces de resolver problemas cada vez más complejos, ha habido un creciente interés por utilizar dichos planificadores para mejorar la eficiencia de tareas robóticas. Sin embargo, los planificadores requieren un modelo del dominio, el cual puede ser creado a mano o aprendido. Aunque aprender modelos automáticamente puede ser costoso, recientemente han aparecido métodos que permiten la interacción persona-máquina y generalizan el conocimiento para reducir la cantidad de experiencias requeridas para aprender. En esta tesis proponemos nuevos métodos que permiten a un agente sin conocimiento previo de la tarea resolver problemas de forma más eficiente mediante el uso de planificación automática. Comenzaremos mostrando cómo aplicar planificación probabilística para mejorar la eficiencia de robots en tareas de manipulación (como limpiar suciedad o recoger una mesa). Los planificadores son capaces de obtener las secuencias de acciones que producen los mejores resultados a largo plazo, superando a las estrategias reactivas. Por otro lado, presentamos nuevos algoritmos de aprendizaje por refuerzo en los que el agente puede solicitar demostraciones a un profesor. Dichas demostraciones permiten al agente acelerar el aprendizaje o aprender nuevas acciones. En particular, proponemos un algoritmo que permite al usuario establecer la mínima suma de recompensas que es aceptable obtener, donde una recompensa más alta implica que se requerirán más demostraciones. Además, el modelo aprendido será analizado para identificar qué partes están incompletas o son problemáticas. Esta información permitirá al agente evitar errores irrecuperables y también guiar al profesor cuando se solicite una demostración. Finalmente, se ha introducido un nuevo método de aprendizaje para modelos de dominios que, además de obtener modelos relacionales de acciones probabilísticas, también puede aprender efectos exógenos. Mostraremos cómo integrar este método en algoritmos de aprendizaje por refuerzo para poder abordar una mayor cantidad de problemas. En resumen, hemos mejorado el uso de técnicas de aprendizaje y planificación para resolver tareas desconocidas a priori. Estas mejoras permiten a un agente aprovechar mejor los planificadores, aprender más rápido, elegir entre reducir el número de acciones ejecutadas o el número de demostraciones solicitadas, evitar errores irrecuperables, interactuar con un profesor para resolver problemas complejos, y adaptarse al comportamiento de otros agentes aprendiendo sus dinámicas. Todos los métodos propuestos han sido comparados con trabajos del estado del arte, y han sido evaluados en distintos escenarios, incluyendo tareas robóticas.

show abstract

Crowdsourced Action-Model Acquisition for Planning

Cited by 13 publications

References 21 publications

Acquiring Planning Knowledge via Crowdsourcing

Acquiring Planning Knowledge via Crowdsourcing

Learning temporal action models from multiple plans: A constraint satisfaction approach

Learning relational models with human interaction for planning in robotics

Contact Info

Product

Resources

About