Towards a Modular and Distributed End-User Development Framework for Human-Robot Interaction

Coronado, Enrique; Deuff, Dominique; Carreno‐Medrano, Pamela; Tian, Leimin; Kulić, Dana; Sumartojo, Shanti; Mastrogiovanni, Fulvio; Venture, Gentiane

doi:10.1109/access.2021.3051605

Cited by 37 publications

(10 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As described in [37], one of the main characteristics of digitalization is its disruptive quality in areas such as the labor market. An alternative for "disrupting disruption" of AI systems is to empower humans, such as is proposed with the concept of digi-grasping [37], end-user development [38], human-robot collaboration [1] and participatory design [39]. We believe that the design of emergent technologies must put humans at the center.…”

Section: Empower Rather Than Replacementioning

confidence: 99%

NEP+: A Human-Centered Framework for Inclusive Human-Machine Interaction Development

Coronado,

Yamanobe,

Venture

2023

Sensors

Self Cite

View full text Add to dashboard Cite

This article presents the Network Empower and Prototyping Platform (NEP+), a flexible framework purposefully crafted to simplify the process of interactive application development, catering to both technical and non-technical users. The name "NEP+" encapsulates the platform’s dual mission: to empower the network-related capabilities of ZeroMQ and to provide software tools and interfaces for prototyping and integration. NEP+ accomplishes this through a comprehensive quality model and an integrated software ecosystem encompassing middleware, user-friendly graphical interfaces, a command-line tool, and an accessible end-user programming interface. This article primarily focuses on presenting the proposed quality model and software architecture, illustrating how they can empower developers to craft cross-platform, accessible, and user-friendly interfaces for various applications, with a particular emphasis on robotics and the Internet of Things (IoT). Additionally, we provide practical insights into the applicability of NEP+ by briefly presenting real-world user cases where human-centered projects have successfully utilized NEP+ to develop robotics systems. To further emphasize the suitability of NEP+ tools and interfaces for developer use, we conduct a pilot study that delves into usability and workload assessment. The outcomes of this study highlight the user-friendly features of NEP+ tools, along with their ease of adoption and cross-platform capabilities. The novelty of NEP+ fundamentally lies in its holistic approach, acting as a bridge across diverse user groups, fostering inclusivity, and promoting collaboration.

show abstract

Section: Empower Rather Than Replacementioning

confidence: 99%

NEP+: A Human-Centered Framework for Inclusive Human-Machine Interaction Development

Coronado,

Yamanobe,

Venture

2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a collaborative industrial scenario, an operator can communicate their intentions to the robot, that keeps track of the actions executed by the human and focuses on the remaining ones. Finally, [20] proposes OpenRIZE, a GUI that allows users to program robot interaction behaviors for HRI applications. The execution layer builds on BTs.…”

Section: Related Workmentioning

confidence: 99%

Interactive Disambiguation for Behavior Tree Execution

Iovino¹,

Doğan²,

Leite³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, robots are used in an increasing variety of tasks, especially by small-and medium sized enterprises. These tasks are usually fast-changing, they have a collaborative scenario and happen in unpredictable environments with possible ambiguities. It is important to have methods capable of generating robot programs easily, that are made as general as possible by handling uncertainties. We present a system that integrates a method to learn Behavior Trees (BTs) from demonstration for pick and place tasks, with a framework that uses verbal interaction to ask followup clarification questions to resolve ambiguities. During the execution of a task, the system asks for user input when there is need to disambiguate an object in the scene, when the targets of the task are objects of a same type that are present in multiple instances. The integrated system is demonstrated on different scenarios of a pick and place task, with increasing level of ambiguities. The code used for this paper is made publicly available 1 .

show abstract

“…Studying human-machine interaction in industrial environments, a lot of research on the robot's perception and hardware for the recognition of human activities emerged [10], [11]. Being an important basis for human-machine interaction success, emotion recognition is also a popular field of exploration.…”

Section: Related Workmentioning

confidence: 99%

Monophonic Music Generation With a Given Emotion Using Conditional Variational Autoencoder

Grekow

Dimitrova-Grekow

2021

IEEE Access

View full text Add to dashboard Cite

The rapid increase in the importance of human-machine interaction and the accelerating pace of life pose various challenges for the creators of digital environments. Continuous improvement of human-machine interaction requires precise modeling of the physical and emotional state of people. By implementing emotional intelligence in machines, robots are expected not only to recognize and track emotions when interacting with humans, but also to respond and behave appropriately. The machine should match its reaction to the mood of the user as precisely as possible. Music generation with a given emotion can be a good start to fulfilling such a requirement. This article presents the process of building a system generating music content of a specified emotion. As the emotion labels, four basic emotions: happy, angry, sad, relaxed, corresponding to the four quarters of Russell's model, were used. Conditional variational autoencoder using a recurrent neural network for sequence processing was used as a generative model. The obtained results in the form of the generated music examples with a specific emotion are convincing in their structure and sound. The generated examples were evaluated with two methods, in the first using metrics for comparison with the training set and in the second using expert annotation.

show abstract

Towards a Modular and Distributed End-User Development Framework for Human-Robot Interaction

Cited by 37 publications

References 55 publications

NEP+: A Human-Centered Framework for Inclusive Human-Machine Interaction Development

NEP+: A Human-Centered Framework for Inclusive Human-Machine Interaction Development

Interactive Disambiguation for Behavior Tree Execution

Monophonic Music Generation With a Given Emotion Using Conditional Variational Autoencoder

Contact Info

Product

Resources

About