Multimodal Reference Resolution In Collaborative Assembly Tasks

Kontogiorgos, Dimosthenis; Sibirtseva, Elena; Pereira, Ana Isabel; Skantze, Gabriel; Gustafson, Joakim

doi:10.1145/3279972.3279976

Cited by 8 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Basing prediction on the longest fixation in continuous modalities such as pointing and gaze [9] are a common way to model them. However, as was shown in our previous study [10], when the complexity of the task is increased, nosiness of these modalities increases accordingly to such an extent that it becomes nearly impossible to make a prediction solely from the longest fixation.…”

Section: Related Worksupporting

confidence: 57%

“…The next step for this project is to test how scalable our approach is to more complex tasks. Our initial motivation to explore temporal dependencies comes from our previous work [10], where participants where building furniture together. The main challenge there came from the nosiness of input modalities.…”

Section: Discussionmentioning

confidence: 99%

“…However, in non-guided natural human-robot interaction this approach has its own limitations. Our previous human study [10] showed that participants often look at each other more than at the target object or spend more time looking at the next object in the sequence while still describing the previous one. This lead us to look for more high level behaviour patterns that consist of events happening in all the modalities.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Sibirtseva

Ghadirzadeh

Leite

et al. 2019

Virtual, Augmented and Mixed Reality. Applications and Case Studies

Self Cite

View full text Add to dashboard Cite

In collaborative tasks, people rely both on verbal and nonverbal cues simultaneously to communicate with each other. For humanrobot interaction to run smoothly and naturally, a robot should be equipped with the ability to robustly disambiguate referring expressions. In this work, we propose a model that can disambiguate multimodal fetching requests using modalities such as head movements, hand gestures, and speech. We analysed the acquired data from mixed reality experiments and formulated a hypothesis that modelling temporal dependencies of events in these three modalities increases the model's predictive power. We evaluated our model on a Bayesian framework to interpret referring expressions with and without exploiting the temporal prior.

show abstract

Section: Related Worksupporting

confidence: 57%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Sibirtseva

Ghadirzadeh

Leite

et al. 2019

Virtual, Augmented and Mixed Reality. Applications and Case Studies

Self Cite

View full text Add to dashboard Cite

show abstract

“…A library of 3D models is developed taking into account the modularity and scalability of components of furniture sets. Human multimodal cues for furniture assembly where analyzed in order to find the most informative so as to enhance the communicational and social skill of collaborative robots (Kontogiorgos et al, 2018).…”

Section: Collaborative Robots Complex Robotic Cells Interconnection and Safety Issuesmentioning

confidence: 99%

Technology enablers for the implementation of Industry 4.0 to traditional manufacturing sectors: A review

Jimeno-Morenilla

Azariadis

Molina-Carmona

et al. 2021

Computers in Industry

View full text Add to dashboard Cite

The traditional manufacturing sectors (footwear, textiles and clothing, furniture and toys, among others) are based on small and medium enterprises with limited capacity on investing in modern production technologies. Although these sectors rely heavily on product customization and short manufacturing cycles, they are still not able to take full advantage of the fourth industrial revolution. Industry 4.0 surfaced to address the current challenges of shorter product life-cycles, highly customized products and stiff global competition. The new manufacturing paradigm supports the development of modular factory structures within a computerized Internet of Things environment. With Industry 4.0, rigid planning and production processes can be revolutionized. However, the computerization of manufacturing has a high degree of complexity and its implementation tends to be expensive, which goes against the reality of SMEs that power the traditional sectors. This paper reviews the main scientific-technological advances that have been developed in recent years in traditional sectors with the aim of facilitating the transition to the new industry standard.

show abstract

The Role of Multimodal Data for Modeling Communication in Artificial Social Agents

Gross

Krenn

2023

Handbook of Human‐Machine Systems

View full text Add to dashboard Cite

Multimodal Reference Resolution In Collaborative Assembly Tasks

Cited by 8 publications

References 15 publications

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality

Technology enablers for the implementation of Industry 4.0 to traditional manufacturing sectors: A review

The Role of Multimodal Data for Modeling Communication in Artificial Social Agents

Contact Info

Product

Resources

About