Classifier for the functional state of the respiratory system via descriptors determined by using multimodal technology

Filist, S. A.; Al-Kasasbeh, Riad Taha; Шаталова, О. В.; Aikeyeva, Altyn Amanzholovna; Al-Habahbeh, Osama M.; Alshamasin, Mahdi Salman; Alekseevich, Korenevskiy Nikolay; Khrisat, Mohammad; Myasnyankin, M B; Maksim, Ilyash

doi:10.1080/10255842.2022.2117551

Cited by 8 publications

(1 citation statement)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous works, we have introduced a Multimodal Interaction Framework based on Firebase Realtime Database [2] and a Biomedical Framework for Enhanced Experimentation -BIOFEE [3]. The interest of such intelligent systems built on a multimodal basis lies in the fact that a decision is made through several independent information channels, with the subsequent aggregation of these decisions [4].…”

Section: Introductionmentioning

confidence: 99%

Multimodal Interaction for Cobot Using MQTT

Rouillard

Vannobel

2023

MTI

View full text Add to dashboard Cite

For greater efficiency, human–machine and human–robot interactions must be designed with the idea of multimodality in mind. To allow the use of several interaction modalities, such as the use of voice, touch, gaze tracking, on several different devices (computer, smartphone, tablets, etc.) and to integrate possible connected objects, it is necessary to have an effective and secure means of communication between the different parts of the system. This is even more important with the use of a collaborative robot (cobot) sharing the same space and very close to the human during their tasks. This study present research work in the field of multimodal interaction for a cobot using the MQTT protocol, in virtual (Webots) and real worlds (ESP microcontrollers, Arduino, IOT2040). We show how MQTT can be used efficiently, with a common publish/subscribe mechanism for several entities of the system, in order to interact with connected objects (like LEDs and conveyor belts), robotic arms (like the Ned Niryo), or mobile robots. We compare the use of MQTT with that of the Firebase Realtime Database used in several of our previous research works. We show how a “pick–wait–choose–and place” task can be carried out jointly by a cobot and a human, and what this implies in terms of communication and ergonomic rules, via health or industrial concerns (people with disabilities, and teleoperation).

show abstract

Section: Introductionmentioning

confidence: 99%