On the Implementation of a Low-Cost Mind-Voice-and-Gesture-Controlled Humanoid Robotic Arm Using Leap Motion and Neurosky Sensor

Pham, Quan K.; Vo, T.V.; Tran, Phuong T.

doi:10.1007/s42835-021-00903-5

Cited by 6 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…8, installed on the remote control, ESP32-Master firmware combined button control and SCR into one framework -with subprograms, headers, and supporting libraries, namely, ESP NOW master register peer & ESP NOW slave callback, Freertos Xtask with semaphore, pulse width modulation (PWM) servo, analog to digital (ADC) compiler, The ESP32-Slave, as shown in Fig. 9, was mounted on the servo controller in this case as a substitution of flexion and extension movement patterns for the Exo-glove poly using a hand prosthetic and servo mechanism from proto 1 [95], using underactuated pulley mechanism with DC servo as artificial MCP joint or flexion and extension to articulate finger movement which has similar as exo-glove poly servo mechanism [96,97,98,99]. The final firmware will be built with C++ programming with a .cpp extension like a pre-trained model design.…”

Section: G Final Embedded Tinyml Master-slavementioning

confidence: 99%

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

Riyanta,

Irianta,

Kamiel

2023

Journal of Robotics and Control

View full text Add to dashboard Cite

Post-stroke dysarthria (PSD) is a widespread outcome of a stroke. To help in the objective evaluation of dysarthria, the development of pathological voice recognition and technology has a lot of attention. Soft robotics therapy devices have been received as an alternative rehabilitation and hand grasp assistance for improving activity daily living (ADL). Despite the significant progress in this field, most soft robotic therapy devices use a complex, bulky, lack of pathological voice recognition model, large computational power, and stationary controller. This study aims to develop a portable wirelessly multi-controller with a simulated dysarthric vowel speech in Bahasa Indonesia and non-dysarthric micro speech recognition, using tiny machine learning (TinyMl) system for hardware efficiency. The speech interface using INMP441, compute with a lightweight Deep Convolutional Neural network (DCNN) design and embedded into ESP-32. Feature model using Short Time Fourier Transform (STFT) and fed into CNN. This method has proven useful in micro-speech recognition with low computational power in both speech scenarios with a level of accuracy above 90%. Realtime inference performance on ESP-32 using hand prosthetics, with 3-level household noise intensity respectively 24db,42db, and 62db, and has respectively resulted from 95%, 85%, and 50% Accuracy. Wireless connectivity success rate with both controllers is around 0.2 - 0.5 ms.

show abstract

Section: G Final Embedded Tinyml Master-slavementioning

confidence: 99%

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

Riyanta,

Irianta,

Kamiel

2023

Journal of Robotics and Control

View full text Add to dashboard Cite

show abstract

“…In those works, an open source CAD design of a hand was printed and a hybrid control (voice-myoelectric) was used to achieve the individual opening and closing of the thumb and the remaining four fingers, however, the applied ON-OFF type control limits the functionality of the robotic hand. In [17] a hand controlled by various types of signals is presented, these are encephalographic, myoelectric, voice and gestures, the project provides a comprehensive explanation of the methods used to acquire the different signals; however, to move the hand, a control law is not used, that is, it is an open loop system. In [18], [19], it was also observed that they use myoelectric sensors as triggers of the control action.…”

Section: Introductionmentioning

confidence: 99%

Optimal Control and Grasping for a Robotic Hand with a Non-linked Double Tendon Arrangement

Sánchez-Garnica,

Rodríguez-Guerrero,

Ortega-Palacios

et al. 2024

IEEE Latin Am. Trans.

View full text Add to dashboard Cite

After comparing different robotic hand projects, a problem is identified: when a finger has a degree of freedom, the hand is unable to grasp irregularly shaped objects. This article proposes a solution. The use of a non-linked doubletendon arrangement in the fingers allows them to have free movement; coupled with the use of Inertial Measurement Units to determine its position, ensures that, despite having one degree of freedom per finger, the hand can effectively grasp irregular objects. Additionally, a web application is developed to control hand movements through voice commands. Finally, due to the necessity for these types of devices to be mobile, an optimal control law is used to minimize energy consumption, thereby increasing autonomy when the hand is powered by batteries. As an additional note, the conducted experiments reveal that the movement of all fingers occurs simultaneously, demonstrating that parallel multitasking programming techniques effectively fulfill that purpose.

show abstract

Application of the MQTT Protocol for the Control of a Scorbot Robot by Means of EGG Electroencephalographic Signals

Salazar

Guamán-Molina

Saltos

et al. 2023

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

On the Implementation of a Low-Cost Mind-Voice-and-Gesture-Controlled Humanoid Robotic Arm Using Leap Motion and Neurosky Sensor

Cited by 6 publications

References 28 publications

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

Optimal Control and Grasping for a Robotic Hand with a Non-linked Double Tendon Arrangement

Application of the MQTT Protocol for the Control of a Scorbot Robot by Means of EGG Electroencephalographic Signals

Contact Info

Product

Resources

About