Object Shape Classification Using Spatial Information in Myoelectric Prosthetic Control

Shima, Ryusei; He, Yunan; Fukuda, Osamu; Bu, Nan; Okumura, Hiroshi; Yamaguchi, Nobuhiko

doi:10.15344/2456-4451/2018/130

Cited by 4 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These techniques have been applied to the control system of prosthetic hands to determine the grasp pattern and grasp timing [14] [18]. For example, in the study of [19], Shima et al attempted to recognize the shape of an object using RGB-D sensor to improve classification accuracy. To control the prosthetic hands in the whole approaching phase, He et al introduced a real-time object detection system and implemented the system in a server-client mode to improve the computation capability.…”

Section: Related Workmentioning

confidence: 99%

Vision-Based Assistance for Myoelectric Hand Control

Kubozono

Fukuda

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Conventional control systems for prosthetic hands use myoelectric signals as an interface, but it is impossible to realize complex and flexible human hand movements with only myoelectric signals. A promising control scheme for prosthetic hands uses computer vision to assist in grasping objects. It features an imaging sensor, and the control system is capable of recognizing an object placed in the environment. Then, a gripping pattern can be selected from some predefined candidates according the recognized object. However, previous studies assumed that only one object exists in the environment. If there are multiple target objects in the environment, the hand could become confused in attempting to find the target object. This study addresses this problem and proposes a method to determine the target object from multiple objects. The proposed method is able to determine the target object by estimating the positional relationship between the artificial hand and the objects, as well as the motion of the hand. To verify the validity and effectiveness, we implemented the proposed method in a vision-based prosthetic hands control system and conducted pick-and-place experiments. The experiments confirm that the proposed method can accurately estimate the target object in accordance with the user's intention.

show abstract

Section: Related Workmentioning

confidence: 99%

Vision-Based Assistance for Myoelectric Hand Control

Kubozono

Fukuda

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Bando et al [11] used a convolutional neural network (CNN 1 ) to classify 20 classes of objects, the classification results help to select the grasp posture from a group of predefined postures. Shima et al [12] takes advantage of object spatial information measured by depth sensor and classifies the objects in terms of their shapes. The shape of the object finally results in the grasp posture.…”

Section: Introductionmentioning

confidence: 99%

Noble Control Scheme for Prosthetic Hands through Spatial Understanding

He¹,

Fukuda²,

Yamaguchi³

et al. 2020

IJACSA

Self Cite

View full text Add to dashboard Cite

A noble control scheme for prosthetic hands through spatial understanding is proposed. The proposed control scheme features an imaging sensor and an inertial measurement unit (IMU) sensor, which makes prosthetic hands capable of visual and motion sensing. The imaging sensor captures the scene where the user is willing to grasp the object. The control system recognizes the target object, extracts its surface features and estimates its pose from the captured images. Then the spatial relationship is constructed between the hand and the target object. With the help of IMU sensors, the relationship can be tracked and kept wherever the prosthetic hand moves even the object is out of the view range of the camera. To interact with the user, this process is visualized using augmented reality (AR) technology. A test platform based on the proposed control scheme is developed and a case study is performed with the platform.

show abstract

“…Shima et al combined a depth sensor with an RGB camera to acquire the spatial information of the object. The spatial information are fed into a CNN for shape classification [14]. This study achieved a higher classification accuracy by feeding depth information as auxiliary input to the CNN network.…”

Section: Introductionmentioning

confidence: 99%

Development of Distributed Control System for Vision-Based Myoelectric Prosthetic Hand

Shima

Fukuda

et al. 2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

Vision-based myoelectric prosthetic hand uses a camera integrated into its body for object detection and environment understanding, where the results provide necessary information for grasp planning. It is expected that the semi-automatic prosthesis control can be realized with this method. However, such a control method usually suffers from heavy computation due to the requirement of real-time image processing to keep up with the arm movements of the user. This paper presents a distributed control system that assigns heavy processing tasks to one or multiple processing nodes through the network, which greatly reduces the computation burdens of the processor embedded in the prosthetic hand. In this control scheme, the embedded system in the prosthetic hand is only used for gathering necessary data for grasp planning, while the processing nodes in the network are responsible for processing and managing the collected data. A test platform is built to verify the proposed control scheme. The test platform streams user electromyography (EMG) signals and images simultaneously to the GPU server. The GPU sever analyzes the received data and generates the corresponding motor commands in real time. A case study that uses a 3-DoF gripper to continuously grasp several objects is performed using this test platform.INDEX TERMS Distributed control system, real-time object detection, vision-based myoelectric hand.

show abstract

Object Shape Classification Using Spatial Information in Myoelectric Prosthetic Control

Cited by 4 publications

References 21 publications

Vision-Based Assistance for Myoelectric Hand Control

Vision-Based Assistance for Myoelectric Hand Control

Noble Control Scheme for Prosthetic Hands through Spatial Understanding

Development of Distributed Control System for Vision-Based Myoelectric Prosthetic Hand

Contact Info

Product

Resources

About