Vision-based Automatic Control of a 5-Fingered Assistive Robotic Manipulator for Activities of Daily Living

Wang, Chen; Freer, Daniel; Liu, Jindong; Yang, Guang‐Zhong

doi:10.1109/iros40897.2019.8967863

Cited by 6 publications

(11 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, there is also a wide applicability of DL techniques in the IoT-Healthcare use cases and Wearables, for instance, to derive short-term and long-term health predictions. • Robotics: In robotics, DNNs served in a wide range of use cases like autonomous vehicles [14], humanoid robots [15], assistive robots [16], swarms [17], and drone control system [18]. • Smart Energy Management: DL can also be used to preserve valuable resources such as electricity.…”

Section: Introductionmentioning

confidence: 99%

Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

et al. 2020

View full text Add to dashboard Cite

Currently, Machine Learning (ML) is becoming ubiquitous in everyday life. Deep Learning (DL) is already present in many applications ranging from computer vision for medicine to autonomous driving of modern cars as well as other sectors in security, healthcare, and finance. However, to achieve impressive performance, these algorithms employ very deep networks, requiring a significant computational power, both during the training and inference time. A single inference of a DL model may require billions of multiply-and-accumulated operations, making the DL extremely compute-and energy-hungry. In a scenario where several sophisticated algorithms need to be executed with limited energy and low latency, the need for cost-effective hardware platforms capable of implementing energy-efficient DL execution arises. This paper first introduces the key properties of two brain-inspired models like Deep Neural Network (DNN), and Spiking Neural Network (SNN), and then analyzes techniques to produce efficient and high-performance designs. This work summarizes and compares the works for four leading platforms for the execution of algorithms such as CPU, GPU, FPGA and ASIC describing the main solutions of the state-of-the-art, giving much prominence to the last two solutions since they offer greater design flexibility and bear the potential of high energy-efficiency, especially for the inference process. In addition to hardware solutions, this paper discusses some of the important security issues that these DNN and SNN models may have during their execution, and offers a comprehensive section on benchmarking, explaining how to assess the quality of different networks and hardware systems designed for them.

show abstract

Section: Introductionmentioning

confidence: 99%

Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The performance of these algorithms ranges from 77.8% to 99.05%. Regarding speed, it is shown that some algorithms were tested without considering the execution time [ 86 , 88 ], one algorithm is reported to run at a speed of 21.62 FPS [ 99 ], and the rest run in real time. Regarding hardware, the algorithms for object recognition were tested on robotic platforms such as the ARMAR-III robot [ 82 ] or the Jaco robot [ 93 , 95 , 100 ], to mention a few.…”

Section: Discussion and Conclusionmentioning

confidence: 99%

“…Then, the most representative points of each object are extracted to store them in a database and use to recognize places in a home. On the contrary, a simulated option is presented by Wang et al in [ 99 ] for an assistive robotic arm capable of helping disabled people to pick up objects on the floor. The authors implemented an RGB-D camera to extract cloud points from the scene.…”

Section: Algorithms Used For Objectsmentioning

confidence: 99%

Artificial Vision Algorithms for Socially Assistive Robot Applications: A Review of the Literature

Montaño-Serrano

Jacinto‐Villegas

Vilchis-González

et al. 2021

Sensors

View full text Add to dashboard Cite

Today, computer vision algorithms are very important for different fields and applications, such as closed-circuit television security, health status monitoring, and recognizing a specific person or object and robotics. Regarding this topic, the present paper deals with a recent review of the literature on computer vision algorithms (recognition and tracking of faces, bodies, and objects) oriented towards socially assistive robot applications. The performance, frames per second (FPS) processing speed, and hardware implemented to run the algorithms are highlighted by comparing the available solutions. Moreover, this paper provides general information for researchers interested in knowing which vision algorithms are available, enabling them to select the one that is most suitable to include in their robotic system applications.

show abstract

“…The CV system was additionally used just before state 4, in which a depth image of the object of interest was taken, and the shape of that object was determined via a 5-layer Deep Convolutional Neural Network (DCNN). In the future, this shape determination step could be used to determine grasp type for the end-effector [18] or determine the final autonomous grasping strategy. The DCNN was trained on about 500 depth images collected in the simulator with the robot in this intermediate state, consistently achieving over 90% accuracy on the validation set.…”

Section: Computer Vision Systemmentioning

confidence: 99%

“…While the visual control system presented here needs improvements to robustly work in a real-world environment, it was adequate to test our current framework for BCI control in simulation. However, the entire system is largely modular, so it should be trivial to upgrade it to include concepts such as classification and localization using deep learning [18], though speed may be an important factor with more complex architectures. Additionally, as the robot autonomously completes the grasp from step 4 onward, other more complex grasping techniques could be implemented that have been learned through techniques such as deep reinforcement learning [19], [20], rather than just tuning a few parameters.…”

Section: Computer Vision Systemmentioning

confidence: 99%

MIndGrasp: A New Training and Testing Framework for Motor Imagery Based 3-Dimensional Assistive Robotic Control

Freer,

Yang

2020

Preprint

Self Cite

View full text Add to dashboard Cite

With increasing global age and disability assistive robots are becoming more necessary, and brain computer interfaces (BCI) are often proposed as a solution to understanding the intent of a disabled person that needs assistance. Most frameworks for electroencephalography (EEG)-based motor imagery (MI) BCI control rely on the direct control of the robot in Cartesian space. However, for 3-dimensional movement, this requires 6 motor imagery classes, which is a difficult distinction even for more experienced BCI users. In this paper, we present a simulated training and testing framework which reduces the number of motor imagery classes to 4 while still grasping objects in three-dimensional space. This is achieved through semiautonomous eye-in-hand vision-based control of the robotic arm, while the user-controlled BCI achieves movement to the left and right, as well as movement toward and away from the object of interest. Additionally, the framework includes a method of training a BCI directly on the assistive robotic system, which should be more easily transferrable to a realworld assistive robot than using a standard training protocol such as Graz-BCI. Presented results do not consider real human EEG data, but are rather shown as a baseline for comparison with future human data and other improvements on the system.

show abstract

Vision-based Automatic Control of a 5-Fingered Assistive Robotic Manipulator for Activities of Daily Living

Cited by 6 publications

References 20 publications

Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

Artificial Vision Algorithms for Socially Assistive Robot Applications: A Review of the Literature

MIndGrasp: A New Training and Testing Framework for Motor Imagery Based 3-Dimensional Assistive Robotic Control

Contact Info

Product

Resources

About