Eye-in-Hand Visual Servoing Enhanced With Sparse Strain Measurement for Soft Continuum Robots

Wang, Xiaomei; Fang, Ge; Wang, Kui; Xie, Xiaochen; Lee, Kit-Hang; Ho, Justin; Tang, Wai Lun; Lam, James; Kwok, Ka‐Wai

doi:10.1109/lra.2020.2969953

Cited by 62 publications

(43 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, for such schemes like the ones in the studies by Yip and Camarillo (2014), Lee et al (2017a), the question of how to characterize and resolve the outliers with low confidence of the ground truth needs to be carefully considered. Although excessive addition of sensors is not recommended, several self-contained measurement devices such as fiber Bragg gratings (FBGs) which can appropriately accommodate the flexible continuum body have the potential for sensor fusion with positional sensors (Lun et al, 2019;Wang et al, 2020;Wang et al, 2021). Meanwhile, the combination of analytical and data-driven models will be another trend for continuum robot control.…”

Section: Discussionmentioning

confidence: 99%

A Survey for Machine Learning-Based Control of Continuum Robots

Wang

Kwok

2021

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Soft continuum robots have been accepted as a promising category of biomedical robots, accredited to the robots’ inherent compliance that makes them safely interact with their surroundings. In its application of minimally invasive surgery, such a continuum concept shares the same view of robotization for conventional endoscopy/laparoscopy. Different from rigid-link robots with accurate analytical kinematics/dynamics, soft robots encounter modeling uncertainties due to intrinsic and extrinsic factors, which would deteriorate the model-based control performances. However, the trade-off between flexibility and controllability of soft manipulators may not be readily optimized but would be demanded for specific kinds of modeling approaches. To this end, data-driven modeling strategies making use of machine learning algorithms would be an encouraging way out for the control of soft continuum robots. In this article, we attempt to overview the current state of kinematic/dynamic model-free control schemes for continuum manipulators, particularly by learning-based means, and discuss their similarities and differences. Perspectives and trends in the development of new control methods are also investigated through the review of existing limitations and challenges.

show abstract

Section: Discussionmentioning

confidence: 99%

A Survey for Machine Learning-Based Control of Continuum Robots

Wang

Kwok

2021

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…The eye-in-hand configuration has several advantages as it enables more flexible and precise viewing of target objects in the workspace, thus augmenting the versatility and accuracy of robotic manipulation [32]. However, as eye-in-hand camera moves with the robot, conventional frame-based cameras suffer from motion blur, which imposes constraints on ambient illumination and maximum operational speeds [20], [21]. event-based vision has the potential to address these challenges in conventional robotic visual servoing.…”

Section: A Related Workmentioning

confidence: 99%

“…In conventional visual servoing, frame-based cameras are mainly used to detect, track and match visual features by processing intensity images at consecutive frames; which in practice, causes delays in visual processing and the consequent robot action. Moreover, they face issues of motion blur and often require increased ambient illumination [20], [21]; which undermines their capabilities in high speed operations and varying light conditions. These shortcomings of frame-based cameras can limit their usage/applicability for visual servoing in both structured and unstructured environments.…”

Section: Introductionmentioning

confidence: 99%

Neuromorphic Eye-in-Hand Visual Servoing

et al. 2021

View full text Add to dashboard Cite

Robotic vision plays a major role in factory automation to service robot applications. However, the traditional use of frame-based cameras sets a limitation on continuous visual feedback due to their low sampling rate, poor performance in low light conditions and redundant data in real-time image processing, especially in the case of high-speed tasks. Neuromorphic event-based vision is a recent technology that gives human-like vision capabilities such as observing the dynamic changes asynchronously at a high temporal resolution (1µs) with low latency and wide dynamic range. In this paper, for the first time, we present a purely event-based visual servoing method using a neuromorphic camera in an eye-in-hand configuration for the grasping pipeline of a robotic manipulator. We devise three surface layers of active events to directly process the incoming stream of events from relative motion. A purely event-based approach is used to detect corner features, localize them robustly using heatmaps and generate virtual features for tracking and grasp alignment. Based on the visual feedback, the motion of the robot is controlled to make the temporal upcoming event features converge to the desired event in Spatiotemporal space. The controller switches its operation such that it explores the workspace, reaches the target object and achieves a stable grasp. The event-based visual servoing (EBVS) method is comprehensively studied and validated experimentally using a commercial robot manipulator in an eye-in-hand configuration for both static and dynamic targets. Experimental results show superior performance of the EBVS method over frame-based vision, especially in high-speed operations and poor lighting conditions. As such, EBVS overcomes the issues of motion blur, lighting and exposure timing that exist in conventional frame-based visual servoing methods.

show abstract

“…These special electronic image sensors can be used to simulate flexible visual systems. [35][36][37] Furthermore, flexible photodetectors are the basis of newer bionic vision applications. The use of flexible electronic technology to simulate biological photoreceptor arrays, can help simulate the applications of bionic vision.…”

Section: Introductionmentioning

confidence: 99%

Flexible Image Sensors with Semiconducting Nanowires for Biomimic Visual Applications

Lou

Shen

2021

Small Structures

View full text Add to dashboard Cite

Flexible image sensors with good detectivity, high mechanical stability, and excellent flexibility have generated great interest in biomimetic visual applications. Semiconductor nanowires (NWs) with a unique geometric structure, good transparency, and excellent electronic/optoelectronic properties can cater to flexible applications associated with image sensors. Herein, a comprehensive review of flexible image sensors with semiconducting NWs is offered, and the key performance parameters of photodetectors are introduced in detail. The excellent performances of heterostructure and NW array photodetectors, as well as organic-inorganic hybrid devices, are also reviewed. Finally, new concepts of flexible image sensors, including visual memory, infrared image sensing, and biomimetic electronic eye, are examined to showcase future research in this exciting field.

show abstract

Eye-in-Hand Visual Servoing Enhanced With Sparse Strain Measurement for Soft Continuum Robots

Cited by 62 publications

References 21 publications

A Survey for Machine Learning-Based Control of Continuum Robots

A Survey for Machine Learning-Based Control of Continuum Robots

Neuromorphic Eye-in-Hand Visual Servoing

Flexible Image Sensors with Semiconducting Nanowires for Biomimic Visual Applications

Contact Info

Product

Resources

About