Vision-based manipulation of deformable and rigid objects using subspace projections of 2D contours

Zhu, Jihong; Navarro-Alarcon, David; Passama, Robin; Cherubini, Andrea

doi:10.1016/j.robot.2021.103798

Cited by 39 publications

(24 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, since it is a local model, it should be continuously updated during task execution. Model updating methods include: Broyden rules [9], receding horizon adaption [24], local gradient descent [25], QP-based optimization [26], and Multi-armed Banditbased methods [27]. Furthermore, one can design a simple controller by inverting the Jacobian model.…”

Section: Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Challenges and Outlook in Robotic Manipulation of Deformable Objects

Zhu

Cherubini

Dune

et al. 2022

IEEE Robot. Automat. Mag.

Self Cite

View full text Add to dashboard Cite

Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and healthcare sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex, and is still an open research problem. Tackling the challenges in DOM demands breakthroughs in almost all aspects of robotics, namely hardware design, sensing, deformation modeling, planning, and control. In this article, we highlight the main challenges that arise by considering deformation and review recent advances in each sub-field. A particular focus of our paper lies in the discussions of these challenges and proposing promising directions of research.

show abstract

Section: Modelingmentioning

confidence: 99%

“…Furthermore, one can design a simple controller by inverting the Jacobian model. However, since this controller only guarantees local convergence, it should operate via a series of intermediate target shapes [24], [26].…”

Section: Modelingmentioning

confidence: 99%

Challenges and Outlook in Robotic Manipulation of Deformable Objects

Zhu

Cherubini

Dune

et al. 2022

IEEE Robot. Automat. Mag.

Self Cite

View full text Add to dashboard Cite

show abstract

“…as in direct visual servoing [16], [17]) and combine them with dimension reduction techniques, as in e.g. [18], [19].…”

Section: Introductionmentioning

confidence: 99%

LaSeSOM: A Latent and Semantic Representation Framework for Soft Object Manipulation

Zhou

Zhu

Huo

et al. 2021

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Soft object manipulation has recently gained popularity within the robotics community due to its potential applications in many economically important areas. Although great progress has been recently achieved in these types of tasks, most state-of-the-art methods are case-specific; They can only be used to perform a single deformation task (e.g. bending), as their shape representation algorithms typically rely on "hard-coded" features. In this paper, we present LaSeSOM, a new feedback latent representation framework for semantic soft object manipulation. Our new method introduces internal latent representation layers between low-level geometric feature extraction and high-level semantic shape analysis; This allows the identification of each compressed semantic function and the formation of a valid shape classifier from different feature extraction levels. The proposed latent framework makes soft object representation more generic (independent from the object's geometry and its mechanical properties) and scalable (it can work with 1D/2D/3D tasks). Its high-level semantic layer enables to perform (quasi) shape planning tasks with soft objects, a valuable and underexplored capability in many soft manipulation tasks. To validate this new methodology, we report a detailed experimental study with robotic manipulators.

show abstract

“…Note that these previous methods have been used to estimate the DJM of objects with "mostly" elastic properties. In [17], a shape servoing controller is proposed for both rigid and deformable objects that estimates the DJM with a leastsquares method over a sliding window. However, in this and most previous works, the stability of such online algorithms is not rigorously proved (the design of their update rules is generally decoupled from the motion control laws).…”

Section: Introductionmentioning

confidence: 99%

Contour Moments Based Manipulation of Composite Rigid-Deformable Objects with Finite Time Model Estimation and Shape/Position Control

Qi¹,

Ma²,

Zhu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The robotic manipulation of composite rigiddeformable objects (i.e. those with mixed non-homogeneous stiffness properties) is a challenging problem with clear practical applications that, despite the recent progress in the field, it has not been sufficiently studied in the literature. To deal with this issue, in this paper we propose a new visual servoing method that has the capability to manipulate this broad class of objects (which varies from soft to rigid) with the same adaptive strategy. To quantify the object's infinite-dimensional configuration, our new approach computes a compact feedback vector of 2D contour moments features. A sliding mode control scheme is then designed to simultaneously ensure the finite-time convergence of both the feedback shape error and the model estimation error. The stability of the proposed framework (including the boundedness of all the signals) is rigorously proved with Lyapunov theory. Detailed simulations and experiments are presented to validate the effectiveness of the proposed approach. To the best of the author's knowledge, this is the first time that contour moments along with finite-time control have been used to solve this difficult manipulation problem.

show abstract

Vision-based manipulation of deformable and rigid objects using subspace projections of 2D contours

Abstract: We present a unique framework for manipulating both rigid and deformable objects.• Our framework is model-free and requires a short initialization phase.• Our framework does not require camera calibration, and works with different camera poses.

Cited by 39 publications

References 38 publications

Challenges and Outlook in Robotic Manipulation of Deformable Objects

Challenges and Outlook in Robotic Manipulation of Deformable Objects

LaSeSOM: A Latent and Semantic Representation Framework for Soft Object Manipulation

Contour Moments Based Manipulation of Composite Rigid-Deformable Objects with Finite Time Model Estimation and Shape/Position Control

Contact Info

Product

Resources

About