Towards real-time modeling and haptic rendering of deformable objects for point cloud-based Model-Mediated Teleoperation

Abstract

We propose a novel radial function-based deformation (RFBD) approach to enable real-time modeling and haptic rendering of deformable objects for point cloud-based ModelMediated Teleoperation (pcbMMT). In the pcbMMT system, a time-of-flight camera is employed on the slave side to capture a 3D point cloud model of the remote environment. This environment model is transmitted to the master along with the estimated model parameters, including the dynamics of the object's surface deformation and the physical properties such as the stiffness, friction coefficient, etc. Due to the simplicity of the RFBD approach, the model parameters of the remote environment can be obtained in real time. On the master side, a local copy of the environment model is maintained according to the received parameters. Both the haptic rendering and deformation simulation are based on this local model, running at 1kHz. Thus, a good compromise is achieved between the model accuracy and the computational time for online parameter identification. Experiments verify the feasibility of the proposed approach, and show the results of the object deformation for different model parameters.

Toward Haptic Communications Over the 5G Tactile Internet

“…Therefore, online environment modeling and model updating are inevitable. In recent decades, online environment modeling (parameter identification) for teleoperation systems has been widely investigated, e.g., for estimating linear [187], [188] / non-linear [189], [190] environment models, rigid [96] / deformable [191] / movable [192] objects, and for estimating unknown environment models using online neural network approaches [193], [194].…”