Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2D to 3D: A Review

Abstract

<p>This article presents a detailed review and categorizing of the marker displacement method (MDM) used in vision-based tactile sensors. Vision-based tactile sensors have been proven to be a promising solution for robot tactile perception. Among such sensors, MDM is one of the most commonly used contact characterization and extraction methods. It uses visual approaches to obtain contact deformation and achieve multimodal tactile perception using physical models and post-processing algorithms. In recent years, many tactile sensors using MDM have been developed. However, the existing research does not strictly distinguish between the different types of methods but is uniformly grouped into MDM. Without differentiation, there might be a lack of systematic and comprehensive guidance in analyzing and optimizing the characteristics of MDM and selecting the most suitable method. This article is the first to classify MDM into three typical categories based on the dimensionality perspective: 2D MDM, 2.5D MDM, and 3D MDM. 2D MDM relies only on the monocular camera to acquire the marker array’s 2D displacement field. 2.5D MDM supplements 2D MDM with selected indirect features reflecting the location of the markers in the third dimension. 3D MDM employs a multi-camera system and can obtain the 3D displacement field using the stereo vision method common. Based on the latest literature, we compare the principles, characteristics, advantages and disadvantages, and applications of the three ways in detail. This work can provide a valuable reference for researchers interested in applying MDM in fields such as vision-based tactile sensors.</p>

Improving the Representation and Extraction of Contact Information in Vision-based Tactile Sensors Using Continuous Marker Pattern

“…In recent years, different types of tactile sensors have been developed [4], [5]. Among them, vision-based tactile sensors (also called visuotactile sensors) [6]- [9] are a class of sensors with the advantages of high-resolution and multi-modal perception, such as GelForce [10], GelSight [11], GelSlim [12], FingerVision [13], TacTip [14], and OmniTact [15]. These sensors use a soft elastomer to contact the objects.…”

Implementation of AI in program-based governance in Azerbaijan

“…Given the extensive volume of reports accessible on the centralized boyukgayidish.gov.az portal, obtaining detailed information efficiently becomes a challenge. In the present case, a customized chatbot [13] model, leveraging advanced natural language processing like GPT, was developed. Specific data is meticulously fine-tuning to enhance the chatbot's [10] customization and the model underwent fine-tuning to increase its accuracy in generating relevant responses.…”