Force prediction by fingernail imaging using active appearance models

Abstract

This paper investigates the effect of two different parameters on the registration and force prediction accuracy of using Active Appearance Models (AAM) to align fingernail images. First, the color channel used to form the AAM is varied between (1) an averaged grayscale image, (2) the red channel, (3) the green channel and (4) the blue channel. Second, the number of landmark points used to create the AAM is varied between 6 and 75.The color channel is found to have an effect on the registration accuracy and the force prediction error. The green, blue and grayscale images are approximately equivalent, while the red images have a larger error across all metrics used. The number of landmark points may be reduced to 25 with no significant effect on either the registration accuracy or the force prediction error, though further reduction has shown some effect. With this information, a simpler registration model can be used that requires fewer calculations.

ThimbleSense: An individual-digit wearable tactile sensor for experimental grasp studies

“…2.5b shows a wearable, force-based approach that uses changes in coloration of fingernails to estimate forces on fingertips. This particular device has been subject of various publications ( [13], [19], [20], [21]), and has its prominent feature in the fact that it can give both shear and normal forces while leaving the fingerpad completely free from occlusion. However it gives no measurement of torques, and no way has been proposed so far to use this tool to estimate position of contacts accurately.…”

“…More advanced approaches exist that measure forces while leaving the fingerpad free from occlusion: in [11] the relationship between nail strain and compression forces on the fingerpad was studied, while in [12] the horizontal deformation of fingerpads was used to estimate normal forces. In [13] fingernail sensors were introduced that correlate blood distribution under the finger nail with forces, which later publications further developed and refined (see for example [14]). This last solution is particularly interesting since it also provides measurements for shear forces; however, it requires fine calibration, and to the best of our knowledge currently provides no information regarding the position of contact points.…”

Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images

“…10 Following preprocessing, various methods have been developed to estimate the force. Model-based methods 11 contain linearized sigmoid models, EigenNail models, 12 and a linear least squared method (LSM). 13 [4] estimates force and torque using Gaussian processes (GP) and neural networks.…”

Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian processes

“…PvdS is also with fortiss. onto the finger [7], rigid body transformation including the Harris feature point based method [8], Canny edge detection [9], template matching using markers [10], non-rigid registration fitting a finger model [8], and Active Appearance Models (AAM) [11]. Other methods use sensors mounted on the finger [10], [12] or require restrictions such as a bracket to support the hand [7] or the finger [9], [11].…”

“…Model-based methods [11] contain linearized sigmoid models, EigenNail models [13], and linearized sigmoid models. In [10] we previously estimated force and torque using Gaussian processes (GP) and neural networks.…”