Spatiotemporal Modeling of Grip Forces Captures Proficiency in Manual Robot Control

Abstract

New technologies for monitoring grip forces during hand and finger movements in non-standard task contexts have provided unprecedented functional insights into somatosensory cognition. Somatosensory cognition is the basis of our ability to manipulate and transform objects of the physical world and to grasp them with the right amount of force. In previous work, the wireless tracking of grip-force signals recorded from biosensors in the palm of the human hand has permitted us to unravel some of the functional synergies that underlie perceptual and motor learning under conditions of non-standard and essentially unreliable sensory input. This paper builds on this previous work and discusses further, functionally motivated, analyses of individual grip-force data in manual robot control. Grip forces were recorded from various loci in the dominant and non-dominant hands of individuals with wearable wireless sensor technology. Statistical analyses bring to the fore skill-specific temporal variations in thousands of grip forces of a complete novice and a highly proficient expert in manual robot control. A brain-inspired neural network model that uses the output metric of a self-organizing pap with unsupervised winner-take-all learning was run on the sensor output from both hands of each user. The neural network metric expresses the difference between an input representation and its model representation at any given moment in time and reliably captures the differences between novice and expert performance in terms of grip-force variability.Functionally motivated spatiotemporal analysis of individual average grip forces, computed for time windows of constant size in the output of a restricted amount of task-relevant sensors in the dominant (preferred) hand, reveal finger-specific synergies reflecting robotic task skill. The analyses lead the way towards grip-force monitoring in real time. This will permit tracking task skill evolution in trainees, or identify individual proficiency levels in human robot-interaction, which represents unprecedented challenges for perceptual and motor adaptation in environmental contexts of high sensory uncertainty. Cross-disciplinary insights from systems neuroscience and cognitive behavioral science, and the predictive modeling of operator skills using parsimonious Artificial Intelligence (AI), will contribute towards improving the outcome of new types of surgery, in particular the single-port approaches such as NOTES (Natural Orifice Transluminal Endoscopic Surgery) and SILS (Single-Incision Laparoscopic Surgery).

Sensory Factors Influence Dynamic and Static Bi-Manual Finger Grip Strength in a Real-World Task Context

“…Data recorded in the dynamic grip condition display a considerably larger variability within and between participants compared with the data recorded in the static grip condition, which is coherent with the grip style deployed and was to be expected. In the single data curve, we see clear maximum-to-minimum peak spatio-temporal patterns characteristic of dynamic grip tasks [6] in the finger forces of the five individuals across experimental conditions. At a first glance on the plots, dynamic finger grip forces deployed by the dominant and non-dominant hands of the five participants do not show any systematic differences between the two hands.…”

“…Adaptive processes in finger force control during motor learning [13,31,32] explain why the contribution of each finger to overall grip strength, coarse adjustments, or finer grip force control may vary across tasks and their requirements for motor planning and execution. The index, middle, and ring fingers are deemed critical for lifting and/or manipulating weighted cylindrical objects, such as the handles in this study, in various directions [2][3][4][5][6]17,33]. The amount of force applied by each digit depends on anthropometric factors in interaction with object constraints, including where exactly the fingers or sensors are placed when picking up or manipulating an object [12,31].…”

“…The sensor glove system is a prototype designed to optimally fit the hands of a selected number of male operators with comparable hand dimensions in the context of a study on grip force control in expert surgeons and novices manipulating a robotic assistant for minimally invasive surgery [2][3][4][5][6]. It consists of two sensor gloves, one for the right and one for the left hand, designed to measure hand grip strength.…”

“…Which sensors will be optimally activated in a given task when gripping objects depends on parameters relative to the size and shape of the object being gripped, and on the type of manipulation required by the task in a given context. This is discussed extensively in our previous work [2][3][4][5][6]. For gripping the cylindrical objects in this experiment, sensors producing consistent and reliable signal sequences during the whole time of task execution were placed on the middle phalanges of the index, middle, and ring fingers of each hand (Figure 1).…”

“…Biosensor systems for the measurement of grip strength variations during dynamic or stationary grip tasks with hand-object interaction provide valuable insight into the somatosensory control functions underlying the cognitive and motor skills linked to grip strength deployment [1]. As shown in previous work from our group, grip strengthbased performance metrics, such as their variance and its evolution in task time, provide indicators of surgical skill [2][3][4][5][6], adding important insights into individual motor behaviour adjustment beyond the classic measures of task completion times or tool-tip positioning and tool-trajectories. Holding an object or professional tool requires sufficient grip to prevent the tool from dropping/slipping, while avoiding potentially damaging excessive force deployment is a critical aspect of motor performance [7].…”