Modelling natural and artificial hands with synergies

We report on recent work in modelling the process of grasping and active touch by natural and artificial hands. Starting from observations made in human hands about the correlation of degrees of freedom in patterns of more frequent use (postural synergies), we consider the implications of a geometrical model accounting for such data, which is applicable to the pre-grasping phase occurring when shaping the hand before actual contact with the grasped object. To extend applicability of the synergy model to study force distribution in the actual grasp, we introduce a modified model including the mechanical compliance of the hand's musculotendinous system. Numerical results obtained by this model indicate that the same principal synergies observed from pre-grasp postural data are also fundamental in achieving proper grasp force distribution. To illustrate the concept of synergies in the dual domain of haptic sensing, we provide a review of models of how the complexity and heterogeneity of sensory information from touch can be harnessed in simplified, tractable abstractions. These abstractions are amenable to fast processing to enable quick reflexes as well as elaboration of high-level percepts. Applications of the synergy model to the design and control of artificial hands and tactile sensors are illustrated.     

Light touch for balance: influence of a time-varying external driving signal

Sensory information about body sway is used to drive corrective muscle action to keep the body's centre of mass located over the base of support provided by the feet. Loss of vision, by closing the eyes, usually results in increased sway as indexed by fluctuations (i.e. standard deviation, s.d.) in the velocity of a marker at C7 on the neck, s.d. dC7. Variability in the rate of change of centre of pressure (s.d. dCoP), which indexes corrective muscle action, also increases during upright standing with eyes closed. Light touch contact by the tip of one finger with an environmental surface can reduce s.d. dC7 and s.d. dCoP as effectively as opening the eyes. We review studies of light touch and balance and then describe a novel paradigm for studying the nature of somatosensory information contributing to effects of light touch balance. We show that 'light tight touch' contact by the index finger held in the thimble of a haptic device results in increased anteroposterior (AP) sway with entraining by either simple or complex AP sinusoidal oscillations of the haptic device. Moreover, sway is also increased when the haptic device plays back the pre-recorded AP sway path of another person. Cross-correlations between hand and C7 motion reveal a 176 ms lead for the hand and we conclude that light tight touch affords an efficient route for somatosensory feedback support for balance. Furthermore, we suggest that the paradigm has potential to contribute to the understanding of interpersonal postural coordination with light touch in future research.     

Haptic object perception: spatial dimensionality and relation to vision

Enabled by the remarkable dexterity of the human hand, specialized haptic exploration is a hallmark of object perception by touch. Haptic exploration normally takes place in a spatial world that is three-dimensional; nevertheless, stimuli of reduced spatial dimensionality are also used to display spatial information. This paper examines the consequences of full (three-dimensional) versus reduced (two-dimensional) spatial dimensionality for object processing by touch, particularly in comparison with vision. We begin with perceptual recognition of common human-made artefacts, then extend our discussion of spatial dimensionality in touch and vision to include faces, drawing from research on haptic recognition of facial identity and emotional expressions. Faces have often been characterized as constituting a specialized input for human perception. We find that contrary to vision, haptic processing of common objects is impaired by reduced spatial dimensionality, whereas haptic face processing is not. We interpret these results in terms of fundamental differences in object perception across the modalities, particularly the special role of manual exploration in extracting a three-dimensional structure.     

Direct and indirect haptic calibration of visual size judgments

It has long been suspected that touch plays a fundamental role in the calibration of visual perception, and much recent evidence supports this idea. However, as the haptic exploration workspace is limited by the kinematics of the body, the contribution of haptic information to the calibration process should occur only within the region of the haptic workspace reachable by a limb (peripersonal space). To test this hypothesis we evaluated visual size perception and showed that it is indeed more accurate inside the peripersonal space. We then show that allowing subjects to touch the (unseen) stimulus after observation restores accurate size perception; the accuracy persists for some time, implying that calibration has occurred. Finally, we show that observing an actor grasp the object also produces accurate (and lasting) size perception, suggesting that the calibration can also occur indirectly by observing goal-directed actions, implicating the involvement of the "mirror system".     

Pain processing by spinal microcircuits: afferent combinatorics

Pain, itch, heat, cold, and touch represent different percepts arising from somatosensory input. How stimuli give rise to these percepts has been debated for over a century. Recent work supports the view that primary afferents are highly specialized to transduce and encode specific stimulus modalities. However, cross-modal interactions (e.g. inhibition or exacerbation of pain by touch) support convergence rather than specificity in central circuits. We outline how peripheral specialization together with central convergence could enable spinal microcircuits to combine inputs from distinctly specialized, co-activated afferents and to modulate the output signals thus formed through computations like normalization. These issues will be discussed alongside recent advances in our understanding of microcircuitry in the superficial dorsal horn.     

Training multi-parameter gaits to reduce the knee adduction moment with data-driven models and haptic feedback

The purpose of this study was to evaluate gait retraining for reducing the knee adduction moment. Our primary objective was to determine whether subject-specific altered gaits aimed at reducing the knee adduction moment by 30% or more could be identified and adopted in a single session through haptic (touch) feedback training on multiple kinematic gait parameters. Nine healthy subjects performed gait retraining, in which data-driven models specific to each subject were determined through experimental trials and were used to train novel gaits involving a combination of kinematic changes to the tibia angle, foot progression and trunk sway angles. Wearable haptic devices were used on the back, knee and foot for real-time feedback. All subjects were able to adopt altered gaits requiring simultaneous changes to multiple kinematic parameters and reduced their knee adduction moments by 29-48%. Analysis of single parameter gait training showed that moving the knee medially by increasing tibia angle, increasing trunk sway and toeing in all reduced the first peak of the knee adduction moment with tibia angle changes having the most dramatic effect. These results suggest that individualized data-driven gait retraining may be a viable option for reducing the knee adduction moment as a treatment method for early-stage knee osteoarthritis patients with sufficient sensation, endurance and motor learning capabilities.     

Haptic guidance improves the visuo-manual tracking of trajectories

BACKGROUND: Learning to perform new movements is usually achieved by following visual demonstrations. Haptic guidance by a force feedback device is a recent and original technology which provides additional proprioceptive cues during visuo-motor learning tasks. The effects of two types of haptic guidances-control in position (HGP) or in force (HGF)-on visuo-manual tracking ("following") of trajectories are still under debate. METHODOLOGY/PRINCIPALS FINDINGS: Three training techniques of haptic guidance (HGP, HGF or control condition, NHG, without haptic guidance) were evaluated in two experiments. Movements produced by adults were assessed in terms of shapes (dynamic time warping) and kinematics criteria (number of velocity peaks and mean velocity) before and after the training sessions. Trajectories consisted of two Arabic and two Japanese-inspired letters in Experiment 1 and ellipses in Experiment 2. We observed that the use of HGF globally improves the fluency of the visuo-manual tracking of trajectories while no significant improvement was found for HGP or NHG. CONCLUSION/SIGNIFICANCE: These results show that the addition of haptic information, probably encoded in force coordinates, play a crucial role on the visuo-manual tracking of new trajectories.     

Effects of Haptic Information on the Perception of Dynamic 3-D Movement

This study examined effects of hand movement on visual perception of 3-D movement. I used an apparatus in which a cursor position in a simulated 3-D space and the position of a stylus on a haptic device could coincide using a mirror. In three experiments, participants touched the center of a rectangle in the visual display with the stylus of the force-feedback device. Then the rectangle''s surface stereoscopically either protruded toward a participant or indented away from the participant. Simultaneously, the stylus either pushed back participant''s hand, pulled away, or remained static. Visual and haptic information were independently manipulated. Participants judged whether the rectangle visually protruded or dented. Results showed that when the hand was pulled away, subjects were biased to perceive rectangles indented; however, when the hand was pushed back, no effect of haptic information was observed (Experiment 1). This effect persisted even when the cursor position was spatially separated from the hand position (Experiment 2). But, when participants touched an object different from the visual stimulus, this effect disappeared (Experiment 3). These results suggest that the visual system tried to integrate the dynamic visual and haptic information when they coincided cognitively, and the effect of haptic information on visually perceived depth was direction-dependent.     

Dopamine mediates context-dependent modulation of sensory plasticity in C. elegans

Dopamine has been implicated in the modulation of diverse forms of behavioral plasticity, including appetitive learning and addiction. An important challenge is to understand how dopamine's effects at the cellular level alter the properties of neural circuits to modify behavior. In the nematode C. elegans, dopamine modulates habituation of an escape reflex triggered by body touch. In the absence of food, animals habituate more rapidly than in the presence of food; this contextual information about food availability is provided by dopaminergic mechanosensory neurons that sense the presence of bacteria. We find that dopamine alters habituation kinetics by selectively modulating the touch responses of the anterior-body mechanoreceptors; this modulation involves a D1-like dopamine receptor, a Gq/PLC-beta signaling pathway, and calcium release within the touch neurons. Interestingly, the body touch mechanoreceptors can themselves excite the dopamine neurons, forming a positive feedback loop capable of integrating context and experience to modulate mechanosensory attention.     

Short-Term Visual Deprivation,Tactile Acuity,and Haptic Solid Shape Discrimination

Previous psychophysical studies have reported conflicting results concerning the effects of short-term visual deprivation upon tactile acuity. Some studies have found that 45 to 90 minutes of total light deprivation produce significant improvements in participants'' tactile acuity as measured with a grating orientation discrimination task. In contrast, a single 2011 study found no such improvement while attempting to replicate these earlier findings. A primary goal of the current experiment was to resolve this discrepancy in the literature by evaluating the effects of a 90-minute period of total light deprivation upon tactile grating orientation discrimination. We also evaluated the potential effect of short-term deprivation upon haptic 3-D shape discrimination using a set of naturally-shaped solid objects. According to previous research, short-term deprivation enhances performance in a tactile 2-D shape discrimination task – perhaps a similar improvement also occurs for haptic 3-D shape discrimination. The results of the current investigation demonstrate that not only does short-term visual deprivation not enhance tactile acuity, it additionally has no effect upon haptic 3-D shape discrimination. While visual deprivation had no effect in our study, there was a significant effect of experience and learning for the grating orientation task – the participants'' tactile acuity improved over time, independent of whether they had, or had not, experienced visual deprivation.     

A Haptic Feedback Scheme to Accurately Position a Virtual Wrist Prosthesis Using a Three-Node Tactor Array

In this paper, a novel haptic feedback scheme, used for accurately positioning a 1DOF virtual wrist prosthesis through sensory substitution, is presented. The scheme employs a three-node tactor array and discretely and selectively modulates the stimulation frequency of each tactor to relay 11 discrete haptic stimuli to the user. Able-bodied participants were able to move the virtual wrist prosthesis via a surface electromyography based controller. The participants evaluated the feedback scheme without visual or audio feedback and relied solely on the haptic feedback alone to correctly position the hand. The scheme was evaluated through both normal (perpendicular) and shear (lateral) stimulations applied on the forearm. Normal stimulations were applied through a prototype device previously developed by the authors while shear stimulations were generated using an ubiquitous coin motor vibrotactor. Trials with no feedback served as a baseline to compare results within the study and to the literature. The results indicated that using normal and shear stimulations resulted in accurately positioning the virtual wrist, but were not significantly different. Using haptic feedback was substantially better than no feedback. The results found in this study are significant since the feedback scheme allows for using relatively few tactors to relay rich haptic information to the user and can be learned easily despite a relatively short amount of training. Additionally, the results are important for the haptic community since they contradict the common conception in the literature that normal stimulation is inferior to shear. From an ergonomic perspective normal stimulation has the potential to benefit upper limb amputees since it can operate at lower frequencies than shear-based vibrotactors while also generating less noise. Through further tuning of the novel haptic feedback scheme and normal stimulation device, a compact and comfortable sensory substitution device for upper limb amputees might be created.     

Vibration influences haptic perception of surface compliance during walking

Background

The haptic perception of ground compliance is used for stable regulation of dynamic posture and the control of locomotion in diverse natural environments. Although rarely investigated in relation to walking, vibrotactile sensory channels are known to be active in the discrimination of material properties of objects and surfaces through touch. This study investigated how the perception of ground surface compliance is altered by plantar vibration feedback.

Methodology/Principal Findings

Subjects walked in shoes over a rigid floor plate that provided plantar vibration feedback, and responded indicating how compliant it felt, either in subjective magnitude or via pairwise comparisons. In one experiment, the compliance of the floor plate was also varied. Results showed that perceived compliance of the plate increased monotonically with vibration feedback intensity, and depended to a lesser extent on the temporal or frequency distribution of the feedback. When both plate stiffness (inverse compliance) and vibration amplitude were manipulated, the effect persisted, with both factors contributing to compliance perception. A significant influence of vibration was observed even for amplitudes close to psychophysical detection thresholds.

Conclusions/Significance

These findings reveal that vibrotactile sensory channels are highly salient to the perception of surface compliance, and suggest that correlations between vibrotactile sensory information and motor activity may be of broader significance for the control of human locomotion than has been previously acknowledged.     

Is there a ‘plenhaptic’ function?

One approach to gauge the complexity of the computational problem underlying haptic perception is to determine the number of dimensions needed to describe it. In vision, the number of dimensions can be estimated to be seven. This observation raises the question of what is the number of dimensions needed to describe touch. Only with certain simplified representations of mechanical interactions can this number be estimated, because it is in general infinite. Organisms must be sensitive to considerably reduced subsets of all possible measurements. These reductions are discussed by considering the sensing apparatuses of some animals and the underlying mechanisms of two haptic illusions.     

3-D finite-element models of human and monkey fingertips to investigate the mechanics of tactile sense

The biomechanics of skin and underlying tissues plays a fundamental role in the human sense of touch. It governs the mechanics of contact between the skin and an object, the transmission of the mechanical signals through the skin, and their transduction into neural signals by the mechanoreceptors. To better understand the mechanics of touch, it is necessary to establish quantitative relationships between the loads imposed on the skin by an object, the state of stresses/strains at mechanoreceptor locations, and the resulting neural response. Towards this goal, 3-D finite-element models of human and monkey fingertips with realistic external geometries were developed. By computing fingertip model deformations under line loads, it was shown that a multi-layered model was necessary to match previously obtained in vivo data on skin surface displacements. An optimal ratio of elastic moduli of the layers was determined through numerical experiments whose results were matched with empirical data. Numerical values of the elastic moduli of the skin layers were obtained by matching computed results with empirically determined force-displacement relationships for a variety of indentors. Finally, as an example of the relevance of the model to the study of tactile neural response, the multilayered 3-D finite-element model was shown to be able to predict the responses of the slowly adapting type I (SA-I) mechanoreceptors to indentations by complex object shapes.     

Running as fast as it can: How spiking dynamics form object groupings in the laminar circuits of visual cortex

How spiking neurons cooperate to control behavioral processes is a fundamental problem in computational neuroscience. Such cooperative dynamics are required during visual perception when spatially distributed image fragments are grouped into emergent boundary contours. Perceptual grouping is a challenge for spiking cells because its properties of collinear facilitation and analog sensitivity occur in response to binary spikes with irregular timing across many interacting cells. Some models have demonstrated spiking dynamics in recurrent laminar neocortical circuits, but not how perceptual grouping occurs. Other models have analyzed the fast speed of certain percepts in terms of a single feedforward sweep of activity, but cannot explain other percepts, such as illusory contours, wherein perceptual ambiguity can take hundreds of milliseconds to resolve by integrating multiple spikes over time. The current model reconciles fast feedforward with slower feedback processing, and binary spikes with analog network-level properties, in a laminar cortical network of spiking cells whose emergent properties quantitatively simulate parametric data from neurophysiological experiments, including the formation of illusory contours; the structure of non-classical visual receptive fields; and self-synchronizing gamma oscillations. These laminar dynamics shed new light on how the brain resolves local informational ambiguities through the use of properly designed nonlinear feedback spiking networks which run as fast as they can, given the amount of uncertainty in the data that they process.     

Haptic discrimination of size and texture in squirrel monkeys ( Saimiri sciureus )

The present study analyzed haptic abilities of four squirrel monkeys. Using a two-alternative forced-choice procedure, stimuli were presented in a visually opaque box, allowing unrestrained test subjects to grab through a small opening and touch the discriminanda. Difference thresholds were determined by a modified method of limits. In the first experiment we determined size difference thresholds for the discrimination of circular cylinders using standard stimuli differing in diameter from 10 mm to 35 mm. In the second experiment a texture difference threshold was obtained for the discrimination of grooved surfaces (groove width 2-7 mm).The squirrel monkeys achieved a mean size difference threshold of 8% stimulus difference. The linear increase of absolute thresholds as a function of the starting stimulus size showed that haptic size discriminations in squirrel monkeys correspond to Weber's law. Three of the animals achieved a texture difference of 10% stimulus difference, while one monkey showed a distinctively lower haptic acuity. An analysis of the exploratory behavior points to a subject-related difference in the significance of cutaneous and kinesthetic information during size discriminations. Whereas differences in the animals' exploratory behavior did not correlate with the size difference threshold a subject achieved, different thresholds for texture discrimination can be explained by the different exploratory procedures the monkeys used to touch grooved surfaces. The low difference thresholds determined for the squirrel monkeys in the present study point to the significance of unrestrained test conditions for the assessment of the haptic capacity of a species.     

Haptic discrimination of size and texture in squirrel monkeys (Saimiri sciureus)

The present study analyzed haptic abilities of four squirrel monkeys. Using a two-alternative forced-choice procedure, stimuli were presented in a visually opaque box, allowing unrestrained test subjects to grab through a small opening and touch the discriminanda. Difference thresholds were determined by a modified method of limits. In the first experiment we determined size difference thresholds for the discrimination of circular cylinders using standard stimuli differing in diameter from 10 mm to 35 mm. In the second experiment a texture difference threshold was obtained for the discrimination of grooved surfaces (groove width 2-7 mm). The squirrel monkeys achieved a mean size difference threshold of 8% stimulus difference. The linear increase of absolute thresholds as a function of the starting stimulus size showed that haptic size discriminations in squirrel monkeys correspond to Weber's law. Three of the animals achieved a texture difference of 10% stimulus difference, while one monkey showed a distinctively lower haptic acuity. An analysis of the exploratory behavior points to a subject-related difference in the significance of cutaneous and kinesthetic information during size discriminations. Whereas differences in the animals' exploratory behavior did not correlate with the size difference threshold a subject achieved, different thresholds for texture discrimination can be explained by the different exploratory procedures the monkeys used to touch grooved surfaces. The low difference thresholds determined for the squirrel monkeys in the present study point to the significance of unrestrained test conditions for the assessment of the haptic capacity of a species.     

Somato-Motor Haptic Processing in Posterior Inner Perisylvian Region (SII/pIC) of the Macaque Monkey

The posterior inner perisylvian region including the secondary somatosensory cortex (area SII) and the adjacent region of posterior insular cortex (pIC) has been implicated in haptic processing by integrating somato-motor information during hand-manipulation, both in humans and in non-human primates. However, motor-related properties during hand-manipulation are still largely unknown. To investigate a motor-related activity in the hand region of SII/pIC, two macaque monkeys were trained to perform a hand-manipulation task, requiring 3 different grip types (precision grip, finger exploration, side grip) both in light and in dark conditions. Our results showed that 70% (n = 33/48) of task related neurons within SII/pIC were only activated during monkeys’ active hand-manipulation. Of those 33 neurons, 15 (45%) began to discharge before hand-target contact, while the remaining neurons were tonically active after contact. Thirty-percent (n = 15/48) of studied neurons responded to both passive somatosensory stimulation and to the motor task. A consistent percentage of task-related neurons in SII/pIC was selectively activated during finger exploration (FE) and precision grasping (PG) execution, suggesting they play a pivotal role in control skilled finger movements. Furthermore, hand-manipulation-related neurons also responded when visual feedback was absent in the dark. Altogether, our results suggest that somato-motor neurons in SII/pIC likely contribute to haptic processing from the initial to the final phase of grasping and object manipulation. Such motor-related activity could also provide the somato-motor binding principle enabling the translation of diachronic somatosensory inputs into a coherent image of the explored object.     

Vibrissa Self-Motion and Touch Are Reliably Encoded along the Same Somatosensory Pathway from Brainstem through Thalamus

Active sensing involves the fusion of internally generated motor events with external sensation. For rodents, active somatosensation includes scanning the immediate environment with the mystacial vibrissae. In doing so, the vibrissae may touch an object at any angle in the whisk cycle. The representation of touch and vibrissa self-motion may in principle be encoded along separate pathways, or share a single pathway, from the periphery to cortex. Past studies established that the spike rates in neurons along the lemniscal pathway from receptors to cortex, which includes the principal trigeminal and ventral-posterior-medial thalamic nuclei, are substantially modulated by touch. In contrast, spike rates along the paralemniscal pathway, which includes the rostral spinal trigeminal interpolaris, posteromedial thalamic, and ventral zona incerta nuclei, are only weakly modulated by touch. Here we find that neurons along the lemniscal pathway robustly encode rhythmic whisking on a cycle-by-cycle basis, while encoding along the paralemniscal pathway is relatively poor. Thus, the representations of both touch and self-motion share one pathway. In fact, some individual neurons carry both signals, so that upstream neurons with a supralinear gain function could, in principle, demodulate these signals to recover the known decoding of touch as a function of vibrissa position in the whisk cycle.     

Perceiving Object Shape from Specular Highlight Deformation,Boundary Contour Deformation,and Active Haptic Manipulation

It is well known that motion facilitates the visual perception of solid object shape, particularly when surface texture or other identifiable features (e.g., corners) are present. Conventional models of structure-from-motion require the presence of texture or identifiable object features in order to recover 3-D structure. Is the facilitation in 3-D shape perception similar in magnitude when surface texture is absent? On any given trial in the current experiments, participants were presented with a single randomly-selected solid object (bell pepper or randomly-shaped “glaven”) for 12 seconds and were required to indicate which of 12 (for bell peppers) or 8 (for glavens) simultaneously visible objects possessed the same shape. The initial single object’s shape was defined either by boundary contours alone (i.e., presented as a silhouette), specular highlights alone, specular highlights combined with boundary contours, or texture. In addition, there was a haptic condition: in this condition, the participants haptically explored with both hands (but could not see) the initial single object for 12 seconds; they then performed the same shape-matching task used in the visual conditions. For both the visual and haptic conditions, motion (rotation in depth or active object manipulation) was present in half of the trials and was not present for the remaining trials. The effect of motion was quantitatively similar for all of the visual and haptic conditions–e.g., the participants’ performance in Experiment 1 was 93.5 percent higher in the motion or active haptic manipulation conditions (when compared to the static conditions). The current results demonstrate that deforming specular highlights or boundary contours facilitate 3-D shape perception as much as the motion of objects that possess texture. The current results also indicate that the improvement with motion that occurs for haptics is similar in magnitude to that which occurs for vision.