Touch Influences Visual Perception with a Tight Orientation-Tuning

Stimuli from different sensory modalities are thought to be processed initially in distinct unisensory brain areas prior to convergence in multisensory areas. However, signals in one modality can influence the processing of signals from other modalities and recent studies suggest this cross-modal influence may occur early on, even in ‘unisensory’ areas. Some recent psychophysical studies have shown specific cross-modal effects between touch and vision during binocular rivalry, but these cannot completely rule out a response bias. To test for genuine cross-modal integration of haptic and visual signals, we investigated whether congruent haptic input could influence visual contrast sensitivity compared to incongruent haptic input in three psychophysical experiments using a two-interval, two-alternative forced-choice method to eliminate response bias. The initial experiment demonstrated that contrast thresholds for a visual grating were lower when exploring a haptic grating that shared the same orientation compared to an orthogonal orientation. Two subsequent experiments mapped the orientation and spatial frequency tunings for the congruent haptic facilitation of vision, finding a clear orientation tuning effect but not a spatial frequency tuning. In addition to an increased contrast sensitivity for iso-oriented visual-haptic gratings, we found a significant loss of sensitivity for orthogonally oriented visual-haptic gratings. We conclude that the tactile influence on vision is a result of a tactile input to orientation-tuned visual areas.     

Haptic discrimination of size and texture in squirrel monkeys ( Saimiri sciureus )

The present study analyzed haptic abilities of four squirrel monkeys. Using a two-alternative forced-choice procedure, stimuli were presented in a visually opaque box, allowing unrestrained test subjects to grab through a small opening and touch the discriminanda. Difference thresholds were determined by a modified method of limits. In the first experiment we determined size difference thresholds for the discrimination of circular cylinders using standard stimuli differing in diameter from 10 mm to 35 mm. In the second experiment a texture difference threshold was obtained for the discrimination of grooved surfaces (groove width 2-7 mm).The squirrel monkeys achieved a mean size difference threshold of 8% stimulus difference. The linear increase of absolute thresholds as a function of the starting stimulus size showed that haptic size discriminations in squirrel monkeys correspond to Weber's law. Three of the animals achieved a texture difference of 10% stimulus difference, while one monkey showed a distinctively lower haptic acuity. An analysis of the exploratory behavior points to a subject-related difference in the significance of cutaneous and kinesthetic information during size discriminations. Whereas differences in the animals' exploratory behavior did not correlate with the size difference threshold a subject achieved, different thresholds for texture discrimination can be explained by the different exploratory procedures the monkeys used to touch grooved surfaces. The low difference thresholds determined for the squirrel monkeys in the present study point to the significance of unrestrained test conditions for the assessment of the haptic capacity of a species.     

Haptic discrimination of size and texture in squirrel monkeys (Saimiri sciureus)

The present study analyzed haptic abilities of four squirrel monkeys. Using a two-alternative forced-choice procedure, stimuli were presented in a visually opaque box, allowing unrestrained test subjects to grab through a small opening and touch the discriminanda. Difference thresholds were determined by a modified method of limits. In the first experiment we determined size difference thresholds for the discrimination of circular cylinders using standard stimuli differing in diameter from 10 mm to 35 mm. In the second experiment a texture difference threshold was obtained for the discrimination of grooved surfaces (groove width 2-7 mm). The squirrel monkeys achieved a mean size difference threshold of 8% stimulus difference. The linear increase of absolute thresholds as a function of the starting stimulus size showed that haptic size discriminations in squirrel monkeys correspond to Weber's law. Three of the animals achieved a texture difference of 10% stimulus difference, while one monkey showed a distinctively lower haptic acuity. An analysis of the exploratory behavior points to a subject-related difference in the significance of cutaneous and kinesthetic information during size discriminations. Whereas differences in the animals' exploratory behavior did not correlate with the size difference threshold a subject achieved, different thresholds for texture discrimination can be explained by the different exploratory procedures the monkeys used to touch grooved surfaces. The low difference thresholds determined for the squirrel monkeys in the present study point to the significance of unrestrained test conditions for the assessment of the haptic capacity of a species.     

The ventriloquist effect results from near-optimal bimodal integration

Ventriloquism is the ancient art of making one's voice appear to come from elsewhere, an art exploited by the Greek and Roman oracles, and possibly earlier. We regularly experience the effect when watching television and movies, where the voices seem to emanate from the actors' lips rather than from the actual sound source. Originally, ventriloquism was explained by performers projecting sound to their puppets by special techniques, but more recently it is assumed that ventriloquism results from vision "capturing" sound. In this study we investigate spatial localization of audio-visual stimuli. When visual localization is good, vision does indeed dominate and capture sound. However, for severely blurred visual stimuli (that are poorly localized), the reverse holds: sound captures vision. For less blurred stimuli, neither sense dominates and perception follows the mean position. Precision of bimodal localization is usually better than either the visual or the auditory unimodal presentation. All the results are well explained not by one sense capturing the other, but by a simple model of optimal combination of visual and auditory information.     

Viewing geometry determines how vision and haptics combine in size perception

Vision and haptics have different limitations and advantages because they obtain information by different methods. If the brain combined information from the two senses optimally, it would rely more on the one providing more precise information for the current task. In this study, human observers judged the distance between two parallel surfaces in two within-modality experiments (vision-alone and haptics-alone) and in an intermodality experiment (vision and haptics together). In the within-modality experiments, the precision of visual estimates varied with surface orientation, as expected from geometric considerations; the precision of haptic estimates did not. An ideal observer that combines visual and haptic information weights them differently as a function of orientation. In the intermodality experiment, humans adjusted visual and haptic weights in a fashion quite similar to that of the ideal observer. As a result, combined size estimates are finer than is possible with either vision or haptics alone; indeed, they approach statistical optimality.     

Receptor noise as a determinant of colour thresholds.

Inferences about mechanisms at one particular stage of a visual pathway may be made from psychophysical thresholds only if the noise at the stage in question dominates that in the others. Spectral sensitivities, measured under bright conditions, for di-, tri-, and tetrachromatic eyes from a range of animals can be modelled by assuming that thresholds are set by colour opponency mechanisms whose performance is limited by photoreceptor noise, the achromatic signal being disregarded. Noise in the opponency channels themselves is therefore not statistically independent, and it is not possible to infer anything more about the channels from psychophysical thresholds. As well as giving insight into mechanisms of vision, the model predicts the performance of colour vision in animals where physiological and anatomical data on the eye are available, but there are no direct measurements of perceptual thresholds. The model, therefore, is widely applicable to comparative studies of eye design and visual ecology.     

Multisensory integration: a late bloomer

Under many circumstances, human adults integrate information from different sensory modalities, such as vision and hearing, in a statistically optimal fashion. New results suggest that optimal multisensory integration only develops in middle childhood.     

Noninformative vision improves haptic spatial perception

Previous studies have attempted to map somatosensory space via haptic matching tasks and have shown that individuals make large and systematic matching errors, the magnitude and angular direction of which vary systematically through the workspace. Based upon such demonstrations, it has been suggested that haptic space is non-Euclidian. This conclusion assumes that spatial perception is modality specific, and it largely ignores the fact that tactile matching tasks involve active, exploratory arm movements. Here we demonstrate that, when individuals match two bar stimuli (i.e., make them parallel) in circumstances favoring extrinsic (visual) coordinates, providing noninformative visual information significantly increases the accuracy of haptic perception. In contrast, when individuals match the same bar stimuli in circumstances favoring the coding of movements in intrinsic (limb-based) coordinates, providing identical noninformative visual information either has no effect or leads to the decreased accuracy of haptic perception. These results are consistent with optimal integration models of sensory integration in which the weighting given to visual and somatosensory signals depends upon the precision of the visual and somatosensory information and provide important evidence for the task-dependent integration of visual and somatosensory signals during the construction of a representation of peripersonal space.     

Within- and Cross-Modal Distance Information Disambiguate Visual Size-Change Perception

Perception is fundamentally underconstrained because different combinations of object properties can generate the same sensory information. To disambiguate sensory information into estimates of scene properties, our brains incorporate prior knowledge and additional “auxiliary” (i.e., not directly relevant to desired scene property) sensory information to constrain perceptual interpretations. For example, knowing the distance to an object helps in perceiving its size. The literature contains few demonstrations of the use of prior knowledge and auxiliary information in combined visual and haptic disambiguation and almost no examination of haptic disambiguation of vision beyond “bistable” stimuli. Previous studies have reported humans integrate multiple unambiguous sensations to perceive single, continuous object properties, like size or position. Here we test whether humans use visual and haptic information, individually and jointly, to disambiguate size from distance. We presented participants with a ball moving in depth with a changing diameter. Because no unambiguous distance information is available under monocular viewing, participants rely on prior assumptions about the ball''s distance to disambiguate their -size percept. Presenting auxiliary binocular and/or haptic distance information augments participants'' prior distance assumptions and improves their size judgment accuracy—though binocular cues were trusted more than haptic. Our results suggest both visual and haptic distance information disambiguate size perception, and we interpret these results in the context of probabilistic perceptual reasoning.     

Colour Vision Impairment in Young Alcohol Consumers

Alcohol consumption among young adults is widely accepted in modern society and may be the starting point for abusive use of alcohol at later stages of life. Chronic alcohol exposure can lead to visual function impairment. In the present study, we investigated the spatial luminance contrast sensitivity, colour arrangement ability, and colour discrimination thresholds on young adults that weekly consume alcoholic beverages without clinical concerns. Twenty-four young adults were evaluated by an ophthalmologist and performed three psychophysical tests to evaluate their vision functions. We estimated the spatial luminance contrast sensitivity function at 11 spatial frequencies ranging from 0.1 to 30 cycles/degree. No difference in contrast sensitivity was observed comparing alcohol consumers and control subjects. For the evaluation of colour vision, we used the Farnsworth-Munsell 100 hue test (FM 100 test) to test subject’s ability to perform a colour arrangement task and the Mollon-Reffin test (MR test) to measure subject’s colour discrimination thresholds. Alcohol consumers made more mistakes than controls in the FM100 test, and their mistakes were diffusely distributed in the FM colour space without any colour axis preference. Alcohol consumers also performed worse than controls in the MR test and had higher colour discrimination thresholds compared to controls around three different reference points of a perceptually homogeneous colour space, the CIE 1976 chromaticity diagram. There was no colour axis preference in the threshold elevation observed among alcoholic subjects. Young adult weekly alcohol consumers showed subclinical colour vision losses with preservation of spatial luminance contrast sensitivity. Adolescence and young adult age are periods of important neurological development and alcohol exposure during this period of life might be responsible for deficits in visual functions, especially colour vision that is very sensitive to neurotoxicants.     

Information in channel-coded systems: correlated receivers

Noise correlation can easily occur in the densely connected systems observed in biological information processing. We study the consequences of noise correlation for a statistically optimal processing of noise-perturbed receptor array outputs. We find a critical importance of the noise correlation length as compared to the receptors' tuning width for both the structure and the performance of the ideal observer. We show the general consistency of our scheme with psychophysical discrimination thresholds obtained in human spatial vision.     

Why seeing is believing: merging auditory and visual worlds

Vision may dominate our perception of space not because of any inherent physiological advantage of visual over other sensory connections in the brain, but because visual information tends to be more reliable than other sources of spatial information, and the central nervous system integrates information in a statistically optimal fashion. This review discusses recent experiments on audiovisual integration that support this hypothesis. We consider candidate neural codes that would enable optimal integration and the implications of optimal integration for perception and plasticity.     

Temporal mechanisms of multimodal binding

The simultaneity of signals from different senses—such as vision and audition—is a useful cue for determining whether those signals arose from one environmental source or from more than one. To understand better the sensory mechanisms for assessing simultaneity, we measured the discrimination thresholds for time intervals marked by auditory, visual or auditory–visual stimuli, as a function of the base interval. For all conditions, both unimodal and cross-modal, the thresholds followed a characteristic ‘dipper function’ in which the lowest thresholds occurred when discriminating against a non-zero interval. The base interval yielding the lowest threshold was roughly equal to the threshold for discriminating asynchronous from synchronous presentations. Those lowest thresholds occurred at approximately 5, 15 and 75 ms for auditory, visual and auditory–visual stimuli, respectively. Thus, the mechanisms mediating performance with cross-modal stimuli are considerably slower than the mechanisms mediating performance within a particular sense. We developed a simple model with temporal filters of different time constants and showed that the model produces discrimination functions similar to the ones we observed in humans. Both for processing within a single sense, and for processing across senses, temporal perception is affected by the properties of temporal filters, the outputs of which are used to estimate time offsets, correlations between signals, and more.     

Visual information gleaned by observing grasping movement in allocentric and egocentric perspectives

One of the major functions of vision is to allow for an efficient and active interaction with the environment. In this study, we investigate the capacity of human observers to extract visual information from observation of their own actions, and those of others, from different viewpoints. Subjects discriminated the size of objects by observing a point-light movie of a hand reaching for an invisible object. We recorded real reach-and-grasp actions in three-dimensional space towards objects of different shape and size, to produce two-dimensional 'point-light display' movies, which were used to measure size discrimination for reach-and-grasp motion sequences, release-and-withdraw sequences and still frames, all in egocentric and allocentric perspectives. Visual size discrimination from action was significantly better in egocentric than in allocentric view, but only for reach-and-grasp motion sequences: release-and-withdraw sequences or still frames derived no advantage from egocentric viewing. The results suggest that the system may have access to an internal model of action that contributes to calibrate visual sense of size for an accurate grasp.     

Touch Interacts with Vision during Binocular Rivalry with a Tight Orientation Tuning

Multisensory integration is a common feature of the mammalian brain that allows it to deal more efficiently with the ambiguity of sensory input by combining complementary signals from several sensory sources. Growing evidence suggests that multisensory interactions can occur as early as primary sensory cortices. Here we present incompatible visual signals (orthogonal gratings) to each eye to create visual competition between monocular inputs in primary visual cortex where binocular combination would normally take place. The incompatibility prevents binocular fusion and triggers an ambiguous perceptual response in which the two images are perceived one at a time in an irregular alternation. One key function of multisensory integration is to minimize perceptual ambiguity by exploiting cross-sensory congruence. We show that a haptic signal matching one of the visual alternatives helps disambiguate visual perception during binocular rivalry by both prolonging the dominance period of the congruent visual stimulus and by shortening its suppression period. Importantly, this interaction is strictly tuned for orientation, with a mismatch as small as 7.5° between visual and haptic orientations sufficient to annul the interaction. These results indicate important conclusions: first, that vision and touch interact at early levels of visual processing where interocular conflicts are first detected and orientation tunings are narrow, and second, that haptic input can influence visual signals outside of visual awareness, bringing a stimulus made invisible by binocular rivalry suppression back to awareness sooner than would occur without congruent haptic input.     

Path curvature discrimination: dependence on gaze direction and optical flow speed

Many experimental approaches to the control of steering rely on the tangent point (TP) as major source of information. The TP is a good candidate to control self-motion. It corresponds to a singular and salient point in the subject's visual field, and its location depends on the road geometry, the direction of self-motion relative to the road and the position of the driver on the road. However, the particular status of the TP in the optical flow, as a local minimum of flow speed, has often been left aside. We therefore assume that the TP is actually an optimal location in the dynamic optical array to perceive a change in the trajectory curvature. In this study, we evaluated the ability of human observers to detect variations in their path curvature from optical flow patterns, as a function of their gaze direction in a virtual environment. We simulated curvilinear self-motion parallel to a ground plane. Using random-dot optic flow stimuli of brief duration and a two-alternative forced-choice adaptive procedure, we determined path curvature discrimination thresholds, as a function of gaze direction. The discrimination thresholds are minimal for a gaze directed toward a local minimum of optical flow speed. A model based on Weber fraction of the foveal velocities (ΔV/V) correctly predicts the relationship between experimental thresholds and local flow velocities. This model was also tested for an optical flow computation integrating larger circular areas in central vision. Averaging the flow over five degrees leads to an even better fit of the model to experimental thresholds. We also found that the minimal optical flow speed direction corresponds to a maximal sensitivity of the visual system, as predicted by our model. The spontaneous gazing strategies observed during driving might thus correspond to an optimal selection of relevant information in the optical flow field.     

Perception of visual image size by school students of different ages

In psychophysical experiments, 209 high school students with normal vision, 8 to 16 years old, were examined to study the perception of visual image sizes. Observers assessed the length of linear arrow-like figures (the Müller-Lyer illusion) or the length of single lines without arrow-like ends. Distortion of line size perception by 17–21% was obtained in the Müller-Lyer illusion independent of the age of observers. Distortion of size perception was absent in the case of estimation of the length of single line segments. The size differentiation thresholds gradually decreased in both cases with increasing age of observers and were correlated with the acuity of vision. For single lines, they were, on average, 1.3 times lower than the thresholds in the Müller-Lyer illusion. The probable mechanisms of the Müller-Lyer illusion are discussed. The experimental results demonstrate stability of illusion for observers aged 8–16 years, which may be connected with preferential contribution of the lower levels of the visual system to the appearance of the illusion.     

Separate attentional resources for vision and audition

Current models of attention, typically claim that vision and audition are limited by a common attentional resource which means that visual performance should be adversely affected by a concurrent auditory task and vice versa. Here, we test this implication by measuring auditory (pitch) and visual (contrast) thresholds in conjunction with cross-modal secondary tasks and find that no such interference occurs. Visual contrast discrimination thresholds were unaffected by a concurrent chord or pitch discrimination, and pitch-discrimination thresholds were virtually unaffected by a concurrent visual search or contrast discrimination task. However, if the dual tasks were presented within the same modality, thresholds were raised by a factor of between two (for visual discrimination) and four (for auditory discrimination). These results suggest that at least for low-level tasks such as discriminations of pitch and contrast, each sensory modality is under separate attentional control, rather than being limited by a supramodal attentional resource. This has implications for current theories of attention as well as for the use of multi-sensory media for efficient informational transmission.     

Integration of Semi-Circular Canal and Otolith Cues for Direction Discrimination during Eccentric Rotations

Humans are capable of moving about the world in complex ways. Every time we move, our self-motion must be detected and interpreted by the central nervous system in order to make appropriate sequential movements and informed decisions. The vestibular labyrinth consists of two unique sensory organs the semi-circular canals and the otoliths that are specialized to detect rotation and translation of the head, respectively. While thresholds for pure rotational and translational self-motion are well understood surprisingly little research has investigated the relative role of each organ on thresholds for more complex motion. Eccentric (off-center) rotations during which the participant faces away from the center of rotation stimulate both organs and are thus well suited for investigating integration of rotational and translational sensory information. Ten participants completed a psychophysical direction discrimination task for pure head-centered rotations, translations and eccentric rotations with 5 different radii. Discrimination thresholds for eccentric rotations reduced with increasing radii, indicating that additional tangential accelerations (which increase with radius length) increased sensitivity. Two competing models were used to predict the eccentric thresholds based on the pure rotation and translation thresholds: one assuming that information from the two organs is integrated in an optimal fashion and another assuming that motion discrimination is solved solely by relying on the sensor which is most strongly stimulated. Our findings clearly show that information from the two organs is integrated. However the measured thresholds for 3 of the 5 eccentric rotations are even more sensitive than predictions from the optimal integration model suggesting additional non-vestibular sources of information may be involved.     

Crossmodal Integration Improves Sensory Detection Thresholds in the Ferret

During the last two decades ferrets (Mustela putorius) have been established as a highly efficient animal model in different fields in neuroscience. Here we asked whether ferrets integrate sensory information according to the same principles established for other species. Since only few methods and protocols are available for behaving ferrets we developed a head-free, body-restrained approach allowing a standardized stimulation position and the utilization of the ferret’s natural response behavior. We established a behavioral paradigm to test audiovisual integration in the ferret. Animals had to detect a brief auditory and/or visual stimulus presented either left or right from their midline. We first determined detection thresholds for auditory amplitude and visual contrast. In a second step, we combined both modalities and compared psychometric fits and the reaction times between all conditions. We employed Maximum Likelihood Estimation (MLE) to model bimodal psychometric curves and to investigate whether ferrets integrate modalities in an optimal manner. Furthermore, to test for a redundant signal effect we pooled the reaction times of all animals to calculate a race model. We observed that bimodal detection thresholds were reduced and reaction times were faster in the bimodal compared to unimodal conditions. The race model and MLE modeling showed that ferrets integrate modalities in a statistically optimal fashion. Taken together, the data indicate that principles of multisensory integration previously demonstrated in other species also apply to crossmodal processing in the ferret.