Vision: Visual space is not what it appears to be

The location of visual objects in the world around us is reconstructed in a complex way from the image falling on the retina. Recent studies have begun to reveal the different ways in which the brain dynamically re-maps retinal information across eye movements to compute object locations for perception and directing actions.     

Eye Tracking the Face in the Crowd Task: Why Are Angry Faces Found More Quickly?

Among a crowd of distractor faces, threatening or angry target faces are identified more quickly and accurately than are nonthreatening or happy target faces, a finding known as the “face in the crowd effect.” Two perceptual explanations of the effect have been proposed: (1) the “target orienting” hypothesis (i.e., threatening targets orient attention more quickly than do nonthreatening targets and (2) the “distractor processing” hypothesis (i.e., nonthreatening distractors paired with a threatening target are processed more efficiently than vice versa, leading to quicker detection of threatening targets). Using a task, with real faces and multiple identities, the current study replicated the face in the crowd effect and then, via eye tracking, found greater support for the target orienting hypothesis. Across both the classical search asymmetry paradigm (i.e., one happy target in a crowd of angry distractors vs. one angry target in a crowd of happy distractors) and the constant distractor paradigm (i.e., one happy target in a crowd of neutral distractors vs. one angry target in a crowd of neutral distractors), fewer distractors were fixated before first fixating angry targets relative to happy targets, with no difference in the processing efficiency of distractors. These results suggest that the face in the crowd effect on this task is supported to a greater degree by attentional patterns associated with properties of target rather those of the crowd.     

Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment

Auditory and visual signals generated by a single source tend to be temporally correlated, such as the synchronous sounds of footsteps and the limb movements of a walker. Continuous tracking and comparison of the dynamics of auditory-visual streams is thus useful for the perceptual binding of information arising from a common source. Although language-related mechanisms have been implicated in the tracking of speech-related auditory-visual signals (e.g., speech sounds and lip movements), it is not well known what sensory mechanisms generally track ongoing auditory-visual synchrony for non-speech signals in a complex auditory-visual environment. To begin to address this question, we used music and visual displays that varied in the dynamics of multiple features (e.g., auditory loudness and pitch; visual luminance, color, size, motion, and organization) across multiple time scales. Auditory activity (monitored using auditory steady-state responses, ASSR) was selectively reduced in the left hemisphere when the music and dynamic visual displays were temporally misaligned. Importantly, ASSR was not affected when attentional engagement with the music was reduced, or when visual displays presented dynamics clearly dissimilar to the music. These results appear to suggest that left-lateralized auditory mechanisms are sensitive to auditory-visual temporal alignment, but perhaps only when the dynamics of auditory and visual streams are similar. These mechanisms may contribute to correct auditory-visual binding in a busy sensory environment.     

Further Explorations of the Facing Bias in Biological Motion Perception: Perspective Cues,Observer Sex,and Response Times

The human visual system has evolved to be highly sensitive to visual information about other persons and their movements as is illustrated by the effortless perception of point-light figures or ‘biological motion’. When presented orthographically, a point-light walker is interpreted in two anatomically plausible ways: As ‘facing the viewer’ or as ‘facing away’ from the viewer. However, human observers show a ‘facing bias’: They perceive such a point-light walker as facing towards them in about 70-80% of the cases. In studies exploring the role of social and biological relevance as a possible account for the facing bias, we found a ‘figure gender effect’: Male point-light figures elicit a stronger facing bias than female point-light figures. Moreover, we also found an ‘observer gender effect’: The ‘figure gender effect’ was stronger for male than for female observers. In the present study we presented to 11 males and 11 females point-light walkers of which, very subtly, the perspective information was manipulated by modifying the earlier reported ‘perspective technique’. Proportions of ‘facing the viewer’ responses and reaction times were recorded. Results show that human observers, even in the absence of local shape or size cues, easily pick up on perspective cues, confirming recent demonstrations of high visual sensitivity to cues on whether another person is potentially approaching. We also found a consistent difference in how male and female observers respond to stimulus variations (figure gender or perspective cues) that cause variations in the perceived in-depth orientation of a point-light walker. Thus, the ‘figure gender effect’ is possibly caused by changes in the relative locations and motions of the dots that the perceptual system tends to interpret as perspective cues. Third, reaction time measures confirmed the existence of the facing bias and recent research showing faster detection of approaching than receding biological motion.     

Fixational eye movements predict visual sensitivity

During steady fixation, observers make small fixational saccades at a rate of around 1–2 per second. Presentation of a visual stimulus triggers a biphasic modulation in fixational saccade rate—an initial inhibition followed by a period of elevated rate and a subsequent return to baseline. Here we show that, during passive viewing, this rate signature is highly sensitive to small changes in stimulus contrast. By training a linear support vector machine to classify trials in which a stimulus is either present or absent, we directly compared the contrast sensitivity of fixational eye movements with individuals'' psychophysical judgements. Classification accuracy closely matched psychophysical performance, and predicted individuals'' threshold estimates with less bias and overall error than those obtained using specific features of the signature. Performance of the classifier was robust to changes in the training set (novel subjects and/or contrasts) and good prediction accuracy was obtained with a practicable number of trials. Our results indicate a tight coupling between the sensitivity of visual perceptual judgements and fixational eye control mechanisms. This raises the possibility that fixational saccades could provide a novel and objective means of estimating visual contrast sensitivity without the need for observers to make any explicit judgement.     

The peeping response of pigeons (Columba livia) to isolation from conspecifics and exposure to a novel environment

The present study examined the acute behavioral responses of pigeons to separation from conspecifics and exposure to an unfamiliar environment (UE). The effects of (1) repeated exposure to the UE; (2) visual isolation from surroundings, or saline injections; and (3) diazepam treatment (i.p., 0.25, 0.75, 2.5 or 7.5 mg/kg) before the trial were also examined. UE exposure evoked intense ballistic head movements (peeping), gradually replaced with angular head movements (AHM), both associated with immobility of the trunk and legs. These behaviors failed to habituate after three trials (7-day intertrial intervals). Visual isolation from the surroundings and saline injection prior to exposure to the UE increased the AHM and reduced peeping. Doses of diazepam (0.25 and 0.75 mg/kg) that have demonstrated anti-conflict effects in other tests did not affect the behavioral responses to the UE. Diazepam at 2.5 and 7.5 mg/kg doses consistently increased time spent in immobility. These data suggest that peeping, although expressed in potentially threatening or harmful situations appears not to be a fear-motivated behavior or, alternatively, this specific behavioral response is not diazepam sensitive.     

Modulation of brain activity during action observation: influence of perspective, transitivity and meaningfulness

The coupling process between observed and performed actions is thought to be performed by a fronto-parietal perception-action system including regions of the inferior frontal gyrus and the inferior parietal lobule. When investigating the influence of the movements' characteristics on this process, most research on action observation has focused on only one particular variable even though the type of movements we observe can vary on several levels. By manipulating the visual perspective, transitivity and meaningfulness of observed movements in a functional magnetic resonance imaging study we aimed at investigating how the type of movements and the visual perspective can modulate brain activity during action observation in healthy individuals. Importantly, we used an active observation task where participants had to subsequently execute or imagine the observed movements. Our results show that the fronto-parietal regions of the perception action system were mostly recruited during the observation of meaningless actions while visual perspective had little influence on the activity within the perception-action system. Simultaneous investigation of several sources of modulation during active action observation is probably an approach that could lead to a greater ecological comprehension of this important sensorimotor process.     

Saccadic facilitation in natural backgrounds

In visual systems with a fovea, only a small portion of the visual field can be analyzed with high accuracy. Saccadic eye movements shift that center of gaze around several times a second. Saccades have been characterized in great detail and depend critically on a number of visual properties of the stimuli. However, typical experiments have used bright spots on dark backgrounds, while our natural environment has a highly characteristic rich spatial structure. Here we show that the saccadic system, unlike the perceptual system, is able to compensate for the masking caused by structured backgrounds. Consequently, saccadic latencies in the context of natural backgrounds are much faster than unstructured backgrounds at equal levels of visibility. The results suggest that whenever a structured background acts to mask the visibility of the saccade target, it simultaneously preactivates saccadic circuitry and thus ensures a fast reaction to potentially critical stimuli that are difficult to detect in our environment.     

Numerical study of the effect of head and eye movement on progression of retinal detachment

Rhegmatogenous retinal detachment (RD) is a sight threatening condition. In this type of RD a break in the retina allows retrohyaloid fluid to enter the subretinal space. The prognosis concerning the patients’ visual acuity is better if the RD has not progressed to the macula. The patient is given a posturing advice of bed rest and semi-supine positioning (with the RD as low as possible) to allow the utilisation of gravity and immobilisation in preventing progression of the RD. It is, however, unknown what external loads on the eye contribute the most to the progression of a RD. The goal of this exploratory study is to elucidate the role of eye movements caused by head movements and saccades on the progression of an RD. A finite element model is produced and evaluated in this study. The model is based on geometric and material properties reported in the literature. The model shows that a mild head movement and a severe eye movement produce similar traction loads on the retina. This implies that head movements—and not eye movements—are able to cause loads that can trigger and progress an RD. These preliminary results suggest that head movements have a larger effect on the progression of an RD than saccadic eye movements. This study is the first to use numerical analysis to investigate the development and progression of RD and shows promise for future work.     

Diagnostic features of emotional expressions are processed preferentially

Diagnostic features of emotional expressions are differentially distributed across the face. The current study examined whether these diagnostic features are preferentially attended to even when they are irrelevant for the task at hand or when faces appear at different locations in the visual field. To this aim, fearful, happy and neutral faces were presented to healthy individuals in two experiments while measuring eye movements. In Experiment 1, participants had to accomplish an emotion classification, a gender discrimination or a passive viewing task. To differentiate fast, potentially reflexive, eye movements from a more elaborate scanning of faces, stimuli were either presented for 150 or 2000 ms. In Experiment 2, similar faces were presented at different spatial positions to rule out the possibility that eye movements only reflect a general bias for certain visual field locations. In both experiments, participants fixated the eye region much longer than any other region in the face. Furthermore, the eye region was attended to more pronouncedly when fearful or neutral faces were shown whereas more attention was directed toward the mouth of happy facial expressions. Since these results were similar across the other experimental manipulations, they indicate that diagnostic features of emotional expressions are preferentially processed irrespective of task demands and spatial locations. Saliency analyses revealed that a computational model of bottom-up visual attention could not explain these results. Furthermore, as these gaze preferences were evident very early after stimulus onset and occurred even when saccades did not allow for extracting further information from these stimuli, they may reflect a preattentive mechanism that automatically detects relevant facial features in the visual field and facilitates the orientation of attention towards them. This mechanism might crucially depend on amygdala functioning and it is potentially impaired in a number of clinical conditions such as autism or social anxiety disorders.     

Neural mechanisms for the recognition of biological movements

The visual recognition of complex movements and actions is crucial for the survival of many species. It is important not only for communication and recognition at a distance, but also for the learning of complex motor actions by imitation. Movement recognition has been studied in psychophysical, neurophysiological and imaging experiments, and several cortical areas involved in it have been identified. We use a neurophysiologically plausible and quantitative model as a tool for organizing and making sense of the experimental data, despite their growing size and complexity. We review the main experimental findings and discuss possible neural mechanisms, and show that a learning-based, feedforward model provides a neurophysiologically plausible and consistent summary of many key experimental results.     

Looking at Op Art from a computational viewpoint

Arts history tells an exciting story about repeated attempts to represent features that are crucial for the understanding of our environment and which, at the same time, go beyond the inherently two-dimensional nature of a flat painting surface: depth and motion. In the twentieth century, Op artists such as Bridget Riley began to experiment with simple black and white patterns that do not represent motion in an artistic way but actually create vivid dynamic illusions in static pictures. The cause of motion illusions in such paintings is still a matter of debate. The role of involuntary eye movements in this phenomenon is studied here with a computational approach. The possible consequences of shifting the retinal image of synthetic wave gratings, dubbed as 'riloids', were analysed by a two-dimensional array of motion detectors (2DMD model), which generates response maps representing the spatial distribution of motion signals generated by such a stimulus. For a two-frame sequence reflecting a saccadic displacement, these motion signal maps contain extended patches in which local directions change only little. These directions, however, do not usually precisely correspond to the direction of pattern displacement that can be expected from the geometry of the curved gratings as an instance of the so-called 'aperture problem'. The patchy structure of the simulated motion detector response to the displacement of riloids resembles the motion illusion, which is not perceived as a coherent shift of the whole pattern but as a wobbling and jazzing of ill-defined regions. Although other explanations are not excluded, this might support the view that the puzzle of Op Art motion illusions could potentially have an almost trivial solution in terms of small involuntary eye movement leading to image shifts that are picked up by well-known motion detectors in the early visual system. This view can have further consequences for our understanding of how the human visual system usually compensates for eye movements, in order to let us perceive a stable world despite continuous image shifts generated by gaze instability.     

Evolution and Optimality of Similar Neural Mechanisms for Perception and Action during Search

A prevailing theory proposes that the brain''s two visual pathways, the ventral and dorsal, lead to differing visual processing and world representations for conscious perception than those for action. Others have claimed that perception and action share much of their visual processing. But which of these two neural architectures is favored by evolution? Successful visual search is life-critical and here we investigate the evolution and optimality of neural mechanisms mediating perception and eye movement actions for visual search in natural images. We implement an approximation to the ideal Bayesian searcher with two separate processing streams, one controlling the eye movements and the other stream determining the perceptual search decisions. We virtually evolved the neural mechanisms of the searchers'' two separate pathways built from linear combinations of primary visual cortex receptive fields (V1) by making the simulated individuals'' probability of survival depend on the perceptual accuracy finding targets in cluttered backgrounds. We find that for a variety of targets, backgrounds, and dependence of target detectability on retinal eccentricity, the mechanisms of the searchers'' two processing streams converge to similar representations showing that mismatches in the mechanisms for perception and eye movements lead to suboptimal search. Three exceptions which resulted in partial or no convergence were a case of an organism for which the targets are equally detectable across the retina, an organism with sufficient time to foveate all possible target locations, and a strict two-pathway model with no interconnections and differential pre-filtering based on parvocellular and magnocellular lateral geniculate cell properties. Thus, similar neural mechanisms for perception and eye movement actions during search are optimal and should be expected from the effects of natural selection on an organism with limited time to search for food that is not equi-detectable across its retina and interconnected perception and action neural pathways.     

Self-recognition of avatar motion: how do I know it's me?

When motion is isolated from form cues and viewed from third-person perspectives, individuals are able to recognize their own whole body movements better than those of friends. Because we rarely see our own bodies in motion from third-person viewpoints, this self-recognition advantage may indicate a contribution to perception from the motor system. Our first experiment provides evidence that recognition of self-produced and friends' motion dissociate, with only the latter showing sensitivity to orientation. Through the use of selectively disrupted avatar motion, our second experiment shows that self-recognition of facial motion is mediated by knowledge of the local temporal characteristics of one's own actions. Specifically, inverted self-recognition was unaffected by disruption of feature configurations and trajectories, but eliminated by temporal distortion. While actors lack third-person visual experience of their actions, they have a lifetime of proprioceptive, somatosensory, vestibular and first-person-visual experience. These sources of contingent feedback may provide actors with knowledge about the temporal properties of their actions, potentially supporting recognition of characteristic rhythmic variation when viewing self-produced motion. In contrast, the ability to recognize the motion signatures of familiar others may be dependent on configural topographic cues.     

Both Physical Exercise and Progressive Muscle Relaxation Reduce the Facing-the-Viewer Bias in Biological Motion Perception

Biological motion stimuli, such as orthographically projected stick figure walkers, are ambiguous about their orientation in depth. The projection of a stick figure walker oriented towards the viewer, therefore, is the same as its projection when oriented away. Even though such figures are depth-ambiguous, however, observers tend to interpret them as facing towards them more often than facing away. Some have speculated that this facing-the-viewer bias may exist for sociobiological reasons: Mistaking another human as retreating when they are actually approaching could have more severe consequences than the opposite error. Implied in this hypothesis is that the facing-towards percept of biological motion stimuli is potentially more threatening. Measures of anxiety and the facing-the-viewer bias should therefore be related, as researchers have consistently found that anxious individuals display an attentional bias towards more threatening stimuli. The goal of this study was to assess whether physical exercise (Experiment 1) or an anxiety induction/reduction task (Experiment 2) would significantly affect facing-the-viewer biases. We hypothesized that both physical exercise and progressive muscle relaxation would decrease facing-the-viewer biases for full stick figure walkers, but not for bottom- or top-half-only human stimuli, as these carry less sociobiological relevance. On the other hand, we expected that the anxiety induction task (Experiment 2) would increase facing-the-viewer biases for full stick figure walkers only. In both experiments, participants completed anxiety questionnaires, exercised on a treadmill (Experiment 1) or performed an anxiety induction/reduction task (Experiment 2), and then immediately completed a perceptual task that allowed us to assess their facing-the-viewer bias. As hypothesized, we found that physical exercise and progressive muscle relaxation reduced facing-the-viewer biases for full stick figure walkers only. Our results provide further support that the facing-the-viewer bias for biological motion stimuli is related to the sociobiological relevance of such stimuli.     

Visual Feedback Dominates the Sense of Agency for Brain-Machine Actions

Recent advances in neuroscience and engineering have led to the development of technologies that permit the control of external devices through real-time decoding of brain activity (brain-machine interfaces; BMI). Though the feeling of controlling bodily movements (sense of agency; SOA) has been well studied and a number of well-defined sensorimotor and cognitive mechanisms have been put forth, very little is known about the SOA for BMI-actions. Using an on-line BMI, and verifying that our subjects achieved a reasonable level of control, we sought to describe the SOA for BMI-mediated actions. Our results demonstrate that discrepancies between decoded neural activity and its resultant real-time sensory feedback are associated with a decrease in the SOA, similar to SOA mechanisms proposed for bodily actions. However, if the feedback discrepancy serves to correct a poorly controlled BMI-action, then the SOA can be high and can increase with increasing discrepancy, demonstrating the dominance of visual feedback on the SOA. Taken together, our results suggest that bodily and BMI-actions rely on common mechanisms of sensorimotor integration for agency judgments, but that visual feedback dominates the SOA in the absence of overt bodily movements or proprioceptive feedback, however erroneous the visual feedback may be.     

An upper-body can improve the stability and efficiency of passive dynamic walking

The compass-gait walker proposed by McGeer can walk down a shallow slope with a self-stabilizing gait that requires no actuation or control. However, as the slope goes to zero so does the walking speed, and dynamic gait stability is only possible over a very narrow range of slopes. Gomes and Ruina have results demonstrating that by adding a torso to the compass-gait walker, it can walk passively on level-ground with a non-infinitesimal constant average speed. However, the gait involves exaggerated joint movements, and for energetic reasons horizontal passive dynamic walking cannot be stable. We show in this research that in addition to collision-free walking, adding a torso improves stability and walking speed when walking downhill. Furthermore, adding arms to the torso results in a collision-free periodic gait with natural-looking torso and limb movements. Overall, in contrast to the suggestions that active control may be needed to balance an upper-body on legs, it turns out that the upper and lower bodies can be integrated to improve the stability, efficiency and speed of a passive dynamic walker.     

Footballer's Migraine

Classical migraine, including incapacitating visual field defects, repeatedly developed in five young men immediately after blows to the head while playing football and in no other circumstances. A similar condition occurred in a professional boxer and an isolated attack in a boy footballer. Prophylactic treatment with ergotamine tartrate may not be wholly successful and it may be necessary to give up the sport. Any unitary theory of causation of attacks of migraine must account for prodromal symptoms immediately after head injury.     

Infants in control: rapid anticipation of action outcomes in a gaze-contingent paradigm

Infants' poor motor abilities limit their interaction with their environment and render studying infant cognition notoriously difficult. Exceptions are eye movements, which reach high accuracy early, but generally do not allow manipulation of the physical environment. In this study, real-time eye tracking is used to put 6- and 8-month-old infants in direct control of their visual surroundings to study the fundamental problem of discovery of agency, i.e. the ability to infer that certain sensory events are caused by one's own actions. We demonstrate that infants quickly learn to perform eye movements to trigger the appearance of new stimuli and that they anticipate the consequences of their actions in as few as 3 trials. Our findings show that infants can rapidly discover new ways of controlling their environment. We suggest that gaze-contingent paradigms offer effective new ways for studying many aspects of infant learning and cognition in an interactive fashion and provide new opportunities for behavioral training and treatment in infants.     

Rapid Perceptual Switching of a Reversible Biological Figure

Certain visual stimuli can give rise to contradictory perceptions. In this paper we examine the temporal dynamics of perceptual reversals experienced with biological motion, comparing these dynamics to those observed with other ambiguous structure from motion (SFM) stimuli. In our first experiment, naïve observers monitored perceptual alternations with an ambiguous rotating walker, a figure that randomly alternates between walking in clockwise (CW) and counter-clockwise (CCW) directions. While the number of reported reversals varied between observers, the observed dynamics (distribution of dominance durations, CW/CCW proportions) were comparable to those experienced with an ambiguous kinetic depth cylinder. In a second experiment, we compared reversal profiles with rotating and standard point-light walkers (i.e. non-rotating). Over multiple test repetitions, three out of four observers experienced consistently shorter mean percept durations with the rotating walker, suggesting that the added rotational component may speed up reversal rates with biomotion. For both stimuli, the drift in alternation rate across trial and across repetition was minimal. In our final experiment, we investigated whether reversals with the rotating walker and a non-biological object with similar global dimensions (rotating cuboid) occur at random phases of the rotation cycle. We found evidence that some observers experience peaks in the distribution of response locations that are relatively stable across sessions. Using control data, we discuss the role of eye movements in the development of these reversal patterns, and the related role of exogenous stimulus characteristics. In summary, we have demonstrated that the temporal dynamics of reversal with biological motion are similar to other forms of ambiguous SFM. We conclude that perceptual switching with biological motion is a robust bistable phenomenon.