Naturalistic Stimulus Structure Determines the Integration of Audiovisual Looming Signals in Binocular Rivalry

Rapid integration of biologically relevant information is crucial for the survival of an organism. Most prominently, humans should be biased to attend and respond to looming stimuli that signal approaching danger (e.g. predator) and hence require rapid action. This psychophysics study used binocular rivalry to investigate the perceptual advantage of looming (relative to receding) visual signals (i.e. looming bias) and how this bias can be influenced by concurrent auditory looming/receding stimuli and the statistical structure of the auditory and visual signals.Subjects were dichoptically presented with looming/receding visual stimuli that were paired with looming or receding sounds. The visual signals conformed to two different statistical structures: (1) a ‘simple’ random-dot kinematogram showing a starfield and (2) a “naturalistic” visual Shepard stimulus. Likewise, the looming/receding sound was (1) a simple amplitude- and frequency-modulated (AM-FM) tone or (2) a complex Shepard tone. Our results show that the perceptual looming bias (i.e. the increase in dominance times for looming versus receding percepts) is amplified by looming sounds, yet reduced and even converted into a receding bias by receding sounds. Moreover, the influence of looming/receding sounds on the visual looming bias depends on the statistical structure of both the visual and auditory signals. It is enhanced when audiovisual signals are Shepard stimuli.In conclusion, visual perception prioritizes processing of biologically significant looming stimuli especially when paired with looming auditory signals. Critically, these audiovisual interactions are amplified for statistically complex signals that are more naturalistic and known to engage neural processing at multiple levels of the cortical hierarchy.     

Multisensory integration of looming signals by rhesus monkeys

Looming objects produce ecologically important signals that can be perceived in both the visual and auditory domains. Using a preferential looking technique with looming and receding visual and auditory stimuli, we examined the multisensory integration of looming stimuli by rhesus monkeys. We found a strong attentional preference for coincident visual and auditory looming but no analogous preference for coincident stimulus recession. Consistent with previous findings, the effect occurred only with tonal stimuli and not with broadband noise. The results suggest an evolved capacity to integrate multisensory looming objects.     

Alternation of Sound Location Induces Visual Motion Perception of a Static Object

Background

Audition provides important cues with regard to stimulus motion although vision may provide the most salient information. It has been reported that a sound of fixed intensity tends to be judged as decreasing in intensity after adaptation to looming visual stimuli or as increasing in intensity after adaptation to receding visual stimuli. This audiovisual interaction in motion aftereffects indicates that there are multimodal contributions to motion perception at early levels of sensory processing. However, there has been no report that sounds can induce the perception of visual motion.

Methodology/Principal Findings

A visual stimulus blinking at a fixed location was perceived to be moving laterally when the flash onset was synchronized to an alternating left-right sound source. This illusory visual motion was strengthened with an increasing retinal eccentricity (2.5 deg to 20 deg) and occurred more frequently when the onsets of the audio and visual stimuli were synchronized.

Conclusions/Significance

We clearly demonstrated that the alternation of sound location induces illusory visual motion when vision cannot provide accurate spatial information. The present findings strongly suggest that the neural representations of auditory and visual motion processing can bias each other, which yields the best estimates of external events in a complementary manner.     

The effect of looming and receding sounds on the perceived in-depth orientation of depth-ambiguous biological motion figures

Background

The focus in the research on biological motion perception traditionally has been restricted to the visual modality. Recent neurophysiological and behavioural evidence, however, supports the idea that actions are not represented merely visually but rather audiovisually. The goal of the present study was to test whether the perceived in-depth orientation of depth-ambiguous point-light walkers (plws) is affected by the presentation of looming or receding sounds synchronized with the footsteps.

Methodology/Principal Findings

In Experiment 1 orthographic frontal/back projections of plws were presented either without sound or with sounds of which the intensity level was rising (looming), falling (receding) or stationary. Despite instructions to ignore the sounds and to only report the visually perceived in-depth orientation, plws accompanied with looming sounds were more often judged to be facing the viewer whereas plws paired with receding sounds were more often judged to be facing away from the viewer. To test whether the effects observed in Experiment 1 act at a perceptual level rather than at the decisional level, in Experiment 2 observers perceptually compared orthographic plws without sound or paired with either looming or receding sounds to plws without sound but with perspective cues making them objectively either facing towards or facing away from the viewer. Judging whether either an orthographic plw or a plw with looming (receding) perspective cues is visually most looming becomes harder (easier) when the orthographic plw is paired with looming sounds.

Conclusions/Significance

The present results suggest that looming and receding sounds alter the judgements of the in-depth orientation of depth-ambiguous point-light walkers. While looming sounds are demonstrated to act at a perceptual level and make plws look more looming, it remains a challenge for future research to clarify at what level in the processing hierarchy receding sounds affect how observers judge the in-depth perception of plws.     

Iterative cooperation between parallel pathways for object and background motion

In a typical visual scene, one or more objects move relative to a larger background, which can itself be in motion as a result of the observer’s eyes moving with respect to the outside world. Here we show that accurate estimation of the background motion from an image velocity field can be accomplished through an iterative cooperation between two modules: one that specializes in calculating a weighted average velocity and another one calculating a velocity contrast map. We build on our analysis to provide a model for the tectum-pretectum loop in the nonmammalian midbrain. Our model accounts for some of the known properties of the tectal neurons (sensitivity to relative motion) and pretectal neurons (sensitivity to whole-field motion). It also agrees with our knowledge of the pretectotectal projection (divergent and inhibitory), and with the results of lesion studies in which the pretectal input to the tectum was removed, leading to hyperactivity of the tectal neurons and the animal. Our model also makes a testable prediction regarding the tectopretectal projection, i.e., that the presence of a larger object and a bigger discrepancy between the directions of motion for the object and the background lead to a larger error by the pretectum in estimating the background motion when the tectal input is abolished.     

Responses of descending neurons to looming stimuli in the praying mantis <Emphasis Type="Italic">Tenodera aridifolia</Emphasis>

Responses to visual stimuli of some neurons that descend the nerve cord from the brain were recorded extracellularly in the mantis Tenodera aridifolia. Most of the recorded neurons showed their largest responses to looming stimuli that simulated a black circle approaching towards the mantis. The neurons showed a transient excitatory response to a gradually darkening or receding circle. The neurons showed sustained excitation to the linearly expanding stimuli, but the spike frequency decreased rapidly. The responses of the neurons were affected by both the diameter and the speed of looming stimuli. Faster or smaller looming stimuli elicited a higher peak frequency. These responses were observed in both recordings from the connective between suboesophageal and prothoracic ganglia and the connective between prothoracic and mesothoracic ganglia. There was a one-to-one correspondence of spike firing between these two recordings with a fixed delay. The neurons had the receptive field on ipsilateral side to its axon at the cervical connective. These results suggest that there is a looming-sensitive descending neuron, with an axon projecting over prothoracic ganglion, in the mantis nervous system.     

Motion processing with wide-field neurons in the retino-tecto-rotundal pathway

The retino-tecto-rotundal pathway is the main visual pathway in non-mammalian vertebrates and has been found to be highly involved in visual processing. Despite the extensive receptive fields of tectal and rotundal wide-field neurons, pattern discrimination tasks suggest a system with high spatial resolution. In this paper, we address the problem of how global processing performed by motion-sensitive wide-field neurons can be brought into agreement with the concept of a local analysis of visual stimuli. As a solution to this problem, we propose a firing-rate model of the retino-tecto-rotundal pathway which describes how spatiotemporal information can be organized and retained by tectal and rotundal wide-field neurons while processing Fourier-based motion in absence of periodic receptive-field structures. The model incorporates anatomical and electrophysiological experimental data on tectal and rotundal neurons, and the basic response characteristics of tectal and rotundal neurons to moving stimuli are captured by the model cells. We show that local velocity estimates may be derived from rotundal-cell responses via superposition in a subsequent processing step. Experimentally testable predictions which are both specific and characteristic to the model are provided. Thus, a conclusive explanation can be given of how the retino-tecto-rotundal pathway enables the animal to detect and localize moving objects or to estimate its self-motion parameters.     

Insect detection of small targets moving in visual clutter

Detection of targets that move within visual clutter is a common task for animals searching for prey or conspecifics, a task made even more difficult when a moving pursuer needs to analyze targets against the motion of background texture (clutter). Despite the limited optical acuity of the compound eye of insects, this challenging task seems to have been solved by their tiny visual system. Here we describe neurons found in the male hoverfly,Eristalis tenax, that respond selectively to small moving targets. Although many of these target neurons are inhibited by the motion of a background pattern, others respond to target motion within the receptive field under a surprisingly large range of background motion stimuli. Some neurons respond whether or not there is a speed differential between target and background. Analysis of responses to very small targets (smaller than the size of the visual field of single photoreceptors) or those targets with reduced contrast shows that these neurons have extraordinarily high contrast sensitivity. Our data suggest that rejection of background motion may result from extreme selectivity for small targets contrasting against local patches of the background, combined with this high sensitivity, such that background patterns rarely contain features that satisfactorily drive the neuron.     

Human Vection Perception Using Inertial Nulling and Certainty Estimation: The Effect of Migraine History

Vection is an illusory perception of self-motion that can occur when visual motion fills the majority of the visual field. This study examines the effect of the duration of visual field movement (VFM) on the perceived strength of self-motion using an inertial nulling (IN) and a magnitude estimation technique based on the certainty that motion occurred (certainty estimation, CE). These techniques were then used to investigate the association between migraine diagnosis and the strength of perceived vection. Visual star-field stimuli consistent with either looming or receding motion were presented for 1, 4, 8 or 16s. Subjects reported the perceived direction of self-motion during the final 1s of the stimulus. For the IN method, an inertial nulling motion was delivered during this final 1s of the visual stimulus, and subjects reported the direction of perceived self-motion during this final second. The magnitude of inertial motion was varied adaptively to determine the point of subjective equality (PSE) at which forward or backward responses were equally likely. For the CE trials the same range of VFM was used but without inertial motion and subjects rated their certainty of motion on a scale of 0–100. PSE determined with the IN technique depended on direction and duration of visual motion and the CE technique showed greater certainty of perceived vection with longer VFM duration. A strong correlation between CE and IN techniques was present for the 8s stimulus. There was appreciable between-subject variation in both CE and IN techniques and migraine was associated with significantly increased perception of self-motion by CE and IN at 8 and 16s. Together, these results suggest that vection may be measured by both CE and IN techniques with good correlation. The results also suggest that susceptibility to vection may be higher in subjects with a history of migraine.     

Adaptive surround modulation in cortical area MT

Visual motion perception relies on two opposing operations: integration and segmentation. Integration overcomes motion ambiguity in the visual image by spatial pooling of motion signals, whereas segmentation identifies differences between adjacent moving objects. For visual motion area MT, previous investigations have reported that stimuli in the receptive field surround, which do not elicit a response when presented alone, can nevertheless modulate responses to stimuli in the receptive field center. The directional tuning of this "surround modulation" has been found to be mainly antagonistic and hence consistent with segmentation. Here, we report that surround modulation in area MT can be either antagonistic or integrative depending upon the visual stimulus. Both types of modulation were delayed relative to response onset. Our results suggest that the dominance of antagonistic modulation in previous MT studies was due to stimulus choice and that segmentation and integration are achieved, in part, via adaptive surround modulation.     

Responses of medullary neurons to moving visual stimuli in the common toad

The concept of coded 'command releasing systems' proposes that visually specialized descending tectal (and pretectal) neurons converge on motor pattern generating medullary circuits and release--in goal-specific combination--specific action patterns. Extracellular recordings from medullary neurons of the medial reticular formation of the awake immobilized toad in response to moving visual stimuli revealed the following main results. (i) Properties of medullary neurons were distinguished by location, shape, and size of visual receptive fields (ranging from relatively small to wide), by trigger features of various moving configural stimulus objects (including prey- and predator-selective properties), by tactile sensitivity, and by firing pattern characteristics (sluggish, tonic, warming-up, and cyclic). (ii) Visual receptive fields of medullary neurons and their responses to moving configural objects suggest converging inputs of tectal (and pretectal) descending neurons. (iii) In contrast to tectal monocular 'small-field' neurons, the excitatory visual receptive fields of comparable medullary neurons were larger, ellipsoidally shaped, mostly oriented horizontally, and not topographically mapped in an obvious fashion. Furthermore, configural feature discrimination was sharper. (iv) The observation of multiple properties in most medullary neurons (partly showing combined visual and cutaneous sensitivities) suggests integration of various inputs by these cells, and this is in principle consistent with the concept of command releasing systems. (v) There is evidence for reciprocal tectal/medullary excitatory pathways suitable for premotor warming-up. (vi) Cyclic bursting of many neurons, spontaneously or as a post-stimulus sustaining event, points to a medullary premotor/motor property.     

Integrating biological motion: the role of grouping in the perception of point-light actions

The human visual system is highly sensitive to biological motion and manages to organize even a highly reduced point-light stimulus into a vivid percept of human action. The current study investigated to what extent the origin of this saliency of point-light displays is related to its intrinsic Gestalt qualities. In particular, we studied whether biological motion perception is facilitated when the elements can be grouped according to good continuation and similarity as Gestalt principles of perceptual organization. We found that both grouping principles enhanced biological motion perception but their effects differed when stimuli were inverted. These results provide evidence that Gestalt principles of good continuity and similarity also apply to more complex and dynamic meaningful stimuli.     

Neural processing of auditory looming in the human brain

Acoustic intensity change, along with interaural, spectral, and reverberation information, is an important cue for the perception of auditory motion. Approaching sound sources produce increases in intensity, and receding sound sources produce corresponding decreases. Human listeners typically overestimate increasing compared to equivalent decreasing sound intensity and underestimate the time to contact of approaching sound sources. These characteristics could provide a selective advantage by increasing the margin of safety for response to looming objects. Here, we used dynamic intensity and functional magnetic resonance imaging to examine the neural underpinnings of the perceptual priority for rising intensity. We found that, consistent with activation by horizontal and vertical auditory apparent motion paradigms, rising and falling intensity activated the right temporal plane more than constant intensity. Rising compared to falling intensity activated a distributed neural network subserving space recognition, auditory motion perception, and attention and comprising the superior temporal sulci and the middle temporal gyri, the right temporoparietal junction, the right motor and premotor cortices, the left cerebellar cortex, and a circumscribed region in the midbrain. This anisotropic processing of acoustic intensity change may reflect the salience of rising intensity produced by looming sources in natural environments.     

Deep tectal cells in pigeons respond to kinematograms

Summary Deep tectal neurons in pigeons respond selectively to moving visual stimuli, and are inhibited by large background patterns moved in-phase with these stimuli. In this investigation we demonstrate that these same deep tectal neurons respond equally well to kinematograms as they do to traditional luminance contrast stimuli typically employed in visual experiments.Computer generated kinematograms, the motion domain equivalents of random dot stereograms, were used as stimuli in these experiments. These kinematograms, where a small centrally located set of random dots is moved coherently in one direction while the remaining dots are moved in a different direction, thus constitute a pure motion stimulus where the stimulus form is only visible in the dynamic pattern, but does not exist on any single frame. Both object configured and hole configured kinematograms were employed; the former appearing as regions of texture moving over, or in front of, the background texture, while the latter appear as windows through which a more distant textured surface is revealed.Extracellular recordings from isolated deep tectal cells showed that all units responded in a very similar manner whether the stimulus was an object configured kinematogram or the more traditional luminance contrast variety. This similarity included directional selectivity, the in-phase inhibition anti-phase facilitation effect, and sensitivity to opposed motion independent of direction. However, when the kinematograms were configured as holes none of the units tested responded to these stimuli. The significance of these observations for tectal functioning, image segmentation through motion and animal camouflage is discussed.     

A simple vision-based algorithm for decision making in flying Drosophila

Animals must quickly recognize objects in their environment and act accordingly. Previous studies indicate that looming visual objects trigger avoidance reflexes in many species [1-5]; however, such reflexes operate over a close range and might not detect a threatening stimulus at a safe distance. We analyzed how fruit flies (Drosophila melanogaster) respond to simple visual stimuli both in free flight and in a tethered-flight simulator. Whereas Drosophila, like many other insects, are attracted toward long vertical objects [6-10], we found that smaller visual stimuli elicit not weak attraction but rather strong repulsion. Because aversion to small spots depends on the vertical size of a moving object, and not on looming, it can function at a much greater distance than expansion-dependent reflexes. The opposing responses to long stripes and small spots reflect a simple but effective object classification system. Attraction toward long stripes would lead flies toward vegetative perches or feeding sites, whereas repulsion from small spots would help them avoid aerial predators or collisions with other insects. The motion of flying Drosophila depends on a balance of these two systems, providing a foundation for studying the neural basis of behavioral choice in a genetic model organism.     

Properties of V1 neurons tuned to conjunctions of visual features: application of the V1 saliency hypothesis to visual search behavior

From a computational theory of V1, we formulate an optimization problem to investigate neural properties in the primary visual cortex (V1) from human reaction times (RTs) in visual search. The theory is the V1 saliency hypothesis that the bottom-up saliency of any visual location is represented by the highest V1 response to it relative to the background responses. The neural properties probed are those associated with the less known V1 neurons tuned simultaneously or conjunctively in two feature dimensions. The visual search is to find a target bar unique in color (C), orientation (O), motion direction (M), or redundantly in combinations of these features (e.g., CO, MO, or CM) among uniform background bars. A feature singleton target is salient because its evoked V1 response largely escapes the iso-feature suppression on responses to the background bars. The responses of the conjunctively tuned cells are manifested in the shortening of the RT for a redundant feature target (e.g., a CO target) from that predicted by a race between the RTs for the two corresponding single feature targets (e.g., C and O targets). Our investigation enables the following testable predictions. Contextual suppression on the response of a CO-tuned or MO-tuned conjunctive cell is weaker when the contextual inputs differ from the direct inputs in both feature dimensions, rather than just one. Additionally, CO-tuned cells and MO-tuned cells are often more active than the single feature tuned cells in response to the redundant feature targets, and this occurs more frequently for the MO-tuned cells such that the MO-tuned cells are no less likely than either the M-tuned or O-tuned neurons to be the most responsive neuron to dictate saliency for an MO target.     

A look into the cockpit of the developing locust: Looming detectors and predator avoidance

For many animals, the visual detection of looming stimuli is crucial at any stage of their lives. For example, human babies of only 6 days old display evasive responses to looming stimuli (Bower et al. [1971]: Percept Psychophys 9: 193–196). This means the neuronal pathways involved in looming detection should mature early in life. Locusts have been used extensively to examine the neural circuits and mechanisms involved in sensing looming stimuli and triggering visually evoked evasive actions, making them ideal subjects in which to investigate the development of looming sensitivity. Two lobula giant movement detectors (LGMD) neurons have been identified in the lobula region of the locust visual system: the LGMD1 neuron responds selectively to looming stimuli and provides information that contributes to evasive responses such as jumping and emergency glides. The LGMD2 responds to looming stimuli and shares many response properties with the LGMD1. Both neurons have only been described in the adult. In this study, we describe a practical method combining classical staining techniques and 3D neuronal reconstructions that can be used, even in small insects, to reveal detailed anatomy of individual neurons. We have used it to analyze the anatomy of the fan‐shaped dendritic tree of the LGMD1 and the LGMD2 neurons in all stages of the post‐embryonic development of Locusta migratoria. We also analyze changes seen during the ontogeny of escape behaviors triggered by looming stimuli, specially the hiding response. © 2014 Wiley Periodicals, Inc. Develop Neurobiol 74: 1078–1095, 2014     

Feature-based attentional modulations in the absence of direct visual stimulation

When faced with a crowded visual scene, observers must selectively attend to behaviorally relevant objects to avoid sensory overload. Often this selection process is guided by prior knowledge of a target-defining feature (e.g., the color red when looking for an apple), which enhances the firing rate of visual neurons that are selective for the attended feature. Here, we used functional magnetic resonance imaging and a pattern classification algorithm to predict the attentional state of human observers as they monitored a visual feature (one of two directions of motion). We find that feature-specific attention effects spread across the visual field-even to regions of the scene that do not contain a stimulus. This spread of feature-based attention to empty regions of space may facilitate the perception of behaviorally relevant stimuli by increasing sensitivity to attended features at all locations in the visual field.     

The antidromic activation of tectal neurons by electrical stimuli applied to the caudal medulla oblongata in the toad,Bufo bufo L.

In order to specify the tectal projection to the bulbar/spinal regions, the antidromic responses of the physiologically identified tectal neurons as well as the gross antidromic field responses in the optic tectum to electrical stimuli applied to the caudal medulla were examined in the paralyzed common toad, Bufo bufo. The antidromic field potential was recorded in the optic tectum in response to electrical stimuli applied to the ventral paramedian portion of the contralateral caudal medulla (where the crossed tecto-spinal pathway of Rubinson (1968) and Lázár (1969) runs), but generally not when they were applied to various parts of the ipsilateral caudal medulla. The antidromic field potential was largest at the superficial part of Layer 6 or at the border between Layers 6 and 7 of the optic tectum, indicating that neurons in these layers project to the contralateral caudal medulla. Mapping experiments of the antidromic field potential over the optic tectum showed that the antidromic field potential was recorded mainly in the lateral part of it, indicating that this part of the optic tectum is the main source of projection neurons to the contralateral caudal medulla. Various classes of tectal neurons as well as retinal ganglion neurons were identified from the characteristics of the response properties to moving visual stimuli and the properties of the receptive fields. Of these, the Class T1, T2, T3, T4, T5(1), T5(2), T5(3), and T5(4) tectal neurons were activated antidromically by stimuli applied to the contralateral caudal medulla. Only a limited proportion of the Class T5(1) neurons was activated antidromically by stimuli applied to the ipsilateral caudal medulla. On the other hand, the Class T7 and T8 neurons, as well as the Class R2, R3, and R4 retinal neurons, were not activated antidromically by stimuli applied to the caudal medulla of either side. These results suggest a possibility that these tectal neurons which project to the medullary regions form the substrate of the sensorimotor interfacing and contribute to the initiation or coordination of the visually guided behavior, such as prey-catching.