Audiovisual integration: an investigation of the "streaming-bouncing" phenomenon

Temporal aspects of the perceptual integration of audiovisual information were investigated by utilizing the visual "streaming-bouncing" phenomenon. When two identical visual objects move towards each other, coincide, and then move away from each other, the objects can either be seen as streaming past one another or bouncing off each other. Although the streaming percept is dominant, the bouncing percept can be induced by presenting an auditory stimulus during the visual coincidence of the moving objects. Here we show that the bounce-inducing effect of the auditory stimulus is paramount when its onset and offset occur in temporal proximity of the onset and offset of the period of visual coincidence of the moving objects. When the duration of the auditory stimulus exceeded this period, visual bouncing disappears. Implications for a temporal window of audiovisual integration and the design of effective audiovisual warning signals are discussed.     

Integration across Time Determines Path Deviation Discrimination for Moving Objects

Background

Human vision is vital in determining our interaction with the outside world. In this study we characterize our ability to judge changes in the direction of motion of objects–a common task which can allow us either to intercept moving objects, or else avoid them if they pose a threat.

Methodology/Principal Findings

Observers were presented with objects which moved across a computer monitor on a linear path until the midline, at which point they changed their direction of motion, and observers were required to judge the direction of change. In keeping with the variety of objects we encounter in the real world, we varied characteristics of the moving stimuli such as velocity, extent of motion path and the object size. Furthermore, we compared performance for moving objects with the ability of observers to detect a deviation in a line which formed the static trace of the motion path, since it has been suggested that a form of static memory trace may form the basis for these types of judgment. The static line judgments were well described by a ‘scale invariant’ model in which any two stimuli which possess the same two-dimensional geometry (length/width) result in the same level of performance. Performance for the moving objects was entirely different. Irrespective of the path length, object size or velocity of motion, path deviation thresholds depended simply upon the duration of the motion path in seconds.

Conclusions/Significance

Human vision has long been known to integrate information across space in order to solve spatial tasks such as judgment of orientation or position. Here we demonstrate an intriguing mechanism which integrates direction information across time in order to optimize the judgment of path deviation for moving objects.     

Neuronal processing delays are compensated in the sensorimotor branch of the visual system

Moving objects change their position until signals from the photoreceptors arrive in the visual cortex. Nonetheless, motor responses to moving objects are accurate and do not lag behind the real-world position. The questions are how and where neural delays are compensated for. It was suggested that compensation is achieved within the visual system by extrapolating the position of moving objects. A visual illusion supports this idea: when a briefly flashed object is presented in the same position as a moving object, it appears to lag behind. However, moving objects do not appear ahead of their final or reversal points. We investigated a situation where participants localized the final position of a moving stimulus. Visual perception and short-term memory of the final target position were accurate, but reaching movements were directed toward future positions of the target beyond the vanishing point. Our results show that neuronal latencies are not compensated for at early stages of visual processing, but at a late stage when retinotopic information is transformed into egocentric space used for motor responses. The sensorimotor system extrapolates the position of moving targets to allow for precise localization of moving targets despite neuronal latencies.     

The role of tracking in the perception of objects, moving toward each other in a narrow vertical slit line

It has been shown by means of calculation and device modeling that the observed subjective pressure of two objects moving toward one another behind a narrow vertical slit corresponds to the pressure of objects retinal projections, induced in its turn by a slowed down tracing. 180e turn of one of the moving objects is also explained by tracing.     

Head-centred motion perception in the ageing visual system

Stationary objects appear to move in the opposite direction to a pursuit eye movement (Filehne illusion) and moving objects appear slower when pursued (Aubert-Fleischl phenomenon). Both illusions imply that extra-retinal, eye-velocity signals lead to lower estimates of speed than corresponding retinal motion signals. Intriguingly, the velocity (i.e. speed and direction) of the Filehne illusion depends on the age of the observer, especially for brief display durations (Wertheim and Bekkering, 1992). This suggests relative signal size changes as the visual system matures. To test the signal-size hypothesis, we compared the Filehne illusion and Aubert-Fleischl phenomenon in young and old observers using short and long display durations. The trends in the Filehne data were similar to those reported by Wertheim and Bekkering. However, we found no evidence for an effect of age or duration in the Aubert-Fleischl phenomenon. The differences between the two illusions could not be reconciled on the basis of actual eye movements made. The findings suggest a more complicated explanation of the combined influence of age and duration on head-centred motion perception than that described by the signal-size hypothesis.     

An Empirical Explanation of the Speed-Distance Effect

Understanding motion perception continues to be the subject of much debate, a central challenge being to account for why the speeds and directions seen accord with neither the physical movements of objects nor their projected movements on the retina. Here we investigate the varied perceptions of speed that occur when stimuli moving across the retina traverse different projected distances (the speed-distance effect). By analyzing a database of moving objects projected onto an image plane we show that this phenomenology can be quantitatively accounted for by the frequency of occurrence of image speeds generated by perspective transformation. These results indicate that speed-distance effects are determined empirically from accumulated past experience with the relationship between image speeds and moving objects.     

Responses of medulla neurons to illumination and movement stimuli in the tiger beetle larvae

Intracellular responses of medulla neurons (second-order visual interneurons) have been examined in the tiger beetle larva. The larva possesses six stemmata on either side of the head, two of which are much larger than the remaining four. Beneath the cuticle housing the stemmata an optic neuropil complex occurs consisting of lamina and medulla neuropils. Response patterns of medulla neurons to illumination and moving objects varied from neurons to neurons. For movement stimuli black discs and a black bar were moved in the rostro-caudal direction above the larva. Comparison of responses to the discs and the bar suggested a spatial summation of responses in some neurons, and tuning to small objects in some neurons. The majority of neurons responded to objects moving at heights of 10 mm and 50 mm with the same discharge pattern. A few neurons, however, showed distance sensitivities responding with an increase of spike discharges to moving objects only at either of the two heights. Such distance sensitivities still remained in one-stemma larvae, three of the four stemmata being occluded. These data are discussed in relation to distinct visual behavior of the larva and with special reference to perception of the hunting range.     

Freely flying bees discriminate between stationary and moving objects: performance and possible mechanisms

Summary Freely flying honeybees are innately attracted to moving objects, as revealed by their spontaneous preference for a moving disc over an identical, but stationary disc. We have exploited this spontaneous preference to explore the visual cues by which a bee, which is herself in motion, recognizes a moving object. We find that the moving disc is not detected on the basis that it produces a more rapidly moving image on the retina. The relevant cue might therefore be the motion of the disc relative to the visual surround. We have attempted to test this hypothesis by artificially rotating the structured environment, together with the moving disc, around the bee. Under these conditions, the image of the stationary disc rather than that of the actually moving disc is in motion relative to the surround. We find that rotation of the surround disrupts the bee's capacity not only to distinguish a moving object from a stationary one, but also to discriminate stationary objects at different ranges. Possible interpretations of these results are discussed.     

Are Stripes Beneficial? Dazzle Camouflage Influences Perceived Speed and Hit Rates

In the animal kingdom, camouflage refers to patterns that help potential prey avoid detection. Mostly camouflage is thought of as helping prey blend in with their background. In contrast, disruptive or dazzle patterns protect moving targets and have been suggested as an evolutionary force in shaping the dorsal patterns of animals. Dazzle patterns, such as stripes and zigzags, are thought to reduce the probability with which moving prey will be captured by impairing predators'' perception of speed. We investigated how different patterns of stripes (longitudinal—i.e., parallel to movement direction–and vertical–i.e., perpendicular to movement direction) affect the probability with which humans can hit moving objects and if differences in hitting probability are caused by a misperception of speed. A first experiment showed that longitudinally striped objects were hit more often than unicolored objects. However, vertically striped objects did not differ from unicolored objects. A second study examining the link between perceived speed and hitting probability showed that longitudinally and vertically striped objects were both perceived as moving faster and were hit more often than unicolored objects. In sum, our results provide evidence that striped patterns disrupt the perception of speed, which in turn influences how often objects are hit. However, the magnitude and the direction of the effects depend on additional factors such as speed and the task setup.     

Tempo rubato : animacy speeds up time in the brain

Background

How do we estimate time when watching an action? The idea that events are timed by a centralized clock has recently been called into question in favour of distributed, specialized mechanisms. Here we provide evidence for a critical specialization: animate and inanimate events are separately timed by humans.

Methodology/Principal Findings

In different experiments, observers were asked to intercept a moving target or to discriminate the duration of a stationary flash while viewing different scenes. Time estimates were systematically shorter in the sessions involving human characters moving in the scene than in those involving inanimate moving characters. Remarkably, the animate/inanimate context also affected randomly intermingled trials which always depicted the same still character.

Conclusions/Significance

The existence of distinct time bases for animate and inanimate events might be related to the partial segregation of the neural networks processing these two categories of objects, and could enhance our ability to predict critically timed actions.     

A simple strategy for detecting moving objects during locomotion revealed by animal-robot interactions

An important role of visual systems is to detect nearby predators, prey, and potential mates, which may be distinguished in part by their motion. When an animal is at rest, an object moving in any direction may easily be detected by motion-sensitive visual circuits. During locomotion, however, this strategy is compromised because the observer must detect a moving object within the pattern of optic flow created by its own motion through the stationary background. However, objects that move creating back-to-front (regressive) motion may be unambiguously distinguished from stationary objects because forward locomotion creates only front-to-back (progressive) optic flow. Thus, moving animals should exhibit an enhanced sensitivity to regressively moving objects. We explicitly tested this hypothesis by constructing a simple fly-sized robot that was programmed to interact with a real fly. Our measurements indicate that whereas walking female flies freeze in response to a regressively moving object, they ignore a progressively moving one. Regressive motion salience also explains observations of behaviors exhibited by pairs of walking flies. Because the assumptions underlying the regressive motion salience hypothesis are general, we suspect that the behavior we have observed in Drosophila may be widespread among eyed, motile organisms.     

Dynamic Spatial Organization of Receptive Fields of Neurons in the 21a Cortical Area

We studied changes in the spatial parameters of receptive fields (RFs) of visually sensitive neurons in the associative area 21a of the cat cortex under conditions of presentation of moving visual stimuli. The results of experiments demonstrated that these parameters are dynamic and depend, from many aspects, on the pattern of the stimulus used for their estimation. Angular lengths of the horizontal and vertical axes of the RFs measured in the case of movement of the visual stimuli exceeded many times those determined by presentation of stationary blinking stimuli. As is supposed, a visual stimulus, when moving along the field of vision, activates a certain number of the neurons synaptically connected with the examined cell and possessing RFs localized along the movement trajectory. As a result, such integrated activity of the neuronal group can change the excitation threshold and discharge frequency of the studied neuron. It seems probable that correlated directed activation of the neuronal groups represents a significant neurophysiological mechanism providing dynamic modifications of the RF parameters of visually sensitive neurons in the course of processes of visual perception and identification of moving objects within the field of vision.     

Behavioral discrimination of water motions caused by moving objects.

We tested whether goldfish, Carassius auratus, discriminate hydrodynamic stimuli caused by moving objects. Blindfolded goldfish responded to a passing object with changes in inter-gill-movement intervals. To learn whether goldfish can discriminate water motions caused by different moving objects we habituated them to a certain object stimulus. If the stimulus was altered, e.g., by altering speed, direction of motion, or size or shape of the object, fish again showed a temporary suspension of breathing when the object passed by. If animals failed to respond to an altered stimulus, we paired this stimulus with a weak electric shock during training. Goldfish discriminated object motion direction. In addition, in two choice experiments goldfish discriminated water motions caused by objects which moved with different speeds (e.g., 5 cm s(-1) versus 6 cm s(-1)), or by objects which differed in size (e.g., 1 cm x 1 cm versus 1.4 cm x 1.4 cm cross section), or shape (e.g., a round versus a triangular object). If object size and/or shape was varied quasi-randomly such that the faster moving object not always caused the greatest water velocities, fish still discriminated object speed.     

Tactile motor learning in the antennal system of the honeybee (Apis mellifera L.)

Honeybees fixed in small tubes scan an object within the range of the antennae by touching it briefly and frequently. In our experiments the animals were able to scan an object for several minutes with the antennae. After moving the object out of the range of the antennae, the animals showed antennal movements for several minutes that were correlated with the position of the removed object. These changes of antennal movements are called “behavioural plasticity” and are interpreted as a form of motor learning. Bees showed behavioural plasticity only for objects with relatively large surfaces. Plasticity was more pronounced in bees whose compound eyes were occluded. Behavioural plasticity was related to the duration of object presentation. Repeated presentations of the object increased the degree of plasticity. After presentation durations of 30 min the animals showed a significant increase of antennal positions related to the surface of the object and avoidance of areas corresponding to the edges. Behavioural plasticity was compared with reward-dependent learning by conditioning bees to objects. The results of motor learning and reward-dependent conditioning suggest that bees have tactile spatial memory. Accepted: 13 May 1997     

A saliency-based bottom-up visual attention model for dynamic scenes analysis

This work proposes a model of visual bottom-up attention for dynamic scene analysis. Our work adds motion saliency calculations to a neural network model with realistic temporal dynamics [(e.g., building motion salience on top of De Brecht and Saiki Neural Networks 19:1467–1474, (2006)]. The resulting network elicits strong transient responses to moving objects and reaches stability within a biologically plausible time interval. The responses are statistically different comparing between earlier and later motion neural activity; and between moving and non-moving objects. We demonstrate the network on a number of synthetic and real dynamical movie examples. We show that the model captures the motion saliency asymmetry phenomenon. In addition, the motion salience computation enables sudden-onset moving objects that are less salient in the static scene to rise above others. Finally, we include strong consideration for the neural latencies, the Lyapunov stability, and the neural properties being reproduced by the model.     

鲶鱼电感觉中枢对静止和运动偶极子电场的反应

在7尾鲶鱼所记录到的11个电感觉中枢神经元中,6个只对运动的偶极子电场有反应;4个对运动和静止的偶极子电场均有反应,但以前者反应最强;另一个神经元不仅对运动的偶极子电场,而且对运动的塑料棒也有反应。研究表明,鲶鱼的电感应中枢最适合于察觉移动的电场目标。     

Macaque Monkeys Perceive the Flash Lag Illusion

Transmission of neural signals in the brain takes time due to the slow biological mechanisms that mediate it. During such delays, the position of moving objects can change substantially. The brain could use statistical regularities in the natural world to compensate neural delays and represent moving stimuli closer to real time. This possibility has been explored in the context of the flash lag illusion, where a briefly flashed stimulus in alignment with a moving one appears to lag behind the moving stimulus. Despite numerous psychophysical studies, the neural mechanisms underlying the flash lag illusion remain poorly understood, partly because it has never been studied electrophysiologically in behaving animals. Macaques are a prime model for such studies, but it is unknown if they perceive the illusion. By training monkeys to report their percepts unbiased by reward, we show that they indeed perceive the illusion qualitatively similar to humans. Importantly, the magnitude of the illusion is smaller in monkeys than in humans, but it increases linearly with the speed of the moving stimulus in both species. These results provide further evidence for the similarity of sensory information processing in macaques and humans and pave the way for detailed neurophysiological investigations of the flash lag illusion in behaving macaques.     

Efficiency of a visual system under conditions of uncertainty of the appearance of a moving object

The effect of uncertainty of a moving object appearance in the noise field upon the coefficient efficiency, we studied. At short durations of presentation (40-80 msec) and high level of external noise, this effect was maximal: magnified 100 times. The efficiency coefficient dependence on the duration of a moving object presentation was shown to be characterized by two maximums. The position of minimum situated between these two maximums was found to be independent of either presence or absence of uncertainty of a number of parameters: such as initial position of the object the image, time of its appearance, noise level, velocity and direction of the movement, and has a latency approximately 120 sec. A functional model of the observed phenomena, has been proposed.     

White Noise Masks Some Temporal Parameters of the Auditory Localization of Radially Moving Targets

The temporal parameters of the perception of radially moving sound sources partly masked with broadband internalized noise at an intensity of 40, 46, or 52 dB above the hearing threshold have been studied. The threshold of sound duration necessary for identifying the direction of movement of the sound source (75% correct answers) increases from 135 ms in silence to 285 ms at all intensities of continuous noise studied. The minimum duration of the stimulus beginning with which a subsequent increase in duration does not increase the number of correct responses is the same (385 ms) under all conditions of stimulus presentation. Broadband noise of any intensity increases the time of response to stimuli in the range of durations studied. At a noise of 52 dB, which is close to the threshold of full masking, the reaction time is not increased significantly compared to its estimation at a noise of 46 dB. The minimum duration of the stimulus has proved to be the stablest temporal parameter of the perception of movement of a sound source. Changes in the temporal parameters of sound perception at noise levels close to the threshold of full masking are discussed.     

Two-step exposure of biological objects to infrared laser and microwave radiation]

The effect of two-step exposure of bacterial objects to infrared laser and microwave pulse radiations was studied. The effect is determined by the time interval between two excitation steps and pulse duration. It was shown that the biologically active dose of microwave radiation is much lower than that of infrared laser radiation; however, laser radiation induces a stronger cellular response. It was found that microwaves enhance the efficiency of infrared laser radiation.