Multi-timescale perceptual history resolves visual ambiguity

When visual input is inconclusive, does previous experience aid the visual system in attaining an accurate perceptual interpretation? Prolonged viewing of a visually ambiguous stimulus causes perception to alternate between conflicting interpretations. When viewed intermittently, however, ambiguous stimuli tend to evoke the same percept on many consecutive presentations. This perceptual stabilization has been suggested to reflect persistence of the most recent percept throughout the blank that separates two presentations. Here we show that the memory trace that causes stabilization reflects not just the latest percept, but perception during a much longer period. That is, the choice between competing percepts at stimulus reappearance is determined by an elaborate history of prior perception. Specifically, we demonstrate a seconds-long influence of the latest percept, as well as a more persistent influence based on the relative proportion of dominance during a preceding period of at least one minute. In case short-term perceptual history and long-term perceptual history are opposed (because perception has recently switched after prolonged stabilization), the long-term influence recovers after the effect of the latest percept has worn off, indicating independence between time scales. We accommodate these results by adding two positive adaptation terms, one with a short time constant and one with a long time constant, to a standard model of perceptual switching.     

United we sense, divided we fail: context-driven perception of ambiguous visual stimuli

Ambiguous visual stimuli provide the brain with sensory information that contains conflicting evidence for multiple mutually exclusive interpretations. Two distinct aspects of the phenomenological experience associated with viewing ambiguous visual stimuli are the apparent stability of perception whenever one perceptual interpretation is dominant, and the instability of perception that causes perceptual dominance to alternate between perceptual interpretations upon extended viewing. This review summarizes several ways in which contextual information can help the brain resolve visual ambiguities and construct temporarily stable perceptual experiences. Temporal context through prior stimulation or internal brain states brought about by feedback from higher cortical processing levels may alter the response characteristics of specific neurons involved in rivalry resolution. Furthermore, spatial or crossmodal context may strengthen the neuronal representation of one of the possible perceptual interpretations and consequently bias the rivalry process towards it. We suggest that contextual influences on perceptual choices with ambiguous visual stimuli can be highly informative about the neuronal mechanisms of context-driven inference in the general processes of perceptual decision-making.     

Temporal dynamics of auditory and visual bistability reveal common principles of perceptual organization

When dealing with natural scenes, sensory systems have to process an often messy and ambiguous flow of information. A stable perceptual organization nevertheless has to be achieved in order to guide behavior. The neural mechanisms involved can be highlighted by intrinsically ambiguous situations. In such cases, bistable perception occurs: distinct interpretations of the unchanging stimulus alternate spontaneously in the mind of the observer. Bistable stimuli have been used extensively for more than two centuries to study visual perception. Here we demonstrate that bistable perception also occurs in the auditory modality. We compared the temporal dynamics of percept alternations observed during auditory streaming with those observed for visual plaids and the susceptibilities of both modalities to volitional control. Strong similarities indicate that auditory and visual alternations share common principles of perceptual bistability. The absence of correlation across modalities for subject-specific biases, however, suggests that these common principles are implemented at least partly independently across sensory modalities. We propose that visual and auditory perceptual organization could rely on distributed but functionally similar neural competition mechanisms aimed at resolving sensory ambiguities.     

Attentional modulation of perceptual stabilization

Perceptual priming is generally regarded as a passive and automatic process, as it is obtained even without awareness of the prime. Recent studies have introduced a more active form of perceptual priming in which priming for a subsequent ambiguous stimulus is triggered by the subjective percept, that is, interpretation of a previous ambiguous stimulus. This phenomenon known as stabilization does not require a conscious effort to actively maintain one perceptual interpretation. In this study, we show that distraction of attention, during and even after the prime presentation, interferes with the build-up of perceptual memory for stabilization. This implies that despite the apparent automaticity, stabilization involves an active attentional process for encoding and retention. The disruption during the encoding can be attributed to the reduction in sensory signals for the prime. However, the disruption during the retention suggests that the implicit memory trace of the prime necessitates the attentional resource to fully develop. The active nature of the build-up of perceptual memory for stabilization is consistent with the idea that perceptual memory increases its strength gradually over a few seconds. These findings suggest that seemingly automatic and effortless cognitive processes can compete with online perceptual processing for common attentional resources.     

Tactile rivalry demonstrated with an ambiguous apparent-motion quartet

When observers view ambiguous visual stimuli, their perception will often alternate between the possible interpretations, a phenomenon termed perceptual rivalry [1]. To induce perceptual rivalry in the tactile domain, we developed a new tactile illusion, based on the visual apparent-motion quartet [2]. Pairs of 200 ms vibrotactile stimuli were applied to the finger pad at intervals separated by 300 ms. The location of each successive stimulus pair alternated between the opposing diagonal corners of the approximately 1 cm(2) stimulation array. This stimulation sequence led all participants to report switches between the perception of motion traveling either up and down or left and right across their fingertip. Adaptation to tactile stimulation biased toward one direction caused subsequent ambiguous stimulation to be experienced in the opposing direction. In contrast, when consecutive trials of ambiguous stimulation were presented, motion was generally perceived in the direction consistent with the motion reported in the previous trial. Voluntary eye movements induced shifts in the tactile perception toward a motion axis aligned along a world-centered coordinate frame. Because the tactile quartet results in switching perceptual states despite unvaried sensory input, it is ideally suited to future studies of the neural processes associated with conscious tactile perception.     

Neural response to perception of volume in the lateral occipital complex

Projection of a 3D scene onto the 2D retina necessarily entails a loss of information, yet perceivers experience a world populated with volumetric objects. Using simultaneous behavioral and neural (fMRI) measures, we identify neural bases of volume perception. Neural activity in the lateral occipital cortex increased with presentation of 3D volumes relative to presentation of 2D shapes. Neural activity also modulated with perceived volume, independent of image information. When behavioral responses indicated that observers saw ambiguous images as 3D volumes, neural response increased; when behavioral data revealed a 2D interpretation, neural response waned. Crucially, the physical stimulus was identical under both interpretations; only the percept of volume can account for the increased neural activity.     

Human perceptual learning: The effect of pre-exposure schedule depends on task demands

The effects of the pre-exposure schedule (concurrent, intermixed, and blocked) to two similar visual stimuli were assessed in three different tasks. Participants were more accurate identifying one of two pre-exposed stimuli as the target by means of same/different judgments after concurrent than intermixed or blocked pre-exposures. Regardless of pre-exposure schedule, participants were accurate in identifying the same target stimulus in a subsequent multiple choice task. However, the other pre-exposed stimulus was incorrectly chosen as the target in a greater proportion after blocked than intermixed or concurrent pre-exposure. Finally, participants who received the blocked schedule showed a greater ability to construct the target in a puzzle test than those who received a concurrent or intermixed schedule. These results suggest that the effect of pre-exposure schedule may depend on task-specific demands. But all these results might be explained by a selective attention mechanism like that proposed by Gibson (1969) to account for perceptual learning.     

The initial phase of auditory and visual scene analysis

Auditory streaming and visual plaids have been used extensively to study perceptual organization in each modality. Both stimuli can produce bistable alternations between grouped (one object) and split (two objects) interpretations. They also share two peculiar features: (i) at the onset of stimulus presentation, organization starts with a systematic bias towards the grouped interpretation; (ii) this first percept has 'inertia'; it lasts longer than the subsequent ones. As a result, the probability of forming different objects builds up over time, a landmark of both behavioural and neurophysiological data on auditory streaming. Here we show that first percept bias and inertia are independent. In plaid perception, inertia is due to a depth ordering ambiguity in the transparent (split) interpretation that makes plaid perception tristable rather than bistable: experimental manipulations removing the depth ambiguity suppressed inertia. However, the first percept bias persisted. We attempted a similar manipulation for auditory streaming by introducing level differences between streams, to bias which stream would appear in the perceptual foreground. Here both inertia and first percept bias persisted. We thus argue that the critical common feature of the onset of perceptual organization is the grouping bias, which may be related to the transition from temporally/spatially local to temporally/spatially global computation.     

Perceptual rivalry: reflexes reveal the gradual nature of visual awareness

Rivalry is a common tool to probe visual awareness: a constant physical stimulus evokes multiple, distinct perceptual interpretations ("percepts") that alternate over time. Percepts are typically described as mutually exclusive, suggesting that a discrete (all-or-none) process underlies changes in visual awareness. Here we follow two strategies to address whether rivalry is an all-or-none process: first, we introduce two reflexes as objective measures of rivalry, pupil dilation and optokinetic nystagmus (OKN); second, we use a continuous input device (analog joystick) to allow observers a gradual subjective report. We find that the "reflexes" reflect the percept rather than the physical stimulus. Both reflexes show a gradual dependence on the time relative to perceptual transitions. Similarly, observers' joystick deflections, which are highly correlated with the reflex measures, indicate gradual transitions. Physically simulating wave-like transitions between percepts suggest piece-meal rivalry (i.e., different regions of space belonging to distinct percepts) as one possible explanation for the gradual transitions. Furthermore, the reflexes show that dominance durations depend on whether or not the percept is actively reported. In addition, reflexes respond to transitions with shorter latencies than the subjective report and show an abundance of short dominance durations. This failure to report fast changes in dominance may result from limited access of introspection to rivalry dynamics. In sum, reflexes reveal that rivalry is a gradual process, rivalry's dynamics is modulated by the required action (response mode), and that rapid transitions in perceptual dominance can slip away from awareness.     

Within- and Cross-Modal Distance Information Disambiguate Visual Size-Change Perception

Perception is fundamentally underconstrained because different combinations of object properties can generate the same sensory information. To disambiguate sensory information into estimates of scene properties, our brains incorporate prior knowledge and additional “auxiliary” (i.e., not directly relevant to desired scene property) sensory information to constrain perceptual interpretations. For example, knowing the distance to an object helps in perceiving its size. The literature contains few demonstrations of the use of prior knowledge and auxiliary information in combined visual and haptic disambiguation and almost no examination of haptic disambiguation of vision beyond “bistable” stimuli. Previous studies have reported humans integrate multiple unambiguous sensations to perceive single, continuous object properties, like size or position. Here we test whether humans use visual and haptic information, individually and jointly, to disambiguate size from distance. We presented participants with a ball moving in depth with a changing diameter. Because no unambiguous distance information is available under monocular viewing, participants rely on prior assumptions about the ball''s distance to disambiguate their -size percept. Presenting auxiliary binocular and/or haptic distance information augments participants'' prior distance assumptions and improves their size judgment accuracy—though binocular cues were trusted more than haptic. Our results suggest both visual and haptic distance information disambiguate size perception, and we interpret these results in the context of probabilistic perceptual reasoning.     

Perception of temporally interleaved ambiguous patterns

BACKGROUND: Continuous viewing of ambiguous patterns is characterized by wavering perception that alternates between two or more equally valid visual solutions. However, when such patterns are viewed intermittently, either by repetitive presentation or by periodic closing of the eyes, perception can become locked or "frozen" in one configuration for several minutes at a time. One aspect of this stabilization is the possible existence of a perceptual memory that persists during periods in which the ambiguous stimulus is absent. Here, we use a novel paradigm of temporally interleaved ambiguous stimuli to explore the nature of this memory, with particular regard to its potential impact on perceptual organization. RESULTS: We found that the persistence of a perceptual configuration was robust to interposed visual patterns, and, further, that at least three ambiguous patterns, when interleaved in time, could undergo parallel, stable time courses. Then, using an interleaved presentation paradigm, we established that the occasional reversal in one pattern could be coupled with that of its interleaved counterpart, and that this coupling was a function of the structural similarity between the patterns. CONCLUSIONS: We postulate that the stabilization observed with repetitive presentation of ambiguous patterns can be at least partially accounted for by processes that retain a recent perceptual interpretation, and we speculate that such memory may be important in natural vision. We further propose that the interleaved paradigm introduced here may be of great value to gauge aspects of stimulus similarity that appeal to particular mechanisms of perceptual organization.     

Neuromechanistic Model of Auditory Bistability

Sequences of higher frequency A and lower frequency B tones repeating in an ABA- triplet pattern are widely used to study auditory streaming. One may experience either an integrated percept, a single ABA-ABA- stream, or a segregated percept, separate but simultaneous streams A-A-A-A- and -B---B--. During minutes-long presentations, subjects may report irregular alternations between these interpretations. We combine neuromechanistic modeling and psychoacoustic experiments to study these persistent alternations and to characterize the effects of manipulating stimulus parameters. Unlike many phenomenological models with abstract, percept-specific competition and fixed inputs, our network model comprises neuronal units with sensory feature dependent inputs that mimic the pulsatile-like A1 responses to tones in the ABA- triplets. It embodies a neuronal computation for percept competition thought to occur beyond primary auditory cortex (A1). Mutual inhibition, adaptation and noise are implemented. We include slow NDMA recurrent excitation for local temporal memory that enables linkage across sound gaps from one triplet to the next. Percepts in our model are identified in the firing patterns of the neuronal units. We predict with the model that manipulations of the frequency difference between tones A and B should affect the dominance durations of the stronger percept, the one dominant a larger fraction of time, more than those of the weaker percept—a property that has been previously established and generalized across several visual bistable paradigms. We confirm the qualitative prediction with our psychoacoustic experiments and use the behavioral data to further constrain and improve the model, achieving quantitative agreement between experimental and modeling results. Our work and model provide a platform that can be extended to consider other stimulus conditions, including the effects of context and volition.     

Visual motion retards alternations between conflicting perceptual interpretations

When the visual system is faced with conflicting or ambiguous stimulus information, visual perception fluctuates over time. We found that perceptual alternations are slowed when inducing stimuli move within the visual field, constantly engaging fresh, unadapted neural tissue. During binocular rivalry, dominance durations were longer when rival figures moved compared to when they were stationary, yielding lower alternation rates. Rate was not reduced, however, when observers tracked the moving targets, keeping the images on approximately the same retinal area. Alternations were reliably triggered when rival targets passed through a local region of the visual field preadapted to one of the rival targets. During viewing of a kinetic globe whose direction of rotation was ambiguous, observers experienced fewer alternations in perceived direction when the globe moved around the visual field or when the globe's axis of rotation changed continuously. Evidently, local neural adaptation is a key ingredient in the instability of perception.     

Interaction between pools of binocular disparity detectors tuned to different disparities

Unambiguous dots (having one binocular disparity) when inserted in an ambiguous random-dot stereogram (with multiple disparity values) could pull the ambiguous percept. The unambiguous bias carried that ambiguous depth percept whose disparity was nearest to the disparity of the bias. The closer the disparities were to each other, the stronger the pulling effect that was observed. Even a physical bias of 4% density was adequate to overcome the natural bias of most observers. The stimulus duration had to be over 50 msec to provide a strong pulling effect. In all experiments the stimulus duration was 160 msec or shorter, indicating that the pulling effect was a product of neural interactions, rather than convergence movement of the eyes. As a result of these findings a parallel model of stereopsis has been proposed, which extends the spring-coupled dipole model of Julesz (1971).     

Predicting the stream of consciousness from activity in human visual cortex

Can the rapid stream of conscious experience be predicted from brain activity alone? Recently, spatial patterns of activity in visual cortex have been successfully used to predict feature-specific stimulus representations for both visible and invisible stimuli. However, because these studies examined only the prediction of static and unchanging perceptual states during extended periods of stimulation, it remains unclear whether activity in early visual cortex can also predict the rapidly and spontaneously changing stream of consciousness. Here, we used binocular rivalry to induce frequent spontaneous and stochastic changes in conscious experience without any corresponding changes in sensory stimulation, while measuring brain activity with fMRI. Using information that was present in the multivariate pattern of responses to stimulus features, we could accurately predict, and therefore track, participants' conscious experience from the fMRI signal alone while it underwent many spontaneous changes. Prediction in primary visual cortex primarily reflected eye-based signals, whereas prediction in higher areas reflected the color of the percept. Furthermore, accurate prediction during binocular rivalry could be established with signals recorded during stable monocular viewing, showing that prediction generalized across viewing conditions and did not require or rely on motor responses. It is therefore possible to predict the dynamically changing time course of subjective experience with only brain activity.     

Multistability in perception: binding sensory modalities, an overview

This special issue presents research concerning multistable perception in different sensory modalities. Multistability occurs when a single physical stimulus produces alternations between different subjective percepts. Multistability was first described for vision, where it occurs, for example, when different stimuli are presented to the two eyes or for certain ambiguous figures. It has since been described for other sensory modalities, including audition, touch and olfaction. The key features of multistability are: (i) stimuli have more than one plausible perceptual organization; (ii) these organizations are not compatible with each other. We argue here that most if not all cases of multistability are based on competition in selecting and binding stimulus information. Binding refers to the process whereby the different attributes of objects in the environment, as represented in the sensory array, are bound together within our perceptual systems, to provide a coherent interpretation of the world around us. We argue that multistability can be used as a method for studying binding processes within and across sensory modalities. We emphasize this theme while presenting an outline of the papers in this issue. We end with some thoughts about open directions and avenues for further research.     

Attention controls multisensory perception via two distinct mechanisms at different levels of the cortical hierarchy

To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via two distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

A combination of psychophysics, computational modelling and fMRI reveals novel insights into how the brain controls the binding of information across the senses, such as the voice and lip movements of a speaker.     

Rapid Perceptual Switching of a Reversible Biological Figure

Certain visual stimuli can give rise to contradictory perceptions. In this paper we examine the temporal dynamics of perceptual reversals experienced with biological motion, comparing these dynamics to those observed with other ambiguous structure from motion (SFM) stimuli. In our first experiment, naïve observers monitored perceptual alternations with an ambiguous rotating walker, a figure that randomly alternates between walking in clockwise (CW) and counter-clockwise (CCW) directions. While the number of reported reversals varied between observers, the observed dynamics (distribution of dominance durations, CW/CCW proportions) were comparable to those experienced with an ambiguous kinetic depth cylinder. In a second experiment, we compared reversal profiles with rotating and standard point-light walkers (i.e. non-rotating). Over multiple test repetitions, three out of four observers experienced consistently shorter mean percept durations with the rotating walker, suggesting that the added rotational component may speed up reversal rates with biomotion. For both stimuli, the drift in alternation rate across trial and across repetition was minimal. In our final experiment, we investigated whether reversals with the rotating walker and a non-biological object with similar global dimensions (rotating cuboid) occur at random phases of the rotation cycle. We found evidence that some observers experience peaks in the distribution of response locations that are relatively stable across sessions. Using control data, we discuss the role of eye movements in the development of these reversal patterns, and the related role of exogenous stimulus characteristics. In summary, we have demonstrated that the temporal dynamics of reversal with biological motion are similar to other forms of ambiguous SFM. We conclude that perceptual switching with biological motion is a robust bistable phenomenon.     

When perceptual time stands still: long percept-memory in binocular rivalry

We have carried out binocular rivalry experiments with a large number of subjects to obtain high quality statistics on probability distribution of dominance duration (PDDD) for two cases where (a) the rival stimulus is continuously presented and (b) the rival stimulus is periodically removed, with stimulus-on and stimulus-off intervals T(on) and T(off) respectively. In the present study we have chosen to study the regime of relatively long stimulus-on time, i.e., T(on)> 1s, where the stimulus presentation duration is significantly longer than the human reaction and recognition time. In the case of periodically removed stimulus, the total probability for percept reversal during each of the successive stimulus-on intervals T(on) can be predicted using the PDDD for continuous viewing. More importantly, this total probability for percept reversal during any stimulus-on interval is independent of the length T(off) of the preceding blank time, which can be quite long. We argue that this suggests that, in the regime of long T(on) and T(off) considered here, the variables representing the perceptual state do not change significantly during long blank intervals. We discuss that these findings impose challenges to theoretical models which aim at describing visual perception.     

Learning to use an invisible visual signal for perception

How does the brain construct a percept from sensory signals? One approach to this fundamental question is to investigate perceptual learning as induced by exposure to statistical regularities in sensory signals [1-7]. Recent studies showed that exposure to novel correlations between sensory signals can cause a signal to have new perceptual effects [2, 3]. In those studies, however, the signals were clearly visible. The automaticity of the learning was therefore difficult to determine. Here we investigate whether learning of this sort, which causes new effects on appearance, can be low level and automatic by employing a visual signal whose perceptual consequences were made invisible-a vertical disparity gradient masked by other depth cues. This approach excluded high-level influences such as attention or consciousness. Our stimulus for probing perceptual appearance was a rotating cylinder. During exposure, we introduced a new contingency between the invisible signal and the rotation direction of the cylinder. When subsequently presenting an ambiguously rotating version of the cylinder, we found that the invisible signal influenced the perceived rotation direction. This demonstrates that perception can rapidly undergo "structure learning" by automatically picking up novel contingencies between sensory signals, thus automatically recruiting signals for novel uses during the construction of a percept.