What we see is how we are: new paradigms in visual research

As most sensory modalities, the visual system needs to deal with very fast changes in the environment. Instead of processing all sensory stimuli, the brain is able to construct a perceptual experience by combining selected sensory input with an ongoing internal activity. Thus, the study of visual perception needs to be approached by examining not only the physical properties of stimuli, but also the brain's ongoing dynamical states onto which these perturbations are imposed. At least three different models account for this internal dynamics. One model is based on cardinal cells where the activity of few cells by itself constitutes the neuronal correlate of perception, while a second model is based on a population coding that states that the neuronal correlate of perception requires distributed activity throughout many areas of the brain. A third proposition, known as the temporal correlation hypothesis states that the distributed neuronal populations that correlate with perception, are also defined by synchronization of the activity on a millisecond time scale. This would serve to encode contextual information by defining relations between the features of visual objects. If temporal properties of neural activity are important to establish the neural mechanisms of perception, then the study of appropriate dynamical stimuli should be instrumental to determine how these systems operate. The use of natural stimuli and natural behaviors such as free viewing, which features fast changes of internal brain states as seen by motor markers, is proposed as a new experimental paradigm to study visual perception.     

Opposite influence of perceptual memory on initial and prolonged perception of sensory ambiguity

Observers continually make unconscious inferences about the state of the world based on ambiguous sensory information. This process of perceptual decision-making may be optimized by learning from experience. We investigated the influence of previous perceptual experience on the interpretation of ambiguous visual information. Observers were pre-exposed to a perceptually stabilized sequence of an ambiguous structure-from-motion stimulus by means of intermittent presentation. At the subsequent re-appearance of the same ambiguous stimulus perception was initially biased toward the previously stabilized perceptual interpretation. However, prolonged viewing revealed a bias toward the alternative perceptual interpretation. The prevalence of the alternative percept during ongoing viewing was largely due to increased durations of this percept, as there was no reliable decrease in the durations of the pre-exposed percept. Moreover, the duration of the alternative percept was modulated by the specific characteristics of the pre-exposure, whereas the durations of the pre-exposed percept were not. The increase in duration of the alternative percept was larger when the pre-exposure had lasted longer and was larger after ambiguous pre-exposure than after unambiguous pre-exposure. Using a binocular rivalry stimulus we found analogous perceptual biases, while pre-exposure did not affect eye-bias. We conclude that previously perceived interpretations dominate at the onset of ambiguous sensory information, whereas alternative interpretations dominate prolonged viewing. Thus, at first instance ambiguous information seems to be judged using familiar percepts, while re-evaluation later on allows for alternative interpretations.     

Tactile rivalry demonstrated with an ambiguous apparent-motion quartet

When observers view ambiguous visual stimuli, their perception will often alternate between the possible interpretations, a phenomenon termed perceptual rivalry [1]. To induce perceptual rivalry in the tactile domain, we developed a new tactile illusion, based on the visual apparent-motion quartet [2]. Pairs of 200 ms vibrotactile stimuli were applied to the finger pad at intervals separated by 300 ms. The location of each successive stimulus pair alternated between the opposing diagonal corners of the approximately 1 cm(2) stimulation array. This stimulation sequence led all participants to report switches between the perception of motion traveling either up and down or left and right across their fingertip. Adaptation to tactile stimulation biased toward one direction caused subsequent ambiguous stimulation to be experienced in the opposing direction. In contrast, when consecutive trials of ambiguous stimulation were presented, motion was generally perceived in the direction consistent with the motion reported in the previous trial. Voluntary eye movements induced shifts in the tactile perception toward a motion axis aligned along a world-centered coordinate frame. Because the tactile quartet results in switching perceptual states despite unvaried sensory input, it is ideally suited to future studies of the neural processes associated with conscious tactile perception.     

Visual motion retards alternations between conflicting perceptual interpretations

When the visual system is faced with conflicting or ambiguous stimulus information, visual perception fluctuates over time. We found that perceptual alternations are slowed when inducing stimuli move within the visual field, constantly engaging fresh, unadapted neural tissue. During binocular rivalry, dominance durations were longer when rival figures moved compared to when they were stationary, yielding lower alternation rates. Rate was not reduced, however, when observers tracked the moving targets, keeping the images on approximately the same retinal area. Alternations were reliably triggered when rival targets passed through a local region of the visual field preadapted to one of the rival targets. During viewing of a kinetic globe whose direction of rotation was ambiguous, observers experienced fewer alternations in perceived direction when the globe moved around the visual field or when the globe's axis of rotation changed continuously. Evidently, local neural adaptation is a key ingredient in the instability of perception.     

A common neuronal code for perceptual processes in visual cortex? Comparing choice and attentional correlates in V5/MT.

In the past two decades, sensory neuroscience has moved from describing response properties to external stimuli in cerebral cortex to establishing connections between neuronal activity and sensory perception. The seminal studies by Newsome, Movshon and colleagues in the awake behaving macaque firmly link single cells in extrastriate area V5/MT and perception of motion. A decade later, extrastriate visual cortex appears awash with neuronal correlates for many different perceptual tasks. Examples are attentional signals, choice signals for ambiguous images, correlates for binocular rivalry, stereo and shape perception, and so on. These diverse paradigms are aimed at elucidating the neuronal code for perceptual processes, but it has been little studied how they directly compare or even interact. In this paper, I explore to what degree the measured neuronal signals in V5/MT for choice and attentional paradigms might reflect a common neuronal mechanism for visual perception.     

Multi-timescale perceptual history resolves visual ambiguity

When visual input is inconclusive, does previous experience aid the visual system in attaining an accurate perceptual interpretation? Prolonged viewing of a visually ambiguous stimulus causes perception to alternate between conflicting interpretations. When viewed intermittently, however, ambiguous stimuli tend to evoke the same percept on many consecutive presentations. This perceptual stabilization has been suggested to reflect persistence of the most recent percept throughout the blank that separates two presentations. Here we show that the memory trace that causes stabilization reflects not just the latest percept, but perception during a much longer period. That is, the choice between competing percepts at stimulus reappearance is determined by an elaborate history of prior perception. Specifically, we demonstrate a seconds-long influence of the latest percept, as well as a more persistent influence based on the relative proportion of dominance during a preceding period of at least one minute. In case short-term perceptual history and long-term perceptual history are opposed (because perception has recently switched after prolonged stabilization), the long-term influence recovers after the effect of the latest percept has worn off, indicating independence between time scales. We accommodate these results by adding two positive adaptation terms, one with a short time constant and one with a long time constant, to a standard model of perceptual switching.     

Perception of temporally interleaved ambiguous patterns

BACKGROUND: Continuous viewing of ambiguous patterns is characterized by wavering perception that alternates between two or more equally valid visual solutions. However, when such patterns are viewed intermittently, either by repetitive presentation or by periodic closing of the eyes, perception can become locked or "frozen" in one configuration for several minutes at a time. One aspect of this stabilization is the possible existence of a perceptual memory that persists during periods in which the ambiguous stimulus is absent. Here, we use a novel paradigm of temporally interleaved ambiguous stimuli to explore the nature of this memory, with particular regard to its potential impact on perceptual organization. RESULTS: We found that the persistence of a perceptual configuration was robust to interposed visual patterns, and, further, that at least three ambiguous patterns, when interleaved in time, could undergo parallel, stable time courses. Then, using an interleaved presentation paradigm, we established that the occasional reversal in one pattern could be coupled with that of its interleaved counterpart, and that this coupling was a function of the structural similarity between the patterns. CONCLUSIONS: We postulate that the stabilization observed with repetitive presentation of ambiguous patterns can be at least partially accounted for by processes that retain a recent perceptual interpretation, and we speculate that such memory may be important in natural vision. We further propose that the interleaved paradigm introduced here may be of great value to gauge aspects of stimulus similarity that appeal to particular mechanisms of perceptual organization.     

Perceptual grouping of biological motion promotes binocular rivalry

Investigation of perceptual rivalry between conflicting stimuli presented one to each eye can further understanding of the neural underpinnings of conscious visual perception. During rivalry, visual awareness fluctuates between perceptions of the two stimuli. Here, we demonstrate that high-level perceptual grouping can promote rivalry between stimulus pairs that would otherwise be perceived as nonrivalrous. Perceptual grouping was generated with point-light walker stimuli that simulate human motion, visible only as lights placed on the joints. Although such walking figures are unrecognizable when stationary, recognition judgments as complex as gender and identity can accurately be made from animated displays, demonstrating the efficiency with which our visual system can group dynamic local signals into a globally coherent walking figure. We find that point-light walker stimuli presented one to each eye and in different colors and configurations results in strong rivalry. However, rivalry is minimal when the two walkers are split between the eyes or both presented to one eye. This pattern of results suggests that processing animated walker figures promotes rivalry between signals from the two eyes rather than between higher-level representations of the walkers. This leads us to hypothesize that awareness during binocular rivalry involves the integrated activity of high-level perceptual mechanisms in conjunction with lower-level ocular suppression modulated via cortical feedback.     

Temporal dynamics of auditory and visual bistability reveal common principles of perceptual organization

When dealing with natural scenes, sensory systems have to process an often messy and ambiguous flow of information. A stable perceptual organization nevertheless has to be achieved in order to guide behavior. The neural mechanisms involved can be highlighted by intrinsically ambiguous situations. In such cases, bistable perception occurs: distinct interpretations of the unchanging stimulus alternate spontaneously in the mind of the observer. Bistable stimuli have been used extensively for more than two centuries to study visual perception. Here we demonstrate that bistable perception also occurs in the auditory modality. We compared the temporal dynamics of percept alternations observed during auditory streaming with those observed for visual plaids and the susceptibilities of both modalities to volitional control. Strong similarities indicate that auditory and visual alternations share common principles of perceptual bistability. The absence of correlation across modalities for subject-specific biases, however, suggests that these common principles are implemented at least partly independently across sensory modalities. We propose that visual and auditory perceptual organization could rely on distributed but functionally similar neural competition mechanisms aimed at resolving sensory ambiguities.     

Visual perception: shaping what we see

A new psychophysical study has examined the free flow of perception as observers viewed stimuli with several possible visual interpretations. The results suggest that our subjective impression of such ambiguous patterns may be more closely linked to the brain's encoding of complex shape than previously appreciated.     

Predicting the stream of consciousness from activity in human visual cortex

Can the rapid stream of conscious experience be predicted from brain activity alone? Recently, spatial patterns of activity in visual cortex have been successfully used to predict feature-specific stimulus representations for both visible and invisible stimuli. However, because these studies examined only the prediction of static and unchanging perceptual states during extended periods of stimulation, it remains unclear whether activity in early visual cortex can also predict the rapidly and spontaneously changing stream of consciousness. Here, we used binocular rivalry to induce frequent spontaneous and stochastic changes in conscious experience without any corresponding changes in sensory stimulation, while measuring brain activity with fMRI. Using information that was present in the multivariate pattern of responses to stimulus features, we could accurately predict, and therefore track, participants' conscious experience from the fMRI signal alone while it underwent many spontaneous changes. Prediction in primary visual cortex primarily reflected eye-based signals, whereas prediction in higher areas reflected the color of the percept. Furthermore, accurate prediction during binocular rivalry could be established with signals recorded during stable monocular viewing, showing that prediction generalized across viewing conditions and did not require or rely on motor responses. It is therefore possible to predict the dynamically changing time course of subjective experience with only brain activity.     

Long-term speeding in perceptual switches mediated by attention-dependent plasticity in cortical visual processing

Binocular rivalry has been extensively studied to understand the mechanisms that control switches in visual awareness and much has been revealed about the contributions of stimulus and cognitive factors. Because visual processes are fundamentally adaptive, however, it is also important to understand how experience alters the dynamics of perceptual switches. When observers viewed binocular rivalry repeatedly over many days, the rate of perceptual switches increased as much as 3-fold. This long-term rivalry speeding exhibited a pattern of image-feature specificity that ruled out primary contributions from strategic and nonsensory factors and implicated neural plasticity occurring in both low- and high-level visual processes in the ventral stream. Furthermore, the speeding occurred only when the rivaling patterns were voluntarily attended, suggesting that the underlying neural plasticity selectively engages when stimuli are behaviorally relevant. Long-term rivalry speeding may thus reflect broader mechanisms that facilitate quick assessments of signals that contain multiple behaviorally relevant interpretations.     

Subjective visual perception: from local processing to emergent phenomena of brain activity

The combination of electrophysiological recordings with ambiguous visual stimulation made possible the detection of neurons that represent the content of subjective visual perception and perceptual suppression in multiple cortical and subcortical brain regions. These neuronal populations, commonly referred to as the neural correlates of consciousness, are more likely to be found in the temporal and prefrontal cortices as well as the pulvinar, indicating that the content of perceptual awareness is represented with higher fidelity in higher-order association areas of the cortical and thalamic hierarchy, reflecting the outcome of competitive interactions between conflicting sensory information resolved in earlier stages. However, despite the significant insights into conscious perception gained through monitoring the activities of single neurons and small, local populations, the immense functional complexity of the brain arising from correlations in the activity of its constituent parts suggests that local, microscopic activity could only partially reveal the mechanisms involved in perceptual awareness. Rather, the dynamics of functional connectivity patterns on a mesoscopic and macroscopic level could be critical for conscious perception. Understanding these emergent spatio-temporal patterns could be informative not only for the stability of subjective perception but also for spontaneous perceptual transitions suggested to depend either on the dynamics of antagonistic ensembles or on global intrinsic activity fluctuations that may act upon explicit neural representations of sensory stimuli and induce perceptual reorganization. Here, we review the most recent results from local activity recordings and discuss the potential role of effective, correlated interactions during perceptual awareness.     

Single units and conscious vision.

Figures that can be seen in more than one way are invaluable tools for the study of the neural basis of visual awareness, because such stimuli permit the dissociation of the neural responses that underlie what we perceive at any given time from those forming the sensory representation of a visual pattern. To study the former type of responses, monkeys were subjected to binocular rivalry, and the response of neurons in a number of different visual areas was studied while the animals reported their alternating percepts by pulling levers. Perception-related modulations of neural activity were found to occur to different extents in different cortical visual areas. The cells that were affected by suppression were almost exclusively binocular, and their proportion was found to increase in the higher processing stages of the visual system. The strongest correlations between neural activity and perception were observed in the visual areas of the temporal lobe. A strikingly large number of neurons in the early visual areas remained active during the perceptual suppression of the stimulus, a finding suggesting that conscious visual perception might be mediated by only a subset of the cells exhibiting stimulus selective responses. These physiological findings, together with a number of recent psychophysical studies, offer a new explanation of the phenomenon of binocular rivalry. Indeed, rivalry has long been considered to be closely linked with binocular fusion and stereopsis, and the sequences of dominance and suppression have been viewed as the result of competition between the two monocular channels. The physiological data presented here are incompatible with this interpretation. Rather than reflecting interocular competition, the rivalry is most probably between the two different central neural representations generated by the dichoptically presented stimuli. The mechanisms of rivalry are probably the same as, or very similar to, those underlying multistable perception in general, and further physiological studies might reveal much about the neural mechanisms of our perceptual organization.     

General validity of Levelt's propositions reveals common computational mechanisms for visual rivalry

The mechanisms underlying conscious visual perception are often studied with either binocular rivalry or perceptual rivalry stimuli. Despite existing research into both types of rivalry, it remains unclear to what extent their underlying mechanisms involve common computational rules. Computational models of binocular rivalry mechanisms are generally tested against Levelt's four propositions, describing the psychophysical relation between stimulus strength and alternation dynamics in binocular rivalry. Here we use a bistable rotating structure-from-motion sphere, a generally studied form of perceptual rivalry, to demonstrate that Levelt's propositions also apply to the alternation dynamics of perceptual rivalry. Importantly, these findings suggest that bistability in structure-from-motion results from active cross-inhibition between neural populations with computational principles similar to those present in binocular rivalry. Thus, although the neural input to the computational mechanism of rivalry may stem from different cortical neurons and different cognitive levels the computational principles just prior to the production of visual awareness appear to be common to the two types of rivalry.     

Within- and Cross-Modal Distance Information Disambiguate Visual Size-Change Perception

Perception is fundamentally underconstrained because different combinations of object properties can generate the same sensory information. To disambiguate sensory information into estimates of scene properties, our brains incorporate prior knowledge and additional “auxiliary” (i.e., not directly relevant to desired scene property) sensory information to constrain perceptual interpretations. For example, knowing the distance to an object helps in perceiving its size. The literature contains few demonstrations of the use of prior knowledge and auxiliary information in combined visual and haptic disambiguation and almost no examination of haptic disambiguation of vision beyond “bistable” stimuli. Previous studies have reported humans integrate multiple unambiguous sensations to perceive single, continuous object properties, like size or position. Here we test whether humans use visual and haptic information, individually and jointly, to disambiguate size from distance. We presented participants with a ball moving in depth with a changing diameter. Because no unambiguous distance information is available under monocular viewing, participants rely on prior assumptions about the ball''s distance to disambiguate their -size percept. Presenting auxiliary binocular and/or haptic distance information augments participants'' prior distance assumptions and improves their size judgment accuracy—though binocular cues were trusted more than haptic. Our results suggest both visual and haptic distance information disambiguate size perception, and we interpret these results in the context of probabilistic perceptual reasoning.     

A Common Neurodynamical Mechanism Could Mediate Externally Induced and Intrinsically Generated Transitions in Visual Awareness

The neural correlates of conscious visual perception are commonly studied in paradigms of perceptual multistability that allow multiple perceptual interpretations during unchanged sensory stimulation. What is the source of this multistability in the content of perception? From a theoretical perspective, a fine balance between deterministic and stochastic forces has been suggested to underlie the spontaneous, intrinsically driven perceptual transitions observed during multistable perception. Deterministic forces are represented by adaptation of feature-selective neuronal populations encoding the competing percepts while stochastic forces are modeled as noise-driven processes. Here, we used a unified neuronal competition model to study the dynamics of adaptation and noise processes in binocular flash suppression (BFS), a form of externally induced perceptual suppression, and compare it with the dynamics of intrinsically driven alternations in binocular rivalry (BR). For the first time, we use electrophysiological, biologically relevant data to constrain a model of perceptual rivalry. Specifically, we show that the mean population discharge pattern of a perceptually modulated neuronal population detected in electrophysiological recordings in the lateral prefrontal cortex (LPFC) during BFS, constrains the dynamical range of externally induced perceptual transitions to a region around the bifurcation separating a noise-driven attractor regime from an adaptation-driven oscillatory regime. Most interestingly, the dynamical range of intrinsically driven perceptual transitions during BR is located in the noise-driven attractor regime, where it overlaps with BFS. Our results suggest that the neurodynamical mechanisms of externally induced and spontaneously generated perceptual alternations overlap in a narrow, noise-driven region just before a bifurcation where the system becomes adaptation-driven.     

Touch Interacts with Vision during Binocular Rivalry with a Tight Orientation Tuning

Multisensory integration is a common feature of the mammalian brain that allows it to deal more efficiently with the ambiguity of sensory input by combining complementary signals from several sensory sources. Growing evidence suggests that multisensory interactions can occur as early as primary sensory cortices. Here we present incompatible visual signals (orthogonal gratings) to each eye to create visual competition between monocular inputs in primary visual cortex where binocular combination would normally take place. The incompatibility prevents binocular fusion and triggers an ambiguous perceptual response in which the two images are perceived one at a time in an irregular alternation. One key function of multisensory integration is to minimize perceptual ambiguity by exploiting cross-sensory congruence. We show that a haptic signal matching one of the visual alternatives helps disambiguate visual perception during binocular rivalry by both prolonging the dominance period of the congruent visual stimulus and by shortening its suppression period. Importantly, this interaction is strictly tuned for orientation, with a mismatch as small as 7.5° between visual and haptic orientations sufficient to annul the interaction. These results indicate important conclusions: first, that vision and touch interact at early levels of visual processing where interocular conflicts are first detected and orientation tunings are narrow, and second, that haptic input can influence visual signals outside of visual awareness, bringing a stimulus made invisible by binocular rivalry suppression back to awareness sooner than would occur without congruent haptic input.     

Fusion and rivalry are dependent on the perceptual meaning of visual stimuli

We view the world with two eyes and yet are typically only aware of a single, coherent image. Arguably the simplest explanation for this is that the visual system unites the two monocular stimuli into a common stream that eventually leads to a single coherent sensation. However, this notion is inconsistent with the well-known phenomenon of rivalry; when physically different stimuli project to the same retinal location, the ensuing perception alternates between the two monocular views in space and time. Although fundamental for understanding the principles of binocular vision and visual awareness, the mechanisms under-lying binocular rivalry remain controversial. Specifically, there is uncertainty about what determines whether monocular images undergo fusion or rivalry. By taking advantage of the perceptual phenomenon of color contrast, we show that physically identical monocular stimuli tend to rival-not fuse-when they signify different objects at the same location in visual space. Conversely, when physically different monocular stimuli are likely to represent the same object at the same location in space, fusion is more likely to result. The data suggest that what competes for visual awareness in the two eyes is not the physical similarity between images but the similarity in their perceptual/empirical meaning.     

Is Attention Based on Spatial Contextual Memory Preferentially Guided by Low Spatial Frequency Signals?

A popular model of visual perception states that coarse information (carried by low spatial frequencies) along the dorsal stream is rapidly transmitted to prefrontal and medial temporal areas, activating contextual information from memory, which can in turn constrain detailed input carried by high spatial frequencies arriving at a slower rate along the ventral visual stream, thus facilitating the processing of ambiguous visual stimuli. We were interested in testing whether this model contributes to memory-guided orienting of attention. In particular, we asked whether global, low-spatial frequency (LSF) inputs play a dominant role in triggering contextual memories in order to facilitate the processing of the upcoming target stimulus. We explored this question over four experiments. The first experiment replicated the LSF advantage reported in perceptual discrimination tasks by showing that participants were faster and more accurate at matching a low spatial frequency version of a scene, compared to a high spatial frequency version, to its original counterpart in a forced-choice task. The subsequent three experiments tested the relative contributions of low versus high spatial frequencies during memory-guided covert spatial attention orienting tasks. Replicating the effects of memory-guided attention, pre-exposure to scenes associated with specific spatial memories for target locations (memory cues) led to higher perceptual discrimination and faster response times to identify targets embedded in the scenes. However, either high or low spatial frequency cues were equally effective; LSF signals did not selectively or preferentially contribute to the memory-driven attention benefits to performance. Our results challenge a generalized model that LSFs activate contextual memories, which in turn bias attention and facilitate perception.