The peri-saccadic perception of objects and space

Eye movements affect object localization and object recognition. Around saccade onset, briefly flashed stimuli appear compressed towards the saccade target, receptive fields dynamically change position, and the recognition of objects near the saccade target is improved. These effects have been attributed to different mechanisms. We provide a unifying account of peri-saccadic perception explaining all three phenomena by a quantitative computational approach simulating cortical cell responses on the population level. Contrary to the common view of spatial attention as a spotlight, our model suggests that oculomotor feedback alters the receptive field structure in multiple visual areas at an intermediate level of the cortical hierarchy to dynamically recruit cells for processing a relevant part of the visual field. The compression of visual space occurs at the expense of this locally enhanced processing capacity.     

Visual saliency and spike timing in the ventral visual pathway.

Visual saliency is a fundamental yet hard to define property of objects or locations in the visual world. In a context where objects and their representations compete to dominate our perception, saliency can be thought of as the "juice" that makes objects win the race. It is often assumed that saliency is extracted and represented in an explicit saliency map, which serves to determine the location of spatial attention at any given time. It is then by drawing attention to a salient object that it can be recognized or categorized. I argue against this classical view that visual "bottom-up" saliency automatically recruits the attentional system prior to object recognition. A number of visual processing tasks are clearly performed too fast for such a costly strategy to be employed. Rather, visual attention could simply act by biasing a saliency-based object recognition system. Under natural conditions of stimulation, saliency can be represented implicitly throughout the ventral visual pathway, independent of any explicit saliency map. At any given level, the most activated cells of the neural population simply represent the most salient locations. The notion of saliency itself grows increasingly complex throughout the system, mostly based on luminance contrast until information reaches visual cortex, gradually incorporating information about features such as orientation or color in primary visual cortex and early extrastriate areas, and finally the identity and behavioral relevance of objects in temporal cortex and beyond. Under these conditions the object that dominates perception, i.e. the object yielding the strongest (or the first) selective neural response, is by definition the one whose features are most "salient"--without the need for any external saliency map. In addition, I suggest that such an implicit representation of saliency can be best encoded in the relative times of the first spikes fired in a given neuronal population. In accordance with our subjective experience that saliency and attention do not modify the appearance of objects, the feed-forward propagation of this first spike wave could serve to trigger saliency-based object recognition outside the realm of awareness, while conscious perceptions could be mediated by the remaining discharges of longer neuronal spike trains.     

The Medawar Lecture 2001 knowledge for vision: vision for knowledge

An evolutionary development of perception is suggested-from passive reception to active perception to explicit conception-earlier stages being largely retained and incorporated in later species. A key is innate and then individually learned knowledge, giving meaning to sensory signals. Inappropriate or misapplied knowledge produces rich cognitive phenomena of illusions, revealing normally hidden processes of vision, tentatively classified here in a 'periodic table'. Phenomena of physiology are distinguished from phenomena of general rules and specific object knowledge. It is concluded that vision uses implicit knowledge, and provides knowledge for intelligent behaviour and for explicit conceptual understanding including science.     

Low-level information and high-level perception: the case of speech in noise

Auditory information is processed in a fine-to-crude hierarchical scheme, from low-level acoustic information to high-level abstract representations, such as phonological labels. We now ask whether fine acoustic information, which is not retained at high levels, can still be used to extract speech from noise. Previous theories suggested either full availability of low-level information or availability that is limited by task difficulty. We propose a third alternative, based on the Reverse Hierarchy Theory (RHT), originally derived to describe the relations between the processing hierarchy and visual perception. RHT asserts that only the higher levels of the hierarchy are immediately available for perception. Direct access to low-level information requires specific conditions, and can be achieved only at the cost of concurrent comprehension. We tested the predictions of these three views in a series of experiments in which we measured the benefits from utilizing low-level binaural information for speech perception, and compared it to that predicted from a model of the early auditory system. Only auditory RHT could account for the full pattern of the results, suggesting that similar defaults and tradeoffs underlie the relations between hierarchical processing and perception in the visual and auditory modalities.     

Two-dimensional substructure of MT receptive fields.

Neurons at progressively higher levels of the visual system have progressively larger, more complicated receptive fields, presumably constructed from simpler antecedent receptive fields. To study this hierarchical organization, we used sparse white noise to map receptive-field substructure (second order Wiener-like kernels) in an extrastriate motion processing area (MT) of alert monkeys. The maps revealed a clear substructure, on a spatial scale comparable to the receptive fields of the V1 inputs. There were both facilitatory and suppressive interactions that differed in spatial organization and time course. Directional interactions were remarkably precise over a very small spatial range, and reversed when successive stimuli reversed contrast--a neural correlate of "reverse phi" motion perception. The maps of some cells had an unexpected, curved shape, which challenges existing models for direction selectivity.     

Low-Level Information and High-Level Perception: The Case of Speech in Noise

Auditory information is processed in a fine-to-crude hierarchical scheme, from low-level acoustic information to high-level abstract representations, such as phonological labels. We now ask whether fine acoustic information, which is not retained at high levels, can still be used to extract speech from noise. Previous theories suggested either full availability of low-level information or availability that is limited by task difficulty. We propose a third alternative, based on the Reverse Hierarchy Theory (RHT), originally derived to describe the relations between the processing hierarchy and visual perception. RHT asserts that only the higher levels of the hierarchy are immediately available for perception. Direct access to low-level information requires specific conditions, and can be achieved only at the cost of concurrent comprehension. We tested the predictions of these three views in a series of experiments in which we measured the benefits from utilizing low-level binaural information for speech perception, and compared it to that predicted from a model of the early auditory system. Only auditory RHT could account for the full pattern of the results, suggesting that similar defaults and tradeoffs underlie the relations between hierarchical processing and perception in the visual and auditory modalities.     

Eye movements modulate visual receptive fields of V4 neurons

The receptive field, defined as the spatiotemporal selectivity of neurons to sensory stimuli, is central to our understanding of the neuronal mechanisms of perception. However, despite the fact that eye movements are critical during normal vision, the influence of eye movements on the structure of receptive fields has never been characterized. Here, we map the receptive fields of macaque area V4 neurons during saccadic eye movements and find that receptive fields are remarkably dynamic. Specifically, before the initiation of a saccadic eye movement, receptive fields shrink and shift towards the saccade target. These spatiotemporal dynamics may enhance information processing of relevant stimuli during the scanning of a visual scene, thereby assisting the selection of saccade targets and accelerating the analysis of the visual scene during free viewing.     

Contextual modulation outside of awareness

Contextual effects are ubiquitous in vision and reveal fundamental principles of sensory coding. Here, we demonstrate that an oriented surround grating can affect the perceived orientation of a central test grating even when backward masking of the surround prevents its orientation from being consciously perceived. The effect survives introduction of a gap between test and surround of over a degree even under masking, suggesting either that contextual information can effectively propagate across early visual cortex in the absence of awareness of the signaled context or that it can proceed undetected to higher processing levels at which such horizontal propagation may not be necessary. The effect under masking also shows partial interocular transfer, demonstrating processing of orientation by binocular neurons in visual cortex in the absence of conscious orientation perception. This pattern of results is consistent with the suggestion that simultaneous orientation contrast is mediated at multiple levels of the visual processing hierarchy, and it supports the view that propagation of signals to and, possibly, back from higher visual areas is necessary for conscious perception.     

Border ownership from intracortical interactions in visual area v2

A border between two image regions normally belongs to only one of the regions; determining which one it belongs to is essential for surface perception and figure-ground segmentation. Border ownership is signaled by a class of V2 neurons, even though its value depends on information coming from well outside their classical receptive fields. I use a model of V2 to show that this visual area is able to generate the ownership signal by itself, without requiring any top-down mechanism or external explicit labels for figures, T junctions, or corners. In the model, neurons have spatially local classical receptive fields, are tuned to orientation, and receive information (from V1) about the location and orientation of borders. Border ownership signals that model physiological observations arise through finite range, intraareal interactions. Additional effects from surface features and attention are discussed. The model licenses testable predictions.     

Implicit attentional selection of bound visual features

Traditionally, research on visual attention has been focused on the processes involved in conscious, explicit selection of task-relevant sensory input. Recently, however, it has been shown that attending to a specific feature of an object automatically increases neural sensitivity to this feature throughout the visual field. Here we show that directing attention to a specific color of an object results in attentional modulation of the processing of task-irrelevant and not consciously perceived motion signals that are spatiotemporally associated with this color throughout the visual field. Such implicit cross-feature spreading of attention takes place according to the veridical physical associations between the color and motion signals, even under special circumstances when they are perceptually misbound. These results imply that the units of implicit attentional selection are spatiotemporally colocalized feature clusters that are automatically bound throughout the visual field.     

Competitive hierarchies in marine benthic communities

Summary Patterns of competitive displacement by over-growth were examined in communities of sessile organisms in the low intertidal zone at three sites in Washington state and Alaska. Cruotose invertebrates and algae can be arranged into a hierarchy such that species of lower competitive rank rarely overgrow any higher ranking species. Erect and solitary species show a wide range of competitive abilities, but whether they fall into a strict hierarchy is unknown. Few of the solitary or erect species occupy substantial amounts of space in the communities examined.An approximate competitive hierarchy is well established in middle to high intertidal areas dominated by mussels, fleshy algae, and barncles, and has been an important concept in developing both an intuitive understanding and specific mathematical models of the dynamics of benthic marine communities. In particular, lower ranking species in such communities are thought to depend upon predation or chronic disturbance to the dominants to avoid competitive displacement. An alternative viewpoint, proposed on the basis of nonstransitive competitive relationships observed in cryptic encrusting communities on the undersides of coral plates, is that specific competitive loops or networks allow the coexistence of a number of competitors. Although the growth forms and higher taxa represented in the low intertidal bear some similarity to those in the cryptic coral reef community, there is little evidence of ecologically important competitive loops in the intertidal. A reanalysis of data from cryptic reef communities suggests that they also do not depart substantially from a competitive hierarchy, although there appear to be many more cases of local reversals in the outcome of competition. It is suggested that the ecological importance of departures from a strict hierarchy depends upon the competitive rankings of the participants, with departures involving competitively dominant species likely to contribute much more to community structure than those involving opportunistic species.     

Is sparse and distributed the coding goal of simple cells?

The question of why the receptive fields of simple cells in the primary visual cortex are Gabor-like is a crucial one in vision research. Many research efforts (Olshausen and Field 1996, 1997; van Hateren and Ruderman 1998; van Hateren and van der Schaaf 1998) that yield a set of localized, oriented, and bandpass Gabor-like receptive fields believe that sparse and distributed is the coding goal of simple cells. This paper investigates a more general coding strategy that measures equally any departure from normality in the simple cells responses. That is, we investigate the possibility that highly kurtotic response histograms may result if simple cells explicitly seek, not maximally kurtotic, but rather maximally non-Gaussian response histograms to natural images. It is found that, under this coding strategy, the simulations produce a majority of localized, oriented, bandpass (Gabor-like) receptive fields. Some receptive fields, however, are spatially distributed and show little oriented structure. Nearly all receptive fields, regardless of whether they are Gabor-like or non-Gabor-like, yield highly kurtotic response histograms to natural images. Thus, in seeking maximally non-Gaussian response histograms, receptive fields spontaneously yield highly kurtotic histograms. The presence in our ensemble of nonlocalized, nonoriented receptive fields may be due to the artificial requirement that receptive fields be orthonormal. We conclude that the high kurtoses observed in the response histograms of simple-cell receptive fields to natural images may reflect a property of natural images themselves rather than an explicit coding goal used to structure simple-cell receptive fields.Acknowledgement This work was supported by the US Office of Naval Research under agreement number N68936-00-2-0002.     

Dissociation of neuronal and psychophysical responses to local and global motion

Most neurons in cortical area MT (V5) are strongly direction selective, and their activity is closely associated with the perception of visual motion. These neurons have large receptive fields built by combining inputs with smaller receptive fields that respond to local motion. Humans integrate motion over large areas and can perceive what has been referred to as global motion. The large size and direction selectivity of MT receptive fields suggests that MT neurons may represent global motion. We have explored this possibility by measuring responses to a stimulus in which the directions of simultaneously presented local and global motion are independently controlled. Surprisingly, MT responses depended only on the local motion and were unaffected by the global motion. Yet, under similar conditions, human observers perceive global motion and are impaired in discriminating local motion. Although local motion perception might depend on MT signals, global motion perception depends on mechanisms qualitatively different from those in MT. Motion perception therefore does not depend on a single cortical area but reflects the action and interaction of multiple brain systems.     

Are V1 Simple Cells Optimized for Visual Occlusions? A Comparative Study

Simple cells in primary visual cortex were famously found to respond to low-level image components such as edges. Sparse coding and independent component analysis (ICA) emerged as the standard computational models for simple cell coding because they linked their receptive fields to the statistics of visual stimuli. However, a salient feature of image statistics, occlusions of image components, is not considered by these models. Here we ask if occlusions have an effect on the predicted shapes of simple cell receptive fields. We use a comparative approach to answer this question and investigate two models for simple cells: a standard linear model and an occlusive model. For both models we simultaneously estimate optimal receptive fields, sparsity and stimulus noise. The two models are identical except for their component superposition assumption. We find the image encoding and receptive fields predicted by the models to differ significantly. While both models predict many Gabor-like fields, the occlusive model predicts a much sparser encoding and high percentages of ‘globular’ receptive fields. This relatively new center-surround type of simple cell response is observed since reverse correlation is used in experimental studies. While high percentages of ‘globular’ fields can be obtained using specific choices of sparsity and overcompleteness in linear sparse coding, no or only low proportions are reported in the vast majority of studies on linear models (including all ICA models). Likewise, for the here investigated linear model and optimal sparsity, only low proportions of ‘globular’ fields are observed. In comparison, the occlusive model robustly infers high proportions and can match the experimentally observed high proportions of ‘globular’ fields well. Our computational study, therefore, suggests that ‘globular’ fields may be evidence for an optimal encoding of visual occlusions in primary visual cortex.     

Invariance of visual operations at the level of receptive fields

The brain is able to maintain a stable perception although the visual stimuli vary substantially on the retina due to geometric transformations and lighting variations in the environment. This paper presents a theory for achieving basic invariance properties already at the level of receptive fields. Specifically, the presented framework comprises (i) local scaling transformations caused by objects of different size and at different distances to the observer, (ii) locally linearized image deformations caused by variations in the viewing direction in relation to the object, (iii) locally linearized relative motions between the object and the observer and (iv) local multiplicative intensity transformations caused by illumination variations. The receptive field model can be derived by necessity from symmetry properties of the environment and leads to predictions about receptive field profiles in good agreement with receptive field profiles measured by cell recordings in mammalian vision. Indeed, the receptive field profiles in the retina, LGN and V1 are close to ideal to what is motivated by the idealized requirements. By complementing receptive field measurements with selection mechanisms over the parameters in the receptive field families, it is shown how true invariance of receptive field responses can be obtained under scaling transformations, affine transformations and Galilean transformations. Thereby, the framework provides a mathematically well-founded and biologically plausible model for how basic invariance properties can be achieved already at the level of receptive fields and support invariant recognition of objects and events under variations in viewpoint, retinal size, object motion and illumination. The theory can explain the different shapes of receptive field profiles found in biological vision, which are tuned to different sizes and orientations in the image domain as well as to different image velocities in space-time, from a requirement that the visual system should be invariant to the natural types of image transformations that occur in its environment.     

The Ferrier Lecture 1995 behind the seen: the functional specialization of the brain in space and time

The visual brain consists of many different visual areas, which are functionally specialized to process and perceive different attributes of the visual scene. However, the time taken to process different attributes varies; consequently, we see some attributes before others. It follows that there is a perceptual asynchrony and hierarchy in visual perception. Because perceiving an attribute is tantamount to becoming conscious of it, it follows that we become conscious of different attributes at different times. Visual consciousness is therefore distributed in time. Given that we become conscious of different visual attributes because of activity at different, functionally specialized, areas of the visual brain, it follows that visual consciousness is also distributed in space. Therefore, visual consciousness is not a single unified entity, but consists of many microconsciousnesses.     

Implicit and explicit processes in social cognition

In this review we consider research on social cognition in which implicit processes can be compared and contrasted with explicit, conscious processes. In each case, their function is distinct, sometimes complementary and sometimes oppositional. We argue that implicit processes in social interaction are automatic and are often opposed to conscious strategies. While we are aware of explicit processes in social interaction, we cannot always use them to override implicit processes. Many studies show that implicit processes facilitate the sharing of knowledge, feelings, and actions, and hence, perhaps surprisingly, serve altruism rather than selfishness. On the other hand, higher-level conscious processes are as likely to be selfish as prosocial.     

Preservation of auditory P300-like potentials in cortical deafness

The phenomenon of blindsight has been largely studied and refers to residual abilities of blind patients without an acknowledged visual awareness. Similarly, "deaf hearing" might represent a further example of dissociation between detection and perception of sounds. Here we report the rare case of a patient with a persistent and complete cortical deafness caused by damage to the bilateral temporo-parietal lobes who occasionally showed unexpected reactions to environmental sounds despite she denied hearing. We applied for the first time electrophysiological techniques to better understand auditory processing and perceptual awareness of the patient. While auditory brainstem responses were within normal limits, no middle- and long-latency waveforms could be identified. However, event-related potentials showed conflicting results. While the Mismatch Negativity could not be evoked, robust P3-like waveforms were surprisingly found in the latency range of 600-700 ms. The generation of P3-like potentials, despite extensive destruction of the auditory cortex, might imply the integrity of independent circuits necessary to process auditory stimuli even in the absence of consciousness of sound. Our results support the reverse hierarchy theory that asserts that the higher levels of the hierarchy are immediately available for perception, while low-level information requires more specific conditions. The accurate characterization in terms of anatomy and neurophysiology of the auditory lesions might facilitate understanding of the neural substrates involved in deaf-hearing.     

The functional impact of mental imagery on conscious perception

Mental imagery has been proposed to contribute to a variety of high-level cognitive functions, including memory encoding and retrieval, navigation, spatial planning, and even social communication and language comprehension. However, it is debated whether mental imagery relies on the same sensory representations as perception, and if so, what functional consequences such an overlap might have on perception itself. We report novel evidence that single instances of imagery can have a pronounced facilitatory influence on subsequent conscious perception. Either seeing or imagining a specific pattern could strongly bias which of two competing stimuli reach awareness during binocular rivalry. Effects of imagery and perception were location and orientation specific, accumulated in strength over time, and survived an intervening visual task lasting several seconds prior to presentation of the rivalry display. Interestingly, effects of imagery differed from those of feature-based attention. The results demonstrate that imagery, in the absence of any incoming visual signals, leads to the formation of a short-term sensory trace that can bias future perception, suggesting a means by which high-level processes that support imagination and memory retrieval may shape low-level sensory representations.