A computational model to link psychophysics and cortical cell activation patterns in human texture processing

The human visual system uses texture information to automatically, or pre-attentively, segregate parts of the visual scene. We investigate the neural substrate underlying human texture processing using a computational model that consists of a hierarchy of bi-directionally linked model areas. The model builds upon two key hypotheses, namely that (i) texture segregation is based on boundary detection--rather than clustering of homogeneous items--and (ii) texture boundaries are detected mainly on the basis of a large scenic context that is analyzed by higher cortical areas within the ventral visual pathway, such as area V4. Here, we focus on the interpretation of key results from psychophysical studies on human texture segmentation. In psychophysical studies, texture patterns were varied along several feature dimensions to systematically characterize human performance. We use simulations to demonstrate that the activation patterns of our model directly correlate with the psychophysical results. This allows us to identify the putative neural mechanisms and cortical key areas which underlie human behavior. In particular, we investigate (i) the effects of varying texture density on target saliency, and the impact of (ii) element alignment and (iii) orientation noise on the detectability of a pop-out bar. As a result, we demonstrate that the dependency of target saliency on texture density is linked to a putative receptive field organization of orientation-selective neurons in V4. The effect of texture element alignment is related to grouping mechanisms in early visual areas. Finally, the modulation of cell activity by feedback activation from higher model areas, interacting with mechanisms of intra-areal center-surround competition, is shown to result in the specific suppression of noise-related cell activities and to improve the overall model capabilities in texture segmentation. In particular, feedback interaction is crucial to raise the model performance to the level of human observers.     

Neural processes in symmetry perception: a parallel spatio-temporal model

Symmetry is usually computationally expensive to detect reliably, while it is relatively easy to perceive. In spite of many attempts to understand the neurofunctional properties of symmetry processing, no symmetry-specific activation was found in earlier cortical areas. Psychophysical evidence relating to the processing mechanisms suggests that the basic processes of symmetry perception would not perform a serial, point-by-point comparison of structural features but rather operate in parallel. Here, modeling of neural processes in psychophysical detection of bilateral texture symmetry is considered. A simple fine-grained algorithm that is capable of performing symmetry estimation without explicit comparison of remote elements is introduced. A computational model of symmetry perception is then described to characterize the underlying mechanisms as one-dimensional spatio-temporal neural processes, each of which is mediated by intracellular horizontal connections in primary visual cortex and adopts the proposed algorithm for the neural computation. Simulated experiments have been performed to show the efficiency and the dynamics of the model. Model and human performances are comparable for symmetry perception of intensity images. Interestingly, the responses of V1 neurons to propagation activities reflecting higher-order perceptual computations have been reported in neurophysiologic experiments.     

The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.     

Pre-attentive segmentation in the primary visual cortex

The activities of neurons in primary visual cortex have been shown to be significantly influenced by stimuli outside their classical receptive fields. We propose that these contextual influences serve pre-attentive visual segmentation by causing relatively higher neural responses to important or conspicuous image locations, making them more salient for perceptual pop-out. These locations include boundaries between regions, smooth contours, and pop-out targets against backgrounds. The mark of these locations is the breakdown of spatial homogeneity in the input. for instance, at the border between two texture regions of equal mean luminance. This breakdown causes changes in contextual influences, often resulting in higher responses at the border than at surrounding locations. This proposal is implemented in a biologically based model of VI in which contextual influences are mediated by intra-cortical horizontal connections. The behavior of the model is demonstrated using examples of texture segmentation, figure-ground segregation, target-distractor asymmetry, and contour enhancement, and is compared with psychophysical and physiological data. The model predicts (1) how neural responses should be tuned to the orientation of nearby texture borders, (2) a set of qualitative constraints on the structure of the intracortical connections, and (3) stimulus-dependent biases in estimating the locations of the region borders by pre-attentive vision.     

Central neural mechanisms for detecting second-order motion.

Single-unit neurophysiology and human psychophysics have begun to reveal distinct neural mechanisms for processing visual stimuli defined by differences in contrast or texture (second-order motion) rather than by luminance (first-order motion). This processing begins in early visual cortical areas, with subsequent extrastriate specialization, and may provide a basis for form-cue invariant analyses of image structure, such as figure-ground segregation and detection of illusory contours.     

Top-Down Inputs Enhance Orientation Selectivity in Neurons of the Primary Visual Cortex during Perceptual Learning

Perceptual learning has been used to probe the mechanisms of cortical plasticity in the adult brain. Feedback projections are ubiquitous in the cortex, but little is known about their role in cortical plasticity. Here we explore the hypothesis that learning visual orientation discrimination involves learning-dependent plasticity of top-down feedback inputs from higher cortical areas, serving a different function from plasticity due to changes in recurrent connections within a cortical area. In a Hodgkin-Huxley-based spiking neural network model of visual cortex, we show that modulation of feedback inputs to V1 from higher cortical areas results in shunting inhibition in V1 neurons, which changes the response properties of V1 neurons. The orientation selectivity of V1 neurons is enhanced without changing orientation preference, preserving the topographic organizations in V1. These results provide new insights to the mechanisms of plasticity in the adult brain, reconciling apparently inconsistent experiments and providing a new hypothesis for a functional role of the feedback connections.     

No evidence for early modulation of evoked responses in primary visual cortex to irrelevant probe stimuli presented during the attentional blink

Background

During rapid serial visual presentation (RSVP), observers often miss the second of two targets if it appears within 500 ms of the first. This phenomenon, called the attentional blink (AB), is widely held to reflect a bottleneck in the processing of rapidly sequential stimuli that arises after initial sensory registration is complete (i.e., at a relatively late, post-perceptual stage of processing). Contrary to this view, recent fMRI studies have found that activity in the primary visual area (V1), which represents the earliest cortical stage of visual processing, is attenuated during the AB. Here we asked whether such changes in V1 activity during the AB arise in the initial feedforward sweep of stimulus input, or instead reflect the influence of feedback signals from higher cortical areas.

Methodology/Principal Findings

EEG signals were recorded while participants monitored a sequential stream of distractor letters for two target digits (T1 and T2). Neural responses associated with an irrelevant probe stimulus presented simultaneously with T2 were measured using an ERP marker – the C1 component – that reflects initial perceptual processing of visual information in V1. As expected, T2 accuracy was compromised when the inter-target interval was brief, reflecting an AB deficit. Critically, however, the magnitude of the early C1 component evoked by the probe was not reduced during the AB.

Conclusions/Significance

Our finding that early sensory processing of irrelevant probe stimuli is not suppressed during the AB is consistent with theoretical models that assume that the bottleneck underlying the AB arises at a post-perceptual stage of processing. This suggests that reduced neural activity in V1 during the AB is driven by re-entrant signals from extrastriate areas that regulate early cortical activity via feedback connections with V1.     

On the role of medial geometry in human vision.

A key challenge underlying theories of vision is how the spatially restricted, retinotopically represented feature analysis can be integrated to form abstract, coordinate-free object models. A resolution likely depends on the use of intermediate-level representations which can on the one hand be populated by local features and on the other hand be used as atomic units underlying the formation of, and interaction with, object hypotheses. The precise structure of this intermediate representation derives from the varied requirements of a range of visual tasks which motivate a significant role for incorporating a geometry of visual form. The need to integrate input from features capturing surface properties such as texture, shading, motion, color, etc., as well as from features capturing surface discontinuities such as silhouettes, T-junctions, etc., implies a geometry which captures both regional and boundary aspects. Curves, as a geometric model of boundaries, have been extensively used as an intermediate representation in computational, perceptual, and physiological studies, while the use of the medial axis (MA) has been popular mainly in computer vision as a geometric region-based model of the interior of closed boundaries. We extend the traditional model of the MA to represent images, where each MA segment represents a region of the image which we call a visual fragment. We present a unified theory of perceptual grouping and object recognition where through various sequences of transformations of the MA representation, visual fragments are grouped in various configurations to form object hypotheses, and are related to stored models. The mechanisms underlying both the computation and the transformation of the MA is a lateral wave propagation model. Recent psychophysical experiments depicting contrast sensitivity map peaks at the medial axes of stimuli, and experiments on perceptual filling-in, and brightness induction and modulation, are consistent with both the use of an MA representation and a propagation-based scheme. Also, recent neurophysiological recordings in V1 correlate with the MA hypothesis and a horizontal propagation scheme. This evidence supports a geometric computational paradigm for processing sensory data where both dynamic in-plane propagation and feedforward-feedback connections play an integral role.     

V1 mechanisms and some figure-ground and border effects.

V1 neurons have been observed to respond more strongly to figure than background regions. Within a figure region, the responses are usually stronger near figure boundaries (the border effect), than further inside the boundaries. Sometimes the medial axes of the figures (e.g., the vertical midline of a vertical figure strip) induce secondary, intermediate, response peaks (the medial axis effect). Related is the physiologically elusive "cross-orientation facilitation", the observation that a cell's response to a grating patch can be facilitated by an orthogonally oriented grating in the surround. Higher center feedbacks have been suggested to cause these figure-ground effects. It has been shown, using a V1 model, that the causes could be intra-cortical interactions within V1 that serve pre-attentive visual segmentation, particularly, object boundary detection. Furthermore, whereas the border effect is robust, the figure-ground effects in the interior of a figure, in particular, the medial axis effect, are by-products of the border effect and are predicted to diminish to zero for larger figures. This model prediction (of the figure size dependence) was subsequently confirmed physiologically, and supported by findings that the response modulations by texture surround do not depend on feedbacks from V2. In addition, the model explains the "cross-orientation facilitation" as caused by a dis-inhibition, to the cell responding to the center of the central grating, by the background grating. Furthermore, the elusiveness of this phenomena was accounted for by the insight that it depends critically on the size of the figure grating. The model is applied to understand some figure-ground effects and segmentation in psychophysics: in particular, that contrast discrimination threshold is lower within and at the center of a closed contour than that in the background, and that a very briefly presented vernier target can perceptually shine through a subsequently presented large grating centered at the same location.     

Neural Mechanisms of Cortical Motion Computation Based on a Neuromorphic Sensory System

The visual cortex analyzes motion information along hierarchically arranged visual areas that interact through bidirectional interconnections. This work suggests a bio-inspired visual model focusing on the interactions of the cortical areas in which a new mechanism of feedforward and feedback processing are introduced. The model uses a neuromorphic vision sensor (silicon retina) that simulates the spike-generation functionality of the biological retina. Our model takes into account two main model visual areas, namely V1 and MT, with different feature selectivities. The initial motion is estimated in model area V1 using spatiotemporal filters to locally detect the direction of motion. Here, we adapt the filtering scheme originally suggested by Adelson and Bergen to make it consistent with the spike representation of the DVS. The responses of area V1 are weighted and pooled by area MT cells which are selective to different velocities, i.e. direction and speed. Such feature selectivity is here derived from compositions of activities in the spatio-temporal domain and integrating over larger space-time regions (receptive fields). In order to account for the bidirectional coupling of cortical areas we match properties of the feature selectivity in both areas for feedback processing. For such linkage we integrate the responses over different speeds along a particular preferred direction. Normalization of activities is carried out over the spatial as well as the feature domains to balance the activities of individual neurons in model areas V1 and MT. Our model was tested using different stimuli that moved in different directions. The results reveal that the error margin between the estimated motion and synthetic ground truth is decreased in area MT comparing with the initial estimation of area V1. In addition, the modulated V1 cell activations shows an enhancement of the initial motion estimation that is steered by feedback signals from MT cells.     

Correction of Distortion in Flattened Representations of the Cortical Surface Allows Prediction of V1-V3 Functional Organization from Anatomy

Several domains of neuroscience offer map-like models that link location on the cortical surface to properties of sensory representation. Within cortical visual areas V1, V2, and V3, algebraic transformations can relate position in the visual field to the retinotopic representation on the flattened cortical sheet. A limit to the practical application of this structure-function model is that the cortex, while topologically a two-dimensional surface, is curved. Flattening of the curved surface to a plane unavoidably introduces local geometric distortions that are not accounted for in idealized models. Here, we show that this limitation is overcome by correcting the geometric distortion induced by cortical flattening. We use a mass-spring-damper simulation to create a registration between functional MRI retinotopic mapping data of visual areas V1, V2, and V3 and an algebraic model of retinotopy. This registration is then applied to the flattened cortical surface anatomy to create an anatomical template that is linked to the algebraic retinotopic model. This registered cortical template can be used to accurately predict the location and retinotopic organization of these early visual areas from cortical anatomy alone. Moreover, we show that prediction accuracy remains when extrapolating beyond the range of data used to inform the model, indicating that the registration reflects the retinotopic organization of visual cortex. We provide code for the mass-spring-damper technique, which has general utility for the registration of cortical structure and function beyond the visual cortex.     

A neural network model for texture discrimination

A model of texture discrimination in visual cortex was built using a feedforward network with lateral interactions among relatively realistic spiking neural elements. The elements have various membrane currents, equilibrium potentials and time constants, with action potentials and synapses. The model is derived from the modified programs of MacGregor (1987). Gabor-like filters are applied to overlapping regions in the original image; the neural network with lateral excitatory and inhibitory interactions then compares and adjusts the Gabor amplitudes in order to produce the actual texture discrimination. Finally, a combination layer selects and groups various representations in the output of the network to form the final transformed image material. We show that both texture segmentation and detection of texture boundaries can be represented in the firing activity of such a network for a wide variety of synthetic to natural images. Performance details depend most strongly on the global balance of strengths of the excitatory and inhibitory lateral interconnections. The spatial distribution of lateral connective strengths has relatively little effect. Detailed temporal firing activities of single elements in the lateral connected network were examined under various stimulus conditions. Results show (as in area 17 of cortex) that a single element's response to image features local to its receptive field can be altered by changes in the global context.     

Separating Fusion from Rivalry

Visual fusion is the process in which differing but compatible binocular information is transformed into a unified percept. Even though this is at the basis of binocular vision, the underlying neural processes are, as yet, poorly understood. In our study we therefore aimed to investigate neural correlates of visual fusion. To this end, we presented binocularly compatible, fusible (BF), and incompatible, rivaling (BR) stimuli, as well as an intermediate stimulus type containing both binocularly fusible and monocular, incompatible elements (BFR). Comparing BFR stimuli with BF and BR stimuli, respectively, we were able to disentangle brain responses associated with either visual fusion or rivalry. By means of functional magnetic resonance imaging, we measured brain responses to these stimulus classes in the visual cortex, and investigated them in detail at various retinal eccentricities. Compared with BF stimuli, the response to BFR stimuli was elevated in visual cortical areas V1 and V2, but not in V3 and V4 – implying that the response to monocular stimulus features decreased from V1 to V4. Compared to BR stimuli, the response to BFR stimuli decreased with increasing eccentricity, specifically within V3 and V4. Taken together, it seems that although the processing of exclusively monocular information decreases from V1 to V4, the processing of binocularly fused information increases from earlier to later visual areas. Our findings suggest the presence of an inhibitory neural mechanism which, depending on the presence of fusion, acts differently on the processing of monocular information.     

Object Segmentation from Motion Discontinuities and Temporal Occlusions–A Biologically Inspired Model

Background

Optic flow is an important cue for object detection. Humans are able to perceive objects in a scene using only kinetic boundaries, and can perform the task even when other shape cues are not provided. These kinetic boundaries are characterized by the presence of motion discontinuities in a local neighbourhood. In addition, temporal occlusions appear along the boundaries as the object in front covers the background and the objects that are spatially behind it.

Methodology/Principal Findings

From a technical point of view, the detection of motion boundaries for segmentation based on optic flow is a difficult task. This is due to the problem that flow detected along such boundaries is generally not reliable. We propose a model derived from mechanisms found in visual areas V1, MT, and MSTl of human and primate cortex that achieves robust detection along motion boundaries. It includes two separate mechanisms for both the detection of motion discontinuities and of occlusion regions based on how neurons respond to spatial and temporal contrast, respectively. The mechanisms are embedded in a biologically inspired architecture that integrates information of different model components of the visual processing due to feedback connections. In particular, mutual interactions between the detection of motion discontinuities and temporal occlusions allow a considerable improvement of the kinetic boundary detection.

Conclusions/Significance

A new model is proposed that uses optic flow cues to detect motion discontinuities and object occlusion. We suggest that by combining these results for motion discontinuities and object occlusion, object segmentation within the model can be improved. This idea could also be applied in other models for object segmentation. In addition, we discuss how this model is related to neurophysiological findings. The model was successfully tested both with artificial and real sequences including self and object motion.     

V1 mechanisms and some figure–ground and border effects

V1 neurons have been observed to respond more strongly to figure than background regions. Within a figure region, the responses are usually stronger near figure boundaries (the border effect), than further inside the boundaries. Sometimes the medial axes of the figures (e.g., the vertical midline of a vertical figure strip) induce secondary, intermediate, response peaks (the medial axis effect). Related is the physiologically elusive “cross-orientation facilitation”, the observation that a cell's response to a grating patch can be facilitated by an orthogonally oriented grating in the surround. Higher center feedbacks have been suggested to cause these figure–ground effects. It has been shown, using a V1 model, that the causes could be intra-cortical interactions within V1 that serve pre-attentive visual segmentation, particularly, object boundary detection. Furthermore, whereas the border effect is robust, the figure–ground effects in the interior of a figure, in particular, the medial axis effect, are by-products of the border effect and are predicted to diminish to zero for larger figures. This model prediction (of the figure size dependence) was subsequently confirmed physiologically, and supported by findings that the response modulations by texture surround do not depend on feedbacks from V2. In addition, the model explains the “cross-orientation facilitation” as caused by a dis-inhibition, to the cell responding to the center of the central grating, by the background grating. Furthermore, the elusiveness of this phenomena was accounted for by the insight that it depends critically on the size of the figure grating. The model is applied to understand some figure–ground effects and segmentation in psychophysics: in particular, that contrast discrimination threshold is lower within and at the center of a closed contour than that in the background, and that a very briefly presented vernier target can perceptually shine through a subsequently presented large grating centered at the same location.     

Extraction of Surface-Related Features in a Recurrent Model of V1-V2 Interactions

Background

Humans can effortlessly segment surfaces and objects from two-dimensional (2D) images that are projections of the 3D world. The projection from 3D to 2D leads partially to occlusions of surfaces depending on their position in depth and on viewpoint. One way for the human visual system to infer monocular depth cues could be to extract and interpret occlusions. It has been suggested that the perception of contour junctions, in particular T-junctions, may be used as cue for occlusion of opaque surfaces. Furthermore, X-junctions could be used to signal occlusion of transparent surfaces.

Methodology/Principal Findings

In this contribution, we propose a neural model that suggests how surface-related cues for occlusion can be extracted from a 2D luminance image. The approach is based on feedforward and feedback mechanisms found in visual cortical areas V1 and V2. In a first step, contours are completed over time by generating groupings of like-oriented contrasts. Few iterations of feedforward and feedback processing lead to a stable representation of completed contours and at the same time to a suppression of image noise. In a second step, contour junctions are localized and read out from the distributed representation of boundary groupings. Moreover, surface-related junctions are made explicit such that they are evaluated to interact as to generate surface-segmentations in static images. In addition, we compare our extracted junction signals with a standard computer vision approach for junction detection to demonstrate that our approach outperforms simple feedforward computation-based approaches.

Conclusions/Significance

A model is proposed that uses feedforward and feedback mechanisms to combine contextually relevant features in order to generate consistent boundary groupings of surfaces. Perceptually important junction configurations are robustly extracted from neural representations to signal cues for occlusion and transparency. Unlike previous proposals which treat localized junction configurations as 2D image features, we link them to mechanisms of apparent surface segregation. As a consequence, we demonstrate how junctions can change their perceptual representation depending on the scene context and the spatial configuration of boundary fragments.     

A neural network model of attention-modulated neurodynamics

Visual attention appears to modulate cortical neurodynamics and synchronization through various cholinergic mechanisms. In order to study these mechanisms, we have developed a neural network model of visual cortex area V4, based on psychophysical, anatomical and physiological data. With this model, we want to link selective visual information processing to neural circuits within V4, bottom-up sensory input pathways, top-down attention input pathways, and to cholinergic modulation from the prefrontal lobe. We investigate cellular and network mechanisms underlying some recent analytical results from visual attention experimental data. Our model can reproduce the experimental findings that attention to a stimulus causes increased gamma-frequency synchronization in the superficial layers. Computer simulations and STA power analysis also demonstrate different effects of the different cholinergic attention modulation action mechanisms.     

Portrait of visual cortical circuits for generating neural oscillation dynamics

The mouse primary visual cortex (V1) has emerged as a classical system to study neural circuit mechanisms underlying visual function and plasticity. A variety of efferent-afferent neuronal connections exists within the V1 and between the V1 and higher visual cortical areas or thalamic nuclei, indicating that the V1 system is more than a mere receiver in information processing. Sensory representations in the V1 are dynamically correlated with neural activity oscillations that are distributed across different cortical layers in an input-dependent manner. Circuits consisting of excitatory pyramidal cells (PCs) and inhibitory interneurons (INs) are the basis for generating neural oscillations. In general, INs are clustered with their adjacent PCs to form specific microcircuits that gate or filter the neural information. The interaction between these two cell populations has to be coordinated within a local circuit in order to preserve neural coding schemes and maintain excitation–inhibition (E–I) balance. Phasic alternations of the E–I balance can dynamically regulate temporal rhythms of neural oscillation. Accumulating experimental evidence suggests that the two major sub-types of INs, parvalbumin-expressing (PV⁺) cells and somatostatin-expressing (SOM⁺) INs, are active in controlling slow and fast oscillations, respectively, in the mouse V1. The review summarizes recent experimental findings on elucidating cellular or circuitry mechanisms for the generation of neural oscillations with distinct rhythms in either developing or matured mouse V1, mainly focusing on visual relaying circuits and distinct local inhibitory circuits.     

Visual event related potentials. The origin of wave P300. A computer model

This five-layered model consisting of 180 neurons is aimed at simulating some elementary functions of primary visual cortex of mammals in form detection. Its main achievements are: 1. Detection of points, lines, simple geometric figures in the V1. 2. Abstraction of 19 different qualities of geometric figures. 3. Simulation and rational explanation of processing of peripheral stimuli in the V1, explanation of mechanism of origin of visual ERPs, including P300 wave. 4. Simulation and explanation of the nature and build up of the cognitive function within V1 and its possible relation to long-term memory. 5. The model is based partly on Hebb-type synapses, illustrates the role of neuronal assemblies, sheds light on the functional relationship of excitatory and inhibitory neurons, in their conformity with special tasks of different cortical layers.