The time course of segmentation and cue-selectivity in the human visual cortex

Texture discontinuities are a fundamental cue by which the visual system segments objects from their background. The neural mechanisms supporting texture-based segmentation are therefore critical to visual perception and cognition. In the present experiment we employ an EEG source-imaging approach in order to study the time course of texture-based segmentation in the human brain. Visual Evoked Potentials were recorded to four types of stimuli in which periodic temporal modulation of a central 3° figure region could either support figure-ground segmentation, or have identical local texture modulations but not produce changes in global image segmentation. The image discontinuities were defined either by orientation or phase differences across image regions. Evoked responses to these four stimuli were analyzed both at the scalp and on the cortical surface in retinotopic and functional regions-of-interest (ROIs) defined separately using fMRI on a subject-by-subject basis. Texture segmentation (tsVEP: segmenting versus non-segmenting) and cue-specific (csVEP: orientation versus phase) responses exhibited distinctive patterns of activity. Alternations between uniform and segmented images produced highly asymmetric responses that were larger after transitions from the uniform to the segmented state. Texture modulations that signaled the appearance of a figure evoked a pattern of increased activity starting at ～143 ms that was larger in V1 and LOC ROIs, relative to identical modulations that didn't signal figure-ground segmentation. This segmentation-related activity occurred after an initial response phase that did not depend on the global segmentation structure of the image. The two cue types evoked similar tsVEPs up to 230 ms when they differed in the V4 and LOC ROIs. The evolution of the response proceeded largely in the feed-forward direction, with only weak evidence for feedback-related activity.     

Border ownership from intracortical interactions in visual area v2

A border between two image regions normally belongs to only one of the regions; determining which one it belongs to is essential for surface perception and figure-ground segmentation. Border ownership is signaled by a class of V2 neurons, even though its value depends on information coming from well outside their classical receptive fields. I use a model of V2 to show that this visual area is able to generate the ownership signal by itself, without requiring any top-down mechanism or external explicit labels for figures, T junctions, or corners. In the model, neurons have spatially local classical receptive fields, are tuned to orientation, and receive information (from V1) about the location and orientation of borders. Border ownership signals that model physiological observations arise through finite range, intraareal interactions. Additional effects from surface features and attention are discussed. The model licenses testable predictions.     

Early visual cortex: smarter than you think

A recent study has provided elegant evidence that the early visual area V2 plays an important role in image segmentation, piecing together parts of an object with the help of stereoscopic clues.     

The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.     

A Structured Model of Video Reproduces Primary Visual Cortical Organisation

The visual system must learn to infer the presence of objects and features in the world from the images it encounters, and as such it must, either implicitly or explicitly, model the way these elements interact to create the image. Do the response properties of cells in the mammalian visual system reflect this constraint? To address this question, we constructed a probabilistic model in which the identity and attributes of simple visual elements were represented explicitly and learnt the parameters of this model from unparsed, natural video sequences. After learning, the behaviour and grouping of variables in the probabilistic model corresponded closely to functional and anatomical properties of simple and complex cells in the primary visual cortex (V1). In particular, feature identity variables were activated in a way that resembled the activity of complex cells, while feature attribute variables responded much like simple cells. Furthermore, the grouping of the attributes within the model closely parallelled the reported anatomical grouping of simple cells in cat V1. Thus, this generative model makes explicit an interpretation of complex and simple cells as elements in the segmentation of a visual scene into basic independent features, along with a parametrisation of their moment-by-moment appearances. We speculate that such a segmentation may form the initial stage of a hierarchical system that progressively separates the identity and appearance of more articulated visual elements, culminating in view-invariant object recognition.     

Segmentation of moving images by the human visual system

 New segments appearing in an image sequence or spontaneously accelerated segments are band limited by the visual system due to a nonperfect tracking of these segments by eye movements. In spite of this band limitation and acceleration of segments, a coarse segmentation (initial segmentation phase) can be performed by the visual system. This is interesting for the development of purely automatic segmentation algorithms for multimedia applications. In this paper the segmentation of the visual system is modelled and used in an automatic coarse initial segmentation. A suitable model for motion processing based on a spectral representation is presented and applied to the segmentation of synthetic and real image sequences with band limited and accelerated moving foreground and background segments. Received: 1 August 1995/Accepted in revised form: 25 February 1997     

Visual attention: spotlight on the primary visual cortex.

Visual search tasks appear to involve spatially selective attention to the target, but evidence for attentional modulation in the visual area with the most precise retinotopic organization V1 has been elusive. Recent imaging studies show that spatial attention can indeed enhance visual responses in human V1.     

Cortical Surround Interactions and Perceptual Salience via Natural Scene Statistics

Spatial context in images induces perceptual phenomena associated with salience and modulates the responses of neurons in primary visual cortex (V1). However, the computational and ecological principles underlying contextual effects are incompletely understood. We introduce a model of natural images that includes grouping and segmentation of neighboring features based on their joint statistics, and we interpret the firing rates of V1 neurons as performing optimal recognition in this model. We show that this leads to a substantial generalization of divisive normalization, a computation that is ubiquitous in many neural areas and systems. A main novelty in our model is that the influence of the context on a target stimulus is determined by their degree of statistical dependence. We optimized the parameters of the model on natural image patches, and then simulated neural and perceptual responses on stimuli used in classical experiments. The model reproduces some rich and complex response patterns observed in V1, such as the contrast dependence, orientation tuning and spatial asymmetry of surround suppression, while also allowing for surround facilitation under conditions of weak stimulation. It also mimics the perceptual salience produced by simple displays, and leads to readily testable predictions. Our results provide a principled account of orientation-based contextual modulation in early vision and its sensitivity to the homogeneity and spatial arrangement of inputs, and lends statistical support to the theory that V1 computes visual salience.     

Motion edges and regions guide image segmentation by colour.

Image segmentation is an important early stage in visual processing in which the visual system groups together parts of the image that belong together, prior to or in conjunction with object recognition. Two principal processes may be involved in image segmentation: an edge-based process that uses feature contrasts to mark boundaries of coherent regions, and a region-based process that groups similar features over a larger scale. Earlier, we have shown that motion and colour interact strongly in image segmentation by the human visual system. Here we explore the nature of this interaction in terms of edge- and region-based processes. We measure performance on a region-based colour segmentation task in the presence of distinct types of motion information, in the form of edges and regions which in themselves do not reveal the location of the colour target. The results show that both motion edges and regions may guide the integrative process required for this colour segmentation task. Motion edges appear to act by delimiting areas over which to integrate colour information, whereas motion similarities define primitive surfaces within which colour grouping and segmentation processes are deployed.     

AMB-Wnet: Embedding attention model in multi-bridge Wnet for exploring the mechanics of disease

In recent years, progressive application of convolutional neural networks in image processing has successfully filtered into medical diagnosis. As a prerequisite for images detection and classification, object segmentation in medical images has attracted a great deal of attention. This study is based on the fact that most of the analysis of pathological diagnoses requires nuclei detection as the starting phase for obtaining an insight into the underlying biological process and further diagnosis. In this paper, we introduce an embedded attention model in multi-bridge Wnet (AMB-Wnet) to achieve suppression of irrelevant background areas and obtain good features for learning image semantics and modality to automatically segment nuclei, inspired by the 2018 Data Science Bowl. The proposed architecture, consisting of the redesigned down sample group, up-sample group, and middle block (a new multiple-scale convolutional layers block), is designed to extract different level features. In addition, a connection group is proposed instead of skip-connection to transfer semantic information among different levels. In addition, the attention model is well embedded in the connection group, and the performance of the model is improved without increasing the amount of calculation. To validate the model's performance, we evaluated it using the BBBC038V1 data sets for nuclei segmentation. Our proposed model achieves 85.83% F1-score, 97.81% accuracy, 86.12% recall, and 83.52% intersection over union. The proposed AMB-Wnet exhibits superior results compared to the original U-Net, MultiResUNet, and recent Attention U-Net architecture.     

Minimizing Manual Image Segmentation Turn-Around Time for Neuronal Reconstruction by Embracing Uncertainty

The ability to automatically segment an image into distinct regions is a critical aspect in many visual processing applications. Because inaccuracies often exist in automatic segmentation, manual segmentation is necessary in some application domains to correct mistakes, such as required in the reconstruction of neuronal processes from microscopic images. The goal of the automated segmentation tool is traditionally to produce the highest-quality segmentation, where quality is measured by the similarity to actual ground truth, so as to minimize the volume of manual correction necessary. Manual correction is generally orders-of-magnitude more time consuming than automated segmentation, often making handling large images intractable. Therefore, we propose a more relevant goal: minimizing the turn-around time of automated/manual segmentation while attaining a level of similarity with ground truth. It is not always necessary to inspect every aspect of an image to generate a useful segmentation. As such, we propose a strategy to guide manual segmentation to the most uncertain parts of segmentation. Our contributions include 1) a probabilistic measure that evaluates segmentation without ground truth and 2) a methodology that leverages these probabilistic measures to significantly reduce manual correction while maintaining segmentation quality.     

Distribution of attention modulates salience signals in early visual cortex

Previous research has shown that the extent to which people spread attention across the visual field plays a crucial role in visual selection and the occurrence of bottom-up driven attentional capture. Consistent with previous findings, we show that when attention was diffusely distributed across the visual field while searching for a shape singleton, an irrelevant salient color singleton captured attention. However, while using the very same displays and task, no capture was observed when observers initially focused their attention at the center of the display. Using event-related fMRI, we examined the modulation of retinotopic activity related to attentional capture in early visual areas. Because the sensory display characteristics were identical in both conditions, we were able to isolate the brain activity associated with exogenous attentional capture. The results show that spreading of attention leads to increased bottom-up exogenous capture and increased activity in visual area V3 but not in V2 and V1.     

Attention modulates contextual influences in the primary visual cortex of alert monkeys

The response properties of cells in the primary visual cortex (V1) were measured while the animals directed their attention either to the position of the neuron's receptive field (RF), to a position away from the RF (focal attention), or to four locations in the visual field (distributed attention). Over the population, varying attentional state had no significant effect on the response to an isolated stimulus within the RF but had a large influence on the facilitatory effects of contextual lines. We propose that the attentional modulation of contextual effects represents a gating of long range horizontal connections within area V1 by feedback connections to V1 and that this gating provides a mechanism for shaping responses under attention to stimulus configuration.     

Oscillatory network with self-organized dynamical connections for synchronization-based image segmentation

An oscillatory network of columnar architecture located in 3D spatial lattice was recently designed by the authors as oscillatory model of the brain visual cortex. Single network oscillator is a relaxational neural oscillator with internal dynamics tunable by visual image characteristics - local brightness and elementary bar orientation. It is able to demonstrate either activity state (stable undamped oscillations) or "silence" (quickly damped oscillations). Self-organized nonlocal dynamical connections of oscillators depend on oscillator activity levels and orientations of cortical receptive fields. Network performance consists in transfer into a state of clusterized synchronization. At current stage grey-level image segmentation tasks are carried out by 2D oscillatory network, obtained as a limit version of the source model. Due to supplemented network coupling strength control the 2D reduced network provides synchronization-based image segmentation. New results on segmentation of brightness and texture images presented in the paper demonstrate accurate network performance and informative visualization of segmentation results, inherent in the model.     

Separate modulations of human V1 associated with spatial attention and task structure

Functional magnetic resonance imaging (fMRI) was used while normal human volunteers engaged in simple detection and discrimination tasks, revealing separable modulations of early visual cortex associated with spatial attention and task structure. Both modulations occur even when there is no change in sensory stimulation. The modulation due to spatial attention is present throughout the early visual areas V1, V2, V3, and VP, and varies with the attended location. The task structure activations are strongest in V1 and are greater in regions that represent more peripheral parts of the visual field. Control experiments demonstrate that the task structure activations cannot be attributed to visual, auditory, or somatosensory processing, the motor response for the detection/discrimination judgment, or oculomotor responses such as blinks or saccades. These findings demonstrate that early visual areas are modulated by at least two types of endogenous signals, each with distinct cortical distributions.     

Preparatory effects of distractor suppression: evidence from visual cortex

Spatial selective attention is the mechanism that facilitates the selection of relevant information over irrelevant information in the visual field. The current study investigated whether foreknowledge of the presence or absence of distractors surrounding an impending target stimulus results in preparatory changes in visual cortex. We cued the location of the target and the presence or absence of distractors surrounding the target while changes in blood oxygen level dependent (BOLD) signals were measured. In line with prior work, we found that top-down spatial attention resulted in an increased contralateral BOLD response, evoked by the cue throughout early visual cortex (areas V1, V2 and V3). In addition, cues indicating distractor presence evoked a substantial increase in the magnitude of the BOLD signal in visual area V3, but not in V2 or V1. This study shows that prior knowledge concerning the presence of a distractor results in enhanced attentional modulation of visual cortex, in visual areas where neuronal receptive fields are large enough to encompass both targets and distractors. We interpret these findings as evidence that top-down attentional control processes include active preparatory suppression mechanisms for irrelevant, distracting information in the visual scene.     

Saccades differentially modulate human LGN and V1 responses in the presence and absence of visual stimulation

Saccades occur several times each second in normal human vision. The visual image moves across the retina at high velocity during a saccade, yet no blurring of the visual scene is perceived . Active suppression of visual input may account for this perceptual continuity, but the neural mechanisms underlying such saccadic suppression remain unclear. We used functional MRI to specifically examine responses in the lateral geniculate nucleus (LGN) and primary visual cortex (V1) during saccades. Activity in both V1 and LGN was strongly modulated by saccades. Furthermore, this modulation depended on whether visual stimulation was present or absent. In complete darkness, saccades led to reliable signal increases in V1 and LGN, whereas in the presence of visual stimulation, saccades led to suppression of visually evoked responses. These findings represent unequivocal evidence for saccadic suppression in human LGN and retinotopically defined V1 and are consistent with the earliest site of saccadic suppression lying at or before V1.     

Primary Visual Cortex as a Saliency Map: A Parameter-Free Prediction and Its Test by Behavioral Data

It has been hypothesized that neural activities in the primary visual cortex (V1) represent a saliency map of the visual field to exogenously guide attention. This hypothesis has so far provided only qualitative predictions and their confirmations. We report this hypothesis’ first quantitative prediction, derived without free parameters, and its confirmation by human behavioral data. The hypothesis provides a direct link between V1 neural responses to a visual location and the saliency of that location to guide attention exogenously. In a visual input containing many bars, one of them saliently different from all the other bars which are identical to each other, saliency at the singleton’s location can be measured by the shortness of the reaction time in a visual search for singletons. The hypothesis predicts quantitatively the whole distribution of the reaction times to find a singleton unique in color, orientation, and motion direction from the reaction times to find other types of singletons. The prediction matches human reaction time data. A requirement for this successful prediction is a data-motivated assumption that V1 lacks neurons tuned simultaneously to color, orientation, and motion direction of visual inputs. Since evidence suggests that extrastriate cortices do have such neurons, we discuss the possibility that the extrastriate cortices play no role in guiding exogenous attention so that they can be devoted to other functions like visual decoding and endogenous attention.     

Attention-dependent representation of a size illusion in human V1

One of the most fundamental properties of human primary visual cortex (V1) is its retinotopic organization, which makes it an ideal candidate for encoding spatial properties, such as size, of objects. However, three-dimensional (3D) contextual information can lead to size illusions that are reflected in the spatial pattern of activity in V1 [1]. A critical question is how complex 3D contextual information can influence spatial activity patterns in V1. Here, we assessed whether changes in the spatial distribution of activity in V1 depend on the focus of attention, which would be suggestive of feedback of 3D contextual information from higher visual areas. We presented two 3D rings at close and far apparent depths in a 3D scene. When subjects fixated its center, the far ring appeared to be larger and occupy a more eccentric portion of the visual field, relative to the close ring. Using functional magnetic resonance imaging, we found that the spatial distribution of V1 activity induced by the far ring was also shifted toward a more eccentric representation of the visual field, whereas that induced by the close ring was shifted toward the foveal representation, consistent with their perceptual appearances. This effect was significantly reduced when the focus of spatial attention was narrowed with a demanding central fixation task. We reason that focusing attention on the fixation task resulted in reduced activity in--and therefore reduced feedback from--higher visual areas that process the 3D depth cues.