The functional geometry of local and horizontal connections in a model of V1.

A mathematical model of interacting hypercolumns in primary visual cortex (V1) is presented that incorporates details concerning the geometry of local and long-range horizontal connections. Each hypercolumn is modeled as a network of interacting excitatory and inhibitory neural populations with orientation and spatial frequency preferences organized around a pair of pinwheels. The pinwheels are arranged on a planar lattice, reflecting the crystalline-like structure of cortex. Local interactions within a hypercolumn generate orientation and spatial frequency tuning curves, which are modulated by horizontal connections between different hypercolumns on the lattice. The symmetry properties of the local and long-range connections play an important role in determining the types of spontaneous activity patterns that can arise in cortex.     

A spherical model for orientation and spatial-frequency tuning in a cortical hypercolumn

A theory is presented of the way in which the hypercolumns in primary visual cortex (V1) are organized to detect important features of visual images, namely local orientation and spatial-frequency. Given the existence in V1 of dual maps for these features, both organized around orientation pinwheels, we constructed a model of a hypercolumn in which orientation and spatial-frequency preferences are represented by the two angular coordinates of a sphere. The two poles of this sphere are taken to correspond, respectively, to high and low spatial-frequency preferences. In Part I of the paper, we use mean-field methods to derive exact solutions for localized activity states on the sphere. We show how cortical amplification through recurrent interactions generates a sharply tuned, contrast-invariant population response to both local orientation and local spatial frequency, even in the case of a weakly biased input from the lateral geniculate nucleus (LGN). A major prediction of our model is that this response is non-separable with respect to the local orientation and spatial frequency of a stimulus. That is, orientation tuning is weaker around the pinwheels, and there is a shift in spatial-frequency tuning towards that of the closest pinwheel at non-optimal orientations. In Part II of the paper, we demonstrate that a simple feed-forward model of spatial-frequency preference, unlike that for orientation preference, does not generate a faithful representation when amplified by recurrent interactions in V1. We then introduce the idea that cortico-geniculate feedback modulates LGN activity to generate a faithful representation, thus providing a new functional interpretation of the role of this feedback pathway. Using linear filter theory, we show that if the feedback from a cortical cell is taken to be approximately equal to the reciprocal of the corresponding feed-forward receptive field (in the two-dimensional Fourier domain), then the mismatch between the feed-forward and cortical frequency representations is eliminated. We therefore predict that cortico-geniculate feedback connections innervate the LGN in a pattern determined by the orientation and spatial-frequency biases of feed-forward receptive fields. Finally, we show how recurrent cortical interactions can generate cross-orientation suppression.     

Parsimony,Exhaustivity and Balanced Detection in Neocortex

The layout of sensory brain areas is thought to subtend perception. The principles shaping these architectures and their role in information processing are still poorly understood. We investigate mathematically and computationally the representation of orientation and spatial frequency in cat primary visual cortex. We prove that two natural principles, local exhaustivity and parsimony of representation, would constrain the orientation and spatial frequency maps to display a very specific pinwheel-dipole singularity. This is particularly interesting since recent experimental evidences show a dipolar structures of the spatial frequency map co-localized with pinwheels in cat. These structures have important properties on information processing capabilities. In particular, we show using a computational model of visual information processing that this architecture allows a trade-off in the local detection of orientation and spatial frequency, but this property occurs for spatial frequency selectivity sharper than reported in the literature. We validated this sharpening on high-resolution optical imaging experimental data. These results shed new light on the principles at play in the emergence of functional architecture of cortical maps, as well as their potential role in processing information.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

A Model of Temporal Encoding of Stimulus Orientation by Neuronal Responses in the Primary Visual Cortex

It has been shown experimentally that the stimulus orientation that elicits the optimal response in an orientation column in the primary visual cortex (area V1) undergoes rapid systemic changes that last 10–100 ms. These changes allow different orientation columns to encode information from multiple items in the visual space (the so-called temporal encoding). However, the mechanism of these changes is still unknown. In addition, most of the modern biophysical models are unable to reproduce these changes; the peak orientation of their responses is constant over time. In this paper, we suggest a method to improve the firing-rate ring model of the orientation hypercolumn by replacing the spatial symmetric distribution of local connections with a spatial anti-symmetric distribution. As a result, we obtained a more perfect model that is capable of reproducing such changes. Moreover, their amplitude is proportional to the extent of asymmetry in the spatial distribution of local connections.     

Classification of Different Tympanic Membrane Conditions Using Fused Deep Hypercolumn Features and Bidirectional LSTM

Objectives: Middle ear inflammatory diseases are global health problem that can have serious consequences such as hearing loss and speech disorders. The high cost of medical devices such as oto-endoscope and oto-microscope used by the specialists for the diagnosis of the disease prevents its widespread use. In addition, the decisions of otolaryngologists may differ due to the subjective visual examinations. For this reason, computer-aided middle ear disease diagnosis systems are needed to eliminate subjective diagnosis and high cost problems. To this aim, a hybrid deep learning approach was proposed for automatic recognition of different tympanic membrane conditions such as earwax plug, myringosclerosis, chronic otitis media and normal from the otoscopy images.Materials and methods: In this study we used public Ear Imagery dataset containing 880 otoscopy images. The proposed approach detects keypoints from the otoscopy images and following the obtained keypoint positions, extracts hypercolumn deep features from 5 different layers of the VGG 16 model. Classification of tympanic membrane conditions were realized by feeding the deep hypercolumn features to Bi-LSTM network in the form of non-time related data.Results: The performance of the proposed model was evaluated in three different color spaces as Red-Green-Blue (RGB), Hue-Saturation-Value (HSV) and Haematoxylin-Eosin-Diaminobenzidine (HED). The proposed model achieved acceptable results in all color spaces, moreover it showed a very successful performance in classifying tympanic membrane conditions especially in RGB space. Experimental studies showed that the proposed model achieved Acc of 99.06%, Sen of 98.13% and Spe of 99.38%.Conclusion: As a result, a robust model with high sensitivity was obtained for classification of tympanic membrane conditions and it was shown that Bi-LSTM network, which is generally used with time-related data, could also be used successfully with non-time related data for diagnosis of tympanic membrane conditions.     

Scene content selected by active vision

The primate visual system actively selects visual information from the environment for detailed processing through mechanisms of visual attention and saccadic eye movements. This study examines the statistical properties of the scene content selected by active vision. Eye movements were recorded while participants free-viewed digitized images of natural and artificial scenes. Fixation locations were determined for each image and image patches were extracted around the observed fixation locations. Measures of local contrast, local spatial correlation and spatial frequency content were calculated on the extracted image patches. Replicating previous results, local contrast was found to be greater at the points of fixation when compared to either the contrast for image patches extracted at random locations or at the observed fixation locations using an image-shuffled database. Contrary to some results and in agreement with other results in the literature, a significant decorrelation of image intensity is observed between the locations of fixation and other neighboring locations. A discussion and analysis of methodological techniques is given that provides an explanation for the discrepancy in results. The results of our analyses indicate that both the local contrast and correlation at the points of fixation are a function of image type and, furthermore, that the magnitude of these effects depend on the levels of contrast and correlation present overall in the images. Finally, the largest effect sizes in local contrast and correlation are found at distances of approximately 1 deg of visual angle, which agrees well with measures of optimal spatial scale selectivity in the visual periphery where visual information for potential saccade targets is processed.     

Spatial structure of complex cell receptive fields measured with natural images

Neuronal receptive fields (RFs) play crucial roles in visual processing. While the linear RFs of early neurons have been well studied, RFs of cortical complex cells are nonlinear and therefore difficult to characterize, especially in the context of natural stimuli. In this study, we used a nonlinear technique to compute the RFs of complex cells from their responses to natural images. We found that each RF is well described by a small number of subunits, which are oriented, localized, and bandpass. These subunits contribute to neuronal responses in a contrast-dependent, polarity-invariant manner, and they can largely predict the orientation and spatial frequency tuning of the cell. Although the RF structures measured with natural images were similar to those measured with random stimuli, natural images were more effective for driving complex cells, thus facilitating rapid identification of the subunits. The subunit RF model provides a useful basis for understanding cortical processing of natural stimuli.     

Motion perception and motion estimation by total-least squares

A computational model of motion perception is proposed. The model, which is gradient-based, adheres to the neural constraint that transmitted signals are positive-valued functions by posing the estimation of image motion as a quadratic programming problem combined with total-least squares: a model that assumes that image signals are contaminated by noise in both the spatial and temporal dimensions. By shrinking motion estimates with a regularizer whose subtractive effect introduces a contrast dependent speed threshold into motion computations, it is shown that the total-least squares model when posed as a quadratic programming problem, is capable of explaining both increases and decreases in perceived speed as these effects were reported by Thompson (1982) to vary as a function of image contrast and temporal frequency. The correlation that exists between the model's contrast speed response and results reported from visual psychophysics is consistent with the view that the visual system assumes that image signals may be contaminated by noise in both the spatial and the temporal domain, and that visual motion is influenced by the consequence of these assumptions.     

Modulation of activity in human visual area V1 during memory masking

Neurons in the primary visual cortex, V1, are specialized for the processing of elemental features of the visual stimulus, such as orientation and spatial frequency. Recent fMRI evidence suggest that V1 neurons are also recruited in visual perceptual memory; a number of studies using multi-voxel pattern analysis have successfully decoded stimulus-specific information from V1 activity patterns during the delay phase in memory tasks. However, consistent fMRI signal modulations reflecting the memory process have not yet been demonstrated. Here, we report evidence, from three subjects, that the low V1 BOLD activity during retention of low-level visual features is caused by competing interactions between neural populations coding for different values along the spectrum of the dimension remembered. We applied a memory masking paradigm in which the memory representation of a masker stimulus interferes with a delayed spatial frequency discrimination task when its frequency differs from the discriminanda with ±1 octave and found that impaired behavioral performance due to masking is reflected in weaker V1 BOLD signals. This cross-channel inhibition in V1 only occurs with retinotopic overlap between the masker and the sample stimulus of the discrimination task. The results suggest that memory for spatial frequency is a local process in the retinotopically organized visual cortex.     

Stereo disparity computation using Gabor filters

A solution to the correspondence problem for stereopsis is proposed using the differences in the complex phase of local spatial frequency components. One-dimensional spatial Gabor filters (Gabor 1946; Marcelja 1980), at different positions and spatial frequencies are convolved with each member of a stereo pair. The difference between the complex phase at corresponding points in the two images is used to find the stereo disparity. Disparity values are combined across spatial frequencies for each image location. Three-dimensional depth maps have been computed from real images under standard lighting conditions, as well as from random-dot stereograms (Julesz 1971). The algorithm can discriminate disparities significantly smaller than the width of a pixel. It is possible that a similar mechanism might be used in the human visual system.     

A theory for the use of visual orientation information which exploits the columnar structure of striate cortex

A neural model is constructed based on the structure of a visual orientation hypercolumn in mammalian striate cortex. It is then assumed that the perceived orientation of visual contours is determined by the pattern of neuronal activity across orientation columns. Using statistical estimation theory, limits on the precision of orientation estimation and discrimination are calculated. These limits are functions of single unit response properties such as orientation tuning width, response amplitude and response variability, as well as the degree of organization in the neural network. It is shown that a network of modest size, consisting of broadly orientation selective units, can reliably discriminate orientation with a precision equivalent to human performance. Of the various network parameters, the discrimination threshold depends most critically on the number of cells in the hypercolumn. The form of the dependence on cell number correctly predicts the results of psychophysical studies of orientation discrimination. The model system's performance is also consistent with psychophysical data in two situations in which human performance is not optimal. First, interference with orientation discrimination occurs when multiple stimuli activate cells in the same hypercolumn. Second, systematic errors in the estimation of orientation can occur when a stimulus is composed of intersecting lines. The results demonstrate that it is possible to relate neural activity to visual performance by an examination of the pattern of activity across orientation columns. This provides support for the hypothesis that perceived orientation is determined by the distributed pattern of neural activity. The results also encourage the view of neural activity. The results also are determined by the responses of many neurons rather than the sensitivity of individual cells.     

Cortical sensitivity to visual features in natural scenes

A central hypothesis concerning sensory processing is that the neuronal circuits are specifically adapted to represent natural stimuli efficiently. Here we show a novel effect in cortical coding of natural images. Using spike-triggered average or spike-triggered covariance analyses, we first identified the visual features selectively represented by each cortical neuron from its responses to natural images. We then measured the neuronal sensitivity to these features when they were present in either natural images or random stimuli. We found that in the responses of complex cells, but not of simple cells, the sensitivity was markedly higher for natural images than for random stimuli. Such elevated sensitivity leads to increased detectability of the visual features and thus an improved cortical representation of natural scenes. Interestingly, this effect is due not to the spatial power spectra of natural images, but to their phase regularities. These results point to a distinct visual-coding strategy that is mediated by contextual modulation of cortical responses tuned to the spatial-phase structure of natural scenes.     

Considering the Spatial Layout Information of Bag of Features (BoF) Framework for Image Classification

The spatial pooling method such as spatial pyramid matching (SPM) is very crucial in the bag of features model used in image classification. SPM partitions the image into a set of regular grids and assumes that the spatial layout of all visual words obey the uniform distribution over these regular grids. However, in practice, we consider that different visual words should obey different spatial layout distributions. To improve SPM, we develop a novel spatial pooling method, namely spatial distribution pooling (SDP). The proposed SDP method uses an extension model of Gauss mixture model to estimate the spatial layout distributions of the visual vocabulary. For each visual word type, SDP can generate a set of flexible grids rather than the regular grids from the traditional SPM. Furthermore, we can compute the grid weights for visual word tokens according to their spatial coordinates. The experimental results demonstrate that SDP outperforms the traditional spatial pooling methods, and is competitive with the state-of-the-art classification accuracy on several challenging image datasets.     

An information integration model of the primary visual cortex under grating stimulations

During the course of information processing, a visual system extracts characteristic information of the visual image and integrates the spatial and temporal visual information simultaneously. In this study, we investigate the integration effect of neurons in the primary visual cortex (V1 area) under the grating stimulation. First, an information integration model was established based on the receptive field properties of the extracted features of the visual images features, the interaction between neurons and the nonlinear integration of those neurons. Then the neuropsychological experiments were designed both to provide parameters for the model and to verify its effect. The experimental results with factual visual image were largely consistent with the model’s forecast output. This demonstrates that our model can truly reflect the integration effect of the primary visual system when being subjected to grating stimulations with different orientations. Our results indicate the primary visual system integrates the visual information in the following manner: it first extracts visual information through different types of receptive field, and then its neurons interact with each other in a non-linear manner, finally the neurons fire spikes recorded as responses to the visual stimulus.     

What Image Properties Regulate Eye Growth?

The growth of the eye, unlike other parts of the body, is not ballistic. It is guided by visual feedback with the eventual aim being optimal focus of the retinal image or emmetropization . It has been shown in animal models that interference with the quality of the retinal image leads to a disruption to the normal growth pattern, resulting in the development of refractive errors and defocused retinal images . While it is clear that retinal images rich in pattern information are needed to control eye growth, it is unclear what particular aspect of image structure is relevant. Retinal images comprise a range of spatial frequencies at different absolute and relative contrasts and in different degrees of spatial alignment. Here we show, by using synthetic images, that it is not the local edge structure produced by relative spatial frequency alignments within an image but rather the spatial frequency composition per se that is used to regulate the growth of the eye. Furthermore, it is the absolute energy at high spatial frequencies regardless of the spectral slope that is most effective. Neither result would be expected from currently accepted ideas of how human observers judge the degree of image "blur" in a scene where both phase alignments and the relative energy distribution across spatial frequency (i.e., spectral slope) are important.     

视觉大细胞通路对汉字识别的影响

我们研究的目的是探查视觉大细胞系统在汉字识别中的作用.研究以24名大学生正常阅读者作为被试,通过调控汉字材料的空间频率和时间频率区分出大细胞通路敏感刺激和正常视觉刺激两种条件,并采用整体/部件字形判断任务来比较在不同视觉条件下,被试加工汉字整体和部件信息的过程,记录反应时和错误率.结果表明,在大细胞条件下,被试对整字信息的判断要显著快于对部件的判断,即出现整体优先效应.而在正常视觉条件下,整体和部件判断的反应时无显著差异.同时,在大细胞条件下,被试进行部件判断的错误率显著高于整体判断,在正常视觉条件下两者差异不显著.研究结果表明视觉大细胞通路促进了汉字整体信息的加工.     

The coordinated mapping of visual space and response features in visual cortex

Whether general principles can explain the layouts of cortical maps remains unresolved. In primary visual cortex of ferret, the relationships between the maps of visual space and response features are predicted by a "dimension-reduction" model. The representation of visual space is anisotropic, with the elevation and azimuth axes having different magnification. This anisotropy is reflected in the orientation, ocular dominance, and spatial frequency domains, which are elongated such that their directions of rapid change, or high-gradient axes, are orthogonal to the high-gradient axis of the visual map. The feature maps are also strongly interdependent-their high-gradient regions avoid one another and intersect orthogonally where essential, so that overlap is minimized. Our results demonstrate a clear influence of the visual map on each feature map. In turn, the local representation of visual space is smooth, as predicted when many features are mapped within a cortical area.     

Orientation hypercolumns of the visual cortex: Ring model

A hypercolumn of the visual cortex is a functional unit formed of neighboring columns whose neurons respond to a stimulus of particular orientation. The function of the hypercolumn is to amplify the orientation tuning of visually evoked responses. According to the conventional simple model of a hypercolumn, neuronal populations with different orientation preferences are distributed on a ring. Every population is described by a firing rate (FR) model. To determine the limitations of the FR-ring model, it was compared with a more detailed ring model, which takes into account the distribution of neurons of each population according to their voltage values. In the case of leaky integrate-and-fire neurons, every neuronal population is described by the Fokker-Planck equation (FPE). The mapping of parameters was obtained. The simulations revealed differences in the behavior of the two models. The FPE-based model reacts faster to a change in stimulus orientation. The FPE ring model gives a steady-state solution in the form of waves of activity traveling on the ring, whereas the FR ring model presents amplitude instability for the same parameter set. The FPE ring model reproduces the characteristic effects of the FR ring model: virtual rotation and symmetry breaking.