Implementation of the template model of vision

Adopting principles learnt from insect vision we have constructed model of a general-purpose front-end visual system for motion detection that is designed to operate in parallel along each photoreceptor axis with only local connections. The model is also designed to assist electrophysiological analysis of visual processing because it puts the response to a moving scene into sets of template responses similar to the distribution of activity among different neurons. An earlier template model divided the visual image into the fields of adjacent receptors, measured as intensity or receptor modulation at small increments of time. As soon as we used this model with natural scenes, however, we found that we had to look at changes in intensity, not intensity itself. Running the new model also generated new insights into the effects of very fast motion, of blurring the image, and the value of lateral inhibition. We also experimented with ways of measuring the angular velocity of the image moving across the eye. The camera eye is moved at a known speed and the range to objects is calculated from the angular velocity of contrasts moving across the receptor array. The original template model is modified so that contrast is saturated in a new representation of the original image data. This reduces the 8-bit grey-scale image to a log, 3 = 1.6-bit image, which becomes the input to a look-up table of templates. The output consists of groups of responding templates in specific ratios that define the input features, and these ratios lead into types of invariance at a higher level of further logic. At any stage, there can be persistent parallel inputs from all earlier stages. This design would enable groups of templates to be tuned to different expected situations, such as different velocities, different directions and different types of edges.     

A template theory to relate visual processing to digital circuitry

Simple stimulus patterns, in this case visual, are represented by spatiotemporal Boolean functions that can be summarized in a 4 x 4 look-up table of 16 templates behind each sensory neuron. These groups of templates correspond to groups of neurons in columns behind each receptor. They abstract specific combinations of input in simple combinations and include two successive states in time. A template is like a neuron field at threshold, and responds as the field is convolved with the stimulus pattern. The same structure can be repeated in successive layers to make progressive categorization and to reject inappropriate combinations. At any level, the templates act in groups, so providing a very large number of combinations that can represent more complex stimulus patterns at deeper levels.     

Rapid dynamics of contrast responses in the cat primary visual cortex

The visual information we receive during natural vision changes rapidly and continuously. The visual system must adapt to the spatiotemporal contents of the environment in order to efficiently process the dynamic signals. However, neuronal responses to luminance contrast are usually measured using drifting or stationary gratings presented for a prolonged duration. Since motion in our visual field is continuous, the signals received by the visual system contain an abundance of transient components in the contrast domain. Here using a modified reverse correlation method, we studied the properties of responses of neurons in the cat primary visual cortex to different contrasts of grating stimuli presented statically and transiently for 40 ms, and showed that neurons can effectively discriminate the rapidly changing contrasts. The change in the contrast response function (CRF) over time mainly consisted of an increment in contrast gain (CRF shifts to left) in the developing phase of temporal responses and a decrement in response gain (CRF shifts downward) in the decay phase. When the distribution range of stimulus contrasts was increased, neurons demonstrated decrement in contrast gain and response gain. Our results suggest that contrast gain control (contrast adaptation) and response gain control mechanisms are well established during the first tens of milliseconds after stimulus onset and may cooperatively mediate the rapid dynamic responses of visual cortical neurons to the continuously changing contrast. This fast contrast adaptation may play a role in detecting contrast contours in the context of visual scenes that are varying rapidly.     

The spatial resolutions of the apposition compound eye and its neuro-sensory feature detectors: observation versus theory

For 100 years three ideas dominated efforts to understand the apposition compound eye. In Müller's theory, the eye viewed the panorama through an array of little windows without overlaps and without gaps, with no details within windows. Spatial resolution then depended on the interommatidial angle (Deltaphi) and the number of ommatidia. In the second proposal, the insect detected the temporal modulation of the light, which was limited by the aperture of the lens and the wavelength, assuming good focus. Modulation is the change of intensity in the receptor, usually caused by motion of a spatial contrast in the stimulus. Thirdly, motion was detected from the successive temporal modulations at adjacent visual axes. Recently, two more principles arose. The light-sensitive elements, called rhabdomeres, project through the nodal point of the lens to the outside world, and the resolution was limited by their grain size, like the pixels in a digital camera. Finally, detection of contrast and colour was limited by the signal/noise ratio (SNR) which was improved by brighter light and more visual pigment. These five physical principles provide satisfying explanations of eye function but they all originated from theory. Actual measurements of resolution depend on the operation of the test. The visual system of the honeybee recognizes a limited variety of simple cues, but there is no evidence that the pattern of ommatidial stimulation is re-assembled, or even seen. The known cues are: the temporal modulation of groups of receptors, the direction and angular velocity of motion, some measure of the spatial disruption of the pattern or the length of edge (related to spatial frequency and contrast), colour, the intensity, the position of the centre and the size of large well-separated areas of black or colour, the angle of orientation of a bar or grating, radial or tangential edges, and bilateral symmetry. Neurons connected to more than two adjacent ommatidia collaborate in the detection of cues, and the resolution depends on the neuro-sensory feature detectors at work at the time. Although some behavioural and electrophysiological measurements give a spatial resolution similar to the interommatidial angle, different spatial properties of neuro-sensory detectors predominate at different light intensities and with a diurnal rhythm. During the long history of this topic, the belief that the resolution ought to be Deltaphi has frequently been overturned by experimental measurement.     

A linear model for symmetric receptive fields: implications for classification tests with flashed and moving images

The purpose of this study was to explore the effects of spatial and temporal properties on the expected responses of visual neurons that have linear receptive fields (RFs), particularly those having a mirror symmetric distribution of spatial subregions. Receptive fields that are symmetric in at least one spatial dimension occur in neurons of the retina, the lateral geniculate nucleus (LGN), and the visual cortex of mammals. Responses to flashing bars, moving bars, and moving edges were studied for different configurations of an analog RF model in which spatial and temporal aspects were varied independently. Responses of the model at intermediate stimulus speeds were found to agree with responses in the literature for X and Y units of the LGN and often for simple units of the visual cortex. In particular, having separated regions of response to light and dark edges, an identifying property of simple cells, was found to be a linear consequence of RF regions responding inversely to stimuli of opposite polarity. Model differences from responses of cortical complex units show that a linear model cannot mimic their responses, and imply that complex units employ major nonlinearities in coding image polarity (light vs dark), which signifies a nonlinearity in coding intensity. Because sudden flux changes inherent in flashing bars test mainly temporal RF properties, and slowly moving edges test mainly spatial properties, these two tests form a useful minimal set with which to describe and classify RFs. The usefulness of this set derives both from its sensitivity to spatial and temporal variables, and from the correlation between the linearity of a cell's processing of stimulus intensity and its RF classification.     

综述: 初级视皮层的亮度信息加工

亮度(luminance)是最基本的视觉信息.与其他视觉特征相比,由于视神经元对亮度刺激的反应较弱,并且许多神经元对均匀亮度无反应,对亮度信息编码的神经机制知之甚少.初级视皮层部分神经元对亮度的反应要慢于对比度反应,被认为是由边界对比度诱导的亮度知觉(brightness)的神经基础.我们的研究表明,初级视皮层许多神经元的亮度反应要快于对比度反应,并且这些神经元偏好低的空间频率、高的时间频率和高的运动速度,提示皮层下具有低空间频率和高运动速度通路的信息输入对产生初级视皮层神经元的亮度反应有贡献.已经知道初级视皮层神经元对空间频率反应的时间过程是从低空间频率到高空间频率,我们发现的早期亮度反应是对极低空间频率的反应,与这一时间过程是一致的,是这一从粗到细的视觉信息加工过程的第一步,揭示了处理最早的粗的视觉信息的神经基础.另外,初级视皮层含有偏好亮度下降和高运动速度的神经元,这群神经元的活动有助于在光照差的环境中检测高速运动的低亮度物体.     

Seeing objects in motion

This paper reports estimates of the conjoint spatiotemporal tuning functions of the neural mechanisms of the human vision system which detect image motion. The functions were derived from measurements of the minimum contrast necessary to detect the direction of drift of a sinusoidal grating, in the presence of phase-reversed masking gratings of various spatial and temporal frequencies. A mask of similar spatial and temporal frequencies to the test grating reduces sensitivity considerably, whereas one differing greatly in spatial or temporal frequency has little or no effect. The results show that for test gratings drifting at 8 Hz, the tuning function is bandpass in both space and time, peaked at the temporal and spatial frequency (SF) of the test (SFs were 0.1, 1 or 5 c deg-1; c represents cycles throughout). For a grating of 5 c deg-1 drifting at 0.3 Hz, the function is bandpass in space but lowpass in time. Fourier transform of the frequency results yields a function in space-time which we term the 'spatiotemporal receptive field'. For movement detectors (bandpass in space and time) the fields comprise alternating ridges of opposing polarity, elongated in space-time along the preferred velocity axis of the detector. We suggest that this organization explains how detectors analyse form and motion concurrently and accounts, at least in part, for a variety of perceptual phenomena, including summation, reduction of motion smear, metacontrast, stroboscopic motion and spatiotemporal interpolation.     

Sex & vision I: Spatio-temporal resolution

ABSTRACT: BACKGROUND: Cerebral cortex has a very large number of testosterone receptors, which could be a basis for sex differences in sensory functions. For example, audition has clear sex differences, which are related to serum testosterone levels. Of all major sensory systems only vision has not been examined for sex differences, which is surprising because occipital lobe (primary visual projection area) may have the highest density of testosterone receptors in the cortex. We have examined a basic visual function: spatial and temporal pattern resolution and acuity. METHODS: We tested large groups of young adults with normal vision. They were screened with a battery of standard tests that examined acuity, color vision, and stereopsis. We sampled the visual system's contrast-sensitivity function (CSF) across the entire spatio-temporal space: 6 spatial frequencies at each of 5 temporal rates. Stimuli were gratings with sinusoidal luminance profiles generated on a special-purpose computer screen; their contrast was also sinusoidally modulated in time. We measured threshold contrasts using a criterion-free (forced-choice), adaptive psychophysical method (QUEST algorithm). Also, each individual's acuity limit was estimated by fitting his or her data with a model and extrapolating to find the spatial frequency corresponding to 100 % contrast. RESULTS: At a very low temporal rate, the spatial CSF was the canonical inverted-U; but for higher temporal rates, the maxima of the spatial CSFs shifted: Observers lost sensitivity at high spatial frequencies and gained sensitivity at low frequencies; also, all the maxima of the CSFs shifted by about the same amount in spatial frequency. Main effect: there was a significant (ANOVA) sex difference. Across the entire spatio-temporal domain, males were more sensitive, especially at higher spatial frequencies; similarly males had significantly better acuity at all temporal rates. CONCLUSION: As with other sensory systems, there are marked sex differences in vision. The CSFs we measure are largely determined by inputs from specific sets of thalamic neurons to individual neurons in primary visual cortex. This convergence from thalamus to cortex is guided by cortex during embryogenesis. We suggest that testosterone plays a major role, leading to different connectivities in males and in females. But, for whatever reasons, we find that males have significantly greater sensitivity for fine detail and for rapidly moving stimuli. One interpretation is that this is consistent with sex roles in hunter-gatherer societies.     

Computational modelling of motion detectors: responses to two-frame displays

Schemes for motion detection fall into two classes. Reichardt correlators compare spatial luminance patterns at two locations at different times; gradient detectors compare spatial and temporal luminance gradients. Both are candidate operators for biological and machine vision systems. A large body of perceptual data exists, defining the properties of motion detectors used by human observers, which can form a basis for determining which class of detector is appropriate for the human visual system. Plausible versions of each detector were implemented, and their responses to a variety of two-frame stimuli were computed. Results indicated that both detectors can predict most of the data, but on balance gradient detectors offer the best working hypothesis for motion detection by human observers. This conclusion is necessarily limited to the type of stimuli used, and may require modification in the light of responses to continuously moving stimuli.     

Visual discriminations of spokes, sectors, and circles by the honeybee (Apis mellifera)

A new cue for visual discrimination by the honeybee has been demonstrated. Bees detected the position of the centre of symmetry of radial patterns of spokes, sectors, and circles relative to their point of choice in the learning process, irrespective of the pattern. When trained with one of these patterns versus a blank target, the bees discriminated a shift in the position of the centre of symmetry by as little as 5 degrees , in some cases with unfamiliar test patterns. A pattern of spokes or rings also stabilized the vision of the bees in the horizontal plane so that the position of a plain black area could then be discriminated. In other experiments, bees discriminated half of a pattern of radial spokes or concentric circles from the other half, cut either vertically or horizontally, and irrespective of scale. Therefore these patterns were not detected by preformed combinations of orientation detectors or global templates with a single output. Instead, the crucial cue for detecting edges as radial or circular was the coincidence of responses of numerous local edge detectors having the appropriate convergence to a hub. Edges that converged towards a hub were detected by the bees as radial, and edges at right angles to these were parts of circles, irrespective of the actual pattern. Breaking the patterns of spokes or circles into rows of squares spoiled the discrimination if the squares were separately resolved. Alternatively, breaking the pattern into short bars that were separately resolved spoiled the discrimination when the bars subtended less than 3 degrees . The local feature detectors for spokes and circles therefore resembled those of the orientation detectors in being short, independent, and unable to span gaps of more than 3 degrees . In conclusion, radial and circular patterns were identified by the regional coincidences and convergence of local detectors of edge orientation, and the positions of the centres of symmetry were remembered as landmarks that helped locate the reward, but the patterns themselves were not remembered.     

Dynamic Electrode-to-Image (DETI) mapping reveals the human brain’s spatiotemporal code of visual information

A number of neuroimaging techniques have been employed to understand how visual information is transformed along the visual pathway. Although each technique has spatial and temporal limitations, they can each provide important insights into the visual code. While the BOLD signal of fMRI can be quite informative, the visual code is not static and this can be obscured by fMRI’s poor temporal resolution. In this study, we leveraged the high temporal resolution of EEG to develop an encoding technique based on the distribution of responses generated by a population of real-world scenes. This approach maps neural signals to each pixel within a given image and reveals location-specific transformations of the visual code, providing a spatiotemporal signature for the image at each electrode. Our analyses of the mapping results revealed that scenes undergo a series of nonuniform transformations that prioritize different spatial frequencies at different regions of scenes over time. This mapping technique offers a potential avenue for future studies to explore how dynamic feedforward and recurrent processes inform and refine high-level representations of our visual world.     

Spectral Characteristics of Phase Sensitivity and Discharge Rate of Neurons in the Ascending Tectofugal Visual System

Drifting gratings can modulate the activity of visual neurons at the temporal frequency of the stimulus. In order to characterize the temporal frequency modulation in the cat’s ascending tectofugal visual system, we recorded the activity of single neurons in the superior colliculus, the suprageniculate nucleus, and the anterior ectosylvian cortex during visual stimulation with drifting sine-wave gratings. In response to such stimuli, neurons in each structure showed an increase in firing rate and/or oscillatory modulated firing at the temporal frequency of the stimulus (phase sensitivity). To obtain a more complete characterization of the neural responses in spatiotemporal frequency domain, we analyzed the mean firing rate and the strength of the oscillatory modulations measured by the standardized Fourier component of the response at the temporal frequency of the stimulus. We show that the spatiotemporal stimulus parameters that elicit maximal oscillations often differ from those that elicit a maximal discharge rate. Furthermore, the temporal modulation and discharge-rate spectral receptive fields often do not overlap, suggesting that the detection range for visual stimuli provided jointly by modulated and unmodulated response components is larger than the range provided by a one response component.     

Retino-cortical information transmission achievable with a retina implant

Blind subjects with photoreceptor degeneration perceive phosphenes when their intact retinal ganglion cells are stimulated electrically. Is this approach suitable for transmitting enough information to the visual cortex for partially restoring vision? We stimulated the retina of anesthetized cats electrically and visually while recording the responses in the visual cortex. Transmission of retino-cortical information T was quantified by information theory. T was 20-160 bit/s (per stimulation and recording site) with random electrical or visual impulse stimulation at rates between 20 and 40 s-1. While increasing spatial density of independent electrical stimulation channels T did not saturate with 7 electrodes/mm2 retina. With seven electrodes up to 500 bit/s was transmitted to 15 cortical recording sites. Electrical stimulation basically employs temporal stimulus patterns. They are intimately linked with intensity/contrast information coded by the spike density of retinal ganglion cells. From the cortical information spread we estimated the spatial resolution as 0.5mm cortex corresponding to 0.5-1.0 degrees visual angle. If the human cortex can receive and decode the information transmitted by a retina implant, our quantitative results measured in cats suggest that visuo-motor coordination and object recognition in many in- and out-door situations will be possible.     

Flexibility of spatial averaging in visual perception

The classical receptive field (RF) concept-the idea that a visual neuron responds to fixed parts and properties of a stimulus-has been challenged by a series of recent physiological results. Here, we extend these findings to human vision, demonstrating that the extent of spatial averaging in contrast perception is also flexible, depending strongly on stimulus contrast and uniformity. At low contrast, spatial averaging is greatest (about 11 min of arc) within uniform regions such as edges, as expected if the relevant neurons have orientation-selective RFs. At high contrast, spatial averaging is minimal. These results can be understood if the visual system is balancing a trade-off between noise reduction, which favours large areas of averaging, and detail preservation, which favours minimal averaging. Two distinct populations of neurons with hard-wired RFs could account for our results, as could the more intriguing possibility of dynamic, contrast-dependent RFs.     

Nasal chemosensory-stimulation evoked activity patterns in the rat trigeminal ganglion visualized by in vivo voltage-sensitive dye imaging

Mammalian nasal chemosensation is predominantly mediated by two independent neuronal pathways, the olfactory and the trigeminal system. Within the early olfactory system, spatiotemporal responses of the olfactory bulb to various odorants have been mapped in great detail. In contrast, far less is known about the representation of volatile chemical stimuli at an early stage in the trigeminal system, the trigeminal ganglion (TG), which contains neurons directly projecting to the nasal cavity. We have established an in vivo preparation that allows high-resolution imaging of neuronal population activity from a large region of the rat TG using voltage-sensitive dyes (VSDs). Application of different chemical stimuli to the nasal cavity elicited distinct, stimulus-category specific, spatiotemporal activation patterns that comprised activated as well as suppressed areas. Thus, our results provide the first direct insights into the spatial representation of nasal chemosensory information within the trigeminal ganglion imaged at high temporal resolution.     

Timing Precision in Population Coding of Natural Scenes in the Early Visual System

The timing of spiking activity across neurons is a fundamental aspect of the neural population code. Individual neurons in the retina, thalamus, and cortex can have very precise and repeatable responses but exhibit degraded temporal precision in response to suboptimal stimuli. To investigate the functional implications for neural populations in natural conditions, we recorded in vivo the simultaneous responses, to movies of natural scenes, of multiple thalamic neurons likely converging to a common neuronal target in primary visual cortex. We show that the response of individual neurons is less precise at lower contrast, but that spike timing precision across neurons is relatively insensitive to global changes in visual contrast. Overall, spike timing precision within and across cells is on the order of 10 ms. Since closely timed spikes are more efficient in inducing a spike in downstream cortical neurons, and since fine temporal precision is necessary to represent the more slowly varying natural environment, we argue that preserving relative spike timing at a ~10-ms resolution is a crucial property of the neural code entering cortex.     

Profound contrast adaptation early in the visual pathway

Prior exposure to a moving grating of high contrast led to a substantial and persistent reduction in the contrast sensitivity of neurons in the lateral geniculate nucleus (LGN) of macaque. This slow contrast adaptation was potent in all magnocellular (M) cells but essentially absent in parvocellular (P) cells and neurons that received input from S cones. Simultaneous recordings of M cells and the potentials of ganglion cells driving them showed that adaptation originated in ganglion cells. As expected from the spatiotemporal tuning of M cells, adaptation was broadly tuned for spatial frequency and lacked orientation selectivity. Adaptation could be induced by high temporal frequencies to which cortical neurons do not respond, but not by low temporal frequencies that can strongly adapt cortical neurons. Our observations confirm that contrast adaptation occurs at multiple levels in the visual system, and they provide a new way to reveal the function and perceptual significance of the M pathway.     

Functional specialization of seven mouse visual cortical areas

To establish the mouse as a genetically tractable model for high-order visual processing, we characterized fine-scale retinotopic organization of visual cortex and determined functional specialization of layer 2/3 neuronal populations in seven retinotopically identified areas. Each area contains a distinct visuotopic representation and encodes a unique combination of spatiotemporal features. Areas LM, AL, RL, and AM prefer up to three times faster temporal frequencies and significantly lower spatial frequencies than V1, while V1 and PM prefer high spatial and low temporal frequencies. LI prefers both high spatial and temporal frequencies. All extrastriate areas except LI increase orientation selectivity compared to V1, and three areas are significantly more direction selective (AL, RL, and AM). Specific combinations of spatiotemporal representations further distinguish areas. These results reveal that mouse higher visual areas are functionally distinct, and separate groups of areas may be specialized for motion-related versus pattern-related computations, perhaps forming pathways analogous to dorsal and ventral streams in other species.     

Relationship between Size Summation Properties,Contrast Sensitivity and Response Latency in the Dorsomedial and Middle Temporal Areas of the Primate Extrastriate Cortex

Analysis of the physiological properties of single neurons in visual cortex has demonstrated that both the extent of their receptive fields and the latency of their responses depend on stimulus contrast. Here, we explore the question of whether there are also systematic relationships between these response properties across different cells in a neuronal population. Single unit recordings were obtained from the middle temporal (MT) and dorsomedial (DM) extrastriate areas of anaesthetized marmoset monkeys. For each cell, spatial integration properties (length and width summation, as well as the presence of end- and side-inhibition within 15° of the receptive field centre) were determined using gratings of optimal direction of motion and spatial and temporal frequencies, at 60% contrast. Following this, contrast sensitivity was assessed using gratings of near-optimal length and width. In both areas, we found a relationship between spatial integration and contrast sensitivity properties: cells that summated over smaller areas of the visual field, and cells that displayed response inhibition at larger stimulus sizes, tended to show higher contrast sensitivity. In a sample of MT neurons, we found that cells showing longer latency responses also tended to summate over larger expanses of visual space in comparison with neurons that had shorter latencies. In addition, longer-latency neurons also tended to show less obvious surround inhibition. Interestingly, all of these effects were stronger and more consistent with respect to the selectivity for stimulus width and strength of side-inhibition than for length selectivity and end-inhibition. The results are partially consistent with a hierarchical model whereby more extensive receptive fields require convergence of information from larger pools of “feedforward” afferent neurons to reach near-optimal responses. They also suggest that a common gain normalization mechanism within MT and DM is involved, the spatial extent of which is more evident along the cell’s preferred axis of motion.     

Visual pattern discrimination by population retinal ganglion cells’ activities during natural movie stimulation

In the visual system, neurons often fire in synchrony, and it is believed that synchronous activities of group neurons are more efficient than single cell response in transmitting neural signals to down-stream neurons. However, whether dynamic natural stimuli are encoded by dynamic spatiotemporal firing patterns of synchronous group neurons still needs to be investigated. In this paper we recorded the activities of population ganglion cells in bullfrog retina in response to time-varying natural images (natural scene movie) using multi-electrode arrays. In response to some different brief section pairs of the movie, synchronous groups of retinal ganglion cells (RGCs) fired with similar but different spike events. We attempted to discriminate the movie sections based on temporal firing patterns of single cells and spatiotemporal firing patterns of the synchronous groups of RGCs characterized by a measurement of subsequence distribution discrepancy. The discrimination performance was assessed by a classification method based on Support Vector Machines. Our results show that different movie sections of the natural movie elicited reliable dynamic spatiotemporal activity patterns of the synchronous RGCs, which are more efficient in discriminating different movie sections than the temporal patterns of the single cells’ spike events. These results suggest that, during natural vision, the down-stream neurons may decode the visual information from the dynamic spatiotemporal patterns of the synchronous group of RGCs’ activities.