A computational theory of visual receptive fields

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.     

A computational theory of visual attention.

A computational theory of visual attention is presented. The basic theory (TVA) combines the biased-choice model for single-stimulus recognition with the fixed-capacity independent race model (FIRM) for selection from multi-element displays. TVA organizes a large body of experimental findings on performance in visual recognition and attention tasks. A recent development (CTVA) combines TVA with a theory of perceptual grouping by proximity. CTVA explains effects of perceptual grouping and spatial distance between items in multi-element displays. A new account of spatial focusing is proposed in this paper. The account provides a framework for understanding visual search as an interplay between serial and parallel processes.     

A computational model of visual anisotropy

Visual anisotropy has been demonstrated in multiple tasks where performance differs between vertical, horizontal, and oblique orientations of the stimuli. We explain some principles of visual anisotropy by anisotropic smoothing, which is based on a variation on Koenderink's approach in [1]. We tested the theory by presenting gaussian elongated luminance profiles and measuring the perceived orientations by means of an adjustment task. Our framework is based on the smoothing of the image with elliptical gaussian kernels and it correctly predicted an illusory orientation bias towards the vertical axis. We discuss the scope of the theory in the context of other anisotropies in perception.     

A neuron-inspired computational architecture for spatiotemporal visual processing

In this article, we present a neurologically motivated computational architecture for visual information processing. The computational architecture’s focus lies in multiple strategies: hierarchical processing, parallel and concurrent processing, and modularity. The architecture is modular and expandable in both hardware and software, so that it can also cope with multisensory integrations – making it an ideal tool for validating and applying computational neuroscience models in real time under real-world conditions. We apply our architecture in real time to validate a long-standing biologically inspired visual object recognition model, HMAX. In this context, the overall aim is to supply a humanoid robot with the ability to perceive and understand its environment with a focus on the active aspect of real-time spatiotemporal visual processing. We show that our approach is capable of simulating information processing in the visual cortex in real time and that our entropy-adaptive modification of HMAX has a higher efficiency and classification performance than the standard model (up to \(\sim \!+6\,\% \) ).     

A computational theory of human stereo vision.

An algorithm is proposed for solving the stereoscopic matching problem. The algorithm consists of five steps: (1) Each image is filtered at different orientations with bar masks of four sizes that increase with eccentricity; the equivalent filters are one or two octaves wide. (2) Zero-crossings in the filtered images, which roughly correspond to edges, are localized. Positions of the ends of lines and edges are also found. (3) For each mask orientation and size, matching takes place between pairs of zero-crossings or terminationss of the same sign in the two images, for a range of disparities up to about the width of the mask's central region. (4) Wide masks can control vergence movements, thus causing small masks to come into correspondence. (5) When a correspondence is achieved, it is stored in a dynamic buffer, called the 2 1/2-D sketch. It is shown that this proposal provides a theoretical framework for most existing psychophysical and neurophysiological data about stereopsis. Several critical experimental predictions are also made, for instance about the size of Panum's area under various conditions. The results of such experiments would tell us whether, for example, cooperativity is necessary for the matching process.     

A computational model of the error detector of human visual accommodation

A mathematical model is proposed for the error detector of the human visual accommodative system. The model supposes that the accommodative error detector derives both the direction and the magnitude of the accommodative error from naturally-occuring oscillations of the lens and their effects on retinal-image contrast. Differential operators take the first derivatives of two time varying functions: lens power and retinal-image contrast. Directional information is obtained by comparing the signs of these two derivatives and magnitude information is obtained by comparing their amplitudes.Research conducted at the School of Optometry, University of California, BerkeleySupported by National Eye Institute grant EYO-3532-04(C.S.) and National Institutes of Health core grant # 1-445420-32011     

A computational model of the vertical anatomical organization of primary visual cortex

A method for modeling anatomical connectivity for a vertically organized slab of cortical tissue in mammalian primary visual cortex has been developed. The modeled slab covers 500 × 500 m of cortical surface and extends vertically throughout the full depth of the cortex. The model slab was divided into 6 laminae and neuronal somata were distributed in three dimensions through the slab in accordance with experimentally derived cell densities. Axonal and dendritic arborizations were modeled as line segments. A total of 17 morphological types of neurons were included. Connectivity was established based on proximity between axonal and dendritic arbors. There is good general agreement between the vertical distribution of connections generated by the model and the vertical distribution of synapses observed for cat area 17. In all layers, fewer connections were generated in the model than synapses in cat area 17. This is due, at least in part, to the exclusion of long range intracortical projections and sources of afferent input other than the dorsal lateral geniculate nucleus from the model. The connection scheme described here will be used in conjunction with a physiology model to model vertical signal flow, and will be expanded further to model receptive fields of cortical neurons.Supported in part by a grant from Cray Research Inc.     

A computational model of vertical signal propagation in the primary visual cortex

A computational model of the flow of activity in a vertically organized slab of cat primary visual cortex (area 17) has been developed. The membrane potential of each cell in the model, as a function of time, is given by the solution of a system of first order, coupled, non-linear differential equations. When firing threshold is exceeded, an action potential waveform is pasted in. The behavior of the model following a brief simulated stimulus to afferents from the dorsal lateral geniculate nucleus (dLGN) is explored. Excitatory and inhibitory post-synaptic potential (E and IPSP) latencies, as a function of cortical depth, were generated by the model. These data were compared with the experimental literature. In general, good agreement was found for EPSPs. Many disynaptic inhibitory inputs were found to be masked by the firing of action potentials in the model. To our knowledge this phenomenon has not been reported in the experimental literature. The model demonstrates that whether a cell exhibits disynaptic or polysynaptic PSP latencies is not a fixed consequence of anatomical connectivity, but rather, can be influenced by connection strengths, and may be influenced by the ongoing pattern of activity in the cortex.Supported by a grant from Cray Research Inc.     

A template theory to relate visual processing to digital circuitry

Simple stimulus patterns, in this case visual, are represented by spatiotemporal Boolean functions that can be summarized in a 4 x 4 look-up table of 16 templates behind each sensory neuron. These groups of templates correspond to groups of neurons in columns behind each receptor. They abstract specific combinations of input in simple combinations and include two successive states in time. A template is like a neuron field at threshold, and responds as the field is convolved with the stimulus pattern. The same structure can be repeated in successive layers to make progressive categorization and to reject inappropriate combinations. At any level, the templates act in groups, so providing a very large number of combinations that can represent more complex stimulus patterns at deeper levels.     

Intermittent control: a computational theory of human control

The paradigm of continuous control using internal models has advanced understanding of human motor control. However, this paradigm ignores some aspects of human control, including intermittent feedback, serial ballistic control, triggered responses and refractory periods. It is shown that event-driven intermittent control provides a framework to explain the behaviour of the human operator under a wider range of conditions than continuous control. Continuous control is included as a special case, but sampling, system matched hold, an intermittent predictor and an event trigger allow serial open-loop trajectories using intermittent feedback. The implementation here may be described as ??continuous observation, intermittent action??. Beyond explaining unimodal regulation distributions in common with continuous control, these features naturally explain refractoriness and bimodal stabilisation distributions observed in double stimulus tracking experiments and quiet standing, respectively. Moreover, given that human control systems contain significant time delays, a biological-cybernetic rationale favours intermittent over continuous control: intermittent predictive control is computationally less demanding than continuous predictive control. A standard continuous-time predictive control model of the human operator is used as the underlying design method for an event-driven intermittent controller. It is shown that when event thresholds are small and sampling is regular, the intermittent controller can masquerade as the underlying continuous-time controller and thus, under these conditions, the continuous-time and intermittent controller cannot be distinguished. This explains why the intermittent control hypothesis is consistent with the continuous control hypothesis for certain experimental conditions.     

Dynamic texture spreading: probing the mechanisms of surface interpolation

Dynamic texture spreading is a filling-in phenomenon where a colored pattern perceptually spreads onto an area confined by virtual contours in a multi-aperture motion display. The spreading effect is qualitatively similar to static texture spreading but widely surpasses it in strength, making it particularly suited for quantitative studies of visual interpolation processes. We first carried out two experiments to establish with objective tasks that texture spreading is a genuine representation of surface qualities and thus goes beyond mere contour interpolation. Two subsequent experiments serve to relate the phenomenon to ongoing discussions about potentially responsible mechanisms for spatiotemporal integration. With a phenomenological method, we examined to what extent simple sensory persistence might be causally involved in the effect under consideration. Most of our findings are consistent with the idea of sensory persistence, and indicate that information fragments are integrated over a time window of about 100 to 180 ms to form a complete surface representation.     

A theory for the development of feature detecting cells in visual cortex

Passive modification of the strength of synaptic junctions that results in the construction of internal mappings with some of the properties of memory is shown to lead to the development of Hubel-Wiesel type feature detectors in visual cortex. With such synaptic modification a cortical cell can become committed to an arbitrary but repeated external pattern, and thus fire every time the pattern is presented even if that cell has no genetic pre-disposition to respond to the particular pattern. The additional assumption of lateral inhibition between cortical cells severely limits the number of cells which respond to one pattern as well as the number of patterns that are picked up by a cell. The introduction of a simple neural mapping from the visual field to the lateral geniculate leads to an interaction between patterns which, combined with our assumptions above, seems to lead to a progression of patterns from column to column of the type observed by Hubel and Wiesel in monkey.     

Applications of Pick-Nevanlinna interpolation theory to retention-solubility studies in the lungs

Computational procedures for retention-solubility studies are given which determine data feasibility and some extreme properties of lung models compatible with given data. The procedures are analytic and are based on the interpolation theory of Pick and Nevanlinna.     

An evolutionary computational theory of prefrontal executive function in decision-making

The prefrontal cortex subserves executive control and decision-making, that is, the coordination and selection of thoughts and actions in the service of adaptive behaviour. We present here a computational theory describing the evolution of the prefrontal cortex from rodents to humans as gradually adding new inferential Bayesian capabilities for dealing with a computationally intractable decision problem: exploring and learning new behavioural strategies versus exploiting and adjusting previously learned ones through reinforcement learning (RL). We provide a principled account identifying three inferential steps optimizing this arbitration through the emergence of (i) factual reactive inferences in paralimbic prefrontal regions in rodents; (ii) factual proactive inferences in lateral prefrontal regions in primates and (iii) counterfactual reactive and proactive inferences in human frontopolar regions. The theory clarifies the integration of model-free and model-based RL through the notion of strategy creation. The theory also shows that counterfactual inferences in humans yield to the notion of hypothesis testing, a critical reasoning ability for approximating optimal adaptive processes and presumably endowing humans with a qualitative evolutionary advantage in adaptive behaviour.     

Information coding in a laminar computational model of cat primary visual cortex

Neural populations across cortical layers perform different computational tasks. However, it is not known whether information in different layers is encoded using a common neural code or whether it depends on the specific layer. Here we studied the laminar distribution of information in a large-scale computational model of cat primary visual cortex. We analyzed the amount of information about the input stimulus conveyed by the different representations of the cortical responses. In particular, we compared the information encoded in four possible neural codes: (1) the information carried by the firing rate of individual neurons; (2) the information carried by spike patterns within a time window; (3) the rate-and-phase information carried by the firing rate labelled by the phase of the Local Field Potentials (LFP); (4) the pattern-and-phase information carried by the spike patterns tagged with the LFP phase. We found that there is substantially more information in the rate-and-phase code compared with the firing rate alone for low LFP frequency bands (less than 30 Hz). When comparing how information is encoded across layers, we found that the extra information contained in a rate-and-phase code may reach 90 % in Layer 4, while in other layers it reaches only 60 %, compared to the information carried by the firing rate alone. These results suggest that information processing in primary sensory cortices could rely on different coding strategies across different layers.     

General validity of Levelt's propositions reveals common computational mechanisms for visual rivalry

The mechanisms underlying conscious visual perception are often studied with either binocular rivalry or perceptual rivalry stimuli. Despite existing research into both types of rivalry, it remains unclear to what extent their underlying mechanisms involve common computational rules. Computational models of binocular rivalry mechanisms are generally tested against Levelt's four propositions, describing the psychophysical relation between stimulus strength and alternation dynamics in binocular rivalry. Here we use a bistable rotating structure-from-motion sphere, a generally studied form of perceptual rivalry, to demonstrate that Levelt's propositions also apply to the alternation dynamics of perceptual rivalry. Importantly, these findings suggest that bistability in structure-from-motion results from active cross-inhibition between neural populations with computational principles similar to those present in binocular rivalry. Thus, although the neural input to the computational mechanism of rivalry may stem from different cortical neurons and different cognitive levels the computational principles just prior to the production of visual awareness appear to be common to the two types of rivalry.