Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

ELECTROPHYSIOLOGICAL MEASURES OF AUDITORY PROCESSING IN ODONTOCETES

ABSTRACT

The preponderance of our knowledge concerning hearing in the Cetacea has come from psychophysical studies. The most widely used alternative to psychophysical studies are neurophysiological studies utilizing auditory evoked potentials (AEPs). Due in part to the hypertrophy of auditory structures exhibited by the Cetacea, AEPs are highly robust and rapidly and easily obtained. Moreover, because AEPs reflect the synchronized activity of large neuronal assemblies, they offer a high level window onto auditory processing and allow for across-species comparison of responses.

Recent studies utilizing AEP techniques have demonstrated that the cetaceans have extremely high temporal resolution with integration times in the order of 300 msec. Remarkably, these animals also exhibit extremely sharp frequency tuning with auditory filters having Q10 of 20–30.     

Paradoxical evidence integration in rapid decision processes

Decisions about noisy stimuli require evidence integration over time. Traditionally, evidence integration and decision making are described as a one-stage process: a decision is made when evidence for the presence of a stimulus crosses a threshold. Here, we show that one-stage models cannot explain psychophysical experiments on feature fusion, where two visual stimuli are presented in rapid succession. Paradoxically, the second stimulus biases decisions more strongly than the first one, contrary to predictions of one-stage models and intuition. We present a two-stage model where sensory information is integrated and buffered before it is fed into a drift diffusion process. The model is tested in a series of psychophysical experiments and explains both accuracy and reaction time distributions.     

A neural network model for visual motion detection that can explain psychophysical and neurophysiological phenomena

This paper proposes a new neural network model for visual motion detection. The model can well explain both psychophysical findings (the changes of displacement thresholds with stimulus velocity and the perception of apparent motion) and neurophysiological findings (the selectivity for the direction and the velocity of a moving stimulus). To confirm the behavior of the model, numerical examinations were conducted. The results were consistent with both psychophysical and neurophysiological findings.     

Fragment-based learning of visual object categories in non-human primates

When we perceive a visual object, we implicitly or explicitly associate it with an object category we know. Recent research has shown that the visual system can use local, informative image fragments of a given object, rather than the whole object, to classify it into a familiar category. We have previously reported, using human psychophysical studies, that when subjects learn new object categories using whole objects, they incidentally learn informative fragments, even when not required to do so. However, the neuronal mechanisms by which we acquire and use informative fragments, as well as category knowledge itself, have remained unclear. Here we describe the methods by which we adapted the relevant human psychophysical methods to awake, behaving monkeys and replicated key previous psychophysical results. This establishes awake, behaving monkeys as a useful system for future neurophysiological studies not only of informative fragments in particular, but also of object categorization and category learning in general.     

A variational approach to behavioral and neuroelectrical laws

Variational methods play a fundamental and unifying role in several fields of physics, chemistry, engineering, economics, and biology, as they allow one to derive the behavior of a system as a consequence of an optimality principle. A possible application of these methods to a model of perception is given by considering a psychophysical law as the solution of an Euler-Lagrange equation. A general class of Lagrangians is identified by requiring the measurability of prothetic continua on interval scales. The associated Hamiltonian (the energy of the process) is tentatively connected with neurophysiological aspects. As an example of the suggested approach a particular choice of the Lagrangian, that is a sufficient condition to obtain classical psychophysical laws, while accounting for psychophysical adaptation and the stationarity of neuronal activity, is used to explore a possible relation between a behavioral law and a neuroelectrical ,response based on the Naka-Rushton model.     

Radial Frequency Analysis of Contour Shapes in the Visual Cortex

Cumulative psychophysical evidence suggests that the shape of closed contours is analysed by means of their radial frequency components (RFC). However, neurophysiological evidence for RFC-based representations is still missing. We investigated the representation of radial frequency in the human visual cortex with functional magnetic resonance imaging. We parametrically varied the radial frequency, amplitude and local curvature of contour shapes. The stimuli evoked clear responses across visual areas in the univariate analysis, but the response magnitude did not depend on radial frequency or local curvature. Searchlight-based, multivariate representational similarity analysis revealed RFC specific response patterns in areas V2d, V3d, V3AB, and IPS0. Interestingly, RFC-specific representations were not found in hV4 or LO, traditionally associated with visual shape analysis. The modulation amplitude of the shapes did not affect the responses in any visual area. Local curvature, SF-spectrum and contrast energy related representations were found across visual areas but without similar specificity for visual area that was found for RFC. The results suggest that the radial frequency of a closed contour is one of the cortical shape analysis dimensions, represented in the early and mid-level visual areas.     

A transduction model of the Meissner corpuscle

I propose a transduction model of the Meissner corpuscle that integrates ideas put forth by Freeman and Johnson and results obtained by Looft. The principal development in the present model is its specification that RA receptor potentials are updated as a linear function of stimulus velocity above baseline; the model thus readily accommodates non-sinusoidal input. It also incorporates modifications to Freeman and Johnson's model proposed by Slavík and Bell, namely a period of refractoriness lasting 1 ms followed by a period of hyperexcitability lasting 13.5 ms. The model is applied to various psychophysical and physiological situations: psychophysical threshold vs. frequency, RA afferent impulse rates vs. intensity, impulse regularity vs. frequency, phase retardation vs. frequency, and responses to non-repeating noise and to complex stimuli. Model output closely matches psychophysical and neurophysiological data. The proposed model thus reliably predicts RA afferent responses to arbitrary stimuli and may facilitate the development of theories relating psychophysical phenomena to their underlying neural representations.     

Sensing and deciding in the somatosensory system.

Combined psychophysical and neurophysiological experiments have revealed some of the neural codes associated with perception and processing of tactile information. Recently, intracortical microstimulation was used to demonstrate a causal link between primary cortical activity and perception. Evidence for a subsequent link, between a sensory decision process and its expression as a movement, has been found in motor areas.     

Recurrent V1–V2 interaction in early visual boundary processing

A majority of cortical areas are connected via feedforward and feedback fiber projections. In feedforward pathways we mainly observe stages of feature detection and integration. The computational role of the descending pathways at different stages of processing remains mainly unknown. Based on empirical findings we suggest that the top-down feedback pathways subserve a context-dependent gain control mechanism. We propose a new computational model for recurrent contour processing in which normalized activities of orientation selective contrast cells are fed forward to the next processing stage. There, the arrangement of input activation is matched against local patterns of contour shape. The resulting activities are subsequently fed back to the previous stage to locally enhance those initial measurements that are consistent with the top-down generated responses. In all, we suggest a computational theory for recurrent processing in the visual cortex in which the significance of local measurements is evaluated on the basis of a broader visual context that is represented in terms of contour code patterns. The model serves as a framework to link physiological with perceptual data gathered in psychophysical experiments. It handles a variety of perceptual phenomena, such as the local grouping of fragmented shape outline, texture surround and density effects, and the interpolation of illusory contours. Received: 28 October 1998 / Accepted in revised form: 19 March 1999     

A neurally plausible model of the dynamics of motion integration in smooth eye pursuit based on recursive Bayesian estimation

In this study, we describe a model of motion integration in smooth eye pursuit based on a recursive Bayesian estimation process, which displays a dynamic behaviour qualitatively similar to the dynamics of the motion integration process observed experimentally, both psychophysically in humans and monkeys, and physiologically in monkeys. By formulating the model as an approximate version of a Kalman filter algorithm, we have been able to show that it can be put into a neurally plausible, distributed recurrent form which coarsely corresponds to the recurrent circuitry of visual cortical areas V1 and MT. The model thus provides further support for the notion that the motion integration process is based on a form of Bayesian estimation, as has been suggested by many psychophysical studies, and moreover suggests that the observed dynamic properties of this process are the result of the recursive nature of the motion estimation.     

Learning to link visual contours

In complex visual scenes, linking related contour elements is important for object recognition. This process, thought to be stimulus driven and hard wired, has substrates in primary visual cortex (V1). Here, however, we find contour integration in V1 to depend strongly on perceptual learning and top-down influences that are specific to contour detection. In naive monkeys, the information about contours embedded in complex backgrounds is absent in V1 neuronal responses and is independent of the locus of spatial attention. Training animals to find embedded contours induces strong contour-related responses specific to the trained retinotopic region. These responses are most robust when animals perform the contour detection task but disappear under anesthesia. Our findings suggest that top-down influences dynamically adapt neural circuits according to specific perceptual tasks. This may serve as a general neuronal mechanism of perceptual learning and reflect top-down mediated changes in cortical states.     

Neural correlates of chromatic motion perception.

A variety of psychophysical and neurophysiological studies suggest that chromatic motion perception in the primate brain may be performed outside the classical motion processing pathway. We addressed this provocative proposal directly by assessing the sensitivity of neurons in motion area MT to moving colored stimuli while simultaneously determining perceptual sensitivity in nonhuman primate observers. The results of these studies demonstrate a strong correspondence between neuronal and perceptual measures. Our findings testify that area MT is indeed a principal component of the neuronal substrate for color-based motion processing.     

A single functional model of drivers and modulators in cortex

A distinction is commonly made between synaptic connections capable of evoking a response (“drivers”) and those that can alter ongoing activity but not initiate it (“modulators”). Here it is proposed that, in cortex, both drivers and modulators are an emergent property of the perceptual inference performed by cortical circuits. Hence, it is proposed that there is a single underlying computational explanation for both forms of synaptic connection. This idea is illustrated using a predictive coding model of cortical perceptual inference. In this model all synaptic inputs are treated identically. However, functionally, certain synaptic inputs drive neural responses while others have a modulatory influence. This model is shown to account for driving and modulatory influences in bottom-up, lateral, and top-down pathways, and is used to simulate a wide range of neurophysiological phenomena including surround suppression, contour integration, gain modulation, spatio-temporal prediction, and attention. The proposed computational model thus provides a single functional explanation for drivers and modulators and a unified account of a diverse range of neurophysiological data.     

Psychophysical and neurophysiological hearing thresholds in the bat <Emphasis Type="Italic">Phyllostomus </Emphasis><Emphasis Type="Italic">discolor</Emphasis>

Absolute hearing thresholds in the spear-nosed bat Phyllostomus discolor have been determined both with psychophysical and neurophysiological methods. Neurophysiological data have been obtained from two different structures of the ascending auditory pathway, the inferior colliculus and the auditory cortex. Minimum auditory thresholds of neurons are very similar in both structures. Lowest absolute thresholds of 0 dB SPL are reached at frequencies from about 35 to 55 kHz in both cases. Overall behavioural sensitivity is roughly 20 dB better than neural sensitivity. The behavioural audiogram shows a first threshold dip around 23 kHz but threshold was lowest at 80 kHz (−10 dB SPL). This high sensitivity at 80 kHz is not reflected in the neural data. The data suggest that P. discolor has considerably better absolute auditory thresholds than estimated previously. The psychophysical and neurophysiological data are compared to other phyllostomid bats and differences are discussed. S. Hoffmann, L. Baier, F. Borina contributed equally to this work.     

Speech cues contribute to audiovisual spatial integration

Speech is the most important form of human communication but ambient sounds and competing talkers often degrade its acoustics. Fortunately the brain can use visual information, especially its highly precise spatial information, to improve speech comprehension in noisy environments. Previous studies have demonstrated that audiovisual integration depends strongly on spatiotemporal factors. However, some integrative phenomena such as McGurk interference persist even with gross spatial disparities, suggesting that spatial alignment is not necessary for robust integration of audiovisual place-of-articulation cues. It is therefore unclear how speech-cues interact with audiovisual spatial integration mechanisms. Here, we combine two well established psychophysical phenomena, the McGurk effect and the ventriloquist's illusion, to explore this dependency. Our results demonstrate that conflicting spatial cues may not interfere with audiovisual integration of speech, but conflicting speech-cues can impede integration in space. This suggests a direct but asymmetrical influence between ventral 'what' and dorsal 'where' pathways.     

Modeling motor cortical operations by an attractor network of stochastic neurons

Understanding the neural computations performed by the motor cortex requires biologically plausible models that account for cell discharge patterns revealed by neurophysiological recordings. In the present study the motor cortical activity underlying movement generation is modeled as the dynamic evolution of a large fully recurrent network of stochastic spiking neurons with noise superimposed on the synaptic transmission. We show that neural representations of the learned movement trajectories can be stored in the connectivity matrix in such a way that, when activated, a particular trajectory evolves in time as a dynamic attractor of the system while individual neurons fire irregularly with large variability in their interspike intervals. Moreover, the encoding of trajectories as attractors ensures high stability of the ensemble dynamics in the presence of synaptic noise. In agreement with neurophysiological findings, the suggested model can provide a wide repertoire of specific motor behaviors, whereas the number of specialized cells and specific connections may be negligibly small if compared with the whole population engaged in trajectory retrieving. To examine the applicability of the model we study quantitatively the relationship between local geometrical and kinematic characteristics of the trajectories generated by the network. The relationship obtained as a result of simulations is close to the 2/3 power law established by psychophysical and neurophysiological studies.     

Context modulates the ERP signature of contour integration

We investigated how the electrophysiological signature of contour integration is changed by the context in which a contour is embedded. Specifically, we manipulated the orientations of Gabor elements surrounding an embedded shape outline. The amplitudes of early visual components over posterior scalp regions were changed by the presence of a contour, and by the orientation of elements surrounding the contour. Differences in context type had an effect on the early P1 and N1 components, but not on the later P2 component. The presence of a contour had an effect on the N1 and P2 components, but not on the earlier P1 component. A modulatory effect of context on contour integration was observed on the N1 component. These results highlight the importance of the context in which contour integration takes place.     

The roles and functions of cutaneous mechanoreceptors

Combined psychophysical and neurophysiological research has resulted in a relatively complete picture of the neural mechanisms of tactile perception. The results support the idea that each of the four mechanoreceptive afferent systems innervating the hand serves a distinctly different perceptual function, and that tactile perception can be understood as the sum of these functions. Furthermore, the receptors in each of those systems seem to be specialized for their assigned perceptual function.     

Contour saliency in primary visual cortex

Contour integration is an important intermediate stage of object recognition, in which line segments belonging to an object boundary are perceptually linked and segmented from complex backgrounds. Contextual influences observed in primary visual cortex (V1) suggest the involvement of V1 in contour integration. Here, we provide direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours. Receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour as reliably as the animal's behavioral responses. We also show that the same visual contours elicited significantly weaker neuronal responses when they were not detected in the detection task, or when they were unattended. Our results demonstrate that contextual interactions in V1 play a pivotal role in contour integration and saliency.