Color categorization and color constancy in a neural network model of V4

We develop a neural network model that instantiates color constancy and color categorization in a single unified framework. Previous models achieve similar effects but ignore important biological constraints. Color constancy in this model is achieved by a new application of the double opponent cells found in the blobs of the visual cortex. Color categorization emerges naturally, as a consequence of processing chromatic stimuli as vectors in a four-dimensional color space. A computer simulation of this model is subjected to the classic psychophysical tests that first uncovered these phenomena, and its response matches psychophysical results very closely.     

A neural network model of semantic memory linking feature-based object representation and words

Recent theories in cognitive neuroscience suggest that semantic memory is a distributed process, which involves many cortical areas and is based on a multimodal representation of objects. The aim of this work is to extend a previous model of object representation to realize a semantic memory, in which sensory-motor representations of objects are linked with words. The model assumes that each object is described as a collection of features, coded in different cortical areas via a topological organization. Features in different objects are segmented via γ-band synchronization of neural oscillators. The feature areas are further connected with a lexical area, devoted to the representation of words. Synapses among the feature areas, and among the lexical area and the feature areas are trained via a time-dependent Hebbian rule, during a period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from acoustic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits).     

Bifurcation analysis of a neural network model

This paper describes the analysis of the well known neural network model by Wilson and Cowan. The neural network is modeled by a system of two ordinary differential equations that describe the evolution of average activities of excitatory and inhibitory populations of neurons. We analyze the dependence of the model's behavior on two parameters. The parameter plane is partitioned into regions of equivalent behavior bounded by bifurcation curves, and the representative phase diagram is constructed for each region. This allows us to describe qualitatively the behavior of the model in each region and to predict changes in the model dynamics as parameters are varied. In particular, we show that for some parameter values the system can exhibit long-period oscillations. A new type of dynamical behavior is also found when the system settles down either to a stationary state or to a limit cycle depending on the initial point.     

A model for neural representation of temporal duration

To address how temporal duration is encoded in neural systems, we put forward a simple model for recurrent neural networks. Particular assumptions are only the following two: (1) neuronal bistability and; (2) environmental effects described by a heat bath. The results of Monte Carlo simulation show that population activity triggered at an initial time continues for a prolonged duration, followed by an abrupt self-termination. This time course seems highly suitable for neural representation of temporal duration. The time scale of this prolonged duration is much longer than the time scale of neuronal firing which is of the order of ms. The former time scale implies that of interval timing in cognition and behaviour. Thus, the model provides a possible explanation for a link between these two separated time scales. The Weber law, a hallmark of humans and animals' interval timing, can also be reproduced in our model.     

Computational antisense oligo prediction with a neural network model

MOTIVATION: The expression of a gene can be selectively inhibited by antisense oligonucleotides (AOs) targeting the mRNA. However, if the target site in the mRNA is picked randomly, typically 20% or less of the AOs are effective inhibitors in vivo. The sequence properties that make an AO effective are not well understood, thus many AOs need to be tested to find good inhibitors, which is time consuming and costly. So far computational models have been based exclusively on RNA structure prediction or motif searches while ignoring information from other aspects of AO design into the model. RESULTS: We present a computational model for AO prediction based on a neural network approach using a broad range of input parameters. Collecting sequence and efficacy data from AO scanning experiments in the literature generated a database of 490 AO molecules. Using a set of derived parameters based on AO sequence properties we trained a neural network model. The best model, an ensemble of 10 networks, gave an overall correlation coefficient of 0.30 (p=10(-8)). This model can predict effective AOs (>50% inhibition of gene expression) with a success rate of 92%. Using these thresholds the model predicts on average 12 effective AOs per 1000 base pairs, making it a stringent yet practical method for AO prediction.     

Experimental verification of the opponent theory of human color vision

The colors observed by the human eye after a short flash of light of different spectral compositions were studied experimentally. The successive images and changes in their color with time confirm the opponent theory of human color vision.     

Recognition of partly occluded patterns: a neural network model

 Human beings are often able to read a letter or word partly occluded by contaminating ink stains. However, if the stains are completely erased and the occluded areas of the letter are changed to white, we usually have difficulty in reading the letter. In this article I propose a hypothesis explaining why a pattern is easier to recognize when it is occluded by visible objects than by invisible opaque objects. A neural network model is constructed based on this hypothesis. The visual system extracts various visual features from the input pattern and then attempts to recognize it. If the occluding objects are not visible, the visual system will have difficulty in distinguishing which features are relevant to the original pattern and which are newly generated by the occlusion. If the occluding objects are visible, however, the visual system can easily discriminate between relevant and irrelevant features and recognize the occluded pattern correctly. The proposed model is an extended version of the neocognitron model. The activity of the feature-extracting cells whose receptive fields cover the occluding objects is suppressed in an early stage of the hierarchical network. Since the irrelevant features generated by the occlusion are thus eliminated, the model can recognize occluded patterns correctly, provided the occlusion is not so large as to prevent recognition even by human beings. Received: 21 February 2000 / Accepted in revised form: 11 September 2000     

A neural network model of attention-modulated neurodynamics

Visual attention appears to modulate cortical neurodynamics and synchronization through various cholinergic mechanisms. In order to study these mechanisms, we have developed a neural network model of visual cortex area V4, based on psychophysical, anatomical and physiological data. With this model, we want to link selective visual information processing to neural circuits within V4, bottom-up sensory input pathways, top-down attention input pathways, and to cholinergic modulation from the prefrontal lobe. We investigate cellular and network mechanisms underlying some recent analytical results from visual attention experimental data. Our model can reproduce the experimental findings that attention to a stimulus causes increased gamma-frequency synchronization in the superficial layers. Computer simulations and STA power analysis also demonstrate different effects of the different cholinergic attention modulation action mechanisms.     

Purposive behavior and cognitive mapping: a neural network model

This study presents a real-time, biologically plausible neural network approach to purposive behavior and cognitive mapping. The system is composed of (a) an action system, consisting of a goal-seeking neural mechanism controlled by a motivational system; and (b) a cognitive system, involving a neural cognitive map. The goal-seeking mechanism displays exploratory behavior until either (a) the goal is found or (b) an adequate prediction of the goal is generated. The cognitive map built by the network is a top logical map, i.e., it represents only the adjacency, but not distances or directions, between places. The network has recurrent and non-recurrent properties that allow the reading of the cognitive map without modifying it. Two types of predictions are introduced: fast-time and real-time predictions. Fast-time predictions are produced in advance of what occurs in real time, when the information stored in the cognitive map is used to predict the remote future. Real-time predictions are generated simultaneously with the occurrence of environmental events, when the information stored in the cognitive map is being updated. Computer simulations show that the network successfully describes latent learning and detour behavior in rats. In addition, simulations demonstrate that the network can be applied to problem-solving paradigms such as the Tower of Hanoi puzzle.     

Transitory memory retrieval in a biologically plausible neural network model

A number of memory models have been proposed. These all have the basic structure that excitatory neurons are reciprocally connected by recurrent connections together with the connections with inhibitory neurons, which yields associative memory (i.e., pattern completion) and successive retrieval of memory. In most of the models, a simple mathematical model for a neuron in the form of a discrete map is adopted. It has not, however, been clarified whether behaviors like associative memory and successive retrieval of memory appear when a biologically plausible neuron model is used. In this paper, we propose a network model for associative memory and successive retrieval of memory based on Pinsky-Rinzel neurons. The state of pattern completion in associative memory can be observed with an appropriate balance of excitatory and inhibitory connection strengths. Increasing of the connection strength of inhibitory interneurons changes the state of memory retrieval from associative memory to successive retrieval of memory. We investigate this transition.     

An experimental verification of the opponent theory of color vision of man

An experimental study of colors observed by a human eye after a short flash of light of different spectral composition was carried out. Sequential images and changes in their color with time support the opponent theory of colored vision of man.     

Feasibility of estimating isokinetic knee torque using a neural network model

Many studies have investigated the relationships between electromyography (EMG) and torque production. A few investigators have used adjusted learning algorithms and feed-forward artificial neural networks (ANNs) to estimate joint torque in the elbow. This study sought to estimate net isokinetic knee torque using ANN models. Isokinetic knee extensor and flexor torque data were measured simultaneously with agonist and antagonist EMG during concentric and eccentric contractions at joint velocities of 30 degrees /s and 60 degrees /s. Age, gender, height, body mass, agonist EMG, antagonist EMG, joint position and joint velocity were entered as predictive variables of net torque. A three-layer ANN model was developed and trained using an adjusted back-propagation algorithm. Accuracy results were compared against those of forward stepwise regression models. Stepwise regression models included body mass, body height and joint position as the most influential predictors, followed by agonist EMG for concentric and eccentric contractions. Estimation of eccentric torque included antagonist EMG following the agonist activation. ANN models resulted in more accurate torque estimation (R=0.96), compared to the stepwise regression models (R=0.71). ANN model accuracy increased greatly when the number of hidden units increased from 5 to 10, continuing to increase gradually with additional hidden units. The average number of training epochs necessary for solution convergence and the relative accuracy of the model indicate a strong ability for the ANN model to generalize these estimations to a broader sample. The ANN model appears to be a feasible technique for estimating joint torque in the knee.     

Testing of information condensation in a model reverberating spiking neural network

Information about external world is delivered to the brain in the form of structured in time spike trains. During further processing in higher areas, information is subjected to a certain condensation process, which results in formation of abstract conceptual images of external world, apparently, represented as certain uniform spiking activity partially independent on the input spike trains details. Possible physical mechanism of condensation at the level of individual neuron was discussed recently. In a reverberating spiking neural network, due to this mechanism the dynamics should settle down to the same uniform/ periodic activity in response to a set of various inputs. Since the same periodic activity may correspond to different input spike trains, we interpret this as possible candidate for information condensation mechanism in a network. Our purpose is to test this possibility in a network model consisting of five fully connected neurons, particularly, the influence of geometric size of the network, on its ability to condense information. Dynamics of 20 spiking neural networks of different geometric sizes are modelled by means of computer simulation. Each network was propelled into reverberating dynamics by applying various initial input spike trains. We run the dynamics until it becomes periodic. The Shannon's formula is used to calculate the amount of information in any input spike train and in any periodic state found. As a result, we obtain explicit estimate of the degree of information condensation in the networks, and conclude that it depends strongly on the net's geometric size.     

A neural network model for trace conditioning

We studied the dynamics of a neural network that has both recurrent excitatory and random inhibitory connections. Neurons started to become active when a relatively weak transient excitatory signal was presented and the activity was sustained due to the recurrent excitatory connections. The sustained activity stopped when a strong transient signal was presented or when neurons were disinhibited. The random inhibitory connections modulated the activity patterns of neurons so that the patterns evolved without recurrence with time. Hence, a time passage between the onsets of the two transient signals was represented by the sequence of activity patterns. We then applied this model to represent the trace eye blink conditioning, which is mediated by the hippocampus. We assumed this model as CA3 of the hippocampus and considered an output neuron corresponding to a neuron in CA1. The activity pattern of the output neuron was similar to that of CA1 neurons during trace eye blink conditioning, which was experimentally observed.     

A neural network model for texture discrimination

A model of texture discrimination in visual cortex was built using a feedforward network with lateral interactions among relatively realistic spiking neural elements. The elements have various membrane currents, equilibrium potentials and time constants, with action potentials and synapses. The model is derived from the modified programs of MacGregor (1987). Gabor-like filters are applied to overlapping regions in the original image; the neural network with lateral excitatory and inhibitory interactions then compares and adjusts the Gabor amplitudes in order to produce the actual texture discrimination. Finally, a combination layer selects and groups various representations in the output of the network to form the final transformed image material. We show that both texture segmentation and detection of texture boundaries can be represented in the firing activity of such a network for a wide variety of synthetic to natural images. Performance details depend most strongly on the global balance of strengths of the excitatory and inhibitory lateral interconnections. The spatial distribution of lateral connective strengths has relatively little effect. Detailed temporal firing activities of single elements in the lateral connected network were examined under various stimulus conditions. Results show (as in area 17 of cortex) that a single element's response to image features local to its receptive field can be altered by changes in the global context.     

Dynamic representation of odours by oscillating neural assemblies

Stimulus evoked oscillatory synchronization of neural assemblies has been most clearly documented in the olfactory and visual systems. Recent results with the olfactory system of locusts show that information about odour identity is contained in spatial and temporal aspects of an oscillatory population response. This suggests that brain oscillations may reflect a common reference for messages encoded in time. Although stimulus-evoked oscillatory phenomena are reliable, their roles in perception, memory and pattern recognition remain to be demonstrated. Using honey bees, we demonstrated that odour encoding involves, as in locusts, the oscillatory synchronization of assemblies of neurons, and that this synchronization is, here also, selectively abolished by the GABA receptor antagonist picrotoxin. In collaboration with Dr Brian Smith's laboratory, we showed, using a behavioural learning paradigm, that picrotoxin-induced desynchronization impairs the discrimination of molecularly similar odourants, but not that of dissimilar odours. It appears, therefore, that oscillatory synchronization of neuronal assemblies is relevant, and essential for fine odour discrimination. Finally, experiments with locust mushroom body neurons, two synapses downstream from the antennal lobe, indicate that their responses to odours become less specific when antennal lobe neurons are desynchronized by picrotoxin injection. These results suggest that oscillatory synchronization and the kind of temporal encoding it affords provide an additional dimension by which the brain can segment spatially overlapping stimulus representations.     

Optimization of fed-batch bioreactor using neural network model

An algorithm using feedforward neural network model for determining optimal substrate feeding policies for fed-batch fermentation process is presented in this work. The algorithm involves developing the neural network model of the process using the sampled data. The trained neural network model in turn is used for optimization purposes. The advantages of this technique is that optimization can be achieved without detailed kinetic model of the process and the computation of gradient of objective function with respect to control variables is straightforward. The application of the technique is demonstrated with two examples, namely, production of secreted protein and invertase. The simulation results show that the discrete-time dynamics of fed-batch bioreactor can be satisfactorily approximated using a feedforward sigmoidal neural network. The optimal policies obtained with the neural network model agree reasonably well with the previously reported results.     

A neural network model of the McCollough effect

A self-organizing, feature-extracting network (von der Malsburg, 1973) is extended to two feature dimensions to encompass line orientation and color. It is applied to McCollough effects, particularly longlasting, contingent-aftereffects. McCollough effects are thought to involve low-level associative memory in the form of synaptic modification. The McCollough-Malsburg Model (MMM) embodies positive synaptic modification with correlated firing of units in an input layer and an excitatory cortical layer. Computer simulation of MMM reproduces orientation-contingent color aftereffects. The model embodies many of the mechanisms thought to be operating in developmental plasticity, suggesting that equivalent mechanisms may be involved in adult long-term adaptation as well.This work was supported in part by NIH Grant No. 5 R01 NS09755-4 COM of the National Institute of Neurological Diseases and Stroke (M.A. Arbib, Principal Investigator)     

A model of neural network extracting binocular parallax

A model of neural network extracting binocular parallax is proposed. It is a multilayered network composed of analog threshold elements. Three types of binocular neurons are included in this model. They are binocular simple neurons, binocular gate neurons and binocular depth neurons. The final layers of this model consist of elements which correspond to the binocular depth neurons. The performance of the model has been simulated on a digital computer. The results of the computer simulation show that every element of this model acts like neurons found in cat's and monkey's visual system and this model extracts binocular parallax caused by simple line components satisfactorily.