Models of Hopfield-type quaternion neural networks and their energy functions

Recently models of neural networks that can directly deal with complex numbers, complex-valued neural networks, have been proposed and several studies on their abilities of information processing have been done. Furthermore models of neural networks that can deal with quaternion numbers, which is the extension of complex numbers, have also been proposed. However they are all multilayer quaternion neural networks. This paper proposes models of fully connected recurrent quaternion neural networks, Hopfield-type quaternion neural networks. Since quaternion numbers are non-commutative on multiplication, some different models can be considered. We investigate dynamics of these proposed models from the point of view of the existence of an energy function and derive their conditions for existence.     

Phase conjugation model for information transfer in the nervous system

In the nervous system, dispersion in propagation time sometimes brings delay distortion or phase distortion on the information transmission. Also in the memory retrieval processes in the brain, some parts of images may be retrieved more slowly than others. For smooth control of fast movements as well as for keeping exact thinking, these distortions have to be taken out. To understand the distortion cancelling mechanism, new neural network models for compensating the phase distortion are proposed. The models stand on the concept of "phase conjugate mirror" which is used in optical image processing. Simulation studies based on the model resulted in successful cancellation of the delay dispersion involved in the information transmission in the nervous system.     

Improved Neural Networks with Random Weights for Short-Term Load Forecasting

An effective forecasting model for short-term load plays a significant role in promoting the management efficiency of an electric power system. This paper proposes a new forecasting model based on the improved neural networks with random weights (INNRW). The key is to introduce a weighting technique to the inputs of the model and use a novel neural network to forecast the daily maximum load. Eight factors are selected as the inputs. A mutual information weighting algorithm is then used to allocate different weights to the inputs. The neural networks with random weights and kernels (KNNRW) is applied to approximate the nonlinear function between the selected inputs and the daily maximum load due to the fast learning speed and good generalization performance. In the application of the daily load in Dalian, the result of the proposed INNRW is compared with several previously developed forecasting models. The simulation experiment shows that the proposed model performs the best overall in short-term load forecasting.     

Curved trajectory prediction using a self-organizing neural network

Existing neural network models are capable of tracking linear trajectories of moving visual objects. This paper describes an additional neural mechanism, disfacilitation, that enhances the ability of a visual system to track curved trajectories. The added mechanism combines information about an object's trajectory with information about changes in the object's trajectory, to improve the estimates for the object's next probable location. Computational simulations are presented that show how the neural mechanism can learn to track the speed of objects and how the network operates to predict the trajectories of accelerating and decelerating objects.     

End-to-end neural system identification with neural information flow

Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.     

Neural integration by short term potentiation

Neurophysiological studies in the oculomotor system suggest that an integrative operation is required in order to derive an eye position signal from a command signal which usually correlates with eye velocity. Several proposed models for a neural integrator are examined. All these models incorporate some form of positive feedback as a basic mechanism. Based on the performance of the models, we argue that such a scheme require extreme high precision in order to work properly. A new model based on potentiation phenomena in synaptic transmission is proposed and is shown to be free from the deficits of most previous models. The proposed model also accounts for various neural behaviors in a very natural way. A possible implementation of the model is also discussed in the context of the vestibulo-ocular reflex (VOR).     

Computer control of pH and DO in a laboratory fermenter using a neural network technique

In this contribution, the advantages of the artificial neural network approach to the identification and control of a laboratory-scale biochemical reactor are demonstrated. It is very important to be able to maintain the levels of two process variables, pH and dissolved oxygen (DO) concentration, over the course of fermentation in biosystems control. A PC-supported, fully automated, multi-task control system has been designed and built by the authors. Forward and inverse neural process models are used to identify and control both the pH and the DO concentration in a fermenter containing a Saccharomyces cerevisiae based-culture. The models are trained off-line, using a modified back-propagation algorithm based on conjugate gradients. The inverse neural controller is augmented by a new adaptive term that results in a system with robust performance. Experimental results have confirmed that the regulatory and tracking performances of the control system proposed are good.     

Identification and estimation algorithm for stochastic neural system

An algorithm for the estimation of stochastic processes in a neural system is presented. This process is defined here as the continuous stochastic process reflecting the dynamics of the neural system which has some inputs and generates output spike trains. The algorithm proposed here is to identify the system parameters and then estimate the stochastic process called neural system process here. These procedures carried out on the basis of the output spike trains which are supposed to be the data observed in the randomly missing way by the threshold time function in the neural system. The algorithm is constructed with the well-known Kalman filters and realizes the estimation of the neural system process by cooperating with the algorithm for the parameter estimation of the threshold time function presented previously (Nakao et al., 1983). The performance of the algorithm is examined by applying it to the various spike trains simulated by some artificial models and also to the neural spike trains recorded in cat's optic tract fibers. The results in these applications are thought to prove the effectiveness of the algorithm proposed here to some extent. Such attempts, we think, will serve to improve the characterizing and modelling techniques of the stochastic neural systems.     

Identification and estimation algorithm for stochastic neural system. II

The algorithm for identifying the stochastic neural system and estimating the system process which reflects the dynamics of the neural network are presented in this papar. The analogous algorithm has been proposed in our preceding paper (Nakao et al., 1984), which was based on the randomly missed observations of a system process only. Since the previous algorithm mentioned above was subject to an unfavorable effect of consecutively missed observations, to reduce such an effect the algorithm proposed here is designed additionally to observe an intensity process in a neural spike train as the information for the estimation.The algorithm is constructed with the extended Kalman filters because it is naturally expected that a nonlinear and time variant structure is necessary for the filters to realize the observation of an intensity process by means of mapping from a system process to an intensity process. The performance of the algorithm is examined by applying it to some artificial neural systems and also to cat's visual nervous systems. The results in these applications are thought to prove the effectiveness of the algorithm proposed here and its superiority to the algorithm proposed previously.     

Protein secondary structure prediction using three neural networks and a segmental semi Markov model

Prediction of protein secondary structure is an important step towards elucidating its three dimensional structure and its function. This is a challenging problem in bioinformatics. Segmental semi Markov models (SSMMs) are one of the best studied methods in this field. However, incorporating evolutionary information to these methods is somewhat difficult. On the other hand, the systems of multiple neural networks (NNs) are powerful tools for multi-class pattern classification which can easily be applied to take these sorts of information into account.To overcome the weakness of SSMMs in prediction, in this work we consider a SSMM as a decision function on outputs of three NNs that uses multiple sequence alignment profiles. We consider four types of observations for outputs of a neural network. Then profile table related to each sequence is reduced to a sequence of four observations. In order to predict secondary structure of each amino acid we need to consider a decision function. We use an SSMM on outputs of three neural networks. The proposed SSMM has discriminative power and weights over different dependency models for outputs of neural networks. The results show that the accuracy of our model in predictions, particularly for strands, is considerably increased.     

Fast and robust image segmentation by small-world neural oscillator networks

Inspired by the temporal correlation theory of brain functions, researchers have presented a number of neural oscillator networks to implement visual scene segmentation problems. Recently, it is shown that many biological neural networks are typical small-world networks. In this paper, we propose and investigate two small-world models derived from the well-known LEGION (locally excitatory and globally inhibitory oscillator network) model. To form a small-world network, we add a proper proportion of unidirectional shortcuts (random long-range connections) to the original LEGION model. With local connections and shortcuts, the neural oscillators can not only communicate with neighbors but also exchange phase information with remote partners. Model 1 introduces excitatory shortcuts to enhance the synchronization within an oscillator group representing the same object. Model 2 goes further to replace the global inhibitor with a sparse set of inhibitory shortcuts. Simulation results indicate that the proposed small-world models could achieve synchronization faster than the original LEGION model and are more likely to bind disconnected image regions belonging together. In addition, we argue that these two models are more biologically plausible.     

Gustatory processing is dynamic and distributed

The process of gustatory coding consists of neural responses that provide information about the quantity and quality of food, its generalized sensation, its hedonic value, and whether it should be swallowed. Many of the models presently used to analyze gustatory signals are static in that they use the average neural firing rate as a measure of activity and are unimodal in the sense they are thought to only involve chemosensory information. We have recently elaborated upon a dynamic model of gustatory coding that involves interactions between neurons in single as well as in spatially separate, gustatory and somatosensory regions. We propose that the specifics of gustatory responses grow not only out of information ascending from taste receptor cells, but also from the cycling of information around a massively interconnected system.     

Adaptive control for mimo uncertain nonlinear systems using recurrent wavelet neural network

Recurrent wavelet neural network (RWNN) has the advantages such as fast learning property, good generalization capability and information storing ability. With these advantages, this paper proposes an RWNN-based adaptive control (RBAC) system for multi-input multi-output (MIMO) uncertain nonlinear systems. The RBAC system is composed of a neural controller and a bounding compensator. The neural controller uses an RWNN to online mimic an ideal controller, and the bounding compensator can provide smooth and chattering-free stability compensation. From the Lyapunov stability analysis, it is shown that all signals in the closed-loop RBAC system are uniformly ultimately bounded. Finally, the proposed RBAC system is applied to the MIMO uncertain nonlinear systems such as a mass-spring-damper mechanical system and a two-link robotic manipulator system. Simulation results verify that the proposed RBAC system can achieve favorable tracking performance with desired robustness without any chattering phenomenon in the control effort.     

Characterization,scaling, and partial representation of neural junctions and coordinated firing patterns by dynamic similarity

 This paper presents a dynamic-similarity-based system for mathematically characterizing the functional connectivity and information flow of neural junctions. This approach allows for quantitative comparison of operations of neural junctions across systems, and an interpretation of their connectivity parameters in terms of the flow of multiunit firing patterns. The paper further uses this characterization to show how to rationally construct reduced operational models of neural junctions. Both uniformly proportional scaling and partial fragmentary representations are developed. The uniformly scaled models are better adapted to overall capacities and broader theoretical conceptualizations; the partial representations are better adapted to direct comparison with microelectrode experimentation. The characterization of information flow is based on coordinated multiunit patterns such as synfire chains or sequential configurations. The system can be applied to component parts of large composite networks including junctions with topographical patchiness and other irregularities. The characterization should be of use to anatomists, physiologists, modelers, and theorists. The theory predicts that the necessity for cooperative confluence of synaptic potentials in sending and receiving sequential configurations across topographically constrained projection fields requires the existence of functional ‘pattern modules’ within the topographical synaptology of the junction. Received: 13 April 1994/Accepted in revised form: 13 January 1995     

Bulbocortical interplay in olfactory information processing via synchronous oscillations

Emergence of synchronous oscillatory activity is an inherent feature of the olfactory systems of insects, mollusks and mammals. A class of simple computational models of the mammalian olfactory system consisting of olfactory bulb and olfactory cortex is constructed to explore possible roles of the related neural circuitry in olfactory information processing via synchronous oscillations. In the models, the bulbar neural circuitry is represented by a chain of oscillators and that of cortex is analogous to an associative memory network with horizontal synaptic connections. The models incorporate the backprojection from cortical units to the bulbar oscillators in particular ways. They exhibit rapid and robust synchronous oscillations in the presence of odorant stimuli, while they show either nonoscillatory states or propagating waves in the absence of stimuli, depending on the values of model parameters. In both models, the backprojection is shown to enhance the establishment of large-scale synchrony. The results suggest that the modulation of neural activity through centrifugal inputs may play an important role at the early stage of cortical information processing.     

Visual perception of surface curvature. The spin variation and its physiological implications

In the context of the models of structure from motion visual processing, we propose that the optic-flow field is a source of information for the perception of the curvature of a smooth surface in motion. In particular, it is shown how the spin variation (SV), a second spatial derivative of the retinal velocity field, is mathematically related to the curvature of the surface. Under the hypothesis that the visual system relies on SV to analyse the structure of a moving surface, a neural scheme for SV detection is proposed and psychophysical predictions are developed. Results obtained on artificial images show that the SV scheme presents a rather weak sensitivity to noise in conditions of low image velocity.     

A Context-Sensitive Mechanism in Hippocampal CA1 Networks

This paper presents a possible context-sensitive mechanism in a neural network and at single neuron levels based on the experiments of hippocampal CA1 and their theoretical models. First, the spatiotemporal learning rule (STLR, non-Hebbian) and the Hebbian rule (HEBB) are experimentally shown to coexist in dendrite–soma interactions in single hippocampal pyramidal cells of CA1. Second, the functional differences between STLR and HEBB are theoretically shown in pattern separation and pattern completion. Third, the interaction between STLR and HEBB in neural levels is proposed to play an important role in forming a selective context determined by value information, which is related to expected reward and behavioral estimation.     

Initial patterning of the central nervous system: how many organizers?

For three-quarters of a century, developmental biologists have been asking how the nervous system is specified as distinct from the rest of the ectoderm during early development, and how it becomes subdivided initially into distinct regions such as forebrain, midbrain, hindbrain and spinal cord. The two events of 'neural induction' and 'early neural patterning' seem to be intertwined, and many models have been put forward to explain how these processes work at a molecular level. Here I consider early neural patterning and discuss the evidence for and against the two most popular models proposed for its explanation: the idea that multiple signalling centres (organizers) are responsible for inducing different regions of the nervous system, and a model first articulated by Nieuwkoop that invokes two steps (activation/transformation) necessary for neural patterning. As recent evidence from several systems challenges both models, I propose a modification of Nieuwkoop's model that most easily accommodates both classical and more recent data, and end by outlining some possible directions for future research.     

Boring's formulation: a scheme for identifying functional neuron groups in a sensory system

In 1935 Edwin Boring proposed that each attribute of sensation reflects the activity of a different neural circuit. If this idea is valid, it could facilitate both psychophysical and neurophysiological research on sensory systems. We think it likely that Boring's formulation is correct for three reasons: 1) Different sensory attributes reflect conscious information about different parameters of a stimulus. To be measured by any device, each of these parameters must be individually computed. Different neural circuits would appear to be necessary for the nervous system to carry out these different computations. 2) Perceived information about different sensory attributes can be made to diverge by appropriate manipulations of the stimuli. If there is a rigorous relationship between conscious sensory experience and neural activity, such a divergence implies that different sensory attributes are served by different neural circuits. 3) Accurate information about a sensory attribute requires that a human observer's attention be focused on that attribute. Changes in direction of attention are thought to involve a process of switching from one neural circuit to another, and provide another way to cause perceived information about different sensory attributes to diverge.     

Knowledge management of the neuroscientific literature: the data model and underlying strategy of the NeuroScholar system.

This paper describes the underlying strategy and system's design of a knowledge management system for the neuroscientific literature called 'NeuroScholar'. The problem that the system is designed to address is to delineate fully the neural circuitry involved in a specific behaviour. The use of this system provides experimental neuroscientists with a new method of building computational models ('knowledge models') of the contents of the published literature. These models may provide input for analysis (conceptual or computational), or be used as constraint sets for conventional neural modelling work. The underlying problems inherent in this approach, the general framework for the proposed solution, the practical issues concerning usage of the system and a detailed, technical account of the system are described. The author uses a widely used software specification language (the Universal Modelling Language) to describe the design of the system and present examples from published work concerned with classical eyeblink conditioning in the rabbit.