A memory-based self-generated basis function neural network.

The paper presents a novel memory-based Self-Generated Basis Function Neural Network (SGBFN) that is composed of small CMACs. The SGBFN requires much smaller memory space than the conventional CMAC and has an excellent learning convergence property compared to multilayer neural networks. Each CMAC in the new structure takes a subset of problem inputs as its inputs. Several CMACs that have different subsets of inputs form a submodule and a group of submodules form a neural network. The output of a submodule is the product of its CMACs' outputs. Each submodule implements a self-generated basis function, which is developed during the learning. The output of the neural network is the sum of the outputs from the submodules. Using only a subset of inputs in each CMAC significantly reduces the required memory space in high-dimensional modeling. With the same size of memory, the new structure is able to achieve a much smaller learning error compared to the conventional CMAC.     

Saccade control in a simulated robot camera-head system: neural net architectures for efficient learning of inverse kinematics

The high speed of saccades means that they cannot be guided by visual feedback, so that any saccadic control system must know in advance the correct output signals to fixate a particular retinal position. To investigate neural-net architectures for learning this inverse-kinematics problem we simulated a 4 deg-of-freedom robot camera-head system, in which the head could pan and tilt and the cameras pan and verge. The main findings were: (1) Linear nets, multilayer perceptrons (MLPs) trained by backpropagation, and cerebellar model arithmetic computers (CMACs) all learnt rapidly to 5–10% accuracy when given perfect error feedback. (2) For additional accuracy (down to 2%) two-layer nets learnt much faster than a single MLP or CMAC: the best combination tried was to have a CMAC learn the errors of a trained linear net. (3) Imperfect error signals were provided by a crude controller whose output was simply proportional to retinal input in the relevant axis, thereby providing a mechanism for (a) controlling the camera-head system when the feedforward neural net controller was wrong or inoperative, and (b) converting sensory error signals into motor error signals as required in supervised learning. It proved possible to train neural-net controllers using these imperfect error signals over a range of learning rates and crude-controller gains. These results suggest that appropriate neural-net architectures can provide practical, accurate and robust adaptive control for saccadic movements. In addition, the arrangement of a crude controller teaching a sophisticated one may be similar to that used by the primate saccadic system, with brainstem circuitry teaching the cerebellum.     

On the choice of noise for the analysis of the peripheral auditory system

The cross-correlation between output and input of a system containing nonlinearities, when that system is stimulated with Gaussian white noise, is a good estimate of the linear properties of the system. In practice, however, when sequences of pseudonoise are used, great errors may be introduced in the estimate of the linear part depending on the properties of the noise. This consideration assumes special importance in the analysis of the linear properties of the peripheral auditory system, where the rectifying properties of the haircells constitute a second order nonlinearity. To explore this problem, a simple model has been designed, consisting of a second order nonlinearity without memory and sandwiched between two bandpass filters. Different types of pseudonoise are used as input whereupon it is shown that noise based on binary m-sequences, which is commonly used in noise generators, will yield totally incorrect information about this system. Somewhat better results are achieved with other types of noise. By using inverse-repeat sequences the results are greatly improved. Furthermore, certain anomalies obtained in the analysis of responses from single fibers in the auditory nerve are viewed in the light of the present results. The theoretical analysis of these anomalies reveals some information about the organization of the peripheral auditory system. For example, the possibility of the existence of a second bandpass filter in the auditory periphery seems to be excluded.     

The chronotron: a neuron that learns to fire temporally precise spike patterns

In many cases, neurons process information carried by the precise timings of spikes. Here we show how neurons can learn to generate specific temporally precise output spikes in response to input patterns of spikes having precise timings, thus processing and memorizing information that is entirely temporally coded, both as input and as output. We introduce two new supervised learning rules for spiking neurons with temporal coding of information (chronotrons), one that provides high memory capacity (E-learning), and one that has a higher biological plausibility (I-learning). With I-learning, the neuron learns to fire the target spike trains through synaptic changes that are proportional to the synaptic currents at the timings of real and target output spikes. We study these learning rules in computer simulations where we train integrate-and-fire neurons. Both learning rules allow neurons to fire at the desired timings, with sub-millisecond precision. We show how chronotrons can learn to classify their inputs, by firing identical, temporally precise spike trains for different inputs belonging to the same class. When the input is noisy, the classification also leads to noise reduction. We compute lower bounds for the memory capacity of chronotrons and explore the influence of various parameters on chronotrons' performance. The chronotrons can model neurons that encode information in the time of the first spike relative to the onset of salient stimuli or neurons in oscillatory networks that encode information in the phases of spikes relative to the background oscillation. Our results show that firing one spike per cycle optimizes memory capacity in neurons encoding information in the phase of firing relative to a background rhythm.     

Mathematische beschreibung der reaktion farbempfindlicher nervennetze auf intensitäts- und farbsprünge

The reaction of color sensitive neural networks to intensity and color steps on logarithmic transformation of the input signals is calculated mathematically. The networks consist of opponent-color cells respectively with (duple system 1) or without a surround (duple system 2) or of double opponent-color cells (quadruple system). The output signals are independent of the intensity level. Both duple systems are able to code the color of homogeneous areas on a dichromatic level. The hue corresponds to the sign, the saturation to the absolute value of the output signal. The coding of saturation becomes incorrect at intensity borders only with duple system 1 (due to a Mach band response) at color borders however with duple system 1 and 2 (due to low-pass properties). The quadruple system (like duple system 2) is insensitive to intensity differences. It only responds to color differences, which are transferred according to a band-pass filter. The system therefore is able to function as a detector of color borders. The results are used in a new model for the processing of color and color borders. A linear transformation has been found to be less suited for color coding.     

Biologically inspired calibration-free adaptive saccade control of a binocular camera-head

This paper describes fast and accurate calibration-free adaptive saccade control of a four-degrees-of-freedom binocular camera-head by means of Dynamic Cell Structures (DCS). The approach has been inspired by biology because primates face a similar problem and there is strong evidence that they have solved it in a similar way, i.e., by error feedback learning of an inverse model. Yet the emphasis of this article is not on detailed biological modeling but on how incremental growth of our artificial neural network model up to a prespecified precision results in very small networks suitable for real-time saccade control. Error-feedback-based training of this network proceeds in two phases. In the first phase we use a crude model of the cameras and the kinematics of the head to learn the topology of the input manifold together with a rough approximation of the control function off-line. In contrast to, for example, Kohonen-type adaptation rules, the distribution of neural units minimizes the control error and does not merely mimic the input probability density. In the second phase, the operating phase, the linear output units of the network continue to adapt on-line. Besides our TRC binocular camera-head we use a Datacube image processing system and a St?ubli R90 robot arm for automated training in the second phase. It will be demonstrated that the controller successfully corrects errors in the model and rapidly adapts to changing parameters. Received: 27 August 1996 / Accepted in revised form: 22 July 1997     

Towards enabling a cardiovascular digital twin for human systemic circulation using inverse analysis

An exponential rise in patient data provides an excellent opportunity to improve the existing health care infrastructure. In the present work, a method to enable cardiovascular digital twin is proposed using inverse analysis. Conventionally, accurate analytical solutions for inverse analysis in linear problems have been proposed and used. However, these methods fail or are not efficient for nonlinear systems, such as blood flow in the cardiovascular system (systemic circulation) that involves high degree of nonlinearity. To address this, a methodology for inverse analysis using recurrent neural network for the cardiovascular system is proposed in this work, using a virtual patient database. Blood pressure waveforms in various vessels of the body are inversely calculated with the help of long short-term memory (LSTM) cells by inputting pressure waveforms from three non-invasively accessible blood vessels (carotid, femoral and brachial arteries). The inverse analysis system built this way is applied to the detection of abdominal aortic aneurysm (AAA) and its severity using neural networks.

     

植物生物钟及其调控生长发育的研究进展

作为植物细胞内部的授时机制, 生物钟系统主要包括信号输入、核心振荡器和信号输出3个主要部分。该系统通过感受外界光照和温度等环境因子的昼夜周期性变化动态, 协调植物生长发育、代谢与生理反应, 赋予植物对生存环境的适应性。植物生物钟系统的核心振荡器通过多层级调控复杂的下游信号转导网络来参与调节植物生长发育及对生物与非生物胁迫的适应性。该文概述了近年来生物钟核心振荡器及其调控植物生长发育过程诸方面的研究进展, 并初步提出了植物时间生物学研究领域一些亟待解决的科学问题, 以期为生物钟领域的研究成果在作物分子育种方面的利用提供理论借鉴。     

A Model of Self-Organizing Head-Centered Visual Responses in Primate Parietal Areas

We present a hypothesis for how head-centered visual representations in primate parietal areas could self-organize through visually-guided learning, and test this hypothesis using a neural network model. The model consists of a competitive output layer of neurons that receives afferent synaptic connections from a population of input neurons with eye position gain modulated retinal receptive fields. The synaptic connections in the model are trained with an associative trace learning rule which has the effect of encouraging output neurons to learn to respond to subsets of input patterns that tend to occur close together in time. This network architecture and synaptic learning rule is hypothesized to promote the development of head-centered output neurons during periods of time when the head remains fixed while the eyes move. This hypothesis is demonstrated to be feasible, and each of the core model components described is tested and found to be individually necessary for successful self-organization.     

Constructive Autoassociative Neural Network for Facial Recognition

Autoassociative artificial neural networks have been used in many different computer vision applications. However, it is difficult to define the most suitable neural network architecture because this definition is based on previous knowledge and depends on the problem domain. To address this problem, we propose a constructive autoassociative neural network called CANet (Constructive Autoassociative Neural Network). CANet integrates the concepts of receptive fields and autoassociative memory in a dynamic architecture that changes the configuration of the receptive fields by adding new neurons in the hidden layer, while a pruning algorithm removes neurons from the output layer. Neurons in the CANet output layer present lateral inhibitory connections that improve the recognition rate. Experiments in face recognition and facial expression recognition show that the CANet outperforms other methods presented in the literature.     

Pattern orthogonalization via channel decorrelation by adaptive networks

The early processing of sensory information by neuronal circuits often includes a reshaping of activity patterns that may facilitate further processing in the brain. For instance, in the olfactory system the activity patterns that related odors evoke at the input of the olfactory bulb can be highly similar. Nevertheless, the corresponding activity patterns of the mitral cells, which represent the output of the olfactory bulb, can differ significantly from each other due to strong inhibition by granule cells and peri-glomerular cells. Motivated by these results we study simple adaptive inhibitory networks that aim to separate or even orthogonalize activity patterns representing similar stimuli. Since the animal experiences the different stimuli at different times it is difficult for the network to learn the connectivity based on their similarity; biologically it is more plausible that learning is driven by simultaneous correlations between the input channels. We investigate the connection between pattern orthogonalization and channel decorrelation and demonstrate that networks can achieve effective pattern orthogonalization through channel decorrelation if they simultaneously equalize their output levels. In feedforward networks biophysically plausible learning mechanisms fail, however, for even moderately similar input patterns. Recurrent networks do not have that limitation; they can orthogonalize the representations of highly similar input patterns. Even when they are optimized for linear neuronal dynamics they perform very well when the dynamics are nonlinear. These results provide insights into fundamental features of simplified inhibitory networks that may be relevant for pattern orthogonalization by neuronal circuits in general.     

Application of the recommendation architecture to telecommunications network management.

The recommendation architecture has been proposed as a system architecture which can enable a system to learn to perform a complex combination of interrelated functions. The capability of a system with the recommendation architecture to learn to manage complex telecommunication backbone networks has been investigated. A network model with a number of nodes and links and carrying realistic but randomly generated traffic was used as the target for the management system. Traffic data taken from the model was used as input to the recommendation architecture system. The traffic data was organized into inputs once every 5 minutes, and the management system organized these inputs into a hierarchy of repetition similarity. It was demonstrated that the outputs of this hierarchy provided information on the condition of the network. This output information was a compressed version of the inputs which correlated with major network conditions.     

On optimal nonlinear associative recall

The problem of determining the nonlinear function (“blackbox”) which optimally associates (on given criteria) two sets of data is considered. The data are given as discrete, finite column vectors, forming two matricesX (“input”) andY (“output”) with the same numbers of columns and an arbitrary numbers of rows. An iteration method based on the concept of the generalized inverse of a matrix provides the polynomial mapping of degreek onX by whichY is retrieved in an optimal way in the least squares sense. The results can be applied to a wide class of problems since such polynomial mappings may approximate any continuous real function from the “input” space to the “output” space to any required degree of accuracy. Conditions under which the optimal estimate is linear are given. Linear transformations on the input key-vectors and analogies with the “whitening” approach are also discussed. Conditions of “stationarity” on the processes of whichX andY are assumed to represent a set of sample sequences can be easily introduced. The optimal linear estimate is given by a discrete counterpart of the Wiener-Hopf equation and, if the key-signals are noise-like, the holographic-like scheme of associative memory is obtained, as the optimal nonlinear estimator. The theory can be applied to the system identification problem. It is finally suggested that the results outlined here may be relevant to the construction of models of associative, distributed memory.     

Simulation studies of a temporal sequence memory model

Models of circuit action in the mammalian hippocampus have led us to a study of habituation circuits. In order to help model the process of habituation we consider here a memory network designed to learn sequences of inputs separated by various time intervals and to repeat these sequences when cued by their initial portions. The structure of the memory is based on the anatomy of the dentate gyrus region of the mammalian hippocampus. The model consists of a number of arrays of cells called lamellae. Each array consists of four lines of model cells coupled uniformly to neighbors within the array and with some randomness to cells in other lamellae. All model cells operate according to first-order differential equations. Two of the lines of cells in each lamella are coupled such that sufficient excitation by a system input generates a wave of activity that travels down the lamella. Such waves effect dynamic storage of the representation of each input, allowing association connections to form that code both the set of cells stimulated by each input and the time interval between successive inputs. Results of simulation of two networks are presented illustrating the model's operating characteristics and memory capacity.     

From sensors to spikes: evolving receptive fields to enhance sensorimotor information in a robot-arm

In biological systems, instead of actual encoders at different joints, proprioception signals are acquired through distributed receptive fields. In robotics, a single and accurate sensor output per link (encoder) is commonly used to track the position and the velocity. Interfacing bio-inspired control systems with spiking neural networks emulating the cerebellum with conventional robots is not a straight forward task. Therefore, it is necessary to adapt this one-dimensional measure (encoder output) into a multidimensional space (inputs for a spiking neural network) to connect, for instance, the spiking cerebellar architecture; i.e. a translation from an analog space into a distributed population coding in terms of spikes. This paper analyzes how evolved receptive fields (optimized towards information transmission) can efficiently generate a sensorimotor representation that facilitates its discrimination from other "sensorimotor states". This can be seen as an abstraction of the Cuneate Nucleus (CN) functionality in a robot-arm scenario. We model the CN as a spiking neuron population coding in time according to the response of mechanoreceptors during a multi-joint movement in a robot joint space. An encoding scheme that takes into account the relative spiking time of the signals propagating from peripheral nerve fibers to second-order somatosensory neurons is proposed. Due to the enormous number of possible encodings, we have applied an evolutionary algorithm to evolve the sensory receptive field representation from random to optimized encoding. Following the nature-inspired analogy, evolved configurations have shown to outperform simple hand-tuned configurations and other homogenized configurations based on the solution provided by the optimization engine (evolutionary algorithm). We have used artificial evolutionary engines as the optimization tool to circumvent nonlinearity responses in receptive fields.     

Feedforward-Feedback Hybrid Control for Magnetic Shape Memory Alloy Actuators Based on the Krasnosel'skii-Pokrovskii Model

As a new type of smart material, magnetic shape memory alloy has the advantages of a fast response frequency and outstanding strain capability in the field of microdrive and microposition actuators. The hysteresis nonlinearity in magnetic shape memory alloy actuators, however, limits system performance and further application. Here we propose a feedforward-feedback hybrid control method to improve control precision and mitigate the effects of the hysteresis nonlinearity of magnetic shape memory alloy actuators. First, hysteresis nonlinearity compensation for the magnetic shape memory alloy actuator is implemented by establishing a feedforward controller which is an inverse hysteresis model based on Krasnosel''skii-Pokrovskii operator. Secondly, the paper employs the classical Proportion Integration Differentiation feedback control with feedforward control to comprise the hybrid control system, and for further enhancing the adaptive performance of the system and improving the control accuracy, the Radial Basis Function neural network self-tuning Proportion Integration Differentiation feedback control replaces the classical Proportion Integration Differentiation feedback control. Utilizing self-learning ability of the Radial Basis Function neural network obtains Jacobian information of magnetic shape memory alloy actuator for the on-line adjustment of parameters in Proportion Integration Differentiation controller. Finally, simulation results show that the hybrid control method proposed in this paper can greatly improve the control precision of magnetic shape memory alloy actuator and the maximum tracking error is reduced from 1.1% in the open-loop system to 0.43% in the hybrid control system.     

Transformation of Stimulus Correlations by the Retina

Redundancies and correlations in the responses of sensory neurons may seem to waste neural resources, but they can also carry cues about structured stimuli and may help the brain to correct for response errors. To investigate the effect of stimulus structure on redundancy in retina, we measured simultaneous responses from populations of retinal ganglion cells presented with natural and artificial stimuli that varied greatly in correlation structure; these stimuli and recordings are publicly available online. Responding to spatio-temporally structured stimuli such as natural movies, pairs of ganglion cells were modestly more correlated than in response to white noise checkerboards, but they were much less correlated than predicted by a non-adapting functional model of retinal response. Meanwhile, responding to stimuli with purely spatial correlations, pairs of ganglion cells showed increased correlations consistent with a static, non-adapting receptive field and nonlinearity. We found that in response to spatio-temporally correlated stimuli, ganglion cells had faster temporal kernels and tended to have stronger surrounds. These properties of individual cells, along with gain changes that opposed changes in effective contrast at the ganglion cell input, largely explained the pattern of pairwise correlations across stimuli where receptive field measurements were possible.     

Self-calibrated collinearity detector

The human visual system can make remarkably precise spatial judgements. There are reasons to believe that this accuracy is achieved and maintained by using processes that calibrate and correct errors in the system. This work investigate this problem of self-calibration and describes an adaptive system for detecting the collinearity of points and the straightness of lines. The system is initially inaccurate, but, by using an error correction mechanism, it eventually becomes highly accurate. The error correction is performed by a simple self calibration process named proportional multi-gain adjustment. The calibration process adjusts the gain values of the system input units. The process utilizes statistical regularities in the input stimuli. It compensate for errors due to noise in the input units receptive fields location and response functions by ensuring that the average deviation from collinearity offset detected by the system is zero. As a by product of the error correction, the system exhibits also adaptation and aftereffect phenomena, similar to those observed in the human visual system.