Fitting of dynamic recurrent neural network models to sensory stimulus-response data

We present a theoretical study aiming at model fitting for sensory neurons. Conventional neural network training approaches are not applicable to this problem due to lack of continuous data. Although the stimulus can be considered as a smooth time-dependent variable, the associated response will be a set of neural spike timings (roughly the instants of successive action potential peaks) that have no amplitude information. A recurrent neural network model can be fitted to such a stimulus-response data pair by using the maximum likelihood estimation method where the likelihood function is derived from Poisson statistics of neural spiking. The universal approximation feature of the recurrent dynamical neuron network models allows us to describe excitatory-inhibitory characteristics of an actual sensory neural network with any desired number of neurons. The stimulus data are generated by a phased cosine Fourier series having a fixed amplitude and frequency but a randomly shot phase. Various values of amplitude, stimulus component size, and sample size are applied in order to examine the effect of the stimulus to the identification process. Results are presented in tabular and graphical forms at the end of this text. In addition, to demonstrate the success of this research, a study involving the same model, nominal parameters and stimulus structure, and another study that works on different models are compared to that of this research.     

Artificial neural networks and their use in quantitative pathology

A brief general introduction to artificial neural networks is presented, examining in detail the structure and operation of a prototype net developed for the solution of a simple pattern recognition problem in quantitative pathology. The process by which a neural network learns through example and gradually embodies its knowledge as a distributed representation is discussed, using this example. The application of neurocomputer technology to problems in quantitative pathology is explored, using real-world and illustrative examples. Included are examples of the use of artificial neural networks for pattern recognition, database analysis and machine vision. In the context of these examples, characteristics of neural nets, such as their ability to tolerate ambiguous, noisy and spurious data and spontaneously generalize from known examples to handle unfamiliar cases, are examined. Finally, the strengths and deficiencies of a connectionist approach are compared to those of traditional symbolic expert system methodology. It is concluded that artificial neural networks, used in conjunction with other nonalgorithmic artificial intelligence techniques and traditional algorithmic processing, may provide useful software engineering tools for the development of systems in quantitative pathology.     

The development of specific visual connections in the monkey and the goldfish: outline of a geometric theory of receptotopic structure.

The anatomical structure of the primate retino-striate system and the goldfish retino-tectal system are characterized by idealized geometrical domains. The physiological retinotopic mappings are then shown to be determined by the boundary conditions of the respective anatomical surfaces. This fact is interpreted as support for the “systems-matching” hypothesis of neural development of Gaze &; Keating. It is suggested that the development of specific neural mappings may be, in part, a variational problem in which two neural surfaces establish connections as “smoothly” as possible. Dirichlet's Principle supplies a quantitative definition of the term “smooth”—the average physiological magnification factor of the neural mapping is minimized, subject to the boundary conditions of the available tissue. Three developmental rules are formulated, which deal respectively with gross specificity, polarity, and detailed map construction. The latter rule, based on Dirichlet's Principle, supplies a link between the classical theory of fields and developmental neurobiology. Specific experimental tests are outlined in the goldfish visual system, and a general discussion of global approaches to neural structure and function is presented.     

Prioritizing candidate peptides for cancer vaccines through predicting peptide presentation by HLA-I proteins

Cancer (treatment) vaccines that are made of neoantigens, or peptides unique to tumor cells due to somatic mutations, have emerged as a promising method to reinvigorate the immune response against cancer. A key step to prioritizing neoantigens for cancer vaccines is computationally predicting which neoantigens are presented on the cell surface by a human leukocyte antigen (HLA). We propose to address this challenge by training a neural network using mass spectrometry (MS) data composed of peptides presented by at least one of several HLAs of a subject. We embed the neural network within a mixture model and train the neural network by maximizing the likelihood of the mixture model. After evaluating our method using data sets where the peptide presentation status was known, we applied it to analyze somatic mutations of 60 melanoma patients and identified a group of neoantigens more immunogenic in tumor cells than in normal cells. Moreover, neoantigen burden estimated by our method was significantly associated with a measurement of the immune system activity, suggesting these neoantigens could induce an immune response.     

一类求解约束非线性规划问题的神经网络模型

提出一类求解闭凸集上非线性规划问题的神经网络模型。理论分析和计算机模拟表明在适当的假设下所提出的神经网络模型大范围指数级收敛于非线性规划问题的解集。本文神经网络所采用的方法属于广义的最速下降法,甚至当规划问题地正定二次时,本文的模型也比已有的神经网络模型简单。     

应用神经网络和多元回归技术预测森林产量

应用传统统计技术常会因样本小和测量数据不符某种分布而受到限制。本文评价一种前馈型神经网络算法以预测落叶阔叶林产量。另外,还介绍一种由定性变为定量的数据变换方法,以用相对小的样本建立多元回归预测模型。数据变换方法有助于改善多元回归模型的预测效果。在本实验的条件下,研究结果表明神经网络技术能够产生最好的预测效果.     

A neural network methodology of quadratic optimization.

According to the basic optimization principle of artificial neural networks, a novel kind of neural network model for solving the quadratic programming problem is presented. The methodology is based on the Lagrange multiplier theory in optimization and seeks to provide solutions satisfying the necessary conditions of optimality. The equilibrium point of the network satisfies the Kuhn-Tucker condition for the problem. The stability and convergency of the neural network is investigated and the strategy of the neural optimization is discussed. The feasibility of the neural network method is verified with the computation examples. Results of the simulation of the neural network to solve optimum problems are presented to illustrate the computational power of the neural network method.     

Incorporating anatomically realistic cellular-level connectivity in neural network models of the rat hippocampus

The specific connectivity patterns among neuronal classes can play an important role in the regulation of firing dynamics in many brain regions. Yet most neural network models are built based on vastly simplified connectivity schemes that do not accurately reflect the biological complexity. Taking the rat hippocampus as an example, we show here that enough quantitative information is available in the neuroanatomical literature to construct neural networks derived from accurate models of cellular connectivity. Computational simulations based on this approach lend themselves to a direct investigation of the potential relationship between cellular connectivity and network activity. We define a set of fundamental parameters to characterize cellular connectivity, and are collecting the related values for the rat hippocampus from published reports. Preliminary simulations based on these data uncovered a novel putative role for feedforward inhibitory neurons. In particular, "mopp" cells in the dentate gyrus are suitable to help maintain the firing rate of granule cells within physiological levels in response to a plausibly noisy input from the entorhinal cortex. The stabilizing effect of feedforward inhibition is further shown to depend on the particular ratio between the relative threshold values of the principal cells and the interneurons. We are freely distributing the connectivity data on which this study is based through a publicly accessible web archive (http://www.krasnow.gmu.edu/L-Neuron).     

Optimization of fed-batch bioreactor using neural network model

An algorithm using feedforward neural network model for determining optimal substrate feeding policies for fed-batch fermentation process is presented in this work. The algorithm involves developing the neural network model of the process using the sampled data. The trained neural network model in turn is used for optimization purposes. The advantages of this technique is that optimization can be achieved without detailed kinetic model of the process and the computation of gradient of objective function with respect to control variables is straightforward. The application of the technique is demonstrated with two examples, namely, production of secreted protein and invertase. The simulation results show that the discrete-time dynamics of fed-batch bioreactor can be satisfactorily approximated using a feedforward sigmoidal neural network. The optimal policies obtained with the neural network model agree reasonably well with the previously reported results.     

How robust are neural network models of stimulus generalization?

Artificial feed-forward neural networks are commonly used as a tool for modelling stimulus selection and animal signalling. A key finding of stimulus selection research has been generalization: if a given behaviour has been established to one stimulus, perceptually similar novel stimuli are likely to induce a similar response. Stimulus generalization, in feed-forward neural networks, automatically arises as a property of the network. This network property raises understandable concern regarding the sensitivity of the network to variation in its internal parameter values used in relation to its structure and to its training process. Researchers must have confidence that the predictions of their model follow from the underlying biology that they deliberately incorporated in the model, and not from often arbitrary choices about model implementation. We study how network training and parameter perturbations influence the qualitative and quantitative behaviour of a simple but general network. Specifically, for models of stimulus control we study the effect that parameter variation has on the shape of the generalization curves produced by the network. We show that certain network and training conditions produce undesirable artifacts that need to be avoided (or at least understood) when modelling stimulus selection.     

在体神经元胞外记录用于大脑皮层可塑性研究

神经元网络可塑性是大脑学习和记忆功能的基础,可塑性的变化也是某些脑功能疾病的成因。研究大脑皮层可塑性不仅可以为认识可塑性机制提供基本方法,也可对自然衰老过程和神经退行性疾病的病理过程进行观测,进而可以为评价抗衰老药物和治疗神经退行性疾病提供新方法。本文基于经典的大鼠胡须配对模型建立了一套实验方案,通过在体细胞外记录实验的数据分析,比较修剪胡须后相同时间内神经元感受野不对称变化程度的差异,衡量不同生理条件下大鼠体感皮层神经元网络可塑性。本文以中年和青年大鼠体感皮层神经元网络可塑性比较为例,详细介绍了实验方法中的关键技术和操作,如皮层D2功能柱的定位和D2功能柱内不同层神经元的定位等,结果和我室以前相关研究证明了此实验方案的可行性。     

Study of a Bionic Pattern Classifier Based on Olfactory Neural System

Simulating biological olfactory neural system, KIII network, which is a high-dimensional chaotic neural network, is designed in this paper. Different from conventional artificial neural network, the KⅢ network works in its chaotic trajectory. It can simulate not only the output EEG waveform observed in electrophysiological experiments, but also the biological intelligence for pattern classification. The simulation analysis and application to the recognition of handwriting numerals are presented here. The classification performance of the KⅢ network at different noise levels was also investigated.     

Adaptively timed conditioned responses and the cerebellum: A neural network approach

Conditioned responses often reflect knowledge about the timing of a US. This knowledge is manifested in the dependance of response topography on the CS-US interval employed in training. A neural network model and set of learning rules capable of simulating temporally adaptive features of conditioned responses is reviewed, and simulations are presented. In addition, we present a neural network implementation of the model which is designed to reconcile empirical studies of long-term synaptic depression in the cerebellum with neurobiological evidence from studies of the classically conditioned nictitating membrane response of the rabbit.     

BP人工神经网络在光谱定量预测中的应用

人工神经网络是模仿大脑神经元网络结构和功能而建立的一种信息处理系统,广泛的应用于各种波谱数据处理。误差反向传播多层前馈式网络(back-propagation network,简称BP网络)应用最广,发展最为迅速。将BP神经网络用于紫外-可见吸收光谱和拉曼光谱数据的定量分析和预测,与原文的一元线性回归模型数据处理方法相比,获得了比较满意的预测结果,预测精度有显著提高。这为相关的光谱分析和数据处理提供了一种更有效、更精确的方法。     

An Equivalent Tree Network Methodology for Efficient Utilization of Front-Ends in Linear Network

In this paper, divisible load scheduling in a linear network of processors is presented. The cases of processing load originating at the boundary and also at the interior of the network are considered. An equivalent tree network for the given linear network is derived. Using this equivalent tree network, we prove all the results obtained in the earlier studies. The equivalent tree network methodology presented in this paper, is more general than the earlier results, because in this approach, we can solve the scheduling problem even in an hetrogeneous linear network. The earlier studies considered only homogeneous linear network.     

A biologically inspired neural network for dynamic programming

An artificial neural network with a two-layer feedback topology and generalized recurrent neurons, for solving nonlinear discrete dynamic optimization problems, is developed. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. The neural network based algorithm is an advantageous approach for dynamic programming due to the inherent parallelism of the neural networks; further it reduces the severity of computational problems that can occur in methods like conventional methods. Some illustrative application examples are presented to show how this approach works out including the shortest path and fuzzy decision making problems.     

A self-organizing neural network with a function of associative memory: Feedback-type cognitron

Previously, one of the authors proposed a new hypothesis on the organization of synaptic connections, and constructed a model of self-organizing multi-layered neural network cognitron (Fukushima, 1975). the cognitron consists of a number of neural layers with similar structure connected in a cascade one after another. We have modified the structure of the cognitron, and have developed a new network having an ability of associative memory. The new network, named a feedback-type cognitron, has not only the feedforward connections as in the conventional cognitron, but also modifiable feedback connections from the last-layer cells to the front-layer ones. This network has been simulated on a digital computer. If several stimulus patterns are repeatedly presented to the network, the interconnections between the cells are gradually organized. The feedback connections, as well as the conventional feedforward ones, are self-organized depending on the characteristies of the externally presented stimulus patterns. After adequate number of stimulus presentations, each cell usually acquires the selective responsiveness to one of the stimulus patterns which have been frequently given. That is, every different stimulus pattern becomes to elicit an individual response to the network. After the completion of the self-organization, several stimulus patterns are presented to the network, and the responses are observed. Once a stimulus is given to the network, the signal keeps circulating in the network even after cutting off the stimulus, and the response gradually changes. Even though an imperfect or an ambiguous pattern is presented, the response usually converges to one of the patterns which have been frequently given during the process of self-organization. In some cases, however, a new pattern which has never been presented before, emerges. It is seen that this feedback-type cognitron has characteristics quite similar to some functions of the brain, such as the associative recall of memory, or the creation of a new idea by intuition.     

Markovian neural networks

The neural network that efficiently and nearly optimally solves difficult optimization problems is defined. The convergence proof for the Markovian neural network that asynchronously updates its neurons' states is also presented. The comparison of the performance of the Markovian neural network with various combinatorial optimization methods in two domains is described. The Markovian neural network is shown to be an efficient tool for solving optimization problems.     

Energy coding in neural network with inhibitory neurons

This paper aimed at assessing and comparing the effects of the inhibitory neurons in the neural network on the neural energy distribution, and the network activities in the absence of the inhibitory neurons to understand the nature of neural energy distribution and neural energy coding. Stimulus, synchronous oscillation has significant difference between neural networks with and without inhibitory neurons, and this difference can be quantitatively evaluated by the characteristic energy distribution. In addition, the synchronous oscillation difference of the neural activity can be quantitatively described by change of the energy distribution if the network parameters are gradually adjusted. Compared with traditional method of correlation coefficient analysis, the quantitative indicators based on nervous energy distribution characteristics are more effective in reflecting the dynamic features of the neural network activities. Meanwhile, this neural coding method from a global perspective of neural activity effectively avoids the current defects of neural encoding and decoding theory and enormous difficulties encountered. Our studies have shown that neural energy coding is a new coding theory with high efficiency and great potential.     

Prediction of Michaelis-Menten constant of beta-glucosidases using nitrophenyl-beta-D-glucopyranoside as substrate

In this study, we attempted to use the neural network to model a quantitative structure-K(m) (Michaelis-Menten constant) relationship for beta-glucosidase, which is an important enzyme to cut the beta-bond linkage in glucose while K(m) is a very important parameter in enzymatic reactions. Eight feedforward backpropagation neural networks with different layers and neurons were applied for the development of predictive model, and twenty-five different features of amino acids were chosen as predictors one by one. The results show that the 20-1 feedforward backpropagation neural network can serve as a predictive model while the normalized polarizability index as well as the amino-acid distribution probability can serve as the predictors. This study threw lights on the possibility of predicting the K(m) in beta-glucosidases based on their amino-acid features.