Recurrent fractal neural networks: a strategy for the exchange of local and global information processing in the brain

The regulation of biological networks relies significantly on convergent feedback signaling loops that render a global output locally accessible. Ideally, the recurrent connectivity within these systems is self-organized by a time-dependent phase-locking mechanism. This study analyzes recurrent fractal neural networks (RFNNs), which utilize a self-similar or fractal branching structure of dendrites and downstream networks for phase-locking of reciprocal feedback loops: output from outer branch nodes of the network tree enters inner branch nodes of the dendritic tree in single neurons. This structural organization enables RFNNs to amplify re-entrant input by over-the-threshold signal summation from feedback loops with equivalent signal traveling times. The columnar organization of pyramidal neurons in the neocortical layers V and III is discussed as the structural substrate for this network architecture. RFNNs self-organize spike trains and render the entire neural network output accessible to the dendritic tree of each neuron within this network. As the result of a contraction mapping operation, the local dendritic input pattern contains a downscaled version of the network output coding structure. RFNNs perform robust, fractal data compression, thus coping with a limited number of feedback loops for signal transport in convergent neural networks. This property is discussed as a significant step toward the solution of a fundamental problem in neuroscience: how is neuronal computation in separate neurons and remote brain areas unified as an instance of experience in consciousness? RFNNs are promising candidates for engaging neural networks into a coherent activity and provide a strategy for the exchange of global and local information processing in the human brain, thereby ensuring the completeness of a transformation from neuronal computation into conscious experience.     

BP神经网络在沙棘生态功能评价中的应用

BP人工神经网络技术在环境评价领域中已经得到越来越广泛的运用,将该法引入到陕蒙砒砂岩区沙棘生态功能综合评价的研究中,以沙棘生态功能评价指标标准值作为样本输入,综合评价级别作为网络输出,建立了一个含有4个输入神经元节点、6个隐含神经元节点和1个输出神经元节点的BP人工神经网络等级模型。将目标年(2008年)各评价指标实际数据作为输入,得到输出值是0.44,大于Ⅱ级标准,研究结果表明:砒砂岩区种植十年沙棘后,其生态效益很好,对砒砂岩地区的生态环境改善作用显著。BP神经网络的评价结果与较成熟的AHP-模糊综合评价结果一致,证明将BP人工神经网络模型用于沙棘生态功能评价是可行的,且评价结论客观。     

The application of an artificial neural network in the identification of medicinal rhubarbs by near-infrared spectroscopy

This paper describes a method to combine near-infrared spectroscopy and a three layer back-propagation artificial neural network in order to identify official and unofficial rhubarbs. Thirty-three samples were taken as the training set, and 62 samples as the test set. The effects of input node number, learning rate and momentum on the final error and recognition accuracy for the training set, and on prediction accuracy for the test set were determined. A neural network with eight input nodes, a 0.5 learning rate, and a momentum of 0.3 can achieve a recognition accuracy of 100% for the training set and a prediction accuracy of 96.8% for the test set. The method described offers a quick and efficient means of identifying rhubarbs.     

An improvement of extreme learning machine for compact single-hidden-layer feedforward neural networks

Recently, a novel learning algorithm called extreme learning machine (ELM) was proposed for efficiently training single-hidden-layer feedforward neural networks (SLFNs). It was much faster than the traditional gradient-descent-based learning algorithms due to the analytical determination of output weights with the random choice of input weights and hidden layer biases. However, this algorithm often requires a large number of hidden units and thus slowly responds to new observations. Evolutionary extreme learning machine (E-ELM) was proposed to overcome this problem; it used the differential evolution algorithm to select the input weights and hidden layer biases. However, this algorithm required much time for searching optimal parameters with iterative processes and was not suitable for data sets with a large number of input features. In this paper, a new approach for training SLFNs is proposed, in which the input weights and biases of hidden units are determined based on a fast regularized least-squares scheme. Experimental results for many real applications with both small and large number of input features show that our proposed approach can achieve good generalization performance with much more compact networks and extremely high speed for both learning and testing.     

Development of feature detectors by self-organization

We present a two-layered network of linear neurons that organizes itself as to extract the complete information contained in a set of presented patterns. The weights between layers obey a Hebbian rule. We propose a local anti-Hebbian rule for lateral, hierarchically organized weights within the output layer. This rule forces the activities of the output units to become uncorrelated and the lateral weights to vanish. The weights between layers converge to the eigenvectors of the covariance matrix of input patterns, i.e., the network performs a principal component analysis, yielding all principal components. As a consequence of the proposed learning scheme, the output units become detectors of orthogonal features, similar to ones found in the brain of mammals.     

A Model of Self-Organizing Head-Centered Visual Responses in Primate Parietal Areas

We present a hypothesis for how head-centered visual representations in primate parietal areas could self-organize through visually-guided learning, and test this hypothesis using a neural network model. The model consists of a competitive output layer of neurons that receives afferent synaptic connections from a population of input neurons with eye position gain modulated retinal receptive fields. The synaptic connections in the model are trained with an associative trace learning rule which has the effect of encouraging output neurons to learn to respond to subsets of input patterns that tend to occur close together in time. This network architecture and synaptic learning rule is hypothesized to promote the development of head-centered output neurons during periods of time when the head remains fixed while the eyes move. This hypothesis is demonstrated to be feasible, and each of the core model components described is tested and found to be individually necessary for successful self-organization.     

A neural network study on the dynamic identification of a fermentation system

The objective of this paper is to propose neural networks for the study of dynamic identification and prediction of a fermentation system which produces mainly 2,3-butanediol (2,3-BDL). The metabolic products of the fermentation, acetic acid, acetoin, ethanol, and 2,3-BDL were measured on-line via a mass spectrometer modified by the insertion of a dimethylvinylsilicone membrane probe. The measured data at different sampling times were included as the input and output nodes, at different learning batches, of the network. A fermentation system is usually nonlinear and dynamic in nature. Measured fermentation data obtained from the complex metabolic pathways are often difficult to be entirely included in a static process model, therefore, a dynamic model was suggested instead. In this work, neural networks were provided by a dynamic learning and prediction process that moved along the time sequence batchwise. In other words, a scheme of two-dimensional moving window (number of input nodes by the number of training data) was proposed for reading in new data while forgetting part of the old data. Proper size of the network including proper number of input/output nodes were determined by trained with the real-time fermentation data. Different number of hidden nodes under the consideration of both learning performance and computation efficiency were tested. The data size for each learning batch was determined. The performance of the learning factors such as the learning coefficient η and the momentum term coefficient α were also discussed. The effect of different dynamic learning intervals, with different starting points and the same ending point, both on the learning and prediction performance were studied. On the other hand, the effect of different dynamic learning intervals, with the same starting point and different ending points, was also investigated. The size of data sampling interval was also discussed. The performance from four different types of transfer functions, x/(1+|x|), sgn(x)·x ²/(1+x ²), 2/(1+e ^{? x} )?1, and 1/(1+e ^{? x} ) was compared. A scaling factor b was added to the transfer function and the effect of this factor on the learning was also evaluated. The prediction results from the time-delayed neural networks were also studied.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

基于BP神经网络与ETM+遥感数据的盐城滨海自然湿地覆被分类

高效而精确的湿地遥感分类是大范围湿地资源动态监测与管理的必要保障。本研究使用ETM 遥感数据,借助Matlab神经网络工具箱,构建了基于BP神经网络的滨海湿地覆被分类模型,并将其应用于江苏盐城沿海湿地珍禽国家级自然保护区的核心区的自然湿地覆被分类研究中。本研究选择3、4、7、8波段作为输入层变量,单隐藏层设为10个节点,输出层变量对应待划分的8种覆被类型,构建三层式BP神经网络滨海湿地覆被分类模型。结果显示,BP分类总精度为85.91%,Kappa系数为0.8328,与最小距离法和极大似然法的分类总精度相比,分别提高了7.99%和6.08%,Kappa系数也相比提高。研究结果表明,BP神经网络分类法是一种较为有效的湿地遥感影像分类技术,能够提高分类精度。     

Determination of compound aminopyrine phenacetin tablets by using artificial neural networks combined with principal components analysis

A method for simultaneous, nondestructive analysis of aminopyrine and phenacetin in compound aminopyrine phenacetin tablets with different concentrations has been developed by principal component artificial neural networks (PC-ANNs) on near-infrared (NIR) spectroscopy. In PC-ANN models, the spectral data were initially analyzed by principal component analysis. Then the scores of the principal components were chosen as input nodes for the input layer instead of the spectral data. The artificial neural network models using the spectral data as input nodes were also established and compared with the PC-ANN models. Four different preprocessing methods (first-derivative, second-derivative, standard normal variate (SNV), and multiplicative scatter correction) were applied to three sets of NIR spectra of compound aminopyrine phenacetin tablets. The PC-ANNs approach with SNV preprocessing spectra was found to provide the best results. The degree of approximation was performed as the selective criterion of the optimum network parameters.     

Computer simulations of synchrony and oscillations evoked by two coherent inputs

Coherent oscillations have been reported in multiple cortical areas. This study examines the characteristics of output spikes through computer simulations when the neural network model receives periodic/aperiodic spatiotemporal spikes with modulated/constant populational activity from two pathways. Synchronous oscillations which have the same period as the input are observed in response to periodic input patterns regardless of populational activity. The results confirm that the output frequency of synchrony is essentially determined by the period of the repeated input patterns. On the other hand, weak periodic outputs are observed when aperiodic spikes are input with modulated populational activity. In this case, higher firing rates are necessary to input for higher frequency oscillations. The spike-timing-dependent plasticity suppresses the spikes which do not contribute to the synchrony for periodic inputs. This effect corresponds to the experimental reports that learning sharpens the synchrony in the motor cortex. These results suggest that spatiotemporal spike patterns should be entrained on modulated populational activity to transmit oscillatory information effectively in the convergent pathway.     

Identification of <Emphasis Type="BoldItalic">b</Emphasis>-/<Emphasis Type="BoldItalic">y</Emphasis>-ions in MS/MS spectra using a two stage neural network

Independent of the approach used, the ability to correctly interpret tandem MS data depends on the quality of the original spectra. Even in the case of the highest quality spectra, the majority of spectral peaks can not be reliably interpreted. The accuracy of sequencing algorithms can be improved by filtering out such 'noise' peaks. Preprocessing MS/MS spectra to select informative ion peaks increases accuracy and reduces the processing time. Intuitively, the mix of informative versus non-informative peaks has a direct effect on the quality and size of the resulting candidate peptide search space. As the number of selected peaks increases, the corresponding search space increases exponentially. If we select too few peaks then the ion-ladder interpretation of the spectrum will contain gaps that can only be explained by permutations of combinations of amino acids. This will result in a larger candidate peptide search space and poorer quality candidates. The dependency that peptide sequencing accuracy has on an initial peak selection regime makes this preprocessing step a crucial facet of any approach, whether de novo or not, to MS/MS spectra interpretation.We have developed a novel approach to address this problem. Our approach uses a staged neural network to model ion fragmentation patterns and estimate the posterior probability of each ion type. Our method improves upon other preprocessing techniques and shows a significant reduction in the search space for candidate peptides without sacrificing candidate peptide quality.     

A multilayer neural network model for perception of rotational motion

A multilayer neural nerwork model for the perception of rotational motion has been developed usingReichardt's motion detector array of correlation type, Kohonen's self-organized feature map and Schuster-Wagner's oscillating neural network. It is shown that the unsupervised learning could make the neurons on the second layer of the network tend to be self-organized in a form resembling columnar organization of selective directions in area MT of the primate's visual cortex. The output layer can interpret rotation information and give the directions and velocities of rotational motion. The computer simulation results are in agreement with some psychophysical observations of rotation-al perception. It is demonstrated that the temporal correlation between the oscillating neurons would be powerful for solving the "binding problem" of shear components of rotational motion.     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Optimal learning with excitatory and inhibitory synapses

Characterizing the relation between weight structure and input/output statistics is fundamental for understanding the computational capabilities of neural circuits. In this work, I study the problem of storing associations between analog signals in the presence of correlations, using methods from statistical mechanics. I characterize the typical learning performance in terms of the power spectrum of random input and output processes. I show that optimal synaptic weight configurations reach a capacity of 0.5 for any fraction of excitatory to inhibitory weights and have a peculiar synaptic distribution with a finite fraction of silent synapses. I further provide a link between typical learning performance and principal components analysis in single cases. These results may shed light on the synaptic profile of brain circuits, such as cerebellar structures, that are thought to engage in processing time-dependent signals and performing on-line prediction.     

BP人工神经网络模拟杨树林冠蒸腾

利用2008和2010年的气温、饱和差、总辐射和叶面积指数作为模型输入,液流法观测的蒸腾速率作为模型输出,建立了用于杨树林冠蒸腾模拟的BP人工神经网络模型,利用2009年的观测数据对模型的模拟能力进行了检验,并应用连接权值计算得到的输入变量对输出变量的相对贡献进行了敏感性分析。结果表明:建立的BP人工神经网络蒸腾模型可以很好的模拟林冠蒸腾大小和季节变化,模拟的绝对误差和绝对相对误差的平均值分别为0.11 mm/d和9.5%,纳什效率系数为0.83;输入变量对蒸腾的相对贡献以及蒸腾与输入变量之间的相关性大小顺序相同,均为总辐射叶面积指数饱和差气温。     

Classification of pathogenic bacteria by Raman spectroscopy combined with variational auto-encoder and deep learning

Rapid and early identification of pathogens is critical to guide antibiotic therapy. Raman spectroscopy as a noninvasive diagnostic technique provides rapid and accurate detection of pathogens. Raman spectrum of single cells serves as the “fingerprint” of the cell, revealing its metabolic characteristics. Rapid identification of pathogens can be achieved by combining Raman spectroscopy and deep learning. Traditional classification techniques frequently require lots of data for training, which is time costing to collect Raman spectra. For trace samples and strains that are difficult to culture, it is difficult to provide an accurate classification model. In order to reduce the number of samples collected and improve the accuracy of the classification model, a new pathogen detection method integrating Raman spectroscopy, variational auto-encoder (VAE), and long short-term memory network (LSTM) is proposed in this paper. We collect the Raman signals of pathogens and input them to VAE for training. VAE will generate a large number of Raman spectral data that cannot be distinguished from the real spectrum, and the signal-to-noise ratio is higher than that of the real spectrum. These spectra are input into the LSTM together with the real spectrum for training, and a good classification model is obtained. The results of the experiments reveal that this method not only improves the average accuracy of pathogen classification to 96.9% but also reduces the number of Raman spectra collected from 1000 to 200. With this technology, the number of Raman spectra collected can be greatly reduced, so that strains that are difficult to culture or trace can be rapidly identified.     

A Network Model of Motion Processing in Area MT of Primates

A simple and biologically plausible model is proposed to simulatethe visual motion processing taking place in the middle temporal (MT) areaof the visual cortex in the primate brain. The model is ahierarchical neural network composed of multiple competitive learninglayers. The input layer of the network simulates the neurons in the primaryvisual cortex (V1), which are sensitive to the orientation and motionvelocity of the visual stimuli, and the middle and output layers of thenetwork simulate the component MT and pattern MT neurons, which areselectively responsive to local and global motions, respectively. Thenetwork model was tested with various simulated motion patterns (random dotsof different direction correlations, transparent motion, grating and plaidpatterns, and so on). The response properties of the model closely resemblemany of the known features of the MT neurons found neurophysiologically.These results show that the sophisticated response behaviors of the MTneurons can emerge naturally from some very simple models, such as acompetitive learning network.     

A graphical simulation model of the entire DNA process associated with the analysis of short tandem repeat loci

The use of expert systems to interpret short tandem repeat DNA profiles in forensic, medical and ancient DNA applications is becoming increasingly prevalent as high-throughput analytical systems generate large amounts of data that are time-consuming to process. With special reference to low copy number (LCN) applications, we use a graphical model to simulate stochastic variation associated with the entire DNA process starting with extraction of sample, followed by the processing associated with the preparation of a PCR reaction mixture and PCR itself. Each part of the process is modelled with input efficiency parameters. Then, the key output parameters that define the characteristics of a DNA profile are derived, namely heterozygote balance (Hb) and the probability of allelic drop-out p(D). The model can be used to estimate the unknown efficiency parameters, such as π_extraction. ‘What-if’ scenarios can be used to improve and optimize the entire process, e.g. by increasing the aliquot forwarded to PCR, the improvement expected to a given DNA profile can be reliably predicted. We demonstrate that Hb and drop-out are mainly a function of stochastic effect of pre-PCR molecular selection. Whole genome amplification is unlikely to give any benefit over conventional PCR for LCN.     

Real value prediction of solvent accessibility in proteins using multiple sequence alignment and secondary structure

The present study is an attempt to develop a neural network-based method for predicting the real value of solvent accessibility from the sequence using evolutionary information in the form of multiple sequence alignment. In this method, two feed-forward networks with a single hidden layer have been trained with standard back-propagation as a learning algorithm. The Pearson's correlation coefficient increases from 0.53 to 0.63, and mean absolute error decreases from 18.2 to 16% when multiple-sequence alignment obtained from PSI-BLAST is used as input instead of a single sequence. The performance of the method further improves from a correlation coefficient of 0.63 to 0.67 when secondary structure information predicted by PSIPRED is incorporated in the prediction. The final network yields a mean absolute error value of 15.2% between the experimental and predicted values, when tested on two different nonhomologous and nonredundant datasets of varying sizes. The method consists of two steps: (1) in the first step, a sequence-to-structure network is trained with the multiple alignment profiles in the form of PSI-BLAST-generated position-specific scoring matrices, and (2) in the second step, the output obtained from the first network and PSIPRED-predicted secondary structure information is used as an input to the second structure-to-structure network. Based on the present study, a server SARpred (http://www.imtech.res.in/raghava/sarpred/) has been developed that predicts the real value of solvent accessibility of residues for a given protein sequence. We have also evaluated the performance of SARpred on 47 proteins used in CASP6 and achieved a correlation coefficient of 0.68 and a MAE of 15.9% between predicted and observed values.