Partial retraining: a new approach to input relevance determination.

In this article we introduce partial retraining, an algorithm to determine the relevance of the input variables of a trained neural network. We place this algorithm in the context of other approaches to relevance determination. Numerical experiments on both artificial and real-world problems show that partial retraining outperforms its competitors, which include methods based on constant substitution, analysis of weight magnitudes, and "optimal brain surgeon".     

Synaptic plasticity and connectivity requirements to produce stimulus-pair specific responses in recurrent networks of spiking neurons

Animals must respond selectively to specific combinations of salient environmental stimuli in order to survive in complex environments. A task with these features, biconditional discrimination, requires responses to select pairs of stimuli that are opposite to responses to those stimuli in another combination. We investigate the characteristics of synaptic plasticity and network connectivity needed to produce stimulus-pair neural responses within randomly connected model networks of spiking neurons trained in biconditional discrimination. Using reward-based plasticity for synapses from the random associative network onto a winner-takes-all decision-making network representing perceptual decision-making, we find that reliably correct decision making requires upstream neurons with strong stimulus-pair selectivity. By chance, selective neurons were present in initial networks; appropriate plasticity mechanisms improved task performance by enhancing the initial diversity of responses. We find long-term potentiation of inhibition to be the most beneficial plasticity rule by suppressing weak responses to produce reliably correct decisions across an extensive range of networks.     

Neural network predicts sequence of TP53 gene based on DNA chip

We have trained an artificial neural network to predict the sequence of the human TP53 tumor suppressor gene based on a p53 GeneChip. The trained neural network uses as input the fluorescence intensities of DNA hybridized to oligonucleotides on the surface of the chip and makes between zero and four errors in the predicted 1300 bp sequence when tested on wild-type TP53 sequence. AVAILABILITY: The trained neural network is available for academic use by contacting steen@cbs.dtu.dk     

Development of predictive models of laboratory animal growth using artificial neural networks

Traditional regression analysis of body weight growth curvesencounters problems .when the data are extremely variable. Whiletransformations are often employed to meet the criteria of theanalysis, some transformations are inadequate for normalizingthe data. Regression analysis also requires presuppositionsregarding the model to be fit and the techniques to be usedin the analysis. An alternative approach using artificial neuralnetworks is presented which may be suitable for developing predictivemodels of growth. Neural networks are simulators of the processesthat occur in the biological brain during the learning process.They are trained on the data, developing the necessary algorithmswithin their internal architecture, and produce a predictivemodel based on the learned facts. A dataset of Sprague–Dawleyrat (Rattus norvegicus) weights is analyzed by both traditionalregression analysis and neural network training. Predictionsof body weight are made from both models. While both methodsproduce models that adequately predict the body weights, theneural network model is superior in that it combines accuracyand precision, being less influenced by longitudinal variabilityin the data. Thus, the neural network provides another toolfor researchers to analyze growth curve data.     

Training back-propagation neural networks to define and detect DNA-binding sites

A three layered back-propagation neural network was trained to recognize E. coli promoters of the 17 base spacing class. To this end, the network was presented with 39 promoter sequences and derivatives of those sequences as positive inputs; 60% A + T random sequences and sequences containing 2 promoter-down point mutations were used as negative inputs. The entire promoter sequence of 58 bases, approximately -50 to +8, was entered as input. The network was asked to associate an output of 1.0 with promoter sequence input and 0.0 with non-promoter input. Generally, after 100,000 input cycles, the network was virtually perfect in classifying the training set. A trained network was about 80% effective in recognizing 'new' promoters which were not in the training set, with a false positive rate below 0.1%. Network searches on pBR322 and on the lambda genome were also performed. Overall the results were somewhat better than the best rule-based procedures. The trained network can be analyzed both for its choice of base and relative weighting, positive and negative, at each position of the sequence. This method, which requires only appropriate input/output training pairs, can be used to define and search for any DNA regulatory sequence for which there are sufficient exemplars.     

Acoustical detection of coronary occlusions using neural networks

A nonlinear neural network classifier was applied to noninvasive acoustic detection of coronary artery disease; the classifier included a feature vector, derived from diastolic heart sounds, and a multi-layered network trained by the backpropagation. The feature vector is based on the linear prediction coefficients of the autoregressive method after an adaptive line enhancement method was used as the input pattern to the neural network. One hundred and twelve recordings (70 abnormal, 42 normal) were studied and the network was trained on a randomly chosen set of six abnormal and six normal patients. It was tested on a database consisting of 100 recordings to which it had not been exposed. The network correctly identified 50 of the 64 patients with coronary artery disease and 32 of the 36 patients without any coronary artery occlusions. These results showed that this neural network is capable of distinguishing normal patients from abnormal patients. In addition, the diagnostic capability of this approach is much better than any other available noninvasive approach.     

A note on the low-dimensional display of multivariate data using neural networks

A novel neural network technique has been proposed (Livingstone et al. J. Mol. Graphics 1991, 9, 115-118) which is useful for a low-dimensional display of multivariate data sets. The method makes use of the activity values of the hidden neurons in a trained three-layer feed-forward network to produce the low-dimensional display. It was claimed that in contrast to conventional techniques, such as principal components analysis or nonlinear mapping, this technique could be used also to reconstruct, from a given point in the low-dimensional display, the corresponding multivariate input vector via the completely known weight matrices of a suitably trained network. We show here that this claim is unjustified in this general form. When previously unknown, grossly different input vectors are presented to the trained network, they can occupy, for example, exactly the same point in the low-dimensional display which is occupied also by a given training vector, if certain linear relationships between the vector components are fulfilled. Thus, an infinite set of different linearly dependent input vectors is projected onto one single point in the low-dimensional display. Reconstruction of a multivariate vector, starting from this point in the low-dimensional display, is able to lead back to only one multivariate vector (in the example given, to the original training vector).     

Bioprocess model building using artificial neural networks

Artificial neural networks are made upon of highly interconnected layers of simple neuron-like nodes. The neurons act as non-linear processing elements within the network. An attractive property of artificial neural networks is that given the appropriate network topology, they are capable of learning and characterising non-linear functional relationships. Furthermore, the structure of the resulting neural network based process model may be considered generic, in the sense that little prior process knowledge is required in its determination. The methodology therefore provides a cost efficient and reliable process modelling technique. One area where such a technique could be useful is biotechnological systems. Here, for example, the use of a process model within an estimation scheme has long been considered an effective means of overcoming inherent on-line measurement problems. However, the development of an accurate process model is extremely time consuming and often results in a model of limited applicability. Artificial neural networks could therefore prove to be a useful model building tool when striving to improve bioprocess operability. Two large scale industrial fermentation systems have been considered as test cases; a fed-batch penicillin fermentation and a continuous mycelial fermentation. Both systems serve to demonstrate the utility, flexibility and potential of the artificial neural network approach to process modelling.     

Non-Linear Neuronal Responses as an Emergent Property of Afferent Networks: A Case Study of the Locust Lobula Giant Movement Detector

In principle it appears advantageous for single neurons to perform non-linear operations. Indeed it has been reported that some neurons show signatures of such operations in their electrophysiological response. A particular case in point is the Lobula Giant Movement Detector (LGMD) neuron of the locust, which is reported to locally perform a functional multiplication. Given the wide ramifications of this suggestion with respect to our understanding of neuronal computations, it is essential that this interpretation of the LGMD as a local multiplication unit is thoroughly tested. Here we evaluate an alternative model that tests the hypothesis that the non-linear responses of the LGMD neuron emerge from the interactions of many neurons in the opto-motor processing structure of the locust. We show, by exposing our model to standard LGMD stimulation protocols, that the properties of the LGMD that were seen as a hallmark of local non-linear operations can be explained as emerging from the dynamics of the pre-synaptic network. Moreover, we demonstrate that these properties strongly depend on the details of the synaptic projections from the medulla to the LGMD. From these observations we deduce a number of testable predictions. To assess the real-time properties of our model we applied it to a high-speed robot. These robot results show that our model of the locust opto-motor system is able to reliably stabilize the movement trajectory of the robot and can robustly support collision avoidance. In addition, these behavioural experiments suggest that the emergent non-linear responses of the LGMD neuron enhance the system''s collision detection acuity. We show how all reported properties of this neuron are consistently reproduced by this alternative model, and how they emerge from the overall opto-motor processing structure of the locust. Hence, our results propose an alternative view on neuronal computation that emphasizes the network properties as opposed to the local transformations that can be performed by single neurons.     

Functional nodes in dynamic neural networks for bioprocess modelling

This contribution presents a novel method for the direct integration of a-priori knowledge in a neural network and its application for the online determination of a secondary metabolite during industrial yeast fermentation. Hereby, existing system knowledge is integrated in an artificial neural network (ANN) by means of 'functional nodes'. A generalized backpropagation algorithm is presented. For illustration, a set of ordinary differential equations describing the diacetyl formation and degradation during the cultivation is incorporated in a functional node and integrated in a dynamic feedforward neural network in a hybrid manner. The results show that a hybrid modelling approach exploiting available a-priori knowledge and experimental data can considerably outperform a pure data-based modelling approach with respect to robustness, generalization and necessary amount of training data. The number of training sets were decreased by 50%, obtaining the same accuracy as in a conventional approach. All incorrect decisions, according to defined cost criteria obtained with the conventional ANN, were avoided.     

人工神经网络模型及其在农业和生态学研究中的应用

对于一些复杂的农业生态系统,人们对其生态过程了解较少,且这些系统的不确定性和模糊性较大,用传统的方法难以模拟这些系统的行为,神经网络模型因为能较精确地模拟这些系统的行为,而引起生态学者们的广泛兴趣。该文着重介绍了误差逆传神经网络模型的结构、算法及其在农业和生态学中的应用研究。误差逆传神经网络模型一般采用三层神经网络模型结构,三层的神经网络模型能模拟任意复杂程度的连续函数,而且因为它的结构小而不容易产生与训练数据的过度吻合。误差逆传神经网络模型算法的主要特征是:利用当前的输入误差对权值进行调整。在生态学和农业研究中,误差逆传神经网络模型通常作为非线性函数模拟器用于预测作物产量、生物生产量、生物与环境之间的关系等。已有的研究表明:误差逆传神经网络模型的模拟精度要远远高于多元线性方程,类似于非线性方程,而在样本量足够的情况下,有一定的外推能力。但是误差逆传神经网络模型需要大量的样本量来保证所求取参数的可靠性,但这在实际研究中很难做到,因而限制了误差逆传神经网络模型的应用。近年来人们提出了强制训练停止、复合模型等多种技术来提高误差逆传神经网络模型的外推能力,也提出了Garson算法、敏感性分析以及随机化检验等技术对误差逆传神经网络模型的机理进行解释。误差逆传神经网络模型的真正优势在于模拟人们了解较少或不确定性和模糊性较大系统的行为,这些是传统模型所无法实现的,因而是对传统机理模型的重要补充。     

Prediction of product formation in 2-keto-l-gulonic acid fermentation through Bayesian combination of multiple neural networks

As the key precursor for l-ascorbic acid synthesis, 2-keto-l-gulonic acid (2-KGA) is widely produced by the mixed culture of Bacillus megaterium and Ketogulonicigenium vulgare. In this study, a Bayesian combination of multiple neural networks is developed to obtain accurate prediction of the product formation. The historical batches are classified into three categories with a batch classification algorithm based on the statistical analysis of the product formation profiles. For each category, an artificial neural network is constructed. The input vector of the neural network consists of a series of time-discretized process variables. The output of the neural network is the predicted product formation. The training database for each neural network is composed of both the input–output data pairs from the historical bathes in the corresponding category, and all the available data pairs collected from the batch of present interest. The prediction of the product formation is practiced through a Bayesian combination of three trained neural networks. Validation was carried out in a Chinese pharmaceutical factory for 140 industrial batches in total, and the average root mean square error (RMSE) is 2.2% and 2.6% for 4 h and 8 h ahead prediction of product formation, respectively.     

Vision-based force measurement using neural networks for biological cell microinjection

This paper presents a vision-based force measurement method using an artificial neural network model. The proposed model is used for measuring the applied load to a spherical biological cell during micromanipulation process. The devised vision-based method is most useful when force measurement capability is required, but it is very challenging or even infeasible to use a force sensor. Artificial neural networks in conjunction with image processing techniques have been used to estimate the applied load to a cell. A bio-micromanipulation system capable of force measurement has also been established in order to collect the training data required for the proposed neural network model. The geometric characterization of zebrafish embryos membranes has been performed during the penetration of the micropipette prior to piercing. The geometric features are extracted from images using image processing techniques. These features have been used to describe the shape and quantify the deformation of the cell at different indentation depths. The neural network is trained by taking the visual data as the input and the measured corresponding force as the output. Once the neural network is trained with sufficient number of data, it can be used as a precise sensor in bio-micromanipulation setups. However, the proposed neural network model is applicable for indentation of any other spherical elastic object. The results demonstrate the capability of the proposed method. The outcomes of this study could be useful for measuring force in biological cell micromanipulation processes such as injection of the mouse oocyte/embryo.     

A Model of Self-Organizing Head-Centered Visual Responses in Primate Parietal Areas

We present a hypothesis for how head-centered visual representations in primate parietal areas could self-organize through visually-guided learning, and test this hypothesis using a neural network model. The model consists of a competitive output layer of neurons that receives afferent synaptic connections from a population of input neurons with eye position gain modulated retinal receptive fields. The synaptic connections in the model are trained with an associative trace learning rule which has the effect of encouraging output neurons to learn to respond to subsets of input patterns that tend to occur close together in time. This network architecture and synaptic learning rule is hypothesized to promote the development of head-centered output neurons during periods of time when the head remains fixed while the eyes move. This hypothesis is demonstrated to be feasible, and each of the core model components described is tested and found to be individually necessary for successful self-organization.     

Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks

The purpose of this study was to develop a method of classifying cancers to specific diagnostic categories based on their gene expression signatures using artificial neural networks (ANNs). We trained the ANNs using the small, round blue-cell tumors (SRBCTs) as a model. These cancers belong to four distinct diagnostic categories and often present diagnostic dilemmas in clinical practice. The ANNs correctly classified all samples and identified the genes most relevant to the classification. Expression of several of these genes has been reported in SRBCTs, but most have not been associated with these cancers. To test the ability of the trained ANN models to recognize SRBCTs, we analyzed additional blinded samples that were not previously used for the training procedure, and correctly classified them in all cases. This study demonstrates the potential applications of these methods for tumor diagnosis and the identification of candidate targets for therapy.     

Prediction of transmembrane regions of beta-barrel proteins using ANN- and SVM-based methods

This article describes a method developed for predicting transmembrane beta-barrel regions in membrane proteins using machine learning techniques: artificial neural network (ANN) and support vector machine (SVM). The ANN used in this study is a feed-forward neural network with a standard back-propagation training algorithm. The accuracy of the ANN-based method improved significantly, from 70.4% to 80.5%, when evolutionary information was added to a single sequence as a multiple sequence alignment obtained from PSI-BLAST. We have also developed an SVM-based method using a primary sequence as input and achieved an accuracy of 77.4%. The SVM model was modified by adding 36 physicochemical parameters to the amino acid sequence information. Finally, ANN- and SVM-based methods were combined to utilize the full potential of both techniques. The accuracy and Matthews correlation coefficient (MCC) value of SVM, ANN, and combined method are 78.5%, 80.5%, and 81.8%, and 0.55, 0.63, and 0.64, respectively. These methods were trained and tested on a nonredundant data set of 16 proteins, and performance was evaluated using "leave one out cross-validation" (LOOCV). Based on this study, we have developed a Web server, TBBPred, for predicting transmembrane beta-barrel regions in proteins (available at http://www.imtech.res.in/raghava/tbbpred).     

芽胞气溶胶表面滞留抗力预测模型研究

针对生物威胁的现场处置工作,建立气溶胶芽胞表面滞留抗力的智能预测模型,以准确预测环境表面芽胞污染状况,为大规模的现场洗消任务提供重要依据,有利于实现及时反应、恰当反应和准确防护的目标。以枯草杆菌芽胞为试验菌,在气溶胶实验室进行芽胞的环境因素暴露及活力测定,以模拟环境中芽胞抗力变化规律数据为依据,采用Matlab6.1软件包中的神经网络工具箱进行抗力预测模型研究。根据研究目的、模拟环境条件和数据训练的平滑曲线等特征,设定了5个输入神经元,8个隐层节点和1个输出神经元。‘tansig’、‘purelin’为传递函数,trainlm为训练函数,网络迭代100次。模型回顾预测效率达到100%,前瞻预测效率达到91%。以实验室数据为依据,利用Matlab平台中的BP神经网络建立的芽胞气溶胶表面滞留抗力预测模型能利用环境因素信息有效预测芽胞抗力。     

An extended model for a spiking neuron class

This paper proposes an extension to the model of a spiking neuron for information processing in artificial neural networks, developing a new approach for the dynamic threshold of the integrate-and-fire neuron. This new approach invokes characteristics of biological neurons such as the behavior of chemical synapses and the receptor field. We demonstrate how such a digital model of spiking neurons can solve complex nonlinear classification with a single neuron, performing experiments for the classical XOR problem. Compared with rate-coded networks and the classical integrate-and-fire model, the trained network demonstrated faster information processing, requiring fewer neurons and shorter learning periods. The extended model validates all the logic functions of biological neurons when such functions are necessary for the proper flow of binary codes through a neural network.     

Artificial intelligence based optimization of exocellular glucansucrase production from Leuconostoc dextranicum NRRL B-1146

Two different artificial intelligence techniques namely artificial neural network (ANN) and genetic algorithm (GA) were integrated for optimizing fermentation medium for the production of glucansucrase. The experimental data reported in a previous study were used to build the neural network. The ANN was trained using the back propagation algorithm. The ANN predicted values showed good agreement with the experimentally reported ones from a response surface based experiment. The concentrations of three medium components: viz Tween 80, sucrose and K(2)HPO(4) served as inputs to the neural network model and the enzyme activity as the output of the model. A model was generated with a coefficient of correlation (R(2)) of 1.0 for the training set and 0.90 for the test data. A genetic algorithm was used to optimize the input space of the neural network model to find the optimum settings for maximum enzyme activity. This artificial neural network supported genetic algorithm predicted a maximum glucansucrase activity of 6.92U/ml at medium composition of 0.54% (v/v) Tween 80, 5.98% (w/v) sucrose and 1.01% (w/v) K(2)HPO(4). ANN-GA predicted model gave a 6.0% increase of enzyme activity over the regression based prediction for optimized enzyme activity. The maximum enzyme activity experimentally obtained using the ANN-GA designed medium was 6.75+/-0.09U/ml which was in good agreement with the predicted value.