Selecting neural networks for a committee decision

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. In contrast to the ordinary approach of utilizing all neural networks available to make a committee decision, we propose creating adaptive committees, which are specific for each input data point. A prediction network is used to identify classification neural networks to be fused for making a committee decision about a given input data point. The jth output value of the prediction network expresses the expectation level that the jth classification neural network will make a correct decision about the class label of a given input data point. The proposed technique is tested in three aggregation schemes, namely majority vote, averaging, and aggregation by the median rule and compared with the ordinary neural networks fusion approach. The effectiveness of the approach is demonstrated on two artificial and three real data sets.     

Backpropagation neural network predictive control and control scheme comparison of 2,3-butanediol fermentation by Klebsiella oxytoca

A backpropagation neural network (BPN) was applied for the control study of 2,3-butanediol fermentation (2,3-BDL) carried by Klebsiella oxytoca. The measurements of cell mass and glucose were not included in the network models, instead, only the on-line measured product concentrations from the MIMS (membrane introduction mass spectrometer) were involved. Oxygen composition was chosen to be the control variable for this fermentation system for the formation of 2,3-BDL is regulated by oxygen. Oxygen composition was directly correlated to the measured product concentrations. A two-dimensional (number of input nodes by number of data sets) moving window to supply data for on-line, dynamic learning of this fermentation system was applied. The input nodes of the networks were also properly selected. Two neural network control schemes for this 2,3-BDL fermentation were discussed and compared in this work. Fermentations often exist time delay due to the measurement and their slow reaction nature. Hence, the order of time delay for the network controller was also investigated.     

Ozone prediction on the basis of neural networks,support vector regression and methods with uncertainty

The article presents modeling of daily average ozone level prediction by means of neural networks, support vector regression and methods based on uncertainty. Based on data measured by a monitoring station of the Pardubice micro-region, the Czech Republic, and optimization of the number of parameters by a defined objective function and genetic algorithm a model of daily average ozone level prediction in a certain time has been designed. The designed model has been optimized in light of its input parameters. The goal of prediction by various methods was to compare the results of prediction with the aim of various recommendations to micro-regional public administration management. It is modeling by means of feed-forward perceptron type neural networks, time delay neural networks, radial basis function neural networks, ε-support vector regression, fuzzy inference systems and Takagi–Sugeno intuitionistic fuzzy inference systems. Special attention is paid to the adaptation of the Takagi–Sugeno intuitionistic fuzzy inference system and adaptation of fuzzy logic-based systems using evolutionary algorithms. Based on data obtained, the daily average ozone level prediction in a certain time is characterized by a root mean squared error. The best possible results were obtained by means of an ε-support vector regression with polynomial kernel functions and Takagi–Sugeno intuitionistic fuzzy inference systems with adaptation by means of a Kalman filter.     

深度学习在药物活性预测研究中的应用

药物从研发到临床应用需要耗费较长的时间，研发期间的投入成本可高达十几亿元。而随着医药研发与人工智能的结合以及生物信息学的飞速发展，药物活性相关数据急剧增加，传统的实验手段进行药物活性预测已经难以满足药物研发的需求。借助算法来辅助药物研发，解决药物研发中的各种问题能够大大推动药物研发进程。传统机器学习方法尤其是随机森林、支持向量机和人工神经网络在药物活性方面能够达到较高的预测精度。深度学习由于具有多层神经网络，模型可以接收高维的输入变量且不需要人工限定数据输入特征，可以拟合较为复杂的函数模型，应用于药物研发可以进一步提高各个环节的效率。在药物活性预测中应用较为广泛的深度学习模型主要是深度神经网络（deep neural networks，DNN）、循环神经网络（recurrent neural networks，RNN）和自编码器（auto encoder，AE），而生成对抗网络（generative adversarial networks，GAN）由于其生成数据的能力常常被用来和其他模型结合进行数据增强。近年来深度学习在药物分子活性预测方面的研究和应用综述表明，深度学习模型的准确度和效率均高于传统实验方法和传统机器学习方法。因此，深度学习模型有望成为药物研发领域未来十年最重要的辅助计算模型。     

Inference of biological pathway from gene expression profiles by time delay boolean networks

One great challenge of genomic research is to efficiently and accurately identify complex gene regulatory networks. The development of high-throughput technologies provides numerous experimental data such as DNA sequences, protein sequence, and RNA expression profiles makes it possible to study interactions and regulations among genes or other substance in an organism. However, it is crucial to make inference of genetic regulatory networks from gene expression profiles and protein interaction data for systems biology. This study will develop a new approach to reconstruct time delay Boolean networks as a tool for exploring biological pathways. In the inference strategy, we will compare all pairs of input genes in those basic relationships by their corresponding [Formula: see text]-scores for every output gene. Then, we will combine those consistent relationships to reveal the most probable relationship and reconstruct the genetic network. Specifically, we will prove that [Formula: see text] state transition pairs are sufficient and necessary to reconstruct the time delay Boolean network of [Formula: see text] nodes with high accuracy if the number of input genes to each gene is bounded. We also have implemented this method on simulated and empirical yeast gene expression data sets. The test results show that this proposed method is extensible for realistic networks.     

Prediction of continuous B-cell epitopes in an antigen using recurrent neural network

B-cell epitopes play a vital role in the development of peptide vaccines, in diagnosis of diseases, and also for allergy research. Experimental methods used for characterizing epitopes are time consuming and demand large resources. The availability of epitope prediction method(s) can rapidly aid experimenters in simplifying this problem. The standard feed-forward (FNN) and recurrent neural network (RNN) have been used in this study for predicting B-cell epitopes in an antigenic sequence. The networks have been trained and tested on a clean data set, which consists of 700 non-redundant B-cell epitopes obtained from Bcipep database and equal number of non-epitopes obtained randomly from Swiss-Prot database. The networks have been trained and tested at different input window length and hidden units. Maximum accuracy has been obtained using recurrent neural network (Jordan network) with a single hidden layer of 35 hidden units for window length of 16. The final network yields an overall prediction accuracy of 65.93% when tested by fivefold cross-validation. The corresponding sensitivity, specificity, and positive prediction values are 67.14, 64.71, and 65.61%, respectively. It has been observed that RNN (JE) was more successful than FNN in the prediction of B-cell epitopes. The length of the peptide is also important in the prediction of B-cell epitopes from antigenic sequences. The webserver ABCpred is freely available at www.imtech.res.in/raghava/abcpred/.     

A neural network for processing olfactory-like stimuli

Several critical issues associated with the processing of olfactory stimuli in animals (but focusing on insects) are discussed with a view to designing a neural network which can process olfactory stimuli. This leads to the construction of a neural network that can learn and identify the quality (direction cosines) of an input vector or extract information from a sequence of correlated input vectors, where the latter corresponds to sampling a time varying olfactory stimulus (or other generically similar pattern recognition problems). The network is constructed around a discrete time content-addressable memory (CAM) module which basically satisfies the Hopfield equations with the addition of a unit time delay feedback. This modification improves the convergence properties of the network and is used to control a switch which activates the learning or template formation process when the input is “unknown”. The network dynamics are embedded within a sniff cycle which includes a larger time delay (i.e. an integert _s <1) that is also used to control the template formation switch. In addition, this time delay is used to modify the input into the CAM module so that the more dominant of two mingling odors or an odor increasing against a background of odors is more readily identified. The performance of the network is evaluated using Monte Carlo simulations and numerical results are presented.     

基于神经网络和遗传算法的木糖醇发酵培养基优化研究

发酵过程机理复杂、影响因素众多。菌种的生理生化特性及发酵的工艺确定之后 ,适宜的培养基配方成了发酵水平、原料成本高低的决定因素。为了优化培养基配方 ,采用遗传算法是一种行之有效的方法。遗传算法 (ＧＡ)是基于达尔文进化论和孟德尔遗传学说来实现随机、自适应、并行性全局搜索的一种无须数学模型的优化算法。与其它搜索方法相比 ,ＧＡ的优越性主要有 :(1)在搜索过程中ＧＡ不易陷入局部最优 ,即使所定义的目标函数非连续、不规则或伴有噪声 ,它也能以很大的概率找到全局最优解 ;(2 )由于ＧＡ固有的并行性 ,使得它非常适合于大规模并…     

Accuracy and response-time distributions for decision-making: linear perfect integrators versus nonlinear attractor-based neural circuits

Animals choose actions based on imperfect, ambiguous data. “Noise” inherent in neural processing adds further variability to this already-noisy input signal. Mathematical analysis has suggested that the optimal apparatus (in terms of the speed/accuracy trade-off) for reaching decisions about such noisy inputs is perfect accumulation of the inputs by a temporal integrator. Thus, most highly cited models of neural circuitry underlying decision-making have been instantiations of a perfect integrator. Here, in accordance with a growing mathematical and empirical literature, we describe circumstances in which perfect integration is rendered suboptimal. In particular we highlight the impact of three biological constraints: (1) significant noise arising within the decision-making circuitry itself; (2) bounding of integration by maximal neural firing rates; and (3) time limitations on making a decision. Under conditions (1) and (2), an attractor system with stable attractor states can easily best an integrator when accuracy is more important than speed. Moreover, under conditions in which such stable attractor networks do not best the perfect integrator, a system with unstable initial states can do so if readout of the system’s final state is imperfect. Ubiquitously, an attractor system with a nonselective time-dependent input current is both more accurate and more robust to imprecise tuning of parameters than an integrator with such input. Given that neural responses that switch stochastically between discrete states can “masquerade” as integration in single-neuron and trial-averaged data, our results suggest that such networks should be considered as plausible alternatives to the integrator model.     

Feature decomposition architectures for neural networks: algorithms, error bounds, and applications

In recent years, systems consisting of multiple modular neural networks have attracted substantial interest in the neural networks community because of various advantages they offer over a single large monolithic network. In this paper, we propose two basic feature decomposition models (namely, parallel model and tandem model) in which each of the neural network modules processes a disjoint subset of the input features. A novel feature decomposition algorithm is introduced to partition the input space into disjoint subsets solely based on the available training data. Under certain assumptions, the approximation error due to decomposition can be proved to be bounded by any desired small value over a compact set. Finally, the performance of feature decomposition networks is compared with that of a monolithic network in real world bench mark pattern recognition and modeling problems.     

Predictive modeling of competitive biosorption equilibrium data

This paper compares regression and neural network modeling approaches to predict competitive biosorption equilibrium data. The regression approach is based on the fitting of modified Langmuir-type isotherm models to experimental data. Neural networks, on the other hand, are non-parametric statistical estimators capable of identifying patterns in data and correlations between input and output. Our results show that the neural network approach outperforms traditional regression-based modeling in correlating and predicting the simultaneous uptake of copper and cadmium by a microbial biosorbent. The neural network is capable of accurately predicting unseen data when provided with limited amounts of data for training. Because neural networks are purely data-driven models, they are more suitable for obtaining accurate predictions than for probing the physical nature of the biosorption process.     

人工神经网络模型在水稻群体分蘖动态模拟中外推能力的测试

研究利用人工神经网络模型 ,以水稻群体分蘖动态为例 ,采用交互验证和独立验证的方式 ,对水稻生长 BP网络模型进行了训练与模拟 ,其结果与水稻群体分蘖的积温统计模型、基本动力学模型和复合分蘖模型进行了比较。研究结果表明 ,神经网络模型具有一定的外推能力 ,但其外推能力依赖于大量的训练样本。神经网络模型具有较好的拟合能力 ,是因为有较多的模型参数 ,因此对神经网络模型的训练需要大量的参数来保证其参数不致过度吻合。具有外推能力神经网络模型的最少训练样本数应大于 6 .75倍于神经网络参数数目 ,小于 13.5倍于神经网络参数数目。因此在应用神经网络模型时 ,如果神经网络模型包括较多的输入变量时 ,可考虑采用主成分分析、对应分析等技术对输入变量进行信息综合 ,相应地减少网络模型的参数。另一方面 ,当训练样本不足时 ,最好只用神经网络模型对同一系统的情况进行模拟 ,应谨慎使用神经网络模型进行外推。神经网络模型给作物模拟研究的科学工作者提供了一个“傻瓜”式工具 ,对数学建模不熟悉的农业研究人员 ,人工神经网络可以替代数学建模进行仿真实验 ;对于精通数学建模的研究人员来说 ,它至少是一种补充和可作为比较的非线性数据处理方法     

人工神经网络模型及其在农业和生态学研究中的应用

对于一些复杂的农业生态系统,人们对其生态过程了解较少,且这些系统的不确定性和模糊性较大,用传统的方法难以模拟这些系统的行为,神经网络模型因为能较精确地模拟这些系统的行为,而引起生态学者们的广泛兴趣。该文着重介绍了误差逆传神经网络模型的结构、算法及其在农业和生态学中的应用研究。误差逆传神经网络模型一般采用三层神经网络模型结构,三层的神经网络模型能模拟任意复杂程度的连续函数,而且因为它的结构小而不容易产生与训练数据的过度吻合。误差逆传神经网络模型算法的主要特征是:利用当前的输入误差对权值进行调整。在生态学和农业研究中,误差逆传神经网络模型通常作为非线性函数模拟器用于预测作物产量、生物生产量、生物与环境之间的关系等。已有的研究表明:误差逆传神经网络模型的模拟精度要远远高于多元线性方程,类似于非线性方程,而在样本量足够的情况下,有一定的外推能力。但是误差逆传神经网络模型需要大量的样本量来保证所求取参数的可靠性,但这在实际研究中很难做到,因而限制了误差逆传神经网络模型的应用。近年来人们提出了强制训练停止、复合模型等多种技术来提高误差逆传神经网络模型的外推能力,也提出了Garson算法、敏感性分析以及随机化检验等技术对误差逆传神经网络模型的机理进行解释。误差逆传神经网络模型的真正优势在于模拟人们了解较少或不确定性和模糊性较大系统的行为,这些是传统模型所无法实现的,因而是对传统机理模型的重要补充。     

On-line estimation of biomass concentration using a neural network and information about metabolic state

This paper deals with the design of a neural network-based biomass concentration estimation system. This system is enhanced by the incorporation of information about the actual metabolism of the microorganism cultivated, which is taken from an on-line knowledge-based system. Two different design approaches have been investigated using the fed-batch cultivation of bakers yeast as the model process. In the first, metabolic state (MS) data were passed as additional input to the neural network; in the second, these data were used to select a neural network suitable for the specific MS. Two neural network types—feed-forward (Levenberg-Marquardt) and cascade correlation—were applied to this system and tested, and the performances of these neural networks were compared.     

Information processing in echo state networks at the edge of chaos

We investigate information processing in randomly connected recurrent neural networks. It has been shown previously that the computational capabilities of these networks are maximized when the recurrent layer is close to the border between a stable and an unstable dynamics regime, the so called edge of chaos. The reasons, however, for this maximized performance are not completely understood. We adopt an information-theoretical framework and are for the first time able to quantify the computational capabilities between elements of these networks directly as they undergo the phase transition to chaos. Specifically, we present evidence that both information transfer and storage in the recurrent layer are maximized close to this phase transition, providing an explanation for why guiding the recurrent layer toward the edge of chaos is computationally useful. As a consequence, our study suggests self-organized ways of improving performance in recurrent neural networks, driven by input data. Moreover, the networks we study share important features with biological systems such as feedback connections and online computation on input streams. A key example is the cerebral cortex, which was shown to also operate close to the edge of chaos. Consequently, the behavior of model systems as studied here is likely to shed light on reasons why biological systems are tuned into this specific regime.     

Feed-forward neural networks for secondary structure prediction

A feed-forward neural network has been employed for protein secondary structure prediction. Attempts were made to improve on previous prediction accuracies using a hierarchical mixture of experts (HME). In this method input data are clustered and used to train a series of different networks. Application of an HME to the prediction of protein secondary structure is shown to provide no advantages over a single network. We have also tried various new input representations, chosen to incorporate the effect of residues a long distance away in the one-dimensional amino acid chain. Prediction accuracy using these methods is comparable to that achieved by other neural networks.^1–4     

Localizing Tortoise Nests by Neural Networks

The goal of this research is to recognize the nest digging activity of tortoises using a device mounted atop the tortoise carapace. The device classifies tortoise movements in order to discriminate between nest digging, and non-digging activity (specifically walking and eating). Accelerometer data was collected from devices attached to the carapace of a number of tortoises during their two-month nesting period. Our system uses an accelerometer and an activity recognition system (ARS) which is modularly structured using an artificial neural network and an output filter. For the purpose of experiment and comparison, and with the aim of minimizing the computational cost, the artificial neural network has been modelled according to three different architectures based on the input delay neural network (IDNN). We show that the ARS can achieve very high accuracy on segments of data sequences, with an extremely small neural network that can be embedded in programmable low power devices. Given that digging is typically a long activity (up to two hours), the application of ARS on data segments can be repeated over time to set up a reliable and efficient system, called Tortoise@, for digging activity recognition.     

Neural networks for clustered and longitudinal data using mixed effects models

Although most statistical methods for the analysis of longitudinal data have focused on retrospective models of association, new advances in mobile health data have presented opportunities for predicting future health status by leveraging an individual's behavioral history alongside data from similar patients. Methods that incorporate both individual-level and sample-level effects are critical to using these data to its full predictive capacity. Neural networks are powerful tools for prediction, but many assume input observations are independent even when they are clustered or correlated in some way, such as in longitudinal data. Generalized linear mixed models (GLMM) provide a flexible framework for modeling longitudinal data but have poor predictive power particularly when the data are highly nonlinear. We propose a generalized neural network mixed model that replaces the linear fixed effect in a GLMM with the output of a feed-forward neural network. The model simultaneously accounts for the correlation structure and complex nonlinear relationship between input variables and outcomes, and it utilizes the predictive power of neural networks. We apply this approach to predict depression and anxiety levels of schizophrenic patients using longitudinal data collected from passive smartphone sensor data.     

A new training algorithm using artificial neural networks to classify gender-specific dynamic gait patterns

The aim of this study was to present a new training algorithm using artificial neural networks called multi-objective least absolute shrinkage and selection operator (MOBJ-LASSO) applied to the classification of dynamic gait patterns. The movement pattern is identified by 20 characteristics from the three components of the ground reaction force which are used as input information for the neural networks in gender-specific gait classification. The classification performance between MOBJ-LASSO (97.4%) and multi-objective algorithm (MOBJ) (97.1%) is similar, but the MOBJ-LASSO algorithm achieved more improved results than the MOBJ because it is able to eliminate the inputs and automatically select the parameters of the neural network. Thus, it is an effective tool for data mining using neural networks. From 20 inputs used for training, MOBJ-LASSO selected the first and second peaks of the vertical force and the force peak in the antero-posterior direction as the variables that classify the gait patterns of the different genders.     

An olfactory neuronal network for vapor recognition in an artificial nose

Odorant sensitivity and discrimination in the olfactory system appear to involve extensive neural processing of the primary sensory inputs from the olfactory epithelium. To test formally the functional consequences of such processing, we implemented in an artificial chemosensing system a new analytical approach that is based directly on neural circuits of the vertebrate olfactory system. An array of fiber-optic chemosensors, constructed with response properties similar to those of olfactory sensory neurons, provide time-varying inputs to a computer simulation of the olfactory bulb (OB). The OB simulation produces spatiotemporal patterns of neuronal firing that vary with vapor type. These patterns are then recognized by a delay line neural network (DLNN). In the final output of these two processing steps, vapor identity is encoded by the spatial patterning of activity across units in the DLNN, and vapor intensity is encoded by response latency. The OB-DLNN combination thus separates identity and intensity information into two distinct codes carried by the same output units, enabling discrimination among organic vapors over a range of input signal intensities. In addition to providing a well-defined system for investigating olfactory information processing, this biologically based neuronal network performs better than standard feed-forward neural networks in discriminating vapors when small amounts of training data are used. Received: 30 June 1997 / Accepted in revised form: 12 January 1998