A Lempel-Ziv complexity-based neural network pruning algorithm

This paper presents a pruning method for artificial neural networks (ANNs) based on the 'Lempel-Ziv complexity' (LZC) measure. We call this method the 'silent pruning algorithm' (SPA). The term 'silent' is used in the sense that SPA prunes ANNs without causing much disturbance during the network training. SPA prunes hidden units during the training process according to their ranks computed from LZC. LZC extracts the number of unique patterns in a time sequence obtained from the output of a hidden unit and a smaller value of LZC indicates higher redundancy of a hidden unit. SPA has a great resemblance to biological brains since it encourages higher complexity during the training process. SPA is similar to, yet different from, existing pruning algorithms. The algorithm has been tested on a number of challenging benchmark problems in machine learning, including cancer, diabetes, heart, card, iris, glass, thyroid, and hepatitis problems. We compared SPA with other pruning algorithms and we found that SPA is better than the 'random deletion algorithm' (RDA) which prunes hidden units randomly. Our experimental results show that SPA can simplify ANNs with good generalization ability.     

Entropic analysis and incremental synthesis of multilayered feedforward neural networks

Neural network architecture optimization is often a critical issue, particularly when VLSI implementation is considered. This paper proposes a new minimization method for multilayered feedforward ANNs and an original approach to their synthesis, both based on the analysis of the information quantity (entropy) flowing through the network. A layer is described as an information filter which selects the relevant characteristics until the complete classification is performed. The basic incremental synthesis method, including the supervised training procedure, is derived to design application-tailored neural paradigms with good generalization capability.     

The superior fault tolerance of artificial neural network training with a fault/noise injectionbased genetic algorithm

Artificial neural networks (ANNs) are powerful computational tools that are designed to replicate the human brain and adopted to solve a variety of problems in many different fields. Fault tolerance (FT), an important property of ANNs, ensures their reliability when significant portions of a network are lost. In this paper, a fault/noise injection-based (FIB) genetic algorithm (GA) is proposed to construct fault-tolerant ANNs. The FT performance of an FIB-GA was compared with that of a common genetic algorithm, the back-propagation algorithm, and the modification of weights algorithm. The FIB-GA showed a slower fitting speed when solving the exclusive OR (XOR) problem and the overlapping classification problem, but it significantly reduced the errors in cases of single or multiple faults in ANN weights or nodes. Further analysis revealed that the fit weights showed no correlation with the fitting errors in the ANNs constructed with the FIB-GA, suggesting a relatively even distribution of the various fitting parameters. In contrast, the output weights in the training of ANNs implemented with the use the other three algorithms demonstrated a positive correlation with the errors. Our findings therefore indicate that a combination of the fault/noise injection-based method and a GA is capable of introducing FT to ANNs and imply that the distributed ANNs demonstrate superior FT performance.     

What are artificial neural networks and what they can do?

The artificial neural networks (ANN) are very often applied in many areas of toxicology for the solving of complex problems, such as the prediction of chemical compound properties and quantitative structure-activity relationship. The aim of this contribution is to give the basic knowledge about conception of ANN, theirs division and finally, the typical application of ANN will be discussed. Due to the diversity of architectures and adaptation algorithms, the ANNs are used in the broad spectrum of applications from the environmental processes modeling, through the optimization to quantitative structure-activity relationship (QSAR) methods. In addition, especially ANNs with Kohonen learning are very effective classification tool. The ANNs are mostly applied in cases, where the commonly used methods does not work.     

Modeling of pain using artificial neural networks

In dealing with human nervous system, the sensation of pain is as sophisticated as other physiological phenomena. To obtain an acceptable model of the pain, physiology of the pain has been analysed in the present paper. Pain mechanisms are explained in block diagram representation form. Because of the nonlinear interactions existing among different sections in the diagram, artificial neural networks (ANNs) have been exploited. The basic patterns associated with chronic and acute pain have been collected and then used to obtain proper features for training the neural networks. Both static and dynamic representations of the ANNs were used in this regard. The trained networks then were employed to predict response of the body when it is exposed to special excitations. These excitations have not been used in the training phase and their behavior is interesting from the physiological view. Some of these predictions can be inferred from clinical experimentations. However, more clinical tests have to be accomplished for some of the predictions.     

Evaluation and amelioration of computer-aided diagnosis with artificial neural networks utilizing small-sized sample sets

Concerns about the specificity and reliability of artificial neural networks (ANNs) impede further application of ANNs in medicine. This is particularly true when developing computer-aided diagnosis (CAD) tools using ANNs for orphan diseases and emerging research areas where only a small-sized sample set is available. It is unreasonable to claim one ANN's performance as better than another simply on the basis of a single output without considering possible output variability due to factors including data noise and ANN training protocols. In this paper, a bootstrap resampling method is proposed to quantitatively analyze ANN output reliability and changing performance as the sample data and training protocols are varied. The method is tested in the area of feature classification for analysis of masses detected on mammograms. Our experiments show that ANNs performance, measured in terms of the area under the receiver operating characteristic (ROC) curve, is not a fixed value, but follows a distribution function sensitive to many factors. We demonstrate that our approach to determining the bootstrap estimates of confidence intervals (CIs) and prediction intervals (PIs) can be used to assure optimal performance in terms of ANN model configuration. We also show that the unintentional inclusion of data noise, which biases ANN results in small task-specific databases, can be accurately detected via the bootstrap estimates.     

Multi-Input Distributed Classifiers for Synthetic Genetic Circuits

For practical construction of complex synthetic genetic networks able to perform elaborate functions it is important to have a pool of relatively simple modules with different functionality which can be compounded together. To complement engineering of very different existing synthetic genetic devices such as switches, oscillators or logical gates, we propose and develop here a design of synthetic multi-input classifier based on a recently introduced distributed classifier concept. A heterogeneous population of cells acts as a single classifier, whose output is obtained by summarizing the outputs of individual cells. The learning ability is achieved by pruning the population, instead of tuning parameters of an individual cell. The present paper is focused on evaluating two possible schemes of multi-input gene classifier circuits. We demonstrate their suitability for implementing a multi-input distributed classifier capable of separating data which are inseparable for single-input classifiers, and characterize performance of the classifiers by analytical and numerical results. The simpler scheme implements a linear classifier in a single cell and is targeted at separable classification problems with simple class borders. A hard learning strategy is used to train a distributed classifier by removing from the population any cell answering incorrectly to at least one training example. The other scheme implements a circuit with a bell-shaped response in a single cell to allow potentially arbitrary shape of the classification border in the input space of a distributed classifier. Inseparable classification problems are addressed using soft learning strategy, characterized by probabilistic decision to keep or discard a cell at each training iteration. We expect that our classifier design contributes to the development of robust and predictable synthetic biosensors, which have the potential to affect applications in a lot of fields, including that of medicine and industry.     

Development of predictive models of laboratory animal growth using artificial neural networks

Traditional regression analysis of body weight growth curvesencounters problems .when the data are extremely variable. Whiletransformations are often employed to meet the criteria of theanalysis, some transformations are inadequate for normalizingthe data. Regression analysis also requires presuppositionsregarding the model to be fit and the techniques to be usedin the analysis. An alternative approach using artificial neuralnetworks is presented which may be suitable for developing predictivemodels of growth. Neural networks are simulators of the processesthat occur in the biological brain during the learning process.They are trained on the data, developing the necessary algorithmswithin their internal architecture, and produce a predictivemodel based on the learned facts. A dataset of Sprague–Dawleyrat (Rattus norvegicus) weights is analyzed by both traditionalregression analysis and neural network training. Predictionsof body weight are made from both models. While both methodsproduce models that adequately predict the body weights, theneural network model is superior in that it combines accuracyand precision, being less influenced by longitudinal variabilityin the data. Thus, the neural network provides another toolfor researchers to analyze growth curve data.     

Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks

The purpose of this study was to develop a method of classifying cancers to specific diagnostic categories based on their gene expression signatures using artificial neural networks (ANNs). We trained the ANNs using the small, round blue-cell tumors (SRBCTs) as a model. These cancers belong to four distinct diagnostic categories and often present diagnostic dilemmas in clinical practice. The ANNs correctly classified all samples and identified the genes most relevant to the classification. Expression of several of these genes has been reported in SRBCTs, but most have not been associated with these cancers. To test the ability of the trained ANN models to recognize SRBCTs, we analyzed additional blinded samples that were not previously used for the training procedure, and correctly classified them in all cases. This study demonstrates the potential applications of these methods for tumor diagnosis and the identification of candidate targets for therapy.     

Characterizing and Predicting Submovements during Human Three-Dimensional Arm Reaches

We have demonstrated that 3D target-oriented human arm reaches can be represented as linear combinations of discrete submovements, where the submovements are a set of minimum-jerk basis functions for the reaches. We have also demonstrated the ability of deterministic feed-forward Artificial Neural Networks (ANNs) to predict the parameters of the submovements. ANNs were trained using kinematic data obtained experimentally from five human participants making target-directed movements that were decomposed offline into minimum-jerk submovements using an optimization algorithm. Under cross-validation, the ANNs were able to accurately predict the parameters (initiation-time, amplitude, and duration) of the individual submovements. We also demonstrated that the ANNs can together form a closed-loop model of human reaching capable of predicting 3D trajectories with VAF >95.9% and RMSE ≤4.32 cm relative to the actual recorded trajectories. This closed-loop model is a step towards a practical arm trajectory generator based on submovements, and should be useful for the development of future arm prosthetic devices that are controlled by brain computer interfaces or other user interfaces.     

大学与国家公园的合作伙伴关系：课堂、户外实验室与世界之窗

2016年美国国家公园系统庆祝其成立100周年。在过去的100年间,作为国家公园的合作伙伴,美国的大学一直致力于为国家公园提供科学、教育和培训方面的支持。在中小学阶段,国家公园就成为学生们的学习场所。大学生在读期间会以实习生和季节工的身份参加国家公园的实践。研究生在完成论文和论文项目时为国家公园提供必要的研究,而所有参与者也就成了传递国家公园价值的更好的老师。这种支持有助于实现公园保护目标、解决公园面临的问题。同时,为在校学生和毕业后的学生提供了丰富学习经验的平台。为此还成立了专门的行政机构,以促进学校与公园的沟通、签订合约、建立合作伙伴。美国的大学与国家公园共同创造了一种有利于学术研究和管理实践的强有力合作伙伴关系,为世界以及中国国家公园体制的建设树立了很好的榜样。     

Training radial basis function neural networks: effects of training set size and imbalanced training sets

Obtaining training data for constructing artificial neural networks (ANNs) to identify microbiological taxa is not always easy. Often, only small data sets with different numbers of observations per taxon are available. Here, the effect of both size of the training data set and of an imbalanced number of training patterns for different taxa is investigated using radial basis function ANNs to identify up to 60 species of marine microalgae. The best networks trained to discriminate 20, 40 and 60 species respectively gave overall percentage correct identification of 92, 84 and 77%. From 100 to 200 patterns per species was sufficient in networks trained to discriminate 20, 40 or 60 species. For 40 and 60 species data sets an imbalance in the number of training patterns per species always affected training success, the greater the imbalance the greater the effect. However, this could be largely compensated for by adjusting the networks using a posteriori probabilities, estimated as network output values.     

Fish distribution predictions from different points of view: comparing associative neural networks,geostatistics and regression models

Accurate prediction of species distributions based on sampling and environmental data is essential for further scientific analysis, such as stock assessment, detection of abundance fluctuation due to climate change or overexploitation, and to underpin management and legislation processes. The evolution of computer science and statistics has allowed the development of sophisticated and well-established modelling techniques as well as a variety of promising innovative approaches for modelling species distribution. The appropriate selection of modelling approach is crucial to the quality of predictions about species distribution. In this study, modelling techniques based on different approaches are compared and evaluated in relation to their predictive performance, utilizing fish density acoustic data. Generalized additive models and mixed models amongst the regression models, associative neural networks (ANNs) and artificial neural networks ensemble amongst the artificial neural networks and ordinary kriging amongst the geostatistical techniques are applied and evaluated. A verification dataset is used for estimating the predictive performance of these models. A combination of outputs from the different models is applied for prediction optimization to exploit the ability of each model to explain certain aspects of variation in species acoustic density. Neural networks and especially ANNs appear to provide more accurate results in fitting the training dataset while generalized additive models appear more flexible in predicting the verification dataset. The efficiency of each technique in relation to certain sampling and output strategies is also discussed.     

Data prevalence matters when assessing species' responses using data-driven species distribution models

The study of species' response is a key to understand the ecology of a species (e.g. critical habitat requirement and biological invasion processes) and design better conservation and management plans (e.g. problem identification, priority assessment and risk analysis). Predictive machine learning methods can be used as a tool for modeling species distributions as well as for describing important variables and specific habitat conditions required for a target species. This study aims (1) to demonstrate how habitat information such as species response curves can be retrieved from a species distribution model (SDM), (2) to assess the effects of data prevalence on model accuracy and habitat information retrieved from SDMs, and (3) to illustrate the differences between three data-driven methods, namely a fuzzy habitat suitability model (FHSM), random forests (RF) and support vector machines (SVMs). Nineteen sets of virtual species data with different data prevalences were generated using field-observed habitat conditions and hypothetical habitat suitability curves under four interaction scenarios governing the species–environment relationship for a virtual species. The effects of data prevalence on species distribution modeling were evaluated based on model accuracy and habitat information such as species response curves. Data prevalence affected both model accuracy and the assessment of species' response, with a stronger influence on the latter. The effects of data prevalence on model accuracy were less pronounced in the case of RF and SVMs which showed a higher performance. While the response curves were similar among the three models, data prevalence markedly affected the shapes of the response curves. Specifically, response curves obtained from a data set with higher prevalence showed higher tolerance to unsuitable habitat conditions, emphasizing the importance of accounting for data prevalence in the assessment of species–environment relationships. In a practical implementation of an SDM, data prevalence should be taken into account when interpreting the model results.     

人工育幼非洲狮、孟加拉虎、豹幼仔的生长曲线拟合比较

<正>生长曲线(Growth curve)是揭示动物生长发育规律的有效手段之一，反映动物发育指标(包括体重、体尺、体表面积等度量值)与时间之间的关系(盛志廉和吴常信，1995)。生长曲线方程大致可以分为三类：第一类是表示报酬递减规律的方程，如指数函数；第二类是描述光滑S型曲线，     

Assessment of respiratory system mechanics by artificial neural networks: an exploratory study.

We evaluated 1) the performance of an artificial neural network (ANN)-based technology in assessing the respiratory system resistance (Rrs) and compliance (Crs) in a porcine model of acute lung injury and 2) the possibility of using, for ANN training, signals coming from an electrical analog (EA) of the lung. Two differently experienced ANNs were compared. One ANN (ANN(BIO)) was trained on tracings recorded at different time points after the administration of oleic acid in 10 anesthetized and paralyzed pigs during constant-flow mechanical ventilation. A second ANN (ANN(MOD)) was trained on EA simulations. Both ANNs were evaluated prospectively on data coming from four different pigs. Linear regression between ANN output and manually computed mechanics showed a regression coefficient (R) of 0.98 for both ANNs in assessing Crs. On Rrs, ANN(BIO) showed a performance expressed by R = 0.40 and ANN(MOD) by R = 0.61. These results suggest that ANNs can learn to assess the respiratory system mechanics during mechanical ventilation but that the assessment of resistance and compliance by ANNs may require different approaches.     

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

To facilitate decision support in freshwater ecosystem protection and restoration management, habitat suitability models can be very valuable. Data driven methods such as artificial neural networks (ANNs) are particularly useful in this context, seen their time-efficient development and relatively high reliability. However, specialized and technical literature on neural network modelling offers a variety of model development criteria to select model architecture, training procedure, etc. This may lead to confusion among ecosystem modellers and managers regarding the optimal training and validation methodology. This paper focuses on the analysis of ANN development and application for predicting macroinvertebrate communities, a species group commonly used in freshwater assessment worldwide. This review reflects on the different aspects regarding model development and application based on a selection of 26 papers reporting the use of ANN models for the prediction of macroinvertebrates. This analysis revealed that the applied model training and validation methodologies can often be improved and moreover crucial steps in the modelling process are often poorly documented. Therefore, suggestions to improve model development, assessment and application in ecological river management are presented. In particular, data pre-processing determines to a high extent the reliability of the induced models and their predictive relevance. This also counts for the validation criteria, that need to be better tuned to the practical simulation requirements. Moreover, the use of sensitivity methods can help to extract knowledge on the habitat preference of species and allow peer-review by ecological experts. The selection of relevant input variables remains a critical challenge as well. Model coupling is a missing crucial step to link human activities, hydrology, physical habitat conditions, water quality and ecosystem status. This last aspect is probably the most valuable aspect to enable decision support in water management based on ANN models.     

String matching artificial neural networks.

Three artificial neural networks (ANNs) are proposed for solving a variety of on- and off-line string matching problems. The ANN structure employed as the building block of these ANNs is derived from the harmony theory (HT) ANN, whereby the resulting string matching ANNs are characterized by fast match-mismatch decisions, low computational complexity, and activation values of the ANN output nodes that can be used as indicators of substitution, insertion (addition) and deletion spelling errors.     

Use of Artificial Neural Networks and a Gamma-Concept-Based Approach To Model Growth of and Bacteriocin Production by Streptococcus macedonicus ACA-DC 198 under Simulated Conditions of Kasseri Cheese Production

Growth of and bacteriocin production by Streptococcus macedonicus ACA-DC 198 were assessed and modeled under conditions simulating Kasseri cheese production. Controlled fermentations were performed in milk supplemented with yeast extract at different combinations of temperature (25, 40, and 55°C), constant pH (pHs 5 and 6), and added NaCl (at concentrations of 0, 2, and 4%, wt/vol). The data obtained were used to construct two types of predictive models, namely, a modeling approach based on the gamma concept, as well as a model based on artificial neural networks (ANNs). The latter computational methods were used on 36 control fermentations to quantify the complex relationships between the conditions applied (temperature, pH, and NaCl) and population behavior and to calculate the associated biokinetic parameters, i.e., maximum specific growth and cell count decrease rates and specific bacteriocin production. The functions obtained were able to estimate these biokinetic parameters for four validation fermentation experiments and obtained good agreement between modeled and experimental values. Overall, these experiments show that both methods can be successfully used to unravel complex kinetic patterns within biological data of this kind and to predict population kinetics. Whereas ANNs yield a better correlation between experimental and predicted results, the gamma-concept-based model is more suitable for biological interpretation. Also, while the gamma-concept-based model has not been designed for modeling of other biokinetic parameters than the specific growth rate, ANNs are able to deal with any parameter of relevance, including specific bacteriocin production.     

Predicting stability of mixed microbial cultures from single species experiments: 1. Phenomenological model

The growth of mixed microbial cultures on mixtures of substrates is a fundamental problem of both theoretical and practical interest. On the one hand, the literature is abundant with experimental studies of mixed-substrate phenomena [T. Egli, The ecological and physiological significance of the growth of heterotrophic microorganisms with mixtures of substrates, Adv. Microbiol. Ecol. 14 (1995) 305-386]. On the other hand, a number of mathematical models of mixed-substrate growth have been analyzed in the last three decades. These models typically assume specific kinetic expressions for substrate uptake and biomass growth rates and their predictions are formulated in terms of parameters of the model. In this work, we formulate and analyze a general mathematical model of mixed microbial growth on mixtures of substitutable substrates. Using this model, we study the effect of mutual inhibition of substrate uptake rates on the stability of the equilibria of the model. Specifically, we address the following question: How much of the dynamics exhibited by two competing species can be inferred from single species data? We provide geometric criteria for stability of various types of equilibria corresponding to non-competitive exclusion, competitive exclusion, and coexistence of two competing species in terms of growth isoclines and consumption curves. A growth isocline is a curve in the plane of substrate concentrations corresponding to the zero net growth of a given species. In [G.T. Reeves, A. Narang, S.S. Pilyugin, Growth of mixed cultures on mixtures of substitutable substrates: The operating diagram for a structured model, J. Theor. Biol. 226 (2004) 143-157], we introduced consumption curves as sets of all possible combinations of substrate concentrations corresponding to balanced growth of a single microbial species. Both types of curves can be obtained in single species experiments.