Automated neuron model optimization techniques: a review

The increase in complexity of computational neuron models makes the hand tuning of model parameters more difficult than ever. Fortunately, the parallel increase in computer power allows scientists to automate this tuning. Optimization algorithms need two essential components. The first one is a function that measures the difference between the output of the model with a given set of parameter and the data. This error function or fitness function makes the ranking of different parameter sets possible. The second component is a search algorithm that explores the parameter space to find the best parameter set in a minimal amount of time. In this review we distinguish three types of error functions: feature-based ones, point-by-point comparison of voltage traces and multi-objective functions. We then detail several popular search algorithms, including brute-force methods, simulated annealing, genetic algorithms, evolution strategies, differential evolution and particle-swarm optimization. Last, we shortly describe Neurofitter, a free software package that combines a phase–plane trajectory density fitness function with several search algorithms.     

Numerical algorithms for estimation and calculation of parameters in modeling pest population dynamics and evolution of resistance

Computational simulation models can provide a way of understanding and predicting insect population dynamics and evolution of resistance, but the usefulness of such models depends on generating or estimating the values of key parameters. In this paper, we describe four numerical algorithms generating or estimating key parameters for simulating four different processes within such models. First, we describe a novel method to generate an offspring genotype table for one- or two-locus genetic models for simulating evolution of resistance, and how this method can be extended to create offspring genotype tables for models with more than two loci. Second, we describe how we use a generalized inverse matrix to find a least-squares solution to an over-determined linear system for estimation of parameters in probit models of kill rates. This algorithm can also be used for the estimation of parameters of Freundlich adsorption isotherms. Third, we describe a simple algorithm to randomly select initial frequencies of genotypes either without any special constraints or with some pre-selected frequencies. Also we give a simple method to calculate the “stable” Hardy–Weinberg equilibrium proportions that would result from these initial frequencies. Fourth we describe how the problem of estimating the intrinsic rate of natural increase of a population can be converted to a root-finding problem and how the bisection algorithm can then be used to find the rate. We implemented all these algorithms using MATLAB and Python code; the key statements in both codes consist of only a few commands and are given in the appendices. The results of numerical experiments are also provided to demonstrate that our algorithms are valid and efficient.     

Parameter Estimation Methods for Single Neuron Models

With the advancement in computer technology, it has become possible to fit complex models to neuronal data. In this work, we test how two methods can estimate parameters of simple neuron models (passive soma) to more complex ones (neuron with one dendritic cylinder and two active conductances). The first method uses classical voltage traces resulting from current pulses injection (time domain), while the second uses measures of the neuron's response to sinusoidal stimuli (frequency domain). Both methods estimate correctly the parameters in all cases studied. However, the time-domain method is slower and more prone to estimation errors in the cable parameters than the frequency-domain method. Because with noisy data the goodness of fit does not distinguish between different solutions, we suggest that running the estimation procedure a large number of times might help find a good solution and can provide information about the interactions between parameters. Also, because the formulation used for the model's response in the frequency domain is analytical, one can derive a local sensitivity analysis for each parameter. This analysis indicates how well a parameter is likely to be estimated and helps choose an optimal stimulation protocol. Finally, the tests suggest a strategy for fitting single-cell models using the two methods examined.     

Global rigid body modeling of macromolecular complexes against small-angle scattering data

New methods to automatically build models of macromolecular complexes from high-resolution structures or homology models of their subunits or domains against x-ray or neutron small-angle scattering data are presented. Depending on the complexity of the object, different approaches are employed for the global search of the optimum configuration of subunits fitting the experimental data. An exhaustive grid search is used for hetero- and homodimeric particles and for symmetric oligomers formed by identical subunits. For the assemblies or multidomain proteins containing more then one subunit/domain per asymmetric unit, heuristic algorithms based on simulated annealing are used. Fast computational algorithms based on spherical harmonics representation of scattering amplitudes are employed. The methods allow one to construct interconnected models without steric clashes, to account for the particle symmetry and to incorporate information from other methods, on distances between specific residues or nucleotides. For multidomain proteins, addition of missing linkers between the domains is possible. Simultaneous fitting of multiple scattering patterns from subcomplexes or deletion mutants is incorporated. The efficiency of the methods is illustrated by their application to complexes of different types in several simulated and practical examples. Limitations and possible ambiguity of rigid body modeling are discussed and simplified docking criteria are provided to rank multiple models. The methods described are implemented in publicly available computer programs running on major hardware platforms.     

The parameters of the stochastic leaky integrate-and-fire neuronal model

Five parameters of one of the most common neuronal models, the diffusion leaky integrate-and-fire model, also known as the Ornstein-Uhlenbeck neuronal model, were estimated on the basis of intracellular recording. These parameters can be classified into two categories. Three of them (the membrane time constant, the resting potential and the firing threshold) characterize the neuron itself. The remaining two characterize the neuronal input. The intracellular data were collected during spontaneous firing, which in this case is characterized by a Poisson process of interspike intervals. Two methods for the estimation were applied, the regression method and the maximum-likelihood method. Both methods permit to estimate the input parameters and the membrane time constant in a short time window (a single interspike interval). We found that, at least in our example, the regression method gave more consistent results than the maximum-likelihood method. The estimates of the input parameters show the asymptotical normality, which can be further used for statistical testing, under the condition that the data are collected in different experimental situations. The model neuron, as deduced from the determined parameters, works in a subthreshold regimen. This result was confirmed by both applied methods. The subthreshold regimen for this model is characterized by the Poissonian firing. This is in a complete agreement with the observed interspike interval data. Action Editor: Nicolas Brunel     

Minimizing model fitting objectives that contain spurious local minima by bootstrap restarting

Objective functions that arise when fitting nonlinear models often contain local minima that are of little significance except for their propensity to trap minimization algorithms. The standard methods for attempting to deal with this problem treat the objective function as fixed and employ stochastic minimization approaches in the hope of randomly jumping out of local minima. This article suggests a simple trick for performing such minimizations that can be employed in conjunction with most conventional nonstochastic fitting methods. The trick is to stochastically perturb the objective function by bootstrapping the data to be fit. Each bootstrap objective shares the large-scale structure of the original objective but has different small-scale structure. Minimizations of bootstrap objective functions are alternated with minimizations of the original objective function starting from the parameter values with which minimization of the previous bootstrap objective terminated. An example is presented, fitting a nonlinear population dynamic model to population dynamic data and including a comparison of the suggested method with simulated annealing. Convergence diagnostics are discussed.     

R法遗传参数估值置信区间的计算方法和重复估计次数的研究

探讨R法遗传参数估值置信区间的计算方法和重复估计次数(NORE)对参数估值的影响,利用4种模型通过模拟产生数据集。基础群中公、母畜数分别为200和2000头,BLUP育种值选择5个世代。利用多变量乘法迭代(MMI)法,结合先决条件的共扼梯度(PCG)法求解混合模型方程组估计方差组分。用经典方法、Box-Cox变换后的经典方法和自助法计算参数估值的均数、标准误和置信区间。结果表明,重复估计次数较多时,3种方法均可;重复估计次数较少时,建议使用自助法。简单模型下需要较少的重复估计,但对于复杂模型则需要较多的重复估计。随模型中随机效应数的增加,直接遗传力高估。随着PCG和MMI轮次的增大,参数估值表现出低估的趋势。     

Parameter inversion estimation in photosynthetic models: Impact of different simulation methods

When we apply ecological models in environmental management, we must assess the accuracy of parameter estimation and its impact on model predictions. Parameters estimated by conventional techniques tend to be nonrobust and require excessive computational resources. However, optimization algorithms are highly robust and generally exhibit convergence of parameter estimation by inversion with nonlinear models. They can simultaneously generate a large number of parameter estimates using an entire data set. In this study, we tested four inversion algorithms (simulated annealing, shuffled complex evolution, particle swarm optimization, and the genetic algorithm) to optimize parameters in photosynthetic models depending on different temperatures. We investigated if parameter boundary values and control variables influenced the accuracy and efficiency of the various algorithms and models. We obtained optimal solutions with all of the inversion algorithms tested if the parameter bounds and control variables were constrained properly. However, the efficiency of processing time use varied with the control variables obtained. In addition, we investigated if temperature dependence formalization impacted optimally the parameter estimation process. We found that the model with a peaked temperature response provided the best fit to the data.     

A Novel Hybrid Classification Model of Genetic Algorithms,Modified k-Nearest Neighbor and Developed Backpropagation Neural Network

Among numerous artificial intelligence approaches, k-Nearest Neighbor algorithms, genetic algorithms, and artificial neural networks are considered as the most common and effective methods in classification problems in numerous studies. In the present study, the results of the implementation of a novel hybrid feature selection-classification model using the above mentioned methods are presented. The purpose is benefitting from the synergies obtained from combining these technologies for the development of classification models. Such a combination creates an opportunity to invest in the strength of each algorithm, and is an approach to make up for their deficiencies. To develop proposed model, with the aim of obtaining the best array of features, first, feature ranking techniques such as the Fisher''s discriminant ratio and class separability criteria were used to prioritize features. Second, the obtained results that included arrays of the top-ranked features were used as the initial population of a genetic algorithm to produce optimum arrays of features. Third, using a modified k-Nearest Neighbor method as well as an improved method of backpropagation neural networks, the classification process was advanced based on optimum arrays of the features selected by genetic algorithms. The performance of the proposed model was compared with thirteen well-known classification models based on seven datasets. Furthermore, the statistical analysis was performed using the Friedman test followed by post-hoc tests. The experimental findings indicated that the novel proposed hybrid model resulted in significantly better classification performance compared with all 13 classification methods. Finally, the performance results of the proposed model was benchmarked against the best ones reported as the state-of-the-art classifiers in terms of classification accuracy for the same data sets. The substantial findings of the comprehensive comparative study revealed that performance of the proposed model in terms of classification accuracy is desirable, promising, and competitive to the existing state-of-the-art classification models.     

Further evidence for the increased power of LOD scores compared with nonparametric methods.

In genetic analysis of diseases in which the underlying model is unknown, "model free" methods-such as affected sib pair (ASP) tests-are often preferred over LOD-score methods, although LOD-score methods under the correct or even approximately correct model are more powerful than ASP tests. However, there might be circumstances in which nonparametric methods will outperform LOD-score methods. Recently, Dizier et al. reported that, in some complex two-locus (2L) models, LOD-score methods with segregation analysis-derived parameters had less power to detect linkage than ASP tests. We investigated whether these particular models, in fact, represent a situation that ASP tests are more powerful than LOD scores. We simulated data according to the parameters specified by Dizier et al. and analyzed the data by using a (a) single locus (SL) LOD-score analysis performed twice, under a simple dominant and a recessive mode of inheritance (MOI), (b) ASP methods, and (c) nonparametric linkage (NPL) analysis. We show that SL analysis performed twice and corrected for the type I-error increase due to multiple testing yields almost as much linkage information as does an analysis under the correct 2L model and is more powerful than either the ASP method or the NPL method. We demonstrate that, even for complex genetic models, the most important condition for linkage analysis is that the assumed MOI at the disease locus being tested is approximately correct, not that the inheritance of the disease per se is correctly specified. In the analysis by Dizier et al., segregation analysis led to estimates of dominance parameters that were grossly misspecified for the locus tested in those models in which ASP tests appeared to be more powerful than LOD-score analyses.     

NeEstimator v2: re‐implementation of software for the estimation of contemporary effective population size (Ne) from genetic data

NeEstimator v2 is a completely revised and updated implementation of software that produces estimates of contemporary effective population size, using several different methods and a single input file. NeEstimator v2 includes three single‐sample estimators (updated versions of the linkage disequilibrium and heterozygote‐excess methods, and a new method based on molecular coancestry), as well as the two‐sample (moment‐based temporal) method. New features include the following: (i) an improved method for accounting for missing data; (ii) options for screening out rare alleles; (iii) confidence intervals for all methods; (iv) the ability to analyse data sets with large numbers of genetic markers (10 000 or more); (v) options for batch processing large numbers of different data sets, which will facilitate cross‐method comparisons using simulated data; and (vi) correction for temporal estimates when individuals sampled are not removed from the population (Plan I sampling). The user is given considerable control over input data and composition, and format of output files. The freely available software has a new JAVA interface and runs under MacOS, Linux and Windows.     

Stepwise Kinetic Equilibrium Models of Quantitative Polymerase Chain Reaction

ABSTRACT: BACKGROUND: Numerous models for use in interpreting quantitative PCR (qPCR) data are present in recent literature. The most commonly used models assume the amplification in qPCR is exponential and fit an exponential model with a constant rate of increase to a select part of the curve. Kinetic theory may be used to model the annealing phase and does not assume constant efficiency of amplification. Mechanistic models describing the annealing phase with kinetic theory offer the most potential for accurate interpretation of qPCR data. Even so, they have not been thoroughly investigated and are rarely used for interpretation of qPCR data. New results for kinetic modeling of qPCR are presented. RESULTS: Two models are presented in which the efficiency of amplification is based on equilibrium solutions for the annealing phase of the qPCR process. Model 1 assumes annealing of complementary targets strands and annealing of target and primers are both reversible reactions and reach a dynamic equilibrium. Model 2 assumes all annealing reactions are nonreversible and equilibrium is static. Both models include the effect of primer concentration during the annealing phase. Analytic formulae are given for the equilibrium values of all single and double stranded molecules at the end of the annealing step. The equilibrium values are then used in a stepwise method to describe the whole qPCR process. Rate constants of kinetic models are the same for solutions that are identical except for possibly having different initial target concentrations. Analysis of qPCR curves from such solutions are thus analyzed by simultaneous non-linear curve fitting with the same rate constant values applying to all curves and each curve having a unique value for initial target concentration. The models were fit to two data sets for which the true initial target concentrations are known. Both models give better fit to observed qPCR data than other kinetic models present in the literature. They also give better estimates of initial target concentration. Model 1 was found to be slightly more robust than model 2 giving better estimates of initial target concentration when estimation of parameters was done for qPCR curves with very different initial target concentration. Both models may be used to estimate the initial absolute concentration of target sequence when a standard curve is not available. CONCLUSIONS: It is argued that the kinetic approach to modeling and interpreting quantitative PCR data has the potential to give more precise estimates of the true initial target concentrations than other methods currently used for analysis of qPCR data. The two models presented here give a unified model of the qPCR process in that they explain the shape of the qPCR curve for a wide variety of initial target concentrations.     

Multiple sequence alignment by parallel simulated annealing

We have developed simulated annealing algorithms to solve theproblem of multiple sequence alignment. The algorithm wns shownto give the optimal solution as confirmed by the rigorous dynamicprogramming algorithm for three-sequence alignment. To overcomelong execution times for simulated annealing, we utilized aparallel computer. A sequential algorithm, a simple parallelalgorithm and the temperature parallel algorithm were testedon a problem. The results were compared with the result obtainedby a conventional tree-based algorithm where alignments weremerged by two-' dynamic programming. Every annealing algorithmproduced a better energy value than the conventional algorithm.The best energy value, which probably represents the optimalsolution, wns reached within a reasonable time by both of theparallel annealing algorithms. We consider the temperature parallelalgorithm of simulated annealing to be the most suitable forfinding the optimal multiple sequence alignment because thealgorithm does not require any scheduling for optimization.The algorithm is also usefiui for refining multiple alignmentsobtained by other hewistic methods.     

Evaluation of the Performance of Information Theory-Based Methods and Cross-Correlation to Estimate the Functional Connectivity in Cortical Networks

Functional connectivity of in vitro neuronal networks was estimated by applying different statistical algorithms on data collected by Micro-Electrode Arrays (MEAs). First we tested these “connectivity methods” on neuronal network models at an increasing level of complexity and evaluated the performance in terms of ROC (Receiver Operating Characteristic) and PPC (Positive Precision Curve), a new defined complementary method specifically developed for functional links identification. Then, the algorithms better estimated the actual connectivity of the network models, were used to extract functional connectivity from cultured cortical networks coupled to MEAs. Among the proposed approaches, Transfer Entropy and Joint-Entropy showed the best results suggesting those methods as good candidates to extract functional links in actual neuronal networks from multi-site recordings.     

Predicting Water Quality Index (WQI) by feature selection and machine learning: A case study of An Kim Hai irrigation system

A variety of water quality indices have been used to assess the state of waterbodies all over the world. In calculating a Water Quality Index (WQI), traditional methods require the evaluation of many water quality parameters, making them costly and time-consuming. In recent years, machine learning (ML) algorithms have emerged as an effective tool to solve many environmental problems, including water quality management. In this study, we investigate the performance of the ML-based method in calculating the WQI. We apply several feature selection techniques to select the key parameters fed the ML models. Experiments are carried out to evaluate the WQI based on a dataset collected from 2007 to 2020 of An Kim Hai system, one of the most important irrigation systems in the north of Vietnam. The obtained results show that the application of selection methods allows reducing significantly the number of water quality parameters fed the ML models without losing their accuracy. In particular, by using the embedded method, we find out four important parameters, including Coliform, DO, Turbidity, and TSS, that have the greatest impact on water quality. Based on these parameters, the Random Forest model provides the best accuracy in predicting the WQI values from the An Kim Hai system with a Similarity of 0.94. The combination of feature selection and ML methods is then considered an effective alternative for calculating the WQI, leading to a desirable performance and a reduction of input parameters. This makes water quality monitoring less costly, substantial effort, and time.     

CONTRAfold: RNA secondary structure prediction without physics-based models

MOTIVATION: For several decades, free energy minimization methods have been the dominant strategy for single sequence RNA secondary structure prediction. More recently, stochastic context-free grammars (SCFGs) have emerged as an alternative probabilistic methodology for modeling RNA structure. Unlike physics-based methods, which rely on thousands of experimentally-measured thermodynamic parameters, SCFGs use fully-automated statistical learning algorithms to derive model parameters. Despite this advantage, however, probabilistic methods have not replaced free energy minimization methods as the tool of choice for secondary structure prediction, as the accuracies of the best current SCFGs have yet to match those of the best physics-based models. RESULTS: In this paper, we present CONTRAfold, a novel secondary structure prediction method based on conditional log-linear models (CLLMs), a flexible class of probabilistic models which generalize upon SCFGs by using discriminative training and feature-rich scoring. In a series of cross-validation experiments, we show that grammar-based secondary structure prediction methods formulated as CLLMs consistently outperform their SCFG analogs. Furthermore, CONTRAfold, a CLLM incorporating most of the features found in typical thermodynamic models, achieves the highest single sequence prediction accuracies to date, outperforming currently available probabilistic and physics-based techniques. Our result thus closes the gap between probabilistic and thermodynamic models, demonstrating that statistical learning procedures provide an effective alternative to empirical measurement of thermodynamic parameters for RNA secondary structure prediction. AVAILABILITY: Source code for CONTRAfold is available at http://contra.stanford.edu/contrafold/.     

Bayesian LASSO for quantitative trait loci mapping

The mapping of quantitative trait loci (QTL) is to identify molecular markers or genomic loci that influence the variation of complex traits. The problem is complicated by the facts that QTL data usually contain a large number of markers across the entire genome and most of them have little or no effect on the phenotype. In this article, we propose several Bayesian hierarchical models for mapping multiple QTL that simultaneously fit and estimate all possible genetic effects associated with all markers. The proposed models use prior distributions for the genetic effects that are scale mixtures of normal distributions with mean zero and variances distributed to give each effect a high probability of being near zero. We consider two types of priors for the variances, exponential and scaled inverse-chi(2) distributions, which result in a Bayesian version of the popular least absolute shrinkage and selection operator (LASSO) model and the well-known Student's t model, respectively. Unlike most applications where fixed values are preset for hyperparameters in the priors, we treat all hyperparameters as unknowns and estimate them along with other parameters. Markov chain Monte Carlo (MCMC) algorithms are developed to simulate the parameters from the posteriors. The methods are illustrated using well-known barley data.     

Accounting for missing data in the estimation of contemporary genetic effective population size (Ne)

Theoretical models are often applied to population genetic data sets without fully considering the effect of missing data. Researchers can deal with missing data by removing individuals that have failed to yield genotypes and/or by removing loci that have failed to yield allelic determinations, but despite their best efforts, most data sets still contain some missing data. As a consequence, realized sample size differs among loci, and this poses a problem for unbiased methods that must explicitly account for random sampling error. One commonly used solution for the calculation of contemporary effective population size (N_e) is to calculate the effective sample size as an unweighted mean or harmonic mean across loci. This is not ideal because it fails to account for the fact that loci with different numbers of alleles have different information content. Here we consider this problem for genetic estimators of contemporary effective population size (N_e). To evaluate bias and precision of several statistical approaches for dealing with missing data, we simulated populations with known N_e and various degrees of missing data. Across all scenarios, one method of correcting for missing data (fixed‐inverse variance‐weighted harmonic mean) consistently performed the best for both single‐sample and two‐sample (temporal) methods of estimating N_e and outperformed some methods currently in widespread use. The approach adopted here may be a starting point to adjust other population genetics methods that include per‐locus sample size components.