New learning automata based algorithms for adaptation of backpropagation algorithm parameters

One popular learning algorithm for feedforward neural networks is the backpropagation (BP) algorithm which includes parameters, learning rate (eta), momentum factor (alpha) and steepness parameter (lambda). The appropriate selections of these parameters have large effects on the convergence of the algorithm. Many techniques that adaptively adjust these parameters have been developed to increase speed of convergence. In this paper, we shall present several classes of learning automata based solutions to the problem of adaptation of BP algorithm parameters. By interconnection of learning automata to the feedforward neural networks, we use learning automata scheme for adjusting the parameters eta, alpha, and lambda based on the observation of random response of the neural networks. One of the important aspects of the proposed schemes is its ability to escape from local minima with high possibility during the training period. The feasibility of proposed methods is shown through simulations on several problems.     

Network Intrusion Detection Based on a General Regression Neural Network Optimized by an Improved Artificial Immune Algorithm

To effectively and accurately detect and classify network intrusion data, this paper introduces a general regression neural network (GRNN) based on the artificial immune algorithm with elitist strategies (AIAE). The elitist archive and elitist crossover were combined with the artificial immune algorithm (AIA) to produce the AIAE-GRNN algorithm, with the aim of improving its adaptivity and accuracy. In this paper, the mean square errors (MSEs) were considered the affinity function. The AIAE was used to optimize the smooth factors of the GRNN; then, the optimal smooth factor was solved and substituted into the trained GRNN. Thus, the intrusive data were classified. The paper selected a GRNN that was separately optimized using a genetic algorithm (GA), particle swarm optimization (PSO), and fuzzy C-mean clustering (FCM) to enable a comparison of these approaches. As shown in the results, the AIAE-GRNN achieves a higher classification accuracy than PSO-GRNN, but the running time of AIAE-GRNN is long, which was proved first. FCM and GA-GRNN were eliminated because of their deficiencies in terms of accuracy and convergence. To improve the running speed, the paper adopted principal component analysis (PCA) to reduce the dimensions of the intrusive data. With the reduction in dimensionality, the PCA-AIAE-GRNN decreases in accuracy less and has better convergence than the PCA-PSO-GRNN, and the running speed of the PCA-AIAE-GRNN was relatively improved. The experimental results show that the AIAE-GRNN has a higher robustness and accuracy than the other algorithms considered and can thus be used to classify the intrusive data.     

Super pairwise alignment (SPA): an efficient approach to global alignment for homologous sequences.

Sequence analysis is the basis of bioinformatics, while sequence alignment is a fundamental task for sequence analysis. The widely used alignment algorithm, Dynamic Programming, though generating optimal alignment, takes too much time due to its high computation complexity O(N(2)). In order to reduce computation complexity without sacrificing too much accuracy, we have developed a new approach to align two homologous sequences. The new approach presented here, adopting our novel algorithm which combines the methods of probabilistic and combinatorial analysis, reduces the computation complexity to as low as O(N). The computation speed by our program is at least 15 times faster than traditional pairwise alignment algorithms without a loss of much accuracy. We hence named the algorithm Super Pairwise Alignment (SPA). The pairwise alignment execution program based on SPA and the detailed results of the aligned sequences discussed in this article are available upon request.     

A fast global fitting algorithm for fluorescence lifetime imaging microscopy based on image segmentation

Global fitting algorithms have been shown to improve effectively the accuracy and precision of the analysis of fluorescence lifetime imaging microscopy data. Global analysis performs better than unconstrained data fitting when prior information exists, such as the spatial invariance of the lifetimes of individual fluorescent species. The highly coupled nature of global analysis often results in a significantly slower convergence of the data fitting algorithm as compared with unconstrained analysis. Convergence speed can be greatly accelerated by providing appropriate initial guesses. Realizing that the image morphology often correlates with fluorophore distribution, a global fitting algorithm has been developed to assign initial guesses throughout an image based on a segmentation analysis. This algorithm was tested on both simulated data sets and time-domain lifetime measurements. We have successfully measured fluorophore distribution in fibroblasts stained with Hoechst and calcein. This method further allows second harmonic generation from collagen and elastin autofluorescence to be differentiated in fluorescence lifetime imaging microscopy images of ex vivo human skin. On our experimental measurement, this algorithm increased convergence speed by over two orders of magnitude and achieved significantly better fits.     

A hybrid swarm-based algorithm for single-objective optimization problems involving high-cost analyses

In many technical fields, single-objective optimization procedures in continuous domains involve expensive numerical simulations. In this context, an improvement of the Artificial Bee Colony (ABC) algorithm, called the Artificial super-Bee enhanced Colony (AsBeC), is presented. AsBeC is designed to provide fast convergence speed, high solution accuracy and robust performance over a wide range of problems. It implements enhancements of the ABC structure and hybridizations with interpolation strategies. The latter are inspired by the quadratic trust region approach for local investigation and by an efficient global optimizer for separable problems. Each modification and their combined effects are studied with appropriate metrics on a numerical benchmark, which is also used for comparing AsBeC with some effective ABC variants and other derivative-free algorithms. In addition, the presented algorithm is validated on two recent benchmarks adopted for competitions in international conferences. Results show remarkable competitiveness and robustness for AsBeC.     

Belief Propagation Algorithm for Portfolio Optimization Problems

The typical behavior of optimal solutions to portfolio optimization problems with absolute deviation and expected shortfall models using replica analysis was pioneeringly estimated by S. Ciliberti et al. [Eur. Phys. B. 57, 175 (2007)]; however, they have not yet developed an approximate derivation method for finding the optimal portfolio with respect to a given return set. In this study, an approximation algorithm based on belief propagation for the portfolio optimization problem is presented using the Bethe free energy formalism, and the consistency of the numerical experimental results of the proposed algorithm with those of replica analysis is confirmed. Furthermore, the conjecture of H. Konno and H. Yamazaki, that the optimal solutions with the absolute deviation model and with the mean-variance model have the same typical behavior, is verified using replica analysis and the belief propagation algorithm.     

Toxic gas dispersion prediction for point source emission using deep learning method

Abstract

Accurate and rapid toxic gas concentration prediction model plays an important role in emergency aid of sudden gas leak. However, it is difficult for existing dispersion model to achieve accuracy and efficiency requirements at the same time. Although some researchers have considered developing new forecasting models with traditional machine learning, such as back propagation (BP) neural network, support vector machine (SVM), the prediction results obtained from such models need to be improved still in terms of accuracy. Then new prediction models based on deep learning are proposed in this paper. Deep learning has obvious advantages over traditional machine learning in prediction and classification. Deep belief networks (DBNs) as well as convolution neural networks (CNNs) are used to build new dispersion models here. Both models are compared with Gaussian plume model, computation fluid dynamics (CFD) model and models based on traditional machine learning in terms of accuracy, prediction time, and computation time. The experimental results turn out that CNNs model performs better considering all evaluation indexes.     

Lagrange Interpolation Learning Particle Swarm Optimization

In recent years, comprehensive learning particle swarm optimization (CLPSO) has attracted the attention of many scholars for using in solving multimodal problems, as it is excellent in preserving the particles’ diversity and thus preventing premature convergence. However, CLPSO exhibits low solution accuracy. Aiming to address this issue, we proposed a novel algorithm called LILPSO. First, this algorithm introduced a Lagrange interpolation method to perform a local search for the global best point (gbest). Second, to gain a better exemplar, one gbest, another two particle’s historical best points (pbest) are chosen to perform Lagrange interpolation, then to gain a new exemplar, which replaces the CLPSO’s comparison method. The numerical experiments conducted on various functions demonstrate the superiority of this algorithm, and the two methods are proven to be efficient for accelerating the convergence without leading the particle to premature convergence.     

Accelerated likelihood surface exploration: the likelihood ratchet

The existence of multiple likelihood maxima necessitates algorithms that explore a large part of the tree space. However, because of computational constraints, stepwise addition-based tree-searching methods do not allow for this exploration in reasonable time. Here, I present an algorithm that increases the speed at which the likelihood landscape can be explored. The iterative algorithm combines the computational speed of distance-based tree construction methods to arrive at approximations of the global optimum with the accuracy of optimality criterion based branch-swapping methods to improve on the result of the starting tree. The algorithm moves between local optima by iteratively perturbing the tree landscape through a process of reweighting randomly drawn samples of the underlying sequence data set. Tests on simulated and real data sets demonstrated that the optimal solution obtained using stepwise addition-based heuristic searches was found faster using the algorithm presented here. Tests on a previously published data set that established the presence of tree islands under maximum likelihood demonstrated that the algorithm identifies the same tree islands in a shorter amount of time than that needed using stepwise addition. The algorithm can be readily applied using standard software for phylogenetic inference.     

The PX-EM algorithm for fast stable fitting of Henderson's mixed model

This paper presents procedures for implementing the PX-EM algorithm of Liu, Rubin and Wu to compute REML estimates of variance covariance components in Henderson''s linear mixed models. The class of models considered encompasses several correlated random factors having the same vector length e.g., as in random regression models for longitudinal data analysis and in sire-maternal grandsire models for genetic evaluation. Numerical examples are presented to illustrate the procedures. Much better results in terms of convergence characteristics (number of iterations and time required for convergence) are obtained for PX-EM relative to the basic EM algorithm in the random regression.     

A fast and convergent stochastic MLP learning algorithm

We propose a stochastic learning algorithm for multilayer perceptrons of linear-threshold function units, which theoretically converges with probability one and experimentally exhibits 100% convergence rate and remarkable speed on parity and classification problems with typical generalization accuracy. For learning the n bit parity function with n hidden units, the algorithm converged on all the trials we tested (n=2 to 12) after 5.8 x 4.1(n) presentations for 0.23 x 4.0(n-6) seconds on a 533MHz Alpha 21164A chip on average, which is five to ten times faster than Levenberg-Marquardt algorithm with restarts. For a medium size classification problem known as Thyroid in UCI repository, the algorithm is faster in speed and comparative in generalization accuracy than the standard backpropagation and Levenberg-Marquardt algorithms.     

Accelerating an Ordered-Subset Low-Dose X-Ray Cone Beam Computed Tomography Image Reconstruction with a Power Factor and Total Variation Minimization

In recent years, there has been increased interest in low-dose X-ray cone beam computed tomography (CBCT) in many fields, including dentistry, guided radiotherapy and small animal imaging. Despite reducing the radiation dose, low-dose CBCT has not gained widespread acceptance in routine clinical practice. In addition to performing more evaluation studies, developing a fast and high-quality reconstruction algorithm is required. In this work, we propose an iterative reconstruction method that accelerates ordered-subsets (OS) reconstruction using a power factor. Furthermore, we combine it with the total-variation (TV) minimization method. Both simulation and phantom studies were conducted to evaluate the performance of the proposed method. Results show that the proposed method can accelerate conventional OS methods, greatly increase the convergence speed in early iterations. Moreover, applying the TV minimization to the power acceleration scheme can further improve the image quality while preserving the fast convergence rate.     

A structural EM algorithm for phylogenetic inference.

A central task in the study of molecular evolution is the reconstruction of a phylogenetic tree from sequences of current-day taxa. The most established approach to tree reconstruction is maximum likelihood (ML) analysis. Unfortunately, searching for the maximum likelihood phylogenetic tree is computationally prohibitive for large data sets. In this paper, we describe a new algorithm that uses Structural Expectation Maximization (EM) for learning maximum likelihood phylogenetic trees. This algorithm is similar to the standard EM method for edge-length estimation, except that during iterations of the Structural EM algorithm the topology is improved as well as the edge length. Our algorithm performs iterations of two steps. In the E-step, we use the current tree topology and edge lengths to compute expected sufficient statistics, which summarize the data. In the M-Step, we search for a topology that maximizes the likelihood with respect to these expected sufficient statistics. We show that searching for better topologies inside the M-step can be done efficiently, as opposed to standard methods for topology search. We prove that each iteration of this procedure increases the likelihood of the topology, and thus the procedure must converge. This convergence point, however, can be a suboptimal one. To escape from such "local optima," we further enhance our basic EM procedure by incorporating moves in the flavor of simulated annealing. We evaluate these new algorithms on both synthetic and real sequence data and show that for protein sequences even our basic algorithm finds more plausible trees than existing methods for searching maximum likelihood phylogenies. Furthermore, our algorithms are dramatically faster than such methods, enabling, for the first time, phylogenetic analysis of large protein data sets in the maximum likelihood framework.     

A fast strong coupling algorithm for the partitioned fluid–structure interaction simulation of BMHVs

The numerical simulation of Bileaflet Mechanical Heart Valves (BMHVs) has gained strong interest in the last years, as a design and optimisation tool. In this paper, a strong coupling algorithm for the partitioned fluid–structure interaction simulation of a BMHV is presented. The convergence of the coupling iterations between the flow solver and the leaflet motion solver is accelerated by using the Jacobian with the derivatives of the pressure and viscous moments acting on the leaflets with respect to the leaflet accelerations. This Jacobian is numerically calculated from the coupling iterations. An error analysis is done to derive a criterion for the selection of useable coupling iterations. The algorithm is successfully tested for two 3D cases of a BMHV and a comparison is made with existing coupling schemes. It is observed that the developed coupling scheme outperforms these existing schemes in needed coupling iterations per time step and CPU time.     

Research on parallelized real-time map matching algorithm for massive GPS data

In construction of smart city, numerous vehicles’ trajectory data are produced by Global Positioning System (GPS) to track their real time location. When these GPS data are processed by map matching, results can be used to support a large number of ITS applications such as real time road condition calculation, inspection of traffic event and emergency treatment. However, as the fast explosive growth of monitored vehicle number, massive GPS data proposes overwhelming challenges for map matching. Consequently, traditional map matching algorithms can hardly satisfy high demands for matching speed and accuracy. Therefore, a real time map matching algorithm for numerous GPS data is proposed to guarantee high matching accuracy and matching efficiency. Meanwhile, it can meet demands of GPS data processing required by the monitor of numerous vehicles within the city. Main contributions of the method are: (1) A Kalman filter based correcting algorithm is proposed to improve the matching accuracy of the traditional topological algorithm on the complicated road sections such as intersections and parallel roads. (2) Based on the Spark streaming framework, the serial map-matching algorithm is converted into a parallelized map-matching algorithm, which significantly improves the processing efficiency of the map matching. (3) A gridding method being applicable to the parallelized algorithm was proposed by the paper. The GPS data in the same grid were allocated to the same computing unit to improve the efficiency of the parallelized computation. Experimental results show that the matching accuracy of the algorithm demonstrated by the paper is increased by 10%; the matching efficiency is 25% higher than same amount of stand-alone computers. A cluster of 15 computers that operates the proposed algorithm is capable for the real time map matching for GPS data produced by 800 thousand vehicles, which can effectively and extensively support the lastingly increased demand for processing numerous GPS data.     

基于人工神经网络的非线性拟合方法ⅡBP学习算法中各因素的影响

本文研究了人工神经网络BP学习算法中动量因子、隐节点数、学习速率、激活因子等对网络学习速度有影响的几个因素,并且找出了最佳值．     

Mixtures of tight-binding enzyme inhibitors. Kinetic analysis by a recursive rate equation.

When two or more tight-binding inhibitors are present in an enzyme assay, the equation that relates the initial velocity v to the concentration of reactants cannot be written in an algebraically explicit form. Rather, for n inhibitors it is an implicit polynomial equation of degree n + 1 with respect to v. The complexity of the polynomial coefficients dramatically increases with each added inhibitor. Solving the transcendental rate equation by traditional methods of numerical mathematics has proven tedious because of the sensitivity of these methods to initial estimates and because of the existence of multiple roots. However, the equation can be rearranged into a convenient recursive form, one in which the velocity appears on both sides and the solution is found iteratively. The algebraic form of the recursive rate equation is remarkably simple and differs from the rate equation for classical rather than tight-binding inhibition only by an added term. The numerical stability and the speed of convergence were tested on the case of two competitive inhibitors. Initial estimates of velocity that spanned 12 orders of magnitude converged within five iterations. The velocities computed with the recursive method for a single tight-binding inhibitor were identical with the values predicted by the Morrison equation. The method is used to analyze experimental data for the inhibition of rat liver dihydrofolate reductase by mixtures of the anticancer drug methotrexate and its metabolic precursor form, methotrexate-alpha-aspartate (a prodrug).     

A computational framework to empower probabilistic protein design

MOTIVATION: The task of engineering a protein to perform a target biological function is known as protein design. A commonly used paradigm casts this functional design problem as a structural one, assuming a fixed backbone. In probabilistic protein design, positional amino acid probabilities are used to create a random library of sequences to be simultaneously screened for biological activity. Clearly, certain choices of probability distributions will be more successful in yielding functional sequences. However, since the number of sequences is exponential in protein length, computational optimization of the distribution is difficult. RESULTS: In this paper, we develop a computational framework for probabilistic protein design following the structural paradigm. We formulate the distribution of sequences for a structure using the Boltzmann distribution over their free energies. The corresponding probabilistic graphical model is constructed, and we apply belief propagation (BP) to calculate marginal amino acid probabilities. We test this method on a large structural dataset and demonstrate the superiority of BP over previous methods. Nevertheless, since the results obtained by BP are far from optimal, we thoroughly assess the paradigm using high-quality experimental data. We demonstrate that, for small scale sub-problems, BP attains identical results to those produced by exact inference on the paradigmatic model. However, quantitative analysis shows that the distributions predicted significantly differ from the experimental data. These findings, along with the excellent performance we observed using BP on the smaller problems, suggest potential shortcomings of the paradigm. We conclude with a discussion of how it may be improved in the future.     

Wind power prediction based on neural network with optimization of adaptive multi-group salp swarm algorithm

Salp swarm algorithm (SSA) is a swarm intelligence algorithm inspired by the swarm behavior of salps in oceans. In this paper, a adaptive multi-group salp swarm algorithm (AMSSA) with three new communication strategies is presented. Adaptive multi-group mechanism is to evenly divide the initial population into several subgroups, and then exchange information among subgroups after each adaptive iteration. Communication strategy is also an important part of adaptive multi-group mechanism. This paper proposes three new communication strategies and focuses on promoting the performance of SSA. These measures significantly improve the cooperative ability of SSA, accelerate convergence speed, and avoid easily falling into local optimum. And the benchmark functions confirm that AMSSA is better than the original SSA in exploration and exploitation. In addition, AMSSA is combined with prediction of wind power based on back propagation (AMSSA-BP) neural network. The simulation results show that the AMSSA-BP neural network prediction model can achieve a better prediction effect of wind power.