Deterministic convergence of chaos injection-based gradient method for training feedforward neural networks

It has been shown that, by adding a chaotic sequence to the weight update during the training of neural networks, the chaos injection-based gradient method (CIBGM) is superior to the standard backpropagation algorithm. This paper presents the theoretical convergence analysis of CIBGM for training feedforward neural networks. We consider both the case of batch learning as well as the case of online learning. Under mild conditions, we prove the weak convergence, i.e., the training error tends to a constant and the gradient of the error function tends to zero. Moreover, the strong convergence of CIBGM is also obtained with the help of an extra condition. The theoretical results are substantiated by a simulation example.     

基于人工神经网络的非线性拟合方法ⅡBP学习算法中各因素的影响

本文研究了人工神经网络BP学习算法中动量因子、隐节点数、学习速率、激活因子等对网络学习速度有影响的几个因素,并且找出了最佳值．     

Nonlinear system identification using a neo fuzzy neuron algorithm: electrical drive application

This work presents a Neo-Fuzzy-Neuron algorithm for the identification of nonlinear dynamic systems at the point of view of a rotor flux observer. The algorithm training is on-line, has low computational cost, does not require previous training and its convergence in one step is proved. The gradient descent method is used for its weights adjustment. Simulation and experimental results demonstrate the effectiveness of the algorithm for flux observer of induction motor drive system.     

Combining efficient conformational sampling with a deformable elastic network model facilitates structure refinement at low resolution

Structural studies of large proteins and protein assemblies are a difficult and pressing challenge in molecular biology. Experiments often yield only low-resolution or sparse data that are not sufficient to fully determine atomistic structures. We have developed a general geometry-based algorithm that efficiently samples conformational space under constraints imposed by low-resolution density maps obtained from electron microscopy or X-ray crystallography experiments. A deformable elastic network (DEN) is used to restrain the sampling to prior knowledge of an approximate structure. The DEN restraints dramatically reduce over-fitting, especially at low resolution. Cross-validation is used to optimally weight the structural information and experimental data. Our algorithm is robust even for noise-added density maps and has a large radius of convergence for our test case. The DEN restraints can also be used to enhance reciprocal space simulated annealing refinement.     

New learning automata based algorithms for adaptation of backpropagation algorithm parameters

One popular learning algorithm for feedforward neural networks is the backpropagation (BP) algorithm which includes parameters, learning rate (eta), momentum factor (alpha) and steepness parameter (lambda). The appropriate selections of these parameters have large effects on the convergence of the algorithm. Many techniques that adaptively adjust these parameters have been developed to increase speed of convergence. In this paper, we shall present several classes of learning automata based solutions to the problem of adaptation of BP algorithm parameters. By interconnection of learning automata to the feedforward neural networks, we use learning automata scheme for adjusting the parameters eta, alpha, and lambda based on the observation of random response of the neural networks. One of the important aspects of the proposed schemes is its ability to escape from local minima with high possibility during the training period. The feasibility of proposed methods is shown through simulations on several problems.     

A linear memory algorithm for Baum-Welch training

Background:  

Baum-Welch training is an expectation-maximisation algorithm for training the emission and transition probabilities of hidden Markov models in a fully automated way. It can be employed as long as a training set of annotated sequences is known, and provides a rigorous way to derive parameter values which are guaranteed to be at least locally optimal. For complex hidden Markov models such as pair hidden Markov models and very long training sequences, even the most efficient algorithms for Baum-Welch training are currently too memory-consuming. This has so far effectively prevented the automatic parameter training of hidden Markov models that are currently used for biological sequence analyses.     

FPGA implementation of a pyramidal Weightless Neural Networks learning system

A hardware architecture of a Probabilistic Logic Neuron (PLN) is presented. The suggested model facilitates the on-chip learning of pyramidal Weightless Neural Networks using a modified probabilistic search reward/penalty training algorithm. The penalization strategy of the training algorithm depends on a predefined parameter called the probabilistic search interval. A complete Weightless Neural Network (WNN) learning system is modeled and implemented on Xilinx XC4005E Field Programmable Gate Array (FPGA), allowing its architecture to be configurable. Various experiments have been conducted to examine the feasibility and performance of the WNN learning system. Results show that the system has a fast convergence rate and good generalization ability.     

模块神经网络及其性能

提出一个不同于实现ｙ＝ｆ（ｘ）的ＢＰ网络的神经网络模型,给出了网络的结构及并行动力学方程,证明了其动力学的稳定性。通过学习算法的建立,证明网络能精确实现输入矢量对（ｘ,ｙ）映入成相联系的输出矢量ｚ,最重要的是网络能同时存诸依时变化的时序模式与静态模式。此外并给出动力学学习算法,证明此学习算法的收敛性,计算机仿真证实理论结果,最后讨论了某些可能的应用。     

Estimation of a covariance matrix with zeros

We consider estimation of the covariance matrix of a multivariaterandom vector under the constraint that certain covariancesare zero. We first present an algorithm, which we call iterativeconditional fitting, for computing the maximum likelihood estimateof the constrained covariance matrix, under the assumption ofmultivariate normality. In contrast to previous approaches,this algorithm has guaranteed convergence properties. Droppingthe assumption of multivariate normality, we show how to estimatethe covariance matrix in an empirical likelihood approach. Theseapproaches are then compared via simulation and on an exampleof gene expression.     

A Novel Algorithm for Independent Component Analysis with Reference and Methods for Its Applications

This paper presents a stable and fast algorithm for independent component analysis with reference (ICA-R). This is a technique for incorporating available reference signals into the ICA contrast function so as to form an augmented Lagrangian function under the framework of constrained ICA (cICA). The previous ICA-R algorithm was constructed by solving the optimization problem via a Newton-like learning style. Unfortunately, the slow convergence and potential misconvergence limit the capability of ICA-R. This paper first investigates and probes the flaws of the previous algorithm and then introduces a new stable algorithm with a faster convergence speed. There are two other highlights in this paper: first, new approaches, including the reference deflation technique and a direct way of obtaining references, are introduced to facilitate the application of ICA-R; second, a new method is proposed that the new ICA-R is used to recover the complete underlying sources with new advantages compared with other classical ICA methods. Finally, the experiments on both synthetic and real-world data verify the better performance of the new algorithm over both previous ICA-R and other well-known methods.     

Machine learning algorithms based on signals from a single wearable inertial sensor can detect surface- and age-related differences in walking

The aim of this study was to investigate if a machine learning algorithm utilizing triaxial accelerometer, gyroscope, and magnetometer data from an inertial motion unit (IMU) could detect surface- and age-related differences in walking. Seventeen older (71.5 ± 4.2 years) and eighteen young (27.0 ± 4.7 years) healthy adults walked over flat and uneven brick surfaces wearing an inertial measurement unit (IMU) over the L5 vertebra. IMU data were binned into smaller data segments using 4-s sliding windows with 1-s step lengths. Ninety percent of the data were used as training inputs and the remaining ten percent were saved for testing. A deep learning network with long short-term memory units was used for training (fully supervised), prediction, and implementation. Four models were trained using the following inputs: all nine channels from every sensor in the IMU (fully trained model), accelerometer signals alone, gyroscope signals alone, and magnetometer signals alone. The fully trained models for surface and age outperformed all other models (area under the receiver operator curve, AUC = 0.97 and 0.96, respectively; p ≤ .045). The fully trained models for surface and age had high accuracy (96.3, 94.7%), precision (96.4, 95.2%), recall (96.3, 94.7%), and f1-score (96.3, 94.6%). These results demonstrate that processing the signals of a single IMU device with machine-learning algorithms enables the detection of surface conditions and age-group status from an individual’s walking behavior which, with further learning, may be utilized to facilitate identifying and intervening on fall risk.     

On the convergence of a clustering algorithm for protein-coding regions in microbial genomes

MOTIVATION: As the number of fully sequenced prokaryotic genomes continues to grow rapidly, computational methods for reliably detecting protein-coding regions become even more important. Audic and Claverie (1998) Proc. Natl Acad. Sci. USA, 95, 10026-10031, have proposed a clustering algorithm for protein-coding regions in microbial genomes. The algorithm is based on three Markov models of order k associated with subsequences extracted from a given genome. The parameters of the three Markov models are recursively updated by the algorithm which, in simulations, always appear to converge to a unique stable partition of the genome. The partition corresponds to three kinds of regions: (1) coding on the direct strand, (2) coding on the complementary strand, (3) non-coding. RESULTS: Here we provide an explanation for the convergence of the algorithm by observing that it is essentially a form of the expectation maximization (EM) algorithm applied to the corresponding mixture model. We also provide a partial justification for the uniqueness of the partition based on identifiability. Other possible variations and improvements are briefly discussed.     

A neural network model for reconstructing EMG signals from eight shoulder muscles: consequences for rehabilitation robotics and biofeedback

This paper demonstrates the ability of a fully connected feed forward neural network (FFNN) using the backpropagation training algorithm to predict the electromyography (EMG) signal from eight muscles of the shoulder for both exercises not used for training and EMG signals from subjects not used for training. The network showed a good predictive ability for subjects not used for training (r(2) between 0.33 and 0.84) and for activities not used for training (r(2) between 0.56 and 0.89). This may have applications for patients, physical therapists and doctors to monitor patient performance by reviewing the level of agreement between the patient EMG and the predicted EMG. Coupled with traditional methods of evaluation, EMG can provide an excellent measure of how well a patient has responded or is responding to treatment. Incorporating robotic technology could facilitate the use of EMG as an input to an intelligent decision making algorithm used to increase or decrease the level of difficulty according to patient performance.     

A Limited-Memory BFGS Algorithm Based on a Trust-Region Quadratic Model for Large-Scale Nonlinear Equations

In this paper, a trust-region algorithm is proposed for large-scale nonlinear equations, where the limited-memory BFGS (L-M-BFGS) update matrix is used in the trust-region subproblem to improve the effectiveness of the algorithm for large-scale problems. The global convergence of the presented method is established under suitable conditions. The numerical results of the test problems show that the method is competitive with the norm method.     

A cellular mechanism for multi-robot construction via evolutionary multi-objective optimization of a gene regulatory network

A major research challenge of multi-robot systems is to predict the emerging behaviors from the local interactions of the individual agents. Biological systems can generate robust and complex behaviors through relatively simple local interactions in a world characterized by rapid changes, high uncertainty, infinite richness, and limited availability of information. Gene Regulatory Networks (GRNs) play a central role in understanding natural evolution and development of biological organisms from cells. In this paper, inspired by biological organisms, we propose a distributed GRN-based algorithm for a multi-robot construction task. Through this algorithm, multiple robots can self-organize autonomously into different predefined shapes, and self-reorganize adaptively under dynamic environments. This developmental process is evolved using a multi-objective optimization algorithm to achieve a shorter travel distance and less convergence time. Furthermore, a theoretical proof of the system's convergence is also provided. Various case studies have been conducted in the simulation, and the results show the efficiency and convergence of the proposed method.     

Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images

We describe a protocol for fully automated detection and segmentation of asymmetric, presumed excitatory, synapses in serial electron microscopy images of the adult mammalian cerebral cortex, taken with the focused ion beam, scanning electron microscope (FIB/SEM). The procedure is based on interactive machine learning and only requires a few labeled synapses for training. The statistical learning is performed on geometrical features of 3D neighborhoods of each voxel and can fully exploit the high z-resolution of the data. On a quantitative validation dataset of 111 synapses in 409 images of 1948×1342 pixels with manual annotations by three independent experts the error rate of the algorithm was found to be comparable to that of the experts (0.92 recall at 0.89 precision). Our software offers a convenient interface for labeling the training data and the possibility to visualize and proofread the results in 3D. The source code, the test dataset and the ground truth annotation are freely available on the website http://www.ilastik.org/synapse-detection.     

Decoupled time-marching schemes in computational cardiac electrophysiology and ECG numerical simulation

This work considers the approximation of the cardiac bidomain equations, either isolated or coupled with the torso, via first order semi-implicit time-marching schemes involving a fully decoupled computation of the unknown fields (ionic state, transmembrane potential, extracellular and torso potentials). For the isolated bidomain system, we show that the Gauss-Seidel and Jacobi like splittings do not compromise energy stability; they simply alter the energy norm. Within the framework of the numerical simulation of electrocardiograms (ECG), these bidomain splittings are combined with an explicit Robin-Robin treatment of the heart-torso coupling conditions. We show that the resulting schemes allow a fully decoupled (energy) stable computation of the heart and torso fields, under an additional hyperbolic-CFL like condition. The accuracy and convergence rate of the considered schemes are investigated numerically with a series of numerical experiments.     

Partly functional temporal process regression with semiparametric profile estimating functions

Summary .  Marginal mean models of temporal processes in event time data analysis are gaining more attention for their milder assumptions than the traditional intensity models. Recent work on fully functional temporal process regression (TPR) offers great flexibility by allowing all the regression coefficients to be nonparametrically time varying. The existing estimation procedure, however, prevents successive goodness-of-fit test for covariate coefficients in comparing a sequence of nested models. This article proposes a partly functional TPR model in the line of marginal mean models. Some covariate effects are time independent while others are completely unspecified in time. This class of models is very rich, including the fully functional model and the semiparametric model as special cases. To estimate the parameters, we propose semiparametric profile estimating equations, which are solved via an iterative algorithm, starting at a consistent estimate from a fully functional model in the existing work. No smoothing is needed, in contrast to other varying-coefficient methods. The weak convergence of the resultant estimators are developed using the empirical process theory. Successive tests of time-varying effects and backward model selection procedure can then be carried out. The practical usefulness of the methodology is demonstrated through a simulation study and a real example of recurrent exacerbation among cystic fibrosis patients.     

Smoothing spline ANOVA frailty model for recurrent event data

Gap time hazard estimation is of particular interest in recurrent event data. This article proposes a fully nonparametric approach for estimating the gap time hazard. Smoothing spline analysis of variance (ANOVA) decompositions are used to model the log gap time hazard as a joint function of gap time and covariates, and general frailty is introduced to account for between-subject heterogeneity and within-subject correlation. We estimate the nonparametric gap time hazard function and parameters in the frailty distribution using a combination of the Newton-Raphson procedure, the stochastic approximation algorithm (SAA), and the Markov chain Monte Carlo (MCMC) method. The convergence of the algorithm is guaranteed by decreasing the step size of parameter update and/or increasing the MCMC sample size along iterations. Model selection procedure is also developed to identify negligible components in a functional ANOVA decomposition of the log gap time hazard. We evaluate the proposed methods with simulation studies and illustrate its use through the analysis of bladder tumor data.