A novel method for predicting protein subcellular localization based on pseudo amino acid composition

In this paper, a novel approach, ELM-PCA, is introduced for the first time to predict protein subcellular localization. Firstly, Protein Samples are represented by the pseudo amino acid composition (PseAAC). Secondly, the principal component analysis (PCA) is employed to extract essential features. Finally, the Elman Recurrent Neural Network (RNN) is used as a classifier to identify the protein sequences. The results demonstrate that the proposed approach is effective and practical.     

Neural networks and linear programming for the satisfiability problem.

First a Linear Programming formulation is considered for the satisfiability problem, in particular for the satisfaction of a Conjunctive Normal Form in the Propositional Calculus and the Simplex algorithm for solving the optimization problem. The use of Recurrent Neural Networks is then described for choosing the best pivot positions and greatly improving the algorithm performance. The result of hard cases testing is reported and shows that the technique can be useful even if it requires a huge amount of size for the constraint array and Neural Network Data Input.     

White Blood Cells Image Classification Using Deep Learning with Canonical Correlation Analysis

White Blood Cells play an important role in observing the health condition of an individual. The opinion related to blood disease involves the identification and characterization of a patient's blood sample. Recent approaches employ Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and merging of CNN and RNN models to enrich the understanding of image content. From beginning to end, training of big data in medical image analysis has encouraged us to discover prominent features from sample images. A single cell patch extraction from blood sample techniques for blood cell classification has resulted in the good performance rate. However, these approaches are unable to address the issues of multiple cells overlap. To address this problem, the Canonical Correlation Analysis (CCA) method is used in this paper. CCA method views the effects of overlapping nuclei where multiple nuclei patches are extracted, learned and trained at a time. Due to overlapping of blood cell images, the classification time is reduced, the dimension of input images gets compressed and the network converges faster with more accurate weight parameters. Experimental results evaluated using publicly available database show that the proposed CNN and RNN merging model with canonical correlation analysis determines higher accuracy compared to other state-of-the-art blood cell classification techniques.     

Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, e.g. fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of RNNs may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics.     

Towards the next generation of recurrent network models for cognitive neuroscience

Recurrent neural networks (RNNs) trained with machine learning techniques on cognitive tasks have become a widely accepted tool for neuroscientists. In this short opinion piece, we discuss fundamental challenges faced by the early work of this approach and recent steps to overcome such challenges and build next-generation RNN models for cognition. We propose several essential questions that practitioners of this approach should address to continue to build future generations of RNN models.     

Training simultaneous recurrent neural network with resilient propagation for static optimization

This paper proposes a non-recurrent training algorithm, resilient propagation, for the Simultaneous Recurrent Neural network operating in relaxation-mode for computing high quality solutions of static optimization problems. Implementation details related to adaptation of the recurrent neural network weights through the non-recurrent training algorithm, resilient backpropagation, are formulated through an algebraic approach. Performance of the proposed neuro-optimizer on a well-known static combinatorial optimization problem, the Traveling Salesman Problem, is evaluated on the basis of computational complexity measures and, subsequently, compared to performance of the Simultaneous Recurrent Neural network trained with the standard backpropagation, and recurrent backpropagation for the same static optimization problem. Simulation results indicate that the Simultaneous Recurrent Neural network trained with the resilient backpropagation algorithm is able to locate superior quality solutions through comparable amount of computational effort for the Traveling Salesman Problem.     

Wheelchair propulsion: Force orientation and amplitude prediction with Recurrent Neural Network

The aim of this study was to use Recurrent Neural Network (RNN) to predict the orientation and amplitude of the applied force during the push phase of manual wheelchair propulsion.Trunk and the right-upper limb kinematics data were assessed with an optoeletronic device (Qualisys) and the force applied on the handrim was recorded with an instrumented wheel (SMARTWheel®). Data acquisitions were performed at 60/80/10/120/140% of the freely chosen frequency at submaximal and maximal conditions. The final database consisted of d = 5708 push phases.The input data were the trunk and right upper-limb kinematics (joint angle, angular velocity and acceleration) and anthropometric data (height, weight, segment length) and the output data were the applied forces orientation and amplitude. A ratio of 70/15/15 was used to train, validate and test the RNN (d_train = 3996, d_validation = 856 and d_test = 856). The angle and amplitude errors between the measured and predicted force was assessed from d_test.Results showed that for most of the push phase (∼70%), the force direction prediction errors were less than 12°. The mean absolute amplitude errors were less than 8 N and the mean absolute amplitude percentage errors were less than 20% for most of the push phase (∼80%).     

Hybrid neural systems for pattern recognition in artificial noses

This work examines the use of Hybrid Intelligent Systems in the pattern recognition system of an artificial nose. The connectionist approaches Multi-Layer Perceptron and Time Delay Neural Networks, and the hybrid approaches Feature-Weighted Detector and Evolving Neural Fuzzy Networks were investigated. A Wavelet Filter is evaluated as a preprocessing method for odor signals. The signals generated by an artificial nose were composed by an array of conducting polymer sensors and exposed to two different odor databases.     

Brain-inspired model for early vocal learning and correspondence matching using free-energy optimization

We propose a developmental model inspired by the cortico-basal system (CX-BG) for vocal learning in babies and for solving the correspondence mismatch problem they face when they hear unfamiliar voices, with different tones and pitches. This model is based on the neural architecture INFERNO standing for Iterative Free-Energy Optimization of Recurrent Neural Networks. Free-energy minimization is used for rapidly exploring, selecting and learning the optimal choices of actions to perform (eg sound production) in order to reproduce and control as accurately as possible the spike trains representing desired perceptions (eg sound categories). We detail in this paper the CX-BG system responsible for linking causally the sound and motor primitives at the order of a few milliseconds. Two experiments performed with a small and a large audio database show the capabilities of exploration, generalization and robustness to noise of our neural architecture in retrieving audio primitives during vocal learning and during acoustic matching with unheared voices (different genders and tones).     

Modular neural networks: a survey.

Modular Neural Networks (MNNs) is a rapidly growing field in artificial Neural Networks (NNs) research. This paper surveys the different motivations for creating MNNs: biological, psychological, hardware, and computational. Then, the general stages of MNN design are outlined and surveyed as well, viz., task decomposition techniques, learning schemes and multi-module decision-making strategies. Advantages and disadvantages of the surveyed methods are pointed out, and an assessment with respect to practical potential is provided. Finally, some general recommendations for future designs are presented.     

A comparative evaluation of neural network classifiers for stress level analysis of automotive drivers using physiological signals

Automotive driving under unacceptable levels of accumulated stress deteriorates their vehicle control and risk-assessment capabilities often inviting road accidents. Design of a safety-critical wearable driver assist system for continuous stress level monitoring requires development of an intelligent algorithm capable of recognizing the drivers’ affective state and cumulatively account for increasing stress level. Task induced modifications in rhythms of physiological signals acquired during a real-time driving are clinically proven hallmarks for quantitative analysis of stress and mental fatigue. The present work proposes a neural network driven based solution to learning driving-induced stress patterns and correlating it with statistical, structural and time-frequency changes observed in the recorded biosignals. Physiological signals like Galvanic Skin Response (GSR) and Photoplethysmography (PPG) were selected for the present work. A comprehensive performance analysis on the selected neural network configurations (both Feed forward and Recurrent) concluded that Layer Recurrent Neural Networks are most optimal for stress level detection. This evaluation achieved an average precision of 89.23%, sensitivity of 88.83% and specificity of 94.92% when tested over 19 automotive drivers. The biofeedback inferred about the driver's ongoing physiological state using this neural network based inference engine would provide crucial information to on-board safety embedded systems to activate accordingly. It is envisaged that such a driver-centric safety system will help save precious lives by way of providing fast and credible real-time alerts to drivers and their coupled cars.     

Reinforcement learning with internal expectation in the random neural networks for cascaded decisions

The reinforcement learning scheme proposed in Halici (J. Biosystems 40 (1997) 83) for the random neural network (RNN) (Neural Computation 1 (1989) 502) is based on reward and performs well for stationary environments. However, when the environment is not stationary it suffers from getting stuck to the previously learned action and extinction is not possible. To overcome the problem, the reinforcement scheme is extended in Halici (Eur. J. Oper. Res., 126(2000) 288) by introducing a new weight update rule (E-rule) which takes into consideration the internal expectation of reinforcement. Although the E-rule is proposed for the RNN, it can be used for training learning automata or other intelligent systems based on reinforcement learning. This paper looks into the behavior of the learning scheme with internal expectation for the environments where the reinforcement is obtained after a sequence of cascaded decisions. The simulation results have shown that the RNN learns well and extinction is possible even for the cases with several decision steps and with hundreds of possible decision paths.     

真菌深层培养过程的房室结构神经网络模型

在对横纹黑蛋巢菌深层培养过程进行分析的基础上,提出一种房室结构的神经网络模型,利用ＲＢＦ网络这各房室的输入,输出关系,并进一步对整个生化过程作了建模型研究,计算结果表明,所建模型性能较佳,对真菌培养过程的观测数据拟合结果令人满意。     

Hardware implementation of stochastic spiking neural networks

Spiking Neural Networks, the last generation of Artificial Neural Networks, are characterized by its bio-inspired nature and by a higher computational capacity with respect to other neural models. In real biological neurons, stochastic processes represent an important mechanism of neural behavior and are responsible of its special arithmetic capabilities. In this work we present a simple hardware implementation of spiking neurons that considers this probabilistic nature. The advantage of the proposed implementation is that it is fully digital and therefore can be massively implemented in Field Programmable Gate Arrays. The high computational capabilities of the proposed model are demonstrated by the study of both feed-forward and recurrent networks that are able to implement high-speed signal filtering and to solve complex systems of linear equations.     

cuConv: CUDA implementation of convolution for CNN inference

Cluster Computing - Convolutions are the core operation of deep learning applications based on Convolutional Neural Networks (CNNs). Current GPU architectures are highly efficient for training and...     

A saliency-based bottom-up visual attention model for dynamic scenes analysis

This work proposes a model of visual bottom-up attention for dynamic scene analysis. Our work adds motion saliency calculations to a neural network model with realistic temporal dynamics [(e.g., building motion salience on top of De Brecht and Saiki Neural Networks 19:1467–1474, (2006)]. The resulting network elicits strong transient responses to moving objects and reaches stability within a biologically plausible time interval. The responses are statistically different comparing between earlier and later motion neural activity; and between moving and non-moving objects. We demonstrate the network on a number of synthetic and real dynamical movie examples. We show that the model captures the motion saliency asymmetry phenomenon. In addition, the motion salience computation enables sudden-onset moving objects that are less salient in the static scene to rise above others. Finally, we include strong consideration for the neural latencies, the Lyapunov stability, and the neural properties being reproduced by the model.     

Book Reviews

Books reviewed in this article:
HOCKING, R. R. Methods and Applications of Linear Models: Regression and the Analysis of Variance.
VERBEKE, G. and MOLENBERGHS, G. (editors). Linear Mixed Models in Practice: A SAS-Oriented Approach.
CHRISTENSEN, R. Analysis of Variance, Design and Regression: Applied Statistical Methods.
BISHOP, C. M. Neural Networks for Pattern Recognition.
RIPLEY, B. D. Pattern Recognition and Neural Networks.
GRENANDER, U. Elements of Pattern Theory.
ZEISEL, H. and KAYE, D. Prove It with Figures: Empirical Methods in Law and Litigation.
AITKEN, C. G. G. Statistics and the Evaluation of Evidence for Forensic Scientists.     

Detection of abnormal fermentations in wine process by multivariate statistics and pattern recognition techniques

Three multivariate statistical techniques (Multiway Principal Component Analysis, Multiway Partial Least Squares, and Stepwise Linear Discriminant Analysis) and one artificial intelligence method (Artificial Neural Networks) were evaluated to detect and predict early abnormal behaviors of wine fermentations. The techniques were tested with data of thirty-two variables at different stages of fermentation from industrial wine fermentations of Cabernet Sauvignon. All the techniques studied considered a pre-treatment to obtain a homogeneous space and reduce the overfitting. The results were encouraging; it was possible to classify at 72h 100% of the fermentation correctly with three variables using Multiway Partial Least Squares and Artificial Neural Networks. Additional and complementary results were obtained with Stepwise Linear Discriminant Analysis, which found that ethanol, sugars and density measurements are able to discriminate abnormal behavior.