Autonomous learning based on cost assumptions: theoretical studies and experiments in robot control

Autonomous learning techniques are based on experience acquisition. In most realistic applications, experience is time-consuming: it implies sensor reading, actuator control and algorithmic update, constrained by the learning system dynamics. The information crudeness upon which classical learning algorithms operate make such problems too difficult and unrealistic. Nonetheless, additional information for facilitating the learning process ideally should be embedded in such a way that the structural, well-studied characteristics of these fundamental algorithms are maintained. We investigate in this article a more general formulation of the Q-learning method that allows for a spreading of information derived from single updates towards a neighbourhood of the instantly visited state and converges to optimality. We show how this new formulation can be used as a mechanism to safely embed prior knowledge about the structure of the state space, and demonstrate it in a modified implementation of a reinforcement learning algorithm in a real robot navigation task.     

Autonomous learning based on cost assumptions: theoretical studies and experiments in robot control

Autonomous learning techniques are based on experience acquisition. In most realistic applications, experience is time-consuming: it implies sensor reading, actuator control and algorithmic update, constrained by the learning system dynamics. The information crudeness upon which classical learning algorithms operate make such problems too difficult and unrealistic. Nonetheless, additional information for facilitating the learning process ideally should be embedded in such a way that the structural, well-studied characteristics of these fundamental algorithms are maintained. We investigate in this article a more general formulation of the Q-learning method that allows for a spreading of information derived from single updates towards a neighbourhood of the instantly visited state and converges to optimality. We show how this new formulation can be used as a mechanism to safely embed prior knowledge about the structure of the state space, and demonstrate it in a modified implementation of a reinforcement learning algorithm in a real robot navigation task.     

H-CORE: enabling genome-scale Bayesian analysis of biological systems without prior knowledge

The Bayesian network is a popular tool for describing relationships between data entities by representing probabilistic (in)dependencies with a directed acyclic graph (DAG) structure. Relationships have been inferred between biological entities using the Bayesian network model with high-throughput data from biological systems in diverse fields. However, the scalability of those approaches is seriously restricted because of the huge search space for finding an optimal DAG structure in the process of Bayesian network learning. For this reason, most previous approaches limit the number of target entities or use additional knowledge to restrict the search space. In this paper, we use the hierarchical clustering and order restriction (H-CORE) method for the learning of large Bayesian networks by clustering entities and restricting edge directions between those clusters, with the aim of overcoming the scalability problem and thus making it possible to perform genome-scale Bayesian network analysis without additional biological knowledge. We use simulations to show that H-CORE is much faster than the widely used sparse candidate method, whilst being of comparable quality. We have also applied H-CORE to retrieving gene-to-gene relationships in a biological system (The 'Rosetta compendium'). By evaluating learned information through literature mining, we demonstrate that H-CORE enables the genome-scale Bayesian analysis of biological systems without any prior knowledge.     

Feedforward neural network with adaptive reference pattern layer.

A hybrid neural network architecture is investigated for modeling purposes. The proposed hybrid is based on the multilayer perceptron (MLP) network. In addition to the usual hidden layers, the first hidden layer is selected to be an adaptive reference pattern layer. Each unit in this new layer incorporates a reference pattern that is located somewhere in the space spanned by the input variables. The outputs of these units are the component wise-squared differences between the elements of a reference pattern and the inputs. The reference pattern layer has some resemblance to the hidden layer of the radial basis function (RBF) networks. Therefore the proposed design can be regarded as a sort of hybrid of MLP and RBF networks. The presented benchmark experiments show that the proposed hybrid can provide significant advantages over standard MLPs and RBFs in terms of fast and efficient learning, and compact network structure.     

Short-term prediction of chaotic time series by using RBF network with regression weights

We propose a framework for constructing and training a radial basis function (RBF) neural network. The structure of the gaussian functions is modified using a pseudo-gaussian function (PG) in which two scaling parameters sigma are introduced, which eliminates the symmetry restriction and provides the neurons in the hidden layer with greater flexibility with respect to function approximation. We propose a modified PG-BF (pseudo-gaussian basis function) network in which the regression weights are used to replace the constant weights in the output layer. For this purpose, a sequential learning algorithm is presented to adapt the structure of the network, in which it is possible to create a new hidden unit and also to detect and remove inactive units. A salient feature of the network systems is that the method used for calculating the overall output is the weighted average of the output associated with each receptive field. The superior performance of the proposed PG-BF system over the standard RBF are illustrated using the problem of short-term prediction of chaotic time series.     

Automatic determination of radial basis functions: an immunity-based approach

The appropriate operation of a radial basis function (RBF) neural network depends mainly upon an adequate choice of the parameters of its basis functions. The simplest approach to train an RBF network is to assume fixed radial basis functions defining the activation of the hidden units. Once the RBF parameters are fixed, the optimal set of output weights can be determined straightforwardly by using a linear least squares algorithm, which generally means reduction in the learning time as compared to the determination of all RBF network parameters using supervised learning. The main drawback of this strategy is the requirement of an efficient algorithm to determine the number, position, and dispersion of the RBFs. The approach proposed here is inspired by models derived from the vertebrate immune system, that will be shown to perform unsupervised cluster analysis. The algorithm is introduced and its performance is compared to that of the random, k-means center selection procedures and other results from the literature. By automatically defining the number of RBF centers, their positions and dispersions, the proposed method leads to parsimonious solutions. Simulation results are reported concerning regression and classification problems.     

基于熵准则的鲁棒的RBF谷胱甘肽发酵建模

在谷胱甘肽的发酵过程建模中, 当试验数据含有噪音时, 往往会导致模型预测精度和泛化能力的下降。针对该问题, 提出了一种新的基于熵准则的RBF神经网络建模方法。与传统的基于MSE准则函数的建模方法相比, 新方法能从训练样本的整体分布结构来进行模型参数学习, 有效地避免了传统的基于MSE准则的RBF网络的过学习和泛化能力差的缺陷。将该模型应用到实际的谷胱甘肽发酵过程建模中, 实验结果表明: 该方法具有较高的预测精度、泛化能力和良好的鲁棒性, 从而对谷胱甘肽的发酵建模有潜在的应用价值。     

基于熵准则的鲁棒的RBF谷胱甘肽发酵建模

在谷胱甘肽的发酵过程建模中, 当试验数据含有噪音时, 往往会导致模型预测精度和泛化能力的下降。针对该问题, 提出了一种新的基于熵准则的RBF神经网络建模方法。与传统的基于MSE准则函数的建模方法相比, 新方法能从训练样本的整体分布结构来进行模型参数学习, 有效地避免了传统的基于MSE准则的RBF网络的过学习和泛化能力差的缺陷。将该模型应用到实际的谷胱甘肽发酵过程建模中, 实验结果表明: 该方法具有较高的预测精度、泛化能力和良好的鲁棒性, 从而对谷胱甘肽的发酵建模有潜在的应用价值。     

Pruning and model-selecting algorithms in the RBF frameworks constructed by support vector learning

This paper presents the pruning and model-selecting algorithms to the support vector learning for sample classification and function regression. When constructing RBF network by support vector learning we occasionally obtain redundant support vectors which do not significantly affect the final classification and function approximation results. The pruning algorithms primarily based on the sensitivity measure and the penalty term. The kernel function parameters and the position of each support vector are updated in order to have minimal increase in error, and this makes the structure of SVM network more flexible. We illustrate this approach with synthetic data simulation and face detection problem in order to demonstrate the pruning effectiveness.     

Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer

Typical regimens for advanced metastatic stage IIIB/IV nonsmall cell lung cancer (NSCLC) consist of multiple lines of treatment. We present an adaptive reinforcement learning approach to discover optimal individualized treatment regimens from a specially designed clinical trial (a "clinical reinforcement trial") of an experimental treatment for patients with advanced NSCLC who have not been treated previously with systemic therapy. In addition to the complexity of the problem of selecting optimal compounds for first- and second-line treatments based on prognostic factors, another primary goal is to determine the optimal time to initiate second-line therapy, either immediately or delayed after induction therapy, yielding the longest overall survival time. A reinforcement learning method called Q-learning is utilized, which involves learning an optimal regimen from patient data generated from the clinical reinforcement trial. Approximating the Q-function with time-indexed parameters can be achieved by using a modification of support vector regression that can utilize censored data. Within this framework, a simulation study shows that the procedure can extract optimal regimens for two lines of treatment directly from clinical data without prior knowledge of the treatment effect mechanism. In addition, we demonstrate that the design reliably selects the best initial time for second-line therapy while taking into account the heterogeneity of NSCLC across patients.     

Parallel reinforcement learning for weighted multi-criteria model with adaptive margin

Reinforcement learning (RL) for a linear family of tasks is described in this paper. The key of our discussion is nonlinearity of the optimal solution even if the task family is linear; we cannot obtain the optimal policy using a naive approach. Although an algorithm exists for calculating the equivalent result to Q-learning for each task simultaneously, it presents the problem of explosion of set sizes. We therefore introduce adaptive margins to overcome this difficulty.     

From biological pathways to regulatory networks

This paper presents a general theoretical framework for generating Boolean networks whose state transitions realize a set of given biological pathways or minor variations thereof. This ill-posed inverse problem, which is of crucial importance across practically all areas of biology, is solved by using Karnaugh maps which are classical tools for digital system design. It is shown that the incorporation of prior knowledge, presented in the form of biological pathways, can bring about a dramatic reduction in the cardinality of the network search space. Constraining the connectivity of the network, the number and relative importance of the attractors, and concordance with observed time-course data are additional factors that can be used to further reduce the cardinality of the search space. The networks produced by the approaches developed here should facilitate the understanding of multivariate biological phenomena and the subsequent design of intervention approaches that are more likely to be successful in practice. As an example, the results of this paper are applied to the widely studied p53 pathway and it is shown that the resulting network exhibits dynamic behavior consistent with experimental observations from the published literature.     

普陀山岛旅游生态安全发展趋势预测

科学地预测海岛目的地旅游生态安全发展趋势,对促进海岛旅游经济和生态环境协调发展具有重要的理论意义和实践价值。基于可持续发展的视角,建立了由承载力-支持力-吸引力-延续力和发展力(CSAED模型)子系统构成的普陀山旅游生态安全指标体系,并在灰色系统GM(1,1)模型和RBF神经网络模型比较选优的基础上,对普陀山岛旅游生态安全发展趋势进行了预测。研究结果表明:(1)和灰色系统GM(1,1)模型相比,RBF神经网络模型的Pearson相关系数和误差均方根值更优,可更精确地拟合普陀山岛旅游生态安全发展趋势;(2)2015—2020年,普陀山岛旅游生态安全指数的RBF神经网络模型预测结果由0.7017增加至0.8135,安全等级由比较安全上升至非常安全。研究结果可为维护普陀山岛旅游生态安全提供决策依据。     

Complex-valued minimal resource allocation network for nonlinear signal processing

This paper presents a sequential learning algorithm and evaluates its performance on complex valued signal processing problems. The algorithm is referred to as Complex Minimal Resource Allocation Network (CMRAN) algorithm and it is an extension of the MRAN algorithm originally developed for online learning in real valued RBF networks. CMRAN has the ability to grow and prune the (complex) RBF network's hidden neurons to ensure a parsimonious network structure. The performance of the learning algorithm is illustrated using two applications from signal processing of communication systems. The first application considers identification of a nonlinear complex channel. The second application considers the use of CMRAN to QAM digital channel equalization problems. Simulation results presented clearly show that CMRAN is very effective in modeling and equalization with performance achieved often being superior to that of some of the well known methods.     

Text-independent speaker verification using Minimal Resource Allocation Networks

This paper presents a text-independent speaker verification system based on an online Radial Basis Function (RBF) network referred to as Minimal Resource Allocation Network (MRAN). MRAN is a sequential learning RBF, in which hidden neurons are added or removed as training progresses. LP-derived cepstral coefficients are used as feature vectors during training and verification phases. The performance of MRAN is compared with other well-known RBF and Elliptical Basis Function (EBF) based speaker verification methods in terms of error rates and computational complexity on a series of speaker verification experiments. The experiments use data from 258 speakers from the phonetically balancedcontinuous speech corpus TIMIT. The results show that MRAN produces comparable error rates to other methods with much less computational complexity.     

Biomimetic Construction of Category Mental Imagery Based on Recognition Mechanism of Visual Cortex of Human Brain

The principle of homology-continuity in Multi-Dimensional Biomimetic Informatics Space is applied to construct the identifying mechanism of category of deep representation of mental imagery. The model of each cerebral region involved in recognizing is established respectively and a feedforward method for establishing category mental imagery is proposed. First, the model of feature acquisition is developed based on Hubel-Wiesel model, and Gaussian function is used to simulate the simple cell receptive field to satisfy the specific function of visual cortex. Second, multiple input aggregation operation is employed to simulate the feature output of complex cells to get the invariance representation in feature space. Then, imagery basis is extracted by unsupervised learning algorithm based on the primary feature and category mental imagery is obtained by building Radial Basis Function (RBF) network. Finally, the system model is tested by training set and test set composed of real images. Experimental results show that the proposed method can establish valid deep representation of these samples, based on which the biomimetic construction of category mental imagery can be achieved. This method provides a new idea for solving imagery problem and studying imagery thinking.     

On the structure and initial parameter identification of Gaussian RBF networks

We consider the efficient initialization of structure and parameters of generalized Gaussian radial basis function (RBF) networks using fuzzy decision trees generated by fuzzy ID3 like induction algorithms. The initialization scheme is based on the proposed functional equivalence property of fuzzy decision trees and generalized Gaussian RBF networks. The resulting RBF network is compact, easy to induce, comprehensible, and has acceptable classification accuracy with stochastic gradient descent learning algorithm.     

EEG source localization: comparative study of classical and neural network methods.

We address the problem of estimating biopotential sources within the brain, based on EEG signals observed on the scalp. This problem, known as the inverse problem of electrophysiology, has no closed-form solution, and requires iterative techniques such as the Levenberg-Marquardt (LM) algorithm. Considering the nonlinear nature of the inverse problem, and the low signal to noise ratio inherent in EEG signals, a backpropagation neural network (BPN) has been recently proposed as a solution. The technique has not been properly compared with classical techniques such as the LM method, or with more recent neural network techniques such as the Radial Basis Function (RBF) network. In this paper, we provide improved strategies based on BPN and consider RBF networks in solving the inverse problem. We compare the performances of BPN, RBF and a hybrid technique with that of the classical LM method.     

Effect of polynomial,radial basis,and Pearson VII function kernels in support vector machine algorithm for classification of crayfish

Freshwater crayfish are one of the most important aquatic organisms that play a pivotal role in the aquatic food chain as well as serving as bioindicators for the aquatic ecosystem health assessment. Hemocytes, the blood cells of crustaceans, can be considered stress and health indicators in crayfish, and are used to evaluate the health response. Therefore, total hemocyte cell numbers (THCs) are useful parameters to show the health of crustaceans and serve as stress indicators to decide the quality of the habitat. Since, catching the fish and the other aquatic organisms, and collecting the data for further assessments are time-consuming and frustrating, today, scientists tend to use swift, more sophisticated, and more reliable methods for modeling the ecosystem stressors based on bioindicators. One tool which has attracted the attention of science communities in the last decades is machine learning algorithms that are reliable and accurate methods to solve classification and regression problems. In this study, a support vector machine is carried out as a machine learning algorithm to classify healthy and unhealthy crayfish based on physiological characteristics. To solve the non-linearity problem of the data by transporting data to high-dimensional space, different kernel functions including polynomial (PK), Pearson VII function-based universal (PUK), and radial basis function (RBF) kernels are used and their effect on the performance of the SVM model was evaluated. Both PK and PUK functions performed well in classifying the crayfish. RBF, however, had an adverse impact on the performance of the model. PUK kernel exhibited an outstanding performance (Accuracy = 100%) for the classification of the healthy and unhealthy crayfish.