On the structure and initial parameter identification of Gaussian RBF networks

We consider the efficient initialization of structure and parameters of generalized Gaussian radial basis function (RBF) networks using fuzzy decision trees generated by fuzzy ID3 like induction algorithms. The initialization scheme is based on the proposed functional equivalence property of fuzzy decision trees and generalized Gaussian RBF networks. The resulting RBF network is compact, easy to induce, comprehensible, and has acceptable classification accuracy with stochastic gradient descent learning algorithm.     

An online self-organizing scheme for Parsimonious and Accurate Fuzzy Neural Networks

In this paper, an online self-organizing scheme for Parsimonious and Accurate Fuzzy Neural Networks (PAFNN), and a novel structure learning algorithm incorporating a pruning strategy into novel growth criteria are presented. The proposed growing procedure without pruning not only simplifies the online learning process but also facilitates the formation of a more parsimonious fuzzy neural network. By virtue of optimal parameter identification, high performance and accuracy can be obtained. The learning phase of the PAFNN involves two stages, namely structure learning and parameter learning. In structure learning, the PAFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In parameter learning, parameters in premises and consequents of fuzzy rules, regardless of whether they are newly created or already in existence, are updated by the extended Kalman filter (EKF) method and the linear least squares (LLS) algorithm, respectively. This parameter adjustment paradigm enables optimization of parameters in each learning epoch so that high performance can be achieved. The effectiveness and superiority of the PAFNN paradigm are demonstrated by comparing the proposed method with state-of-the-art methods. Simulation results on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification and chaotic time-series prediction demonstrate that the proposed PAFNN algorithm can achieve more parsimonious network structure, higher approximation accuracy and better generalization simultaneously.     

Comprehensive decision tree models in bioinformatics

Purpose

Classification is an important and widely used machine learning technique in bioinformatics. Researchers and other end-users of machine learning software often prefer to work with comprehensible models where knowledge extraction and explanation of reasoning behind the classification model are possible.

Methods

This paper presents an extension to an existing machine learning environment and a study on visual tuning of decision tree classifiers. The motivation for this research comes from the need to build effective and easily interpretable decision tree models by so called one-button data mining approach where no parameter tuning is needed. To avoid bias in classification, no classification performance measure is used during the tuning of the model that is constrained exclusively by the dimensions of the produced decision tree.

Results

The proposed visual tuning of decision trees was evaluated on 40 datasets containing classical machine learning problems and 31 datasets from the field of bioinformatics. Although we did not expected significant differences in classification performance, the results demonstrate a significant increase of accuracy in less complex visually tuned decision trees. In contrast to classical machine learning benchmarking datasets, we observe higher accuracy gains in bioinformatics datasets. Additionally, a user study was carried out to confirm the assumption that the tree tuning times are significantly lower for the proposed method in comparison to manual tuning of the decision tree.

Conclusions

The empirical results demonstrate that by building simple models constrained by predefined visual boundaries, one not only achieves good comprehensibility, but also very good classification performance that does not differ from usually more complex models built using default settings of the classical decision tree algorithm. In addition, our study demonstrates the suitability of visually tuned decision trees for datasets with binary class attributes and a high number of possibly redundant attributes that are very common in bioinformatics.     

FiberID--a technique to identify fibrous protein subclasses

Fibrous proteins such as collagen, silk, and elastin play critical biological roles, yet they have been the subject of few projects that use computational techniques to predict either their class or their structure. In this article, we present FiberID, a simple yet effective method for identifying and distinguishing three fibrous protein subclasses from their primary sequences. Using a combination of amino acid composition and fast Fourier measurements, FiberID can classify fibrous proteins belonging to these subclasses with high accuracy by using two standard machine learning techniques (decision trees and Naïve Bayesian classifiers). After presenting our results, we present several fibrous sequences that are regularly misclassified by FiberID as sequences of potential interest for further study. Finally, we analyze the decision trees developed by FiberID for potential insights regarding the structure of these proteins. Proteins 2007. © 2006 Wiley‐Liss, Inc.     

Risk-Based Evaluation of Turkish Agricultural Strategies using Fuzzy AHP and Fuzzy ANP

In this article, we propose a model for selection of an agricultural strategy for Turkey by using fuzzy analytical hierarchy process (AHP) and analytical network process (ANP) based on linguistic terms. Fuzzy AHP-based methodology will be discussed to tackle the different decision criteria such as risk factors, ecological structure, socioeconomic structure, and technological structure involved in the selection of an agricultural strategy for Turkey and ANP will represent an effective tool for providing a suitable solution for managers and government administrators of this case. The linguistic levels of comparisons produced by the experts for each comparison are tapped in the form of triangular fuzzy numbers in order to construct fuzzy pairwise comparison matrices. The implementation of the system is demonstrated by a problem having four stages of hierarchy that contains 4 criteria and 15 sub-criteria.     

A novel efficient learning algorithm for self-generating fuzzy neural network with applications

In this paper, a novel efficient learning algorithm towards self-generating fuzzy neural network (SGFNN) is proposed based on ellipsoidal basis function (EBF) and is functionally equivalent to a Takagi-Sugeno-Kang (TSK) fuzzy system. The proposed algorithm is simple and efficient and is able to generate a fuzzy neural network with high accuracy and compact structure. The structure learning algorithm of the proposed SGFNN combines criteria of fuzzy-rule generation with a pruning technology. The Kalman filter (KF) algorithm is used to adjust the consequent parameters of the SGFNN. The SGFNN is employed in a wide range of applications ranging from function approximation and nonlinear system identification to chaotic time-series prediction problem and real-world fuel consumption prediction problem. Simulation results and comparative studies with other algorithms demonstrate that a more compact architecture with high performance can be obtained by the proposed algorithm. In particular, this paper presents an adaptive modeling and control scheme for drug delivery system based on the proposed SGFNN. Simulation study demonstrates the ability of the proposed approach for estimating the drug's effect and regulating blood pressure at a prescribed level.     

New learning automata based algorithms for adaptation of backpropagation algorithm parameters

One popular learning algorithm for feedforward neural networks is the backpropagation (BP) algorithm which includes parameters, learning rate (eta), momentum factor (alpha) and steepness parameter (lambda). The appropriate selections of these parameters have large effects on the convergence of the algorithm. Many techniques that adaptively adjust these parameters have been developed to increase speed of convergence. In this paper, we shall present several classes of learning automata based solutions to the problem of adaptation of BP algorithm parameters. By interconnection of learning automata to the feedforward neural networks, we use learning automata scheme for adjusting the parameters eta, alpha, and lambda based on the observation of random response of the neural networks. One of the important aspects of the proposed schemes is its ability to escape from local minima with high possibility during the training period. The feasibility of proposed methods is shown through simulations on several problems.     

Automatic detection of gait events: a case study using inductive learning techniques

One of the problems which occurs in the development of a control system for functional electrical stimulation of the lower limbs is to detect accurately specific events within the gait cycle. We present a method for the classification of phases of the gait cycle using the artificial intelligence technique of inductive learning. Both the terminology of inductive learning and the algorithm used for the analyses are fully explained. Given a set of examples of sensor data from the gait events that are to be delected, the inductive learning algorithm is able to produce a decision tree (or set of rules) which classify the data using a minimum number of sensors. The nature of the redundancy of the sensor set is examined by progressively removing combinations of sensors and noting the effect on both the size of the decision trees produced and their classification accuracy on ‘unseen’ testing data. Since the algorithm is able to calculate which sensors are more important (informative), comparisons with the intuitive appreciation of sensor importance of five researchers in the fields were made, revealing that those sensors which appear intuitively most informative may, in fact, provide the least information. Comparison results with the standard statistical classification technique of linear discriminant analysis are also presented, showing the relative simplicity of the inductively derived rules together with their good classification accuracy. In addition to the control of FES, such techniques are also applicable to automatic gait analysis and the construction of expert systems for diagnosis of gait pathologies.     

Inference of biological S-system using the separable estimation method and the genetic algorithm

Reconstruction of a biological system from its experimental time series data is a challenging task in systems biology. The S-system which consists of a group of nonlinear ordinary differential equations (ODEs) is an effective model to characterize molecular biological systems and analyze the system dynamics. However, inference of S-systems without the knowledge of system structure is not a trivial task due to its nonlinearity and complexity. In this paper, a pruning separable parameter estimation algorithm (PSPEA) is proposed for inferring S-systems. This novel algorithm combines the separable parameter estimation method (SPEM) and a pruning strategy, which includes adding an l? regularization term to the objective function and pruning the solution with a threshold value. Then, this algorithm is combined with the continuous genetic algorithm (CGA) to form a hybrid algorithm that owns the properties of these two combined algorithms. The performance of the pruning strategy in the proposed algorithm is evaluated from two aspects: the parameter estimation error and structure identification accuracy. The results show that the proposed algorithm with the pruning strategy has much lower estimation error and much higher identification accuracy than the existing method.     

Hierarchical Gene Selection and Genetic Fuzzy System for Cancer Microarray Data Classification

This paper introduces a novel approach to gene selection based on a substantial modification of analytic hierarchy process (AHP). The modified AHP systematically integrates outcomes of individual filter methods to select the most informative genes for microarray classification. Five individual ranking methods including t-test, entropy, receiver operating characteristic (ROC) curve, Wilcoxon and signal to noise ratio are employed to rank genes. These ranked genes are then considered as inputs for the modified AHP. Additionally, a method that uses fuzzy standard additive model (FSAM) for cancer classification based on genes selected by AHP is also proposed in this paper. Traditional FSAM learning is a hybrid process comprising unsupervised structure learning and supervised parameter tuning. Genetic algorithm (GA) is incorporated in-between unsupervised and supervised training to optimize the number of fuzzy rules. The integration of GA enables FSAM to deal with the high-dimensional-low-sample nature of microarray data and thus enhance the efficiency of the classification. Experiments are carried out on numerous microarray datasets. Results demonstrate the performance dominance of the AHP-based gene selection against the single ranking methods. Furthermore, the combination of AHP-FSAM shows a great accuracy in microarray data classification compared to various competing classifiers. The proposed approach therefore is useful for medical practitioners and clinicians as a decision support system that can be implemented in the real medical practice.     

Hierarchical Dirichlet process model for gene expression clustering

Clustering is an important data processing tool for interpreting microarray data and genomic network inference. In this article, we propose a clustering algorithm based on the hierarchical Dirichlet processes (HDP). The HDP clustering introduces a hierarchical structure in the statistical model which captures the hierarchical features prevalent in biological data such as the gene express data. We develop a Gibbs sampling algorithm based on the Chinese restaurant metaphor for the HDP clustering. We apply the proposed HDP algorithm to both regulatory network segmentation and gene expression clustering. The HDP algorithm is shown to outperform several popular clustering algorithms by revealing the underlying hierarchical structure of the data. For the yeast cell cycle data, we compare the HDP result to the standard result and show that the HDP algorithm provides more information and reduces the unnecessary clustering fragments.     

Variable selection in nonlinear modeling based on RBF networks and evolutionary computation

In this paper a novel variable selection method based on Radial Basis Function (RBF) neural networks and genetic algorithms is presented. The fuzzy means algorithm is utilized as the training method for the RBF networks, due to its inherent speed, the deterministic approach of selecting the hidden node centers and the fact that it involves only a single tuning parameter. The trade-off between the accuracy and parsimony of the produced model is handled by using Final Prediction Error criterion, based on the RBF training and validation errors, as a fitness function of the proposed genetic algorithm. The tuning parameter required by the fuzzy means algorithm is treated as a free variable by the genetic algorithm. The proposed method was tested in benchmark data sets stemming from the scientific communities of time-series prediction and medicinal chemistry and produced promising results.     

Hierarchical genetic algorithm for near optimal feedforward neural network design

In this paper, we propose a genetic algorithm based design procedure for a multi layer feed forward neural network. A hierarchical genetic algorithm is used to evolve both the neural networks topology and weighting parameters. Compared with traditional genetic algorithm based designs for neural networks, the hierarchical approach addresses several deficiencies, including a feasibility check highlighted in literature. A multi objective cost function is used herein to optimize the performance and topology of the evolved neural network simultaneously. In the prediction of Mackey Glass chaotic time series, the networks designed by the proposed approach prove to be competitive, or even superior, to traditional learning algorithms for the multi layer Perceptron networks and radial basis function networks. Based upon the chosen cost function, a linear weight combination decision making approach has been applied to derive an approximated Pareto optimal solution set. Therefore, designing a set of neural networks can be considered as solving a two objective optimization problem.     

Adaptive Control Based Harvesting Strategy for a Predator–Prey Dynamical System

This paper deals with designing a harvesting control strategy for a predator–prey dynamical system, with parametric uncertainties and exogenous disturbances. A feedback control law for the harvesting rate of the predator is formulated such that the population dynamics is asymptotically stabilized at a positive operating point, while maintaining a positive, steady state harvesting rate. The hierarchical block strict feedback structure of the dynamics is exploited in designing a backstepping control law, based on Lyapunov theory. In order to account for unknown parameters, an adaptive control strategy has been proposed in which the control law depends on an adaptive variable which tracks the unknown parameter. Further, a switching component has been incorporated to robustify the control performance against bounded disturbances. Proofs have been provided to show that the proposed adaptive control strategy ensures asymptotic stability of the dynamics at a desired operating point, as well as exact parameter learning in the disturbance-free case and learning with bounded error in the disturbance prone case. The dynamics, with uncertainty in the death rate of the predator, subjected to a bounded disturbance has been simulated with the proposed control strategy.     

Ranking of Business Process Simulation Software Tools with DEX/QQ Hierarchical Decision Model

The omnipresent need for optimisation requires constant improvements of companies’ business processes (BPs). Minimising the risk of inappropriate BP being implemented is usually performed by simulating the newly developed BP under various initial conditions and “what-if” scenarios. An effectual business process simulations software (BPSS) is a prerequisite for accurate analysis of an BP. Characterisation of an BPSS tool is a challenging task due to the complex selection criteria that includes quality of visual aspects, simulation capabilities, statistical facilities, quality reporting etc. Under such circumstances, making an optimal decision is challenging. Therefore, various decision support models are employed aiding the BPSS tool selection. The currently established decision support models are either proprietary or comprise only a limited subset of criteria, which affects their accuracy. Addressing this issue, this paper proposes a new hierarchical decision support model for ranking of BPSS based on their technical characteristics by employing DEX and qualitative to quantitative (QQ) methodology. Consequently, the decision expert feeds the required information in a systematic and user friendly manner. There are three significant contributions of the proposed approach. Firstly, the proposed hierarchical model is easily extendible for adding new criteria in the hierarchical structure. Secondly, a fully operational decision support system (DSS) tool that implements the proposed hierarchical model is presented. Finally, the effectiveness of the proposed hierarchical model is assessed by comparing the resulting rankings of BPSS with respect to currently available results.     

A critical analysis of parameter adaptation in ant colony optimization

Applying parameter adaptation means operating on parameters of an algorithm while it is tackling an instance. For ant colony optimization, several parameter adaptation methods have been proposed. In the literature, these methods have been shown to improve the quality of the results achieved in some particular contexts. In particular, they proved to be successful when applied to novel ant colony optimization algorithms for tackling problems that are not a classical testbed for optimization algorithms. In this paper, we show that the adaptation methods proposed so far do not improve, and often even worsen the performance when applied to high performing ant colony optimization algorithms for some classical combinatorial optimization problems.     

Automated nonlinear system modeling with multiple fuzzy neural networks and kernel smoothing

This paper, presents a novel identification approach using fuzzy neural networks. It focuses on structure and parameters uncertainties which have been widely explored in the literatures. The main contribution of this paper is that an integrated analytic framework is proposed for automated structure selection and parameter identification. A kernel smoothing technique is used to generate a model structure automatically in a fixed time interval. To cope with structural change, a hysteresis strategy is proposed to guarantee finite times switching and desired performance.     

A hierarchical screening approach to enantiomeric separation

The screening of a number of chiral stationary phases (CSPs) with different modifiers in supercritical fluid chromatography to find a chromatographic method for separation of enantiomers can be time‐consuming. Computational methods for data analysis were utilized to establish a hierarchical screening strategy, using a dataset of 110 drug‐like chiral compounds with diverse structures tested on 15 CSPs with two different modifiers. This dataset was analyzed using a combinatorial algorithm, principal component analysis (PCA), and a correlation matrix. The primary goal was to find a set of eight columns resolving a large number of compounds, but also having complementary enantioselective properties. In addition to the hereby defined hierarchical experimental strategy, quantitative structure enantioselective models (QSERs) were evaluated. The diverse chemical space and relatively limited size of the training set reduced the accuracy of the QSERs. However, including separation factors from other CSPs increased the accuracies of the QSERs substantially. Hence, such combined models can support the experimental strategy in prioritizing the CSPs of the second screening phase, when a compound is not separated by the primary set of columns.     

Data mining of the GAW14 simulated data using rough set theory and tree-based methods

Rough set theory and decision trees are data mining methods used for dealing with vagueness and uncertainty. They have been utilized to unearth hidden patterns in complicated datasets collected for industrial processes. The Genetic Analysis Workshop 14 simulated data were generated using a system that implemented multiple correlations among four consequential layers of genetic data (disease-related loci, endophenotypes, phenotypes, and one disease trait). When information of one layer was blocked and uncertainty was created in the correlations among these layers, the correlation between the first and last layers (susceptibility genes and the disease trait in this case), was not easily directly detected. In this study, we proposed a two-stage process that applied rough set theory and decision trees to identify genes susceptible to the disease trait. During the first stage, based on phenotypes of subjects and their parents, decision trees were built to predict trait values. Phenotypes retained in the decision trees were then advanced to the second stage, where rough set theory was applied to discover the minimal subsets of genes associated with the disease trait. For comparison, decision trees were also constructed to map susceptible genes during the second stage. Our results showed that the decision trees of the first stage had accuracy rates of about 99% in predicting the disease trait. The decision trees and rough set theory failed to identify the true disease-related loci.     

Detection of exon location in eukaryotic DNA using a fuzzy adaptive Gabor wavelet transform

The existing model-independent methods for the detection of exons in DNA could not prove to be ideal as commonly employed fixed window length strategy produces spectral leakage causing signal noise The Modified-Gabor-wavelet-transform exploits a multiscale strategy to deal with the issue to some extent. Yet, no rule regarding the occurrence of small and large exons has been specified. To overcome this randomness, scaling-factor of GWT has been adapted based on a fuzzy rule. Due to the nucleotides' genetic code and fuzzy behaviors in DNA configuration, this work could adopt the fuzzy approach. Two fuzzy membership functions (large and small) take care of the variation in the coding regions. The fuzzy-based learning parameter adaptively tunes the scale factor for fast and precise prediction of exons. The proposed approach has an immense plus point of being capable of isolating detailed sub-regions in each exon efficiently proving its efficacy comparing with existing techniques.