A novel approach for short-term load forecasting using support vector machines

A support vector machine (SVM) modeling approach for short-term load forecasting is proposed. The SVM learning scheme is applied to the power load data, forcing the network to learn the inherent internal temporal property of power load sequence. We also study the performance when other related input variables such as temperature and humidity are considered. The performance of our proposed SVM modeling approach has been tested and compared with feed-forward neural network and cosine radial basis function neural network approaches. Numerical results show that the SVM approach yields better generalization capability and lower prediction error compared to those neural network approaches.     

HYPLOSP: a knowledge-based approach to protein local structure prediction

Local structure prediction can facilitate ab initio structure prediction, protein threading, and remote homology detection. However, the accuracy of existing methods is limited. In this paper, we propose a knowledge-based prediction method that assigns a measure called the local match rate to each position of an amino acid sequence to estimate the confidence of our method. Empirically, the accuracy of the method correlates positively with the local match rate; therefore, we employ it to predict the local structures of positions with a high local match rate. For positions with a low local match rate, we propose a neural network prediction method. To better utilize the knowledge-based and neural network methods, we design a hybrid prediction method, HYPLOSP (HYbrid method to Protein LOcal Structure Prediction) that combines both methods. To evaluate the performance of the proposed methods, we first perform cross-validation experiments by applying our knowledge-based method, a neural network method, and HYPLOSP to a large dataset of 3,925 protein chains. We test our methods extensively on three different structural alphabets and evaluate their performance by two widely used criteria, Maximum Deviation of backbone torsion Angle (MDA) and Q(N), which is similar to Q(3) in secondary structure prediction. We then compare HYPLOSP with three previous studies using a dataset of 56 new protein chains. HYPLOSP shows promising results in terms of MDA and Q(N) accuracy and demonstrates its alphabet-independent capability.     

Nonlinear system modelling via optimal design of neural trees

This paper introduces a flexible neural tree model. The model is computed as a flexible multi-layer feed-forward neural network. A hybrid learning/evolutionary approach to automatically optimize the neural tree model is also proposed. The approach includes a modified probabilistic incremental program evolution algorithm (MPIPE) to evolve and determine a optimal structure of the neural tree and a parameter learning algorithm to optimize the free parameters embedded in the neural tree. The performance and effectiveness of the proposed method are evaluated using function approximation, time series prediction and system identification problems and compared with the related methods.     

Hierarchical genetic algorithm for near optimal feedforward neural network design

In this paper, we propose a genetic algorithm based design procedure for a multi layer feed forward neural network. A hierarchical genetic algorithm is used to evolve both the neural networks topology and weighting parameters. Compared with traditional genetic algorithm based designs for neural networks, the hierarchical approach addresses several deficiencies, including a feasibility check highlighted in literature. A multi objective cost function is used herein to optimize the performance and topology of the evolved neural network simultaneously. In the prediction of Mackey Glass chaotic time series, the networks designed by the proposed approach prove to be competitive, or even superior, to traditional learning algorithms for the multi layer Perceptron networks and radial basis function networks. Based upon the chosen cost function, a linear weight combination decision making approach has been applied to derive an approximated Pareto optimal solution set. Therefore, designing a set of neural networks can be considered as solving a two objective optimization problem.     

An Approach Based on Mutually Informed Neural Networks to Optimize the Generalization Capabilities of Decision Support Systems Developed for Heart Failure Prediction

Available clinical methods for heart failure (HF) diagnosis are expensive and require a high-level of experts intervention. Recently, various machine learning models have been developed for the prediction of HF where most of them have an issue of over-fitting. Over-fitting occurs when machine learning based predictive models show better performance on the training data yet demonstrate a poor performance on the testing data and the other way around. Developing a machine learning model which is able to produce generalization capabilities (such that the model exhibits better performance on both the training and the testing data sets) could overall minimize the prediction errors. Hence, such prediction models could potentially be helpful to cardiologists for the effective diagnose of HF. This paper proposes a two-stage decision support system to overcome the over-fitting issue and to optimize the generalization factor. The first stage uses a mutual information based statistical model while the second stage uses a neural network. We applied our approach to the HF subset of publicly available Cleveland heart disease database. Our experimental results show that the proposed decision support system has optimized the generalization capabilities and has reduced the mean percent error (MPE) to 8.8% which is significantly less than the recently published studies. In addition, our model exhibits a 93.33% accuracy rate which is higher than twenty eight recently developed HF risk prediction models that achieved accuracy in the range of 57.85% to 92.31%. We can hope that our decision support system will be helpful to cardiologists if deployed in clinical setup.     

基于支持向量机(SVM)的剪接位点识别

剪接位点的识别作为基因识别中的一个重要环节, 一直受到研究人员的关注。考虑到剪接位点附近存在的序列保守性,已有一些基于统计特性的方法被用于剪接位点的识别中,但效果仍有待进一步改进。支持向量机（Ｓｕｐｐｏｒｔ Ｖｅｃｔｏｒ Ｍａｃｈｉｎｅｓ） 作为一种新的基于统计学习理论的学习机,近几年有了很大的发展,已被应用在模式识别的许多问题中。文中将其用于剪接位点的识别中,并针对满足ＧＴ－ ＡＧ 规则的序列样本中虚假剪接位点的样本数远大于真实位点这一特性, 提出了一种基于ＳＶＭ 的平衡取小法以获得更好的识别效果。实验结果表明,应用支持向量机进行剪接位点的识别能更好地提取位点附近保守序列的统计特征,对测试集具有更好的推广能力,并且使用上更加简单。这一结果为剪接位点的识别提供了一种新的方法,同时也为生物大分子研究中结构和位点的识别问题的解决提供了新的线索。     

Bursting dynamics remarkably improve the performance of neural networks on liquid computing

Burst firings are functionally important behaviors displayed by neural circuits, which plays a primary role in reliable transmission of electrical signals for neuronal communication. However, with respect to the computational capability of neural networks, most of relevant studies are based on the spiking dynamics of individual neurons, while burst firing is seldom considered. In this paper, we carry out a comprehensive study to compare the performance of spiking and bursting dynamics on the capability of liquid computing, which is an effective approach for intelligent computation of neural networks. The results show that neural networks with bursting dynamic have much better computational performance than those with spiking dynamics, especially for complex computational tasks. Further analysis demonstrate that the fast firing pattern of bursting dynamics can obviously enhance the efficiency of synaptic integration from pre-neurons both temporally and spatially. This indicates that bursting dynamic can significantly enhance the complexity of network activity, implying its high efficiency in information processing.     

Neural network modeling in optimisation of continuous fermentation processes

The capability of self-recurrent neural networks in dynamic modeling of continuous fermentation is investigated in this simulation study. In the past, feedforward neural networks have been successfully used as one-step-ahead predictors. However, in steady-state optimisation of continuous fermentations the neural network model has to be iterated to predict many time steps ahead into the future in order to get steady-state values of the variables involved in objective cost function, and this iteration may result in increasing errors. Therefore, as an alternative to classical feedforward neural network trained by using backpropagation method, self-recurrent multilayer neural net trained by backpropagation through time method was chosen in order to improve accuracy of long-term predictions. Prediction capabilities of the resulting neural network model is tested by implementing this into the Integrated System Optimisation and Parameter Estimation (ISOPE) optimisation algorithm. Maximisation of cellular productivity of the baker's yeast continuous fermentation was used as the goal of the proposed optimising control problem. The training and prediction results of proposed neural network and performances of resulting optimisation structure are demonstrated.     

Neural network modeling for on-line estimation of nutrient dynamics in a sequentially-operated batch reactor.

In monitoring and controlling wastewater treatment processes, on-line information of nutrient dynamics is very important. However, these variables are determined with a significant time delay. Although the final effluent quality can be analyzed after this delay, it is often too late to make proper adjustments. In this paper, a neural network approach, a software sensor, was proposed to overcome this problem. Software sensor refers to a modeling approach inferring hard-to-measure process variables from other on-line measurable process variables. A bench-scale sequentially-operated batch reactor (SBR) used for advanced wastewater treatment (BOD plus nutrient removal) was employed to develop the neural network model. In order to improve the network performance, the structure of neural network was arranged in such a way of reflecting the change of operational conditions within a cycle. Real-time estimation of PO3-(4), NO-3, and NH+4 concentrations was successfully carried out with the on-line information of the SBR system only.     

Bayesian Neural Network Models for Censored Data

Neural networks are considered by many to be very promising tools for classification and prediction. The flexibility of the neural network models often result in over-fit. Shrinking the parameters using a penalized likelihood is often used in order to overcome such over-fit. In this paper we extend the approach proposed by FARAGGI and SIMON (1995a) to modeling censored survival data using the input-output relationship associated with a single hidden layer feed-forward neural network. Instead of estimating the neural network parameters using the method of maximum likelihood, we place normal prior distributions on the parameters and make inferences based on derived posterior distributions of the parameters. This Bayesian formulation will result in shrinking the parameters of the neural network model and will reduce the over-fit compared with the maximum likelihood estimators. We illustrate our proposed method on a simulated and a real example.     

SPSRG: a prediction approach for correlated failures in distributed computing systems

Failure instances in distributed computing systems (DCSs) have exhibited temporal and spatial correlations, where a single failure instance can trigger a set of failure instances simultaneously or successively within a short time interval. In this work, we propose a correlated failure prediction approach (CFPA) to predict correlated failures of computing elements in DCSs. The approach models correlated-failure patterns using the concept of probabilistic shared risk groups and makes a prediction for correlated failures by exploiting an association rule mining approach in a parallel way. We conduct extensive experiments to evaluate the feasibility and effectiveness of CFPA using both failure traces from Los Alamos National Lab and simulated datasets. The experimental results show that the proposed approach outperforms other approaches in both the failure prediction performance and the execution time, and can potentially provide better prediction performance in a larger system.     

Selecting neural networks for a committee decision

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. In contrast to the ordinary approach of utilizing all neural networks available to make a committee decision, we propose creating adaptive committees, which are specific for each input data point. A prediction network is used to identify classification neural networks to be fused for making a committee decision about a given input data point. The jth output value of the prediction network expresses the expectation level that the jth classification neural network will make a correct decision about the class label of a given input data point. The proposed technique is tested in three aggregation schemes, namely majority vote, averaging, and aggregation by the median rule and compared with the ordinary neural networks fusion approach. The effectiveness of the approach is demonstrated on two artificial and three real data sets.     

Dynamic prediction of cumulative incidence functions by direct binomial regression

In recent years there have been a series of advances in the field of dynamic prediction. Among those is the development of methods for dynamic prediction of the cumulative incidence function in a competing risk setting. These models enable the predictions to be updated as time progresses and more information becomes available, for example when a patient comes back for a follow‐up visit after completing a year of treatment, the risk of death, and adverse events may have changed since treatment initiation. One approach to model the cumulative incidence function in competing risks is by direct binomial regression, where right censoring of the event times is handled by inverse probability of censoring weights. We extend the approach by combining it with landmarking to enable dynamic prediction of the cumulative incidence function. The proposed models are very flexible, as they allow the covariates to have complex time‐varying effects, and we illustrate how to investigate possible time‐varying structures using Wald tests. The models are fitted using generalized estimating equations. The method is applied to bone marrow transplant data and the performance is investigated in a simulation study.     

Prediction of protein secondary structure at 80% accuracy

Secondary structure prediction involving up to 800 neural network predictions has been developed, by use of novel methods such as output expansion and a unique balloting procedure. An overall performance of 77.2%-80.2% (77.9%-80.6% mean per-chain) for three-state (helix, strand, coil) prediction was obtained when evaluated on a commonly used set of 126 protein chains. The method uses profiles made by position-specific scoring matrices as input, while at the output level it predicts on three consecutive residues simultaneously. The predictions arise from tenfold, cross validated training and testing of 1032 protein sequences, using a scheme with primary structure neural networks followed by structure filtering neural networks. With respect to blind prediction, this work is preliminary and awaits evaluation by CASP4.     

A joint modeling approach for analyzing marker data in the presence of a terminal event

In many medical studies, markers are contingent on recurrent events and the cumulative markers are usually of interest. However, the recurrent event process is often interrupted by a dependent terminal event, such as death. In this article, we propose a joint modeling approach for analyzing marker data with informative recurrent and terminal events. This approach introduces a shared frailty to specify the explicit dependence structure among the markers, the recurrent, and terminal events. Estimation procedures are developed for the model parameters and the degree of dependence, and a prediction of the covariate‐specific cumulative markers is provided. The finite sample performance of the proposed estimators is examined through simulation studies. An application to a medical cost study of chronic heart failure patients from the University of Virginia Health System is illustrated.     

A novel efficient learning algorithm for self-generating fuzzy neural network with applications

In this paper, a novel efficient learning algorithm towards self-generating fuzzy neural network (SGFNN) is proposed based on ellipsoidal basis function (EBF) and is functionally equivalent to a Takagi-Sugeno-Kang (TSK) fuzzy system. The proposed algorithm is simple and efficient and is able to generate a fuzzy neural network with high accuracy and compact structure. The structure learning algorithm of the proposed SGFNN combines criteria of fuzzy-rule generation with a pruning technology. The Kalman filter (KF) algorithm is used to adjust the consequent parameters of the SGFNN. The SGFNN is employed in a wide range of applications ranging from function approximation and nonlinear system identification to chaotic time-series prediction problem and real-world fuel consumption prediction problem. Simulation results and comparative studies with other algorithms demonstrate that a more compact architecture with high performance can be obtained by the proposed algorithm. In particular, this paper presents an adaptive modeling and control scheme for drug delivery system based on the proposed SGFNN. Simulation study demonstrates the ability of the proposed approach for estimating the drug's effect and regulating blood pressure at a prescribed level.     

Prediction of Higher Heating Value of Solid Biomass Fuels Using Artificial Intelligence Formalisms

The higher heating value (HHV) is an important property defining the energy content of biomass fuels. A number of proximate and/or ultimate analysis based predominantly linear correlations have been proposed for predicting the HHV of biomass fuels. A scrutiny of the relationships between the constituents of the proximate and ultimate analyses and the corresponding HHVs suggests that all relationships are not linear and thus nonlinear models may be more appropriate. Accordingly, a novel artificial intelligence (AI) formalism, namely genetic programming (GP) has been employed for the first time for developing two biomass HHV prediction models, respectively using the constituents of the proximate and ultimate analyses as the model inputs. The prediction and generalization performance of these models was compared rigorously with the corresponding multilayer perceptron (MLP) neural network based as also currently available high-performing linear and nonlinear HHV models. This comparison reveals that the HHV prediction performance of the GP and MLP models is consistently better than that of their existing linear and/or nonlinear counterparts. Specifically, the GP- and MLP-based models exhibit an excellent overall prediction accuracy and generalization performance with high (>0.95) magnitudes of the coefficient of correlation and low (<4.5 %) magnitudes of mean absolute percentage error in respect of the experimental and model-predicted HHVs. It is also found that the proximate analysis-based GP model has outperformed all the existing high-performing linear biomass HHV prediction models. In the case of ultimate analysis-based HHV models, the MLP model has exhibited best prediction accuracy and generalization performance when compared with the existing linear and nonlinear models. The AI-based models introduced in this paper due to their excellent performance have the potential to replace the existing biomass HHV prediction models.     

Sequential modelling of a full-scale wastewater treatment plant using an artificial neural network

This work proposes a sequential modelling approach using an artificial neural network (ANN) to develop four independent multivariate models that are able to predict the dynamics of biochemical oxygen demand (BOD), chemical oxygen demand (COD), suspended solid (SS), and total nitrogen (TN) removal in a wastewater treatment plant (WWTP). Suitable structures of ANN models were automatically and conveniently optimized by a genetic algorithm rather than the conventional trial and error method. The sequential modelling approach, which is composed of two parts, a process disturbance estimator and a process behaviour predictor, was also presented to develop multivariate dynamic models. In particular, the process disturbance estimator was first employed to estimate the influent quality. The process behaviour predictor then sequentially predicted the effluent quality based on the estimated influent quality from the process disturbance estimator with other process variables. The efficiencies of the developed ANN models with a sequential modelling approach were demonstrated with a practical application using a data set collected from a full-scale WWTP during 2 years. The results show that the ANN with the sequential modelling approach successfully developed multivariate dynamic models of BOD, COD, SS, and TN removal with satisfactory estimation and prediction capability. Thus, the proposed method could be used as a powerful tool for the prediction of complex and nonlinear WWTP performance.     

Short-term prediction of chaotic time series by using RBF network with regression weights

We propose a framework for constructing and training a radial basis function (RBF) neural network. The structure of the gaussian functions is modified using a pseudo-gaussian function (PG) in which two scaling parameters sigma are introduced, which eliminates the symmetry restriction and provides the neurons in the hidden layer with greater flexibility with respect to function approximation. We propose a modified PG-BF (pseudo-gaussian basis function) network in which the regression weights are used to replace the constant weights in the output layer. For this purpose, a sequential learning algorithm is presented to adapt the structure of the network, in which it is possible to create a new hidden unit and also to detect and remove inactive units. A salient feature of the network systems is that the method used for calculating the overall output is the weighted average of the output associated with each receptive field. The superior performance of the proposed PG-BF system over the standard RBF are illustrated using the problem of short-term prediction of chaotic time series.