A novel approach for short-term load forecasting using support vector machines

A support vector machine (SVM) modeling approach for short-term load forecasting is proposed. The SVM learning scheme is applied to the power load data, forcing the network to learn the inherent internal temporal property of power load sequence. We also study the performance when other related input variables such as temperature and humidity are considered. The performance of our proposed SVM modeling approach has been tested and compared with feed-forward neural network and cosine radial basis function neural network approaches. Numerical results show that the SVM approach yields better generalization capability and lower prediction error compared to those neural network approaches.     

Neural network modeling in optimisation of continuous fermentation processes

The capability of self-recurrent neural networks in dynamic modeling of continuous fermentation is investigated in this simulation study. In the past, feedforward neural networks have been successfully used as one-step-ahead predictors. However, in steady-state optimisation of continuous fermentations the neural network model has to be iterated to predict many time steps ahead into the future in order to get steady-state values of the variables involved in objective cost function, and this iteration may result in increasing errors. Therefore, as an alternative to classical feedforward neural network trained by using backpropagation method, self-recurrent multilayer neural net trained by backpropagation through time method was chosen in order to improve accuracy of long-term predictions. Prediction capabilities of the resulting neural network model is tested by implementing this into the Integrated System Optimisation and Parameter Estimation (ISOPE) optimisation algorithm. Maximisation of cellular productivity of the baker's yeast continuous fermentation was used as the goal of the proposed optimising control problem. The training and prediction results of proposed neural network and performances of resulting optimisation structure are demonstrated.     

Hybrid neural network modeling of a full-scale industrial wastewater treatment process

In recent years, hybrid neural network approaches, which combine mechanistic and neural network models, have received considerable attention. These approaches are potentially very efficient for obtaining more accurate predictions of process dynamics by combining mechanistic and neural network models in such a way that the neural network model properly accounts for unknown and nonlinear parts of the mechanistic model. In this work, a full-scale coke-plant wastewater treatment process was chosen as a model system. Initially, a process data analysis was performed on the actual operational data by using principal component analysis. Next, a simplified mechanistic model and a neural network model were developed based on the specific process knowledge and the operational data of the coke-plant wastewater treatment process, respectively. Finally, the neural network was incorporated into the mechanistic model in both parallel and serial configurations. Simulation results showed that the parallel hybrid modeling approach achieved much more accurate predictions with good extrapolation properties as compared with the other modeling approaches even in the case of process upset caused by, for example, shock loading of toxic compounds. These results indicate that the parallel hybrid neural modeling approach is a useful tool for accurate and cost-effective modeling of biochemical processes, in the absence of other reasonably accurate process models.     

Software reliability prediction using recurrent neural network with Bayesian regularization

A recurrent neural network modeling approach for software reliability prediction with respect to cumulative failure time is proposed. Our proposed network structure has the capability of learning and recognizing the inherent internal temporal property of cumulative failure time sequence. Further, by adding a penalty term of sum of network connection weights, Bayesian regularization is applied to our network training scheme to improve the generalization capability and lower the susceptibility of overfitting. The performance of our proposed approach has been tested using four real-time control and flight dynamic application data sets. Numerical results show that our proposed approach is robust across different software projects, and has a better performance with respect to both goodness-of-fit and next-step-predictability compared to existing neural network models for failure time prediction.     

人工神经网络在发酵工业中的应用

人工神经网络技术具有很强的非线性映射能力,用于系统的非线性建模,具有无可比拟的优势,广泛应用于发酵过程中培养基的优化和系统建模与控制方面,本主要介绍了人工神经网络的基本原理与使用方法,以及BP神经网络在非线性函数逼近的优点,详细介绍了其在发酵培养基优化,连续搅拌反应器神经网络估计,分批发酵及补料分批发酵过程建模与控制优化中的应用实例。     

Predictive modeling of competitive biosorption equilibrium data

This paper compares regression and neural network modeling approaches to predict competitive biosorption equilibrium data. The regression approach is based on the fitting of modified Langmuir-type isotherm models to experimental data. Neural networks, on the other hand, are non-parametric statistical estimators capable of identifying patterns in data and correlations between input and output. Our results show that the neural network approach outperforms traditional regression-based modeling in correlating and predicting the simultaneous uptake of copper and cadmium by a microbial biosorbent. The neural network is capable of accurately predicting unseen data when provided with limited amounts of data for training. Because neural networks are purely data-driven models, they are more suitable for obtaining accurate predictions than for probing the physical nature of the biosorption process.     

BP-MSM混合算法及其在森林自疏规律研究中的应用

森林自然稀疏机制一般是非线性的、动态的.人工神经网络具有逼近任意非线性映射的特性.本文阐述了人工神经网络模拟森林自疏机制的可行性和不足之处,并提出了基于改进单纯形法的神经网络模型(BP-MSM混合算法)的基本原理和算法,结合山杨天然林和杉木人工林自疏实例说明了其应用.森林自疏实例应用结果表明,BP-MSM混合算法模拟森林自然稀疏机制是理想的,模拟精度较高,从而继承和发展了人工神经网络方法与理论,丰富了森林自然稀疏规律研究方法.     

Memory blocking: Facts,problems, and models

The tip-of-the-tongue state, or memory blocking, is considered with regard to the feasibility of its neural network modeling. The results of psycholinguistic and neurobiological studies on memory blocking are reviewed, and basic problems that need be solved to comprehend this phenomenon are formulated. One such point is the dramatic discrepancy between the subjective assurance that an image is familiar and the inability to recollect it fully. To explain this discrepancy, we propose a biologically plausible neural network model of recognition, demonstrating cardinal superiority in the capacity of image recognition over its recollection.     

On-line prediction of fermentation variables using neural networks

This article presents an introduction to the use of neural network computational algorithms for the dynamic modeling of bioprocesses. The dynamic neural model is used for the prediction of key fermentation variables. This relatively hew method is compared with a more traditional prediction technique to judge its performance for prediction. Illustrative simulation results of a continuous stirred tank fermentor are used for this comparison. It is shown that neural network models are accurate with a certain degree of noise immunity. They offer the distinctive ability over more traditional methods to learn very naturally complex relationships without requiring the knowledge of the model structure.     

Optimization of a fermentation medium using neural networks and genetic algorithms

Artificial neural networks and genetic algorithms are used to model and optimize a fermentation medium for the production of the enzyme hydantoinase by Agrobacterium radiobacter. Experimental data reported in the literature were used to build two neural network models. The concentrations of four medium components served as inputs to the neural network models, and hydantoinase or cell concentration served as a single output of each model. Genetic algorithms were used to optimize the input space of the neural network models to find the optimum settings for maximum enzyme and cell production. Using this procedure, two artificial intelligence techniques have been effectively integrated to create a powerful tool for process modeling and optimization.     

时滞BAM神经网络模型的稳定性分析

本文研究了一个含有n+1个神经元的多时滞BAM型神经网络模型.利用儒歇定理及其推论分析了该动力特征方程根的分布情况,进而得到动力系统稳定和Hopf分支存在的条件,画出了分支图.研究了该动力系统的Pitchfork分支,得到了Pitchfork分支曲线,讨论了在分支曲线的不同区域里平衡点的稳定性.     

Bayesian Neural Network Models for Censored Data

Neural networks are considered by many to be very promising tools for classification and prediction. The flexibility of the neural network models often result in over-fit. Shrinking the parameters using a penalized likelihood is often used in order to overcome such over-fit. In this paper we extend the approach proposed by FARAGGI and SIMON (1995a) to modeling censored survival data using the input-output relationship associated with a single hidden layer feed-forward neural network. Instead of estimating the neural network parameters using the method of maximum likelihood, we place normal prior distributions on the parameters and make inferences based on derived posterior distributions of the parameters. This Bayesian formulation will result in shrinking the parameters of the neural network model and will reduce the over-fit compared with the maximum likelihood estimators. We illustrate our proposed method on a simulated and a real example.     

Artificial neural network based soft estimator for estimation of transducer static nonlinearity

In this work, the development of an Artificial Neural Network (ANN) based soft estimator is reported for the estimation of static-nonlinearity associated with the transducers. Under the realm of ANN based transducer modeling, only two neural models have been suggested to estimate the static-nonlinearity associated with the transducers with quite successful results. The first existing model is based on the concept of a functional link artificial neural network (FLANN) trained with mu-LMS (Least Mean Squares) learning algorithm. The second one is based on the architecture of a single layer linear ANN trained with alpha-LMS learning algorithm. However, both these models suffer from the problem of slow convergence (learning). In order to circumvent this problem, it is proposed to synthesize the direct model of transducers using the concept of a Polynomial-ANN (polynomial artificial neural network) trained with Levenberg-Marquardt (LM) learning algorithm. The proposed Polynomial-ANN oriented transducer model is implemented based on the topology of a single-layer feed-forward back-propagation-ANN. The proposed neural modeling technique provided an extremely fast convergence speed with increased accuracy for the estimation of transducer static nonlinearity. The results of convergence are very stimulating with the LM learning algorithm.     

Comparison of different artificial neural network architectures in modeling of Chlorella sp. flocculation

Biodiesel production from microalgae feedstock should be performed after growth and harvesting of the cells, and the most feasible method for harvesting and dewatering of microalgae is flocculation. Flocculation modeling can be used for evaluation and prediction of its performance under different affective parameters. However, the modeling of flocculation in microalgae is not simple and has not performed yet, under all experimental conditions, mostly due to different behaviors of microalgae cells during the process under different flocculation conditions. In the current study, the modeling of microalgae flocculation is studied with different neural network architectures. Microalgae species, Chlorella sp., was flocculated with ferric chloride under different conditions and then the experimental data modeled using artificial neural network. Neural network architectures of multilayer perceptron (MLP) and radial basis function architectures, failed to predict the targets successfully, though, modeling was effective with ensemble architecture of MLP networks. Comparison between the performances of the ensemble and each individual network explains the ability of the ensemble architecture in microalgae flocculation modeling.     

Analysis and implementation of a neural extended Kalman filter for target tracking

Having a better motion model in the state estimator is one way to improve target tracking performance. Since the motion model of the target is not known a priori, either robust modeling techniques or adaptive modeling techniques are required. The neural extended Kalman filter is a technique that learns unmodeled dynamics while performing state estimation in the feedback loop of a control system. This coupled system performs the standard estimation of the states of the plant while estimating a function to approximate the difference between the given state-coupling function model and the behavior of the true plant dynamics. At each sample step, this new model is added to the existing model to improve the state estimate. The neural extended Kalman filter has also been investigated as a target tracking estimation routine. Implementation issues for this adaptive modeling technique, including neural network training parameters, were investigated and an analysis was made of the quality of performance that the technique can have for tracking maneuvering targets.     

A neural network model of the cerebellar cortex performing dynamic associations

The present paper proposes a model which applies formal neural network modeling techniques to construct a theoretical representation of the cerebellar cortex and its performances in motor control. A schema that makes explicit use of propagation delays of neural signals, is introduced to describe the ability to store temporal sequences of patterns in the Golgi-granule cell system. A perceptron association is then performed on these sequences of patterns by the Purkinje cell layer. The model conforms with important biological constraints, such as the known excitatory or inhibitory nature of the various synapses. Also, as suggested by experimental evidence, the synaptic plasticity underlying the learning ability of the model, is confined to the parallel fiber — Purkinje cell synapses, and takes place under the control of the climbing fibers. The result is a neural network model, constructed according to the anatomy of the cerebellar cortex, and capable of learning and retrieval of temporal sequences of patterns. It provides a framework to represent and interpret properties of learning and control of movements by the cerebellum, and to assess the capacity of formal neural network techniques for modeling of real neural systems.     

Analysis of reflex modulation with a biologically realistic neural network

In this study, a neuromusculoskeletal model was built to give insight into the mechanisms behind the modulation of reflexive feedback strength as experimentally identified in the human shoulder joint. The model is an integration of a biologically realistic neural network consisting of motoneurons and interneurons, modeling 12 populations of spinal neurons, and a one degree-of-freedom musculoskeletal model, including proprioceptors. The model could mimic the findings of human postural experiments, using presynaptic inhibition of the Ia afferents to modulate the feedback gains. In a pathological case, disabling one specific neural connection between the inhibitory interneurons and the motoneurons could mimic the experimental findings in complex regional pain syndrome patients. It is concluded that the model is a valuable tool to gain insight into the spinal contributions to human motor control. Applications lay in the fields of human motor control and neurological disorders, where hypotheses on motor dysfunction can be tested, like spasticity, clonus, and tremor. Action Editor: Karen Sigvardt     

Modeling and optimization of microbial hyaluronic acid production by Streptococcus zooepidemicus using radial basis function neural network coupling quantum‐behaved particle swarm optimization algorithm

Hyaluronic acid (HA) is a natural biopolymer with unique physiochemical and biological properties and finds a wide range of applications in biomedical and cosmetic fields. It is important to increase HA production to meet the increasing HA market demand. This work is aimed to model and optimize the amino acids addition to enhance HA production of Streptococcus zooepidemicus with radial basis function (RBF) neural network coupling quantum‐behaved particle swarm optimization (QPSO) algorithm. In the RBF‐QPSO approach, RBF neural network is used as a bioprocess modeling tool and QPSO algorithm is applied to conduct the optimization with the established RBF neural network black model as the objective function. The predicted maximum HA yield was 6.92 g/L under the following conditions: arginine 0.062 g/L, cysteine 0.036 g/L, and lysine 0.043 g/L. The optimal amino acids addition allowed HA yield increased from 5.0 g/L of the control to 6.7 g/L in the validation experiments. Moreover, the modeling and optimization capacity of the RBF‐QPSO approach was compared with that of response surface methodology (RSM). It was indicated that the RBF‐QPSO approach gave a slightly better modeling and optimization result compared with RSM. The developed RBF‐QPSO approach in this work may be helpful for the modeling and optimization of the other multivariable, nonlinear, time‐variant bioprocesses. © 2009 American Institute of Chemical Engineers Biotechnol. Prog., 2009     

Mathematical modeling and optimization of cellulase protein production using Trichoderma reesei RL-P37

The enzyme cellulase, a multienzyme complex made up of several proteins, catalyzes the conversion of cellulose to glucose in an enzymatic hydrolysis-based biomass-to-ethanol process. Production of cellulase enzyme proteins in large quantities using the fungus Trichoderma reesei requires understanding the dynamics of growth and enzyme production. The method of neural network parameter function modeling, which combines the approximation capabilities of neural networks with fundamental process knowledge, is utilized to develop a mathematical model of this dynamic system. In addition, kinetic models are also developed. Laboratory data from bench-scale fermentations involving growth and protein production by T. reesei on lactose and xylose are used to estimate the parameters in these models. The relative performances of the various models and the results of optimizing these models on two different performance measures are presented. An approximately 33% lower root-mean-squared error (RMSE) in protein predictions and about 40% lower total RMSE is obtained with the neural network-based model as opposed to kinetic models. Using the neural network-based model, the RMSE in predicting optimal conditions for two performance indices, is about 67% and 40% lower, respectively, when compared with the kinetic models. Thus, both model predictions and optimization results from the neural network-based model are found to be closer to the experimental data than the kinetic models developed in this work. It is shown that the neural network parameter function modeling method can be useful as a "macromodeling" technique to rapidly develop dynamic models of a process.     

Optimization of auto-induction medium for G-CSF production by Escherichia coli using artificial neural networks coupled with genetic algorithm

Granulocyte colony-stimulating factor (G-CSF) is a cytokine widely used in cancer patients receiving high doses of chemotherapeutic drugs to prevent the chemotherapy-induced suppression of white blood cells. The production of recombinant G-CSF should be increased to meet the increasing market demand. This study aims to model and optimize the carbon source of auto-induction medium to enhance G-CSF production using artificial neural networks coupled with genetic algorithm. In this approach, artificial neural networks served as bioprocess modeling tools, and genetic algorithm (GA) was applied to optimize the established artificial neural network models. Two artificial neural network models were constructed: the back-propagation (BP) network and the radial basis function (RBF) network. The root mean square error, coefficient of determination, and standard error of prediction of the BP model were 0.0375, 0.959, and 8.49 %, respectively, whereas those of the RBF model were 0.0257, 0.980, and 5.82 %, respectively. These values indicated that the RBF model possessed higher fitness and prediction accuracy than the BP model. Under the optimized auto-induction medium, the predicted maximum G-CSF yield by the BP-GA approach was 71.66 %, whereas that by the RBF-GA approach was 75.17 %. These predicted values are in agreement with the experimental results, with 72.4 and 76.014 % for the BP-GA and RBF-GA models, respectively. These results suggest that RBF-GA is superior to BP-GA. The developed approach in this study may be helpful in modeling and optimizing other multivariable, non-linear, and time-variant bioprocesses.