An improved genetic algorithm based fuzzy-tuned neural network

This paper presents a fuzzy-tuned neural network, which is trained by an improved genetic algorithm (GA). The fuzzy-tuned neural network consists of a neural-fuzzy network and a modified neural network. In the modified neural network, a neuron model with two activation functions is used so that the degree of freedom of the network function can be increased. The neural-fuzzy network governs some of the parameters of the neuron model. It will be shown that the performance of the proposed fuzzy-tuned neural network is better than that of the traditional neural network with a similar number of parameters. An improved GA is proposed to train the parameters of the proposed network. Sets of improved genetic operations are presented. The performance of the improved GA will be shown to be better than that of the traditional GA. Some application examples are given to illustrate the merits of the proposed neural network and the improved GA.     

A neural network methodology of quadratic optimization.

According to the basic optimization principle of artificial neural networks, a novel kind of neural network model for solving the quadratic programming problem is presented. The methodology is based on the Lagrange multiplier theory in optimization and seeks to provide solutions satisfying the necessary conditions of optimality. The equilibrium point of the network satisfies the Kuhn-Tucker condition for the problem. The stability and convergency of the neural network is investigated and the strategy of the neural optimization is discussed. The feasibility of the neural network method is verified with the computation examples. Results of the simulation of the neural network to solve optimum problems are presented to illustrate the computational power of the neural network method.     

A Bionic Neural Network for Fish-Robot Locomotion

A bionic neural network for fish-robot locomotion is presented. The bionic neural network inspired from fish neural net- work consists of one high level controller and one chain of central pattern generators （CPGs）. Each CPG contains a nonlinear neural Zhang oscillator which shows properties similar to sine-cosine model. Simulation re, suits show that the bionic neural network presents a good performance in controlling the fish-robot to execute various motions such as startup, stop, forward swimming, backward swimming, turn right and turn left.     

The uniqueness theorem for complex-valued neural networks with threshold parameters and the redundancy of the parameters

This paper will prove the uniqueness theorem for 3-layered complex-valued neural networks where the threshold parameters of the hidden neurons can take non-zeros. That is, if a 3-layered complex-valued neural network is irreducible, the 3-layered complex-valued neural network that approximates a given complex-valued function is uniquely determined up to a finite group on the transformations of the learnable parameters of the complex-valued neural network.     

基于径向基函数神经网络的心电图ST段形态识别

心电图的ST段是指QRS波的终点至T波的起点间的一个子波,其时间长度与心率有关,对ST段形态的识别有助于分析ST段变化的原因和确定缺血的部位。将模糊逻辑系统与神经网络相结合,利用基于自适应模糊系统的径向基函数神经网络对心电信号ST段的形态识别进行了研究。该网络比BP网络学习进度快,具有增量学习的能力,它能够识别学习外的新模式。研究取得了较好的识别结果。     

一类求解约束非线性规划问题的神经网络模型

提出一类求解闭凸集上非线性规划问题的神经网络模型。理论分析和计算机模拟表明在适当的假设下所提出的神经网络模型大范围指数级收敛于非线性规划问题的解集。本文神经网络所采用的方法属于广义的最速下降法,甚至当规划问题地正定二次时,本文的模型也比已有的神经网络模型简单。     

Multi/infinite dimensional neural networks, multi/infinite dimensional logic theory

A mathematical model of an arbitrary multi-dimensional neural network is developed and a convergence theorem for an arbitrary multi-dimensional neural network represented by a fully symmetric tensor is stated and proved. The input and output signal states of a multi-dimensional neural network/logic gate are related through an energy function, defined over the fully symmetric tensor (representing the connection structure of a multi-dimensional neural network). The inputs and outputs are related such that the minimum/maximum energy states correspond to the output states of the logic gate/neural network realizing a logic function. Similarly, a logic circuit consisting of the interconnection of logic gates, represented by a block symmetric tensor, is associated with a quadratic/higher degree energy function. Infinite dimensional logic theory is discussed through the utilization of infinite dimension/order tensors.     

Energy coding in neural network with inhibitory neurons

This paper aimed at assessing and comparing the effects of the inhibitory neurons in the neural network on the neural energy distribution, and the network activities in the absence of the inhibitory neurons to understand the nature of neural energy distribution and neural energy coding. Stimulus, synchronous oscillation has significant difference between neural networks with and without inhibitory neurons, and this difference can be quantitatively evaluated by the characteristic energy distribution. In addition, the synchronous oscillation difference of the neural activity can be quantitatively described by change of the energy distribution if the network parameters are gradually adjusted. Compared with traditional method of correlation coefficient analysis, the quantitative indicators based on nervous energy distribution characteristics are more effective in reflecting the dynamic features of the neural network activities. Meanwhile, this neural coding method from a global perspective of neural activity effectively avoids the current defects of neural encoding and decoding theory and enormous difficulties encountered. Our studies have shown that neural energy coding is a new coding theory with high efficiency and great potential.     

Hybrid neural network modeling of a full-scale industrial wastewater treatment process

In recent years, hybrid neural network approaches, which combine mechanistic and neural network models, have received considerable attention. These approaches are potentially very efficient for obtaining more accurate predictions of process dynamics by combining mechanistic and neural network models in such a way that the neural network model properly accounts for unknown and nonlinear parts of the mechanistic model. In this work, a full-scale coke-plant wastewater treatment process was chosen as a model system. Initially, a process data analysis was performed on the actual operational data by using principal component analysis. Next, a simplified mechanistic model and a neural network model were developed based on the specific process knowledge and the operational data of the coke-plant wastewater treatment process, respectively. Finally, the neural network was incorporated into the mechanistic model in both parallel and serial configurations. Simulation results showed that the parallel hybrid modeling approach achieved much more accurate predictions with good extrapolation properties as compared with the other modeling approaches even in the case of process upset caused by, for example, shock loading of toxic compounds. These results indicate that the parallel hybrid neural modeling approach is a useful tool for accurate and cost-effective modeling of biochemical processes, in the absence of other reasonably accurate process models.     

Bayesian neural networks

A neural network that uses the basic Hebbian learning rule and the Bayesian combination function is defined. Analogously to Hopfield's neural network, the convergence for the Bayesian neural network that asynchronously updates its neurons' states is proved. The performance of the Bayesian neural network in four medical domains is compared with various classification methods. The Bayesian neural network uses more sophisticated combination function than Hopfield's neural network and uses more economically the available information. The naive Bayesian classifier typically outperforms the basic Bayesian neural network since iterations in network make too many mistakes. By restricting the number of iterations and increasing the number of fixed points the network performs better than the naive Bayesian classifier. The Bayesian neural network is designed to learn very quickly and incrementally.     

Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic Nodes

Large-scale artificial neural networks have many redundant structures, making the network fall into the issue of local optimization and extended training time. Moreover, existing neural network topology optimization algorithms have the disadvantage of many calculations and complex network structure modeling. We propose a Dynamic Node-based neural network Structure optimization algorithm (DNS) to handle these issues. DNS consists of two steps: the generation step and the pruning step. In the generation step, the network generates hidden layers layer by layer until accuracy reaches the threshold. Then, the network uses a pruning algorithm based on Hebb’s rule or Pearson’s correlation for adaptation in the pruning step. In addition, we combine genetic algorithm to optimize DNS (GA-DNS). Experimental results show that compared with traditional neural network topology optimization algorithms, GA-DNS can generate neural networks with higher construction efficiency, lower structure complexity, and higher classification accuracy.     

Learning-Induced Synchronization and Plasticity of a Developing Neural Network

Learning-induced synchronization of a neural network at various developing stages is studied by computer simulations using a pulse-coupled neural network model in which the neuronal activity is simulated by a one-dimensional map. Two types of Hebbian plasticity rules are investigated and their differences are compared. For both models, our simulations show a logarithmic increase in the synchronous firing frequency of the network with the culturing time of the neural network. This result is consistent with recent experimental observations. To investigate how to control the synchronization behavior of a neural network after learning, we compare the occurrence of synchronization for four networks with different designed patterns under the influence of an external signal. The effect of such a signal on the network activity highly depends on the number of connections between neurons. We discuss the synaptic plasticity and enhancement effects for a random network after learning at various developing stages.     

A neural network model with feature selection for Korean speech act classification

A speech act is a linguistic action intended by a speaker. Speech act classification is an essential part of a dialogue understanding system because the speech act of an utterance is closely tied with the user's intention in the utterance. We propose a neural network model for Korean speech act classification. In addition, we propose a method that extracts morphological features from surface utterances and selects effective ones among the morphological features. Using the feature selection method, the proposed neural network can partially increase precision and decrease training time. In the experiment, the proposed neural network showed better results than other models using comparatively high-level linguistic features. Based on the experimental result, we believe that the proposed neural network model is suitable for real field applications because it is easy to expand the neural network model into other domains. Moreover, we found that neural networks can be useful in speech act classification if we can convert surface sentences into vectors with fixed dimensions by using an effective feature selection method.     

Markovian neural networks

The neural network that efficiently and nearly optimally solves difficult optimization problems is defined. The convergence proof for the Markovian neural network that asynchronously updates its neurons' states is also presented. The comparison of the performance of the Markovian neural network with various combinatorial optimization methods in two domains is described. The Markovian neural network is shown to be an efficient tool for solving optimization problems.     

Optimization of fed-batch bioreactor using neural network model

An algorithm using feedforward neural network model for determining optimal substrate feeding policies for fed-batch fermentation process is presented in this work. The algorithm involves developing the neural network model of the process using the sampled data. The trained neural network model in turn is used for optimization purposes. The advantages of this technique is that optimization can be achieved without detailed kinetic model of the process and the computation of gradient of objective function with respect to control variables is straightforward. The application of the technique is demonstrated with two examples, namely, production of secreted protein and invertase. The simulation results show that the discrete-time dynamics of fed-batch bioreactor can be satisfactorily approximated using a feedforward sigmoidal neural network. The optimal policies obtained with the neural network model agree reasonably well with the previously reported results.     

Optimizationof neural network architecture using genetic programming improvesdetection and modeling of gene-gene interactions in studies of humandiseases

Background

Appropriate definitionof neural network architecture prior to data analysis is crucialfor successful data mining. This can be challenging when the underlyingmodel of the data is unknown. The goal of this study was to determinewhether optimizing neural network architecture using genetic programmingas a machine learning strategy would improve the ability of neural networksto model and detect nonlinear interactions among genes in studiesof common human diseases.

Results

Using simulateddata, we show that a genetic programming optimized neural network approachis able to model gene-gene interactions as well as a traditionalback propagation neural network. Furthermore, the genetic programmingoptimized neural network is better than the traditional back propagationneural network approach in terms of predictive ability and powerto detect gene-gene interactions when non-functional polymorphismsare present.

Conclusion

This study suggeststhat a machine learning strategy for optimizing neural network architecturemay be preferable to traditional trial-and-error approaches forthe identification and characterization of gene-gene interactionsin common, complex human diseases.

     

Automatic identification of algae: neural network analysis of flow cytometric data

The performance of an artificial neural network for automaticidentification of phytoplankton was investigated with data fromalgal laboratory cultures, analysed on the Optical PlanktonAnalyser (OPA), a flow cytometer especially developed for theanalysis of phytoplankton. Data from monocultures of eight algalspecies were used to train a neural network. The performanceof the trained network was tested with OPA data from mixturesof laboratory cultures. The network could distinguish Cyanobacteriafrom other algae with 99% accuracy. The identification of specieswas performed with less accuracy, but was generally >90%.This indicates that a neural network under supervised learningcan be used for automatic identification of species in relativelycomplex mixtures. Incorporation of such a system may also increasethe operational size range of a flow cytometer. The combinationof the OPA and neural network data analysis offers the elementsto build an operational automatic algal identification system.     

An Attractor-Based Complexity Measurement for Boolean Recurrent Neural Networks

We provide a novel refined attractor-based complexity measurement for Boolean recurrent neural networks that represents an assessment of their computational power in terms of the significance of their attractor dynamics. This complexity measurement is achieved by first proving a computational equivalence between Boolean recurrent neural networks and some specific class of -automata, and then translating the most refined classification of -automata to the Boolean neural network context. As a result, a hierarchical classification of Boolean neural networks based on their attractive dynamics is obtained, thus providing a novel refined attractor-based complexity measurement for Boolean recurrent neural networks. These results provide new theoretical insights to the computational and dynamical capabilities of neural networks according to their attractive potentialities. An application of our findings is illustrated by the analysis of the dynamics of a simplified model of the basal ganglia-thalamocortical network simulated by a Boolean recurrent neural network. This example shows the significance of measuring network complexity, and how our results bear new founding elements for the understanding of the complexity of real brain circuits.     

Study of a Bionic Pattern Classifier Based on Olfactory Neural System

Simulating biological olfactory neural system, KIII network, which is a high-dimensional chaotic neural network, is designed in this paper. Different from conventional artificial neural network, the KⅢ network works in its chaotic trajectory. It can simulate not only the output EEG waveform observed in electrophysiological experiments, but also the biological intelligence for pattern classification. The simulation analysis and application to the recognition of handwriting numerals are presented here. The classification performance of the KⅢ network at different noise levels was also investigated.     

A hybrid neural network of addressable and content-addressable memory

We investigate the memory structure and retrieval of the brain and propose a hybrid neural network of addressable and content-addressable memory which is a special database model and can memorize and retrieve any piece of information (a binary pattern) both addressably and content-addressably. The architecture of this hybrid neural network is hierarchical and takes the form of a tree of slabs which consist of binary neurons with the same array. Simplex memory neural networks are considered as the slabs of basic memory units, being distributed on the terminal vertexes of the tree. It is shown by theoretical analysis that the hybrid neural network is able to be constructed with Hebbian and competitive learning rules, and some other important characteristics of its learning and memory behavior are also consistent with those of the brain. Moreover, we demonstrate the hybrid neural network on a set of ten binary numeral patters