An approach to improve Wang-Smith chaotic simulated annealing

Previous research shows that Wang-Smith chaotic simulated annealing, which employs a gradually decreasing time-step, has only a scaling effect to computational energy of the Hopfield model without changing its shape. This makes the net has sensitive dependence on the value of damping factor. Considering Chen-Aihara chaotic simulated annealing with decaying self-coupling has a shape effect to computational energy of the Hopfield model, a novel approach to improve Wang-Smith chaotic simulated annealing, which reaps the benefits of Wang-Smith model and Chen-Aihara model, is proposed in this paper. With the aid of this method the improved model can affect on computational energy of the Hopfield model from scaling and shape. By adjusting the time-step, the improved neural network can also pass from a chaotic to a non-chaotic state. From numerical simulation experiments, we know that the improved model can escape from local minima more efficiently than original Wang-Smith model.     

Balancing robustness against the dangers of multiple attractors in a Hopfield-type model of biological attractors

Background

Many chronic human diseases are of unclear origin, and persist long beyond any known insult or instigating factor. These diseases may represent a structurally normal biologic network that has become trapped within the basin of an abnormal attractor.

Methodology/Principal Findings

We used the Hopfield net as the archetypical example of a dynamic biological network. By progressively removing the links of fully connected Hopfield nets, we found that a designated attractor of the nets could still be supported until only slightly more than 1 link per node remained. As the number of links approached this minimum value, the rate of convergence to this attractor from an arbitrary starting state increased dramatically. Furthermore, with more than about twice the minimum of links, the net became increasingly able to support a second attractor.

Conclusions/Significance

We speculate that homeostatic biological networks may have evolved to assume a degree of connectivity that balances robustness and agility against the dangers of becoming trapped in an abnormal attractor.     

A goal programming network for linear programming

Tank and Hopfield have shown that networks of analog neurons can be used to solve linear programming (LP) problems. We have re-examined their approach and found that their network model frequently computes solutions that are only suboptimal or that violate the LP problem's constraints. As their approach has proven unreliable, we have developed a new network model: the goal programming network. To this end, a network model was first developed for goal programming problems, a particular type of LP problems. From the manner the network operates on such problems, it was concluded that overconstrainedness, which is possibly present in an LP formulation, should be removed, and we have provided a simple procedure to accomplish this.     

Associative recognition and storage in a model network of physiological neurons

We consider a neural network model in which the single neurons are chosen to closely resemble known physiological properties. The neurons are assumed to be linked by synapses which change their strength according to Hebbian rules on a short time scale (100ms). The dynamics of the network — the time evolution of the cell potentials and the synapses — is investigated by computer simulation. As in more abstract network models (Cooper 1973; Hopfield 1982; Kohonen 1984) it is found that the local dynamics of the cell potentials and the synaptic strengths result in global cooperative properties of the network and enable the network to process an incoming flux of information and to learn and store patterns associatively. A trained net can associate missing details of a pattern, can correct wrong details and can suppress noise in a pattern. The network can further abstract the prototype from a series of patterns with variations. A suitable coupling constant connecting the dynamics of the cell potentials with the synaptic strengths is derived by a mean field approximation. This coupling constant controls the neural sensitivity and thereby avoids both extremes of the network state, the state of permanent inactivity and the state of epileptic hyperactivity.     

On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

Optimization with neural networks: a recipe for improving convergence and solution quality

Artificial Neural Networks, particularly the Hopfield Network have been applied to the solution of a variety of tasks formulated as optimization problems. However, the network often converges to invalid solutions, which have been attributed to an improper choice of parameters and energy functions. In this letter, we propose a fundamental change of viewpoint. We assert that the problem is not due to the bad choice of parameters or the form of the energy function chosen. Instead, we show that the Hopfield Net essentially performs only one iteration of a Sequential Unconstrained Minimization Technique (SUMT). Thus, it is not surprising that unsatisfactory results are obtained. We present results on an SUMT-based formulation for the Travelling Salesman Problem, where we consistently obtained valid tours. We also show how shorter tours can be systematically obtained.     

An artificial modular neural network and its basic dynamical characteristics

This work contains a proposition of an artificial modular neural network (MNN) in which every module network exchanges input/output information with others simultaneously. It further studies the basic dynamical characteristics of this network through both computer simulations and analytical considerations. A notable feature of this model is that it has generic representation with regard to the number of composed modules, network topologies, and classes of introduced interactions. The information processing of the MNN is described as the minimization of a total-energy function that consists of partial-energy functions for modules and their interactions, and the activity and weight dynamics are derived from the total-energy function under the Lyapunov stability condition. This concept was realized by Cross-Coupled Hopfield Nets (CCHN) that one of the authors proposed. In this paper, in order to investigate the basic dynamical properties of CCHN, we offer a representative model called Cross-Coupled Hopfield Nets with Local And Global Interactions (CCHN-LAGI) to which two distinct classes of interactions – local and global interactions – are introduced. Through a conventional test for associative memories, it is confirmed that our energy-function-based approach gives us proper dynamics of CCHN-LAGI even if the networks have different modularity. We also discuss the contribution of a single interaction and the joint contribution of the two distinct interactions through the eigenvalue analysis of connection matrices. Received: 18 July 1995 / Accepted in revised form: 2 October 1997     

Efficient and robust associative memory from a generalized Bloom filter

We develop a variant of a Bloom filter that is robust to hardware failure and show how it can be used as an efficient associative memory. We define a measure of the information recall and show that our new associative memory is able to recall more than twice as much information as a Hopfield network. The extra efficiency of our associative memory is all the more remarkable as it uses only bits while the Hopfield network uses integers.     

Chaotic synchronization using a network of neural oscillators

Synchronization of chaotic low-dimensional systems has been a topic of much recent research. Such systems have found applications for secure communications. In this work we show how synchronization can be achieved in a high-dimensional chaotic neural network. The network used in our studies is an extension of the Hopfield Network, known as the Complex Hopfield Network (CHN). The CHN, also an associative memory, has both fixed point and limit cycle or oscillatory behavior. In the oscillatory mode, the network wanders chaotically from one stored pattern to another. We show how a pair of identical high-dimensional CHNs can be synchronized by communicating only a subset of state vector components. The synchronizability of such a system is characterized through simulations.     

Do proteins learn to evolve? The Hopfield network as a basis for the understanding of protein evolution

Correlations between amino-acid residues can be observed in sets of aligned protein sequences, and the analysis of their statistical and evolutionary significance and distribution has been thoroughly investigated. In this paper, we present a model based on such covariations in protein sequences in which the pairs of residues that have mutual influence combine to produce a system analogous to a Hopfield neural network. The emergent properties of such a network, such as soft failure and the connection between network architecture and stored memory, have close parallels in known proteins. This model suggests that an explanation for observed characters of proteins such as the diminution of function by substitutions distant from the active site, the existence of protein folds (superfolds) that can perform several functions based on one architecture, and structural and functional resilience to destabilizing substitutions might derive from their inherent network-like structure. This model may also provide a basis for mapping the relationship between structure, function and evolutionary history of a protein family, and thus be a powerful tool for rational engineering.     

Pattern storage in a sparsely coded neural network with cyclic activation

We investigate an artificial neural network model with a modified Hebb rule. It is an auto-associative neural network similar to the Hopfield model and to the Willshaw model. It has properties of both of these models. Another property is that the patterns are sparsely coded and are stored in cycles of synchronous neural activities. The cycles of activity for some ranges of parameter increase the capacity of the model. We discuss basic properties of the model and some of the implementation issues, namely optimizing of the algorithms. We describe the modification of the Hebb learning rule, the learning algorithm, the generation of patterns, decomposition of patterns into cycles and pattern recall.     

选择方向强化学习的神经网络模型

提出了神经网络模型的一种选择方向强化学习规则,定义并导出了新模型与Hopfield模型两种不同的筛选曲线,由此表明新模型对相关图样的分辨力优于Hopfield模型。在微机上模拟了由100个神经元构成的网络,结果显示新模型具有重复记忆这一神经生理学特点。定义并分行了记忆强度因子,模拟结果表明记忆强度因子愈大的记忆态,联想性能愈好,学习周期愈短。     

Recognition of general patterns using neural networks

The Hopfield model of neural network stores memory in its symmetric synaptic connections and can only learn to recognize sets of nearly orthogonal patterns. A new algorithm is put forth to permit the recognition of general (non-orthogonal) patterns. The algorithm specifies the construction of the new network's memory matrix T _ij, which is, in general, asymmetrical and contains the Hopfield neural network (Hopfield 1982) as a special case. We find further that in addition to this new algorithm for general pattern recognition, there exists in fact a large class of T _ij memory matrices which permit the recognition of non-orthogonal patterns. The general form of this class of T _ij memory matrix is presented, and the projection matrix neural network (Personnaz et al. 1985) is found as a special case of this general form. This general form of memory matrix extends the library of memory matrices which allow a neural network to recognize non-orthogonal patterns. A neural network which followed this general form of memory matrix was modeled on a computer and successfully recognized a set of non-orthogonal patterns. The new network also showed a tolerance for altered and incomplete data. Through this new method, general patterns may be taught to the neural network.     

Models from experiments: combinatorial drug perturbations of cancer cells

We present a novel method for deriving network models from molecular profiles of perturbed cellular systems. The network models aim to predict quantitative outcomes of combinatorial perturbations, such as drug pair treatments or multiple genetic alterations. Mathematically, we represent the system by a set of nodes, representing molecular concentrations or cellular processes, a perturbation vector and an interaction matrix. After perturbation, the system evolves in time according to differential equations with built‐in nonlinearity, similar to Hopfield networks, capable of representing epistasis and saturation effects. For a particular set of experiments, we derive the interaction matrix by minimizing a composite error function, aiming at accuracy of prediction and simplicity of network structure. To evaluate the predictive potential of the method, we performed 21 drug pair treatment experiments in a human breast cancer cell line (MCF7) with observation of phospho‐proteins and cell cycle markers. The best derived network model rediscovered known interactions and contained interesting predictions. Possible applications include the discovery of regulatory interactions, the design of targeted combination therapies and the engineering of molecular biological networks.     

A novel approach for solving constrained nonlinear optimization problems using neurofuzzy systems

A neural network model for solving constrained nonlinear optimization problems with bounded variables is presented in this paper. More specifically, a modified Hopfield network is developed and its internal parameters are computed using the valid-subspace technique. These parameters guarantee the convergence of the network to the equilibrium points. The network is shown to be completely stable and globally convergent to the solutions of constrained nonlinear optimization problems. A fuzzy logic controller is incorporated in the network to minimize convergence time. Simulation results are presented to validate the proposed approach.     

Influence of noise on the function of a “physiological” neural network

A model neural network with stochastic elements in its millisecond dynamics is investigated. The network consists of neuronal units which are modelled in close analogy to physiological neurons. Dynamical variables of the network are the cellular potentials, axonic currents and synaptic efficacies. The dynamics of the synapses obeys a modified Hebbian rule and, as proposed by v. d. Malsburg (1981, 1985), develop on a time scale of a tenth of a second. In a previous publication (Buhmann and Schulten 1986) we have confirmed that the resulting noiseless autoassociative network is capable of the well-known computational tasks of formal associative networks (Cooper 1973; Kohonen et al. 1984, 1981; Hopfield 1982). In the present paper we demonstrate that random fluctuations of the membrane potential improve the performance of the network. In comparison to a deterministic network a noisy neural network can learn at lower input frequencies and with lower average neural firing rates. The electrical activity of a noisy network is very reminiscent of that observed by physiological recordings. We demonstrate furthermore that associative storage reduces the effective dimension of the phase space in which the electrical activity of the network develops.     

Application of Petri net theory for modelling and validation of the sucrose breakdown pathway in the potato tuber

MOTIVATION: Because of the complexity of metabolic networks and their regulation, formal modelling is a useful method to improve the understanding of these systems. An essential step in network modelling is to validate the network model. Petri net theory provides algorithms and methods, which can be applied directly to metabolic network modelling and analysis in order to validate the model. The metabolism between sucrose and starch in the potato tuber is of great research interest. Even if the metabolism is one of the best studied in sink organs, it is not yet fully understood. RESULTS: We provide an approach for model validation of metabolic networks using Petri net theory, which we demonstrate for the sucrose breakdown pathway in the potato tuber. We start with hierarchical modelling of the metabolic network as a Petri net and continue with the analysis of qualitative properties of the network. The results characterize the net structure and give insights into the complex net behaviour.     

Familiarity recognition and recollection: A neural network model

The capacities of a specially designed neural network for familiarity recognition and recollection have been compared. Recognition is based on calculating “image familiarity” as a modified Hopfield energy function in which the value of the inner sum is replaced by the sign of this value. This replacement makes the calculation of familiarity compatible with the basic dynamic equations of the Hopfield network and is in fact reduced to calculating the scalar product of the neuronet state vectors at two successive time steps.