An annealed chaotic maximum neural network for bipartite subgraph problem

In this paper, based on maximum neural network, we propose a new parallel algorithm that can help the maximum neural network escape from local minima by including a transient chaotic neurodynamics for bipartite subgraph problem. The goal of the bipartite subgraph problem, which is an NP- complete problem, is to remove the minimum number of edges in a given graph such that the remaining graph is a bipartite graph. Lee et al. presented a parallel algorithm using the maximum neural model (winner-take-all neuron model) for this NP- complete problem. The maximum neural model always guarantees a valid solution and greatly reduces the search space without a burden on the parameter-tuning. However, the model has a tendency to converge to a local minimum easily because it is based on the steepest descent method. By adding a negative self-feedback to the maximum neural network, we proposed a new parallel algorithm that introduces richer and more flexible chaotic dynamics and can prevent the network from getting stuck at local minima. After the chaotic dynamics vanishes, the proposed algorithm is then fundamentally reined by the gradient descent dynamics and usually converges to a stable equilibrium point. The proposed algorithm has the advantages of both the maximum neural network and the chaotic neurodynamics. A large number of instances have been simulated to verify the proposed algorithm. The simulation results show that our algorithm finds the optimum or near-optimum solution for the bipartite subgraph problem superior to that of the best existing parallel algorithms.     

An olfactory recognition model based on spatio-temporal encoding of odor quality in the olfactory bulb

In order to study the problem how the olfactory neural system processes the odorant molecular information for constructing the olfactory image of each object, we present a dynamic model of the olfactory bulb constructed on the basis of well-established experimental and theoretical results. The information relevant to a single odor, i.e. its constituent odorant molecules and their mixing ratios, are encoded into a spatio-temporal pattern of neural activity in the olfactory bulb, where the activity pattern corresponds to a limit cycle attractor in the mitral cell network. The spatio-temporal pattern consists of a temporal sequence of spatial firing patterns: each constituent molecule is encoded into a single spatial pattern, and the order of magnitude of the mixing ratio is encoded into the temporal sequence. The formation of a limit cycle attractor under the application of a novel odor is carried out based on the intensity-to-time-delay encoding scheme. The dynamic state of the olfactory bulb, which has learned many odors, becomes a randomly itinerant state in which the current firing state of the bulb itinerates randomly among limit cycle attractors corresponding to the learned odors. The recognition of an odor is generated by the dynamic transition in the network from the randomly itinerant state to a limit cycle attractor state relevant to the odor, where the transition is induced by the short-term synaptic changes made according to the Hebbian rule under the application of the odor stimulus. Received: 28 July 1997 / Accepted in revised form: 6 May 1998     

Spike correlations in a songbird agree with a simple markov population model

The relationships between neural activity at the single-cell and the population levels are of central importance for understanding neural codes. In many sensory systems, collective behaviors in large cell groups can be described by pairwise spike correlations. Here, we test whether in a highly specialized premotor system of songbirds, pairwise spike correlations themselves can be seen as a simple corollary of an underlying random process. We test hypotheses on connectivity and network dynamics in the motor pathway of zebra finches using a high-level population model that is independent of detailed single-neuron properties. We assume that neural population activity evolves along a finite set of states during singing, and that during sleep population activity randomly switches back and forth between song states and a single resting state. Individual spike trains are generated by associating with each of the population states a particular firing mode, such as bursting or tonic firing. With an overall modification of one or two simple control parameters, the Markov model is able to reproduce observed firing statistics and spike correlations in different neuron types and behavioral states. Our results suggest that song- and sleep-related firing patterns are identical on short time scales and result from random sampling of a unique underlying theme. The efficiency of our population model may apply also to other neural systems in which population hypotheses can be tested on recordings from small neuron groups.     

A neural network for the Steiner minimal tree problem

The problem of finding the shortest tree connecting a set of points is called the Steiner minimal tree problem and is nearly three centuries old. It has applications in transportation, computer networks, agriculture, telephony, building layout and very large scale integrated circuit (VLSI) design, among others, and is known to be NP-complete. We propose a neural network which self-organizes to find a minimal tree. Solutions found by the network compare favourably with the best known or optimal results on test problems from the literature. To the best of our knowledge, the proposed network is the first neural-based solution to the problem. We show that the neural network has a built-in mechanism to escape local minima.     

Designing connected nature reserve networks using a graph theory approach

Habitat fragmentation has been cited as one of the critical reasons for biodiversity loss. Establishing connected nature reserve networks is an effective way to reduce habit fragmentation. However, the resources devoted to nature reserves have always been scarce. Therefore it is important to allocate our scarce resources in an optimal way. The optimal design of a reserve network which is effective both ecologically and economically has become an important research topic in the reserve design literature. The problem of optimal selection of a subset from a larger group of potential habitat sites is solved using either heuristic or formal optimization methods. The heuristic methods, although flexible and computationally fast, can not guarantee the solution is optimal therefore may lead to scarce resources being used in an ineffective way. The formal optimization methods, on the other hand, guarantees the solution is optimal, but it has been argued that it would be difficult to model site selection process using optimization models, especially when spatial attributes of the reserve have to be taken into account. This paper presents a linear integer programming model for the design of a minimal connected reserve network using a graph theory approach. A connected tree is determined corresponding to a connected reserve. Computational performance of the model is tested using datasets randomly generated by the software GAMS. Results show that the model can solve a connected reserve design problem which includes 100 potential sites and 30 species in a reasonable period of time. As an empirical application, the model is applied to the protection of endangered and threatened bird species in the Cache River basin area in Illinois, US. Two connected reserve networks are determined for 13 bird species.     

Designing connected nature reserve networks using a graph theory approach

Habitat fragmentation has been cited as one of the critical reasons for biodiversity loss. Establishing connected nature reserve networks is an effective way to reduce habit fragmentation. However, the resources devoted to nature reserves have always been scarce. Therefore it is important to allocate our scarce resources in an optimal way. The optimal design of a reserve network which is effective both ecologically and economically has become an important research topic in the reserve design literature. The problem of optimal selection of a subset from a larger group of potential habitat sites is solved using either heuristic or formal optimization methods. The heuristic methods, although flexible and computationally fast, can not guarantee the solution is optimal therefore may lead to scarce resources being used in an ineffective way. The formal optimization methods, on the other hand, guarantees the solution is optimal, but it has been argued that it would be difficult to model site selection process using optimization models, especially when spatial attributes of the reserve have to be taken into account. This paper presents a linear integer programming model for the design of a minimal connected reserve network using a graph theory approach. A connected tree is determined corresponding to a connected reserve. Computational performance of the model is tested using datasets randomly generated by the software GAMS. Results show that the model can solve a connected reserve design problem which includes 100 potential sites and 30 species in a reasonable period of time. As an empirical application, the model is applied to the protection of endangered and threatened bird species in the Cache River basin area in Illinois, US. Two connected reserve networks are determined for 13 bird species.     

Predicting protein complex geometries with a neural network

A major challenge of the protein docking problem is to define scoring functions that can distinguish near‐native protein complex geometries from a large number of non‐native geometries (decoys) generated with noncomplexed protein structures (unbound docking). In this study, we have constructed a neural network that employs the information from atom‐pair distance distributions of a large number of decoys to predict protein complex geometries. We found that docking prediction can be significantly improved using two different types of polar hydrogen atoms. To train the neural network, 2000 near‐native decoys of even distance distribution were used for each of the 185 considered protein complexes. The neural network normalizes the information from different protein complexes using an additional protein complex identity input neuron for each complex. The parameters of the neural network were determined such that they mimic a scoring funnel in the neighborhood of the native complex structure. The neural network approach avoids the reference state problem, which occurs in deriving knowledge‐based energy functions for scoring. We show that a distance‐dependent atom pair potential performs much better than a simple atom‐pair contact potential. We have compared the performance of our scoring function with other empirical and knowledge‐based scoring functions such as ZDOCK 3.0, ZRANK, ITScore‐PP, EMPIRE, and RosettaDock. In spite of the simplicity of the method and its functional form, our neural network‐based scoring function achieves a reasonable performance in rigid‐body unbound docking of proteins. Proteins 2010. © 2009 Wiley‐Liss, Inc.     

Using Synchronous Boolean Networks to Model Several Phenomena of Collective Behavior

In this paper, we propose an approach for modeling and analysis of a number of phenomena of collective behavior. By collectives we mean multi-agent systems that transition from one state to another at discrete moments of time. The behavior of a member of a collective (agent) is called conforming if the opinion of this agent at current time moment conforms to the opinion of some other agents at the previous time moment. We presume that at each moment of time every agent makes a decision by choosing from the set (where 1-decision corresponds to action and 0-decision corresponds to inaction). In our approach we model collective behavior with synchronous Boolean networks. We presume that in a network there can be agents that act at every moment of time. Such agents are called instigators. Also there can be agents that never act. Such agents are called loyalists. Agents that are neither instigators nor loyalists are called simple agents. We study two combinatorial problems. The first problem is to find a disposition of instigators that in several time moments transforms a network from a state where the majority of simple agents are inactive to a state with the majority of active agents. The second problem is to find a disposition of loyalists that returns the network to a state with the majority of inactive agents. Similar problems are studied for networks in which simple agents demonstrate the contrary to conforming behavior that we call anticonforming. We obtained several theoretical results regarding the behavior of collectives of agents with conforming or anticonforming behavior. In computational experiments we solved the described problems for randomly generated networks with several hundred vertices. We reduced corresponding combinatorial problems to the Boolean satisfiability problem (SAT) and used modern SAT solvers to solve the instances obtained.     

An artificial hysteresis binary neuron: a model suppressing the oscillatory behaviors of neural dynamics

A hysteresis binary McCulloch-Pitts neuron model is proposed in order to suppress the complicated oscillatory behaviors of neural dynamics. The artificial hysteresis binary neural network is used for scheduling time-multiplex crossbar switches in order to demonstrate the effects of hysteresis. Time-multiplex crossbar switching systems must control traffic on demand such that packet blocking probability and packet waiting time are minimized. The system using n×n processing elements solves an n×n crossbar-control problem with O(1) time, while the best existing parallel algorithm requires O(n) time. The hysteresis binary neural network maximizes the throughput of packets through a crossbar switch. The solution quality of our system does not degrade with the problem size.     

Rule extraction from minimal neural networks for credit card screening

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.     

A maximum neural network approach for N-queens problems

 A novel neural network approach using the maximum neuron model is presented for N-queens problems. The goal of the N-queens problem is to find a set of locations of N queens on an N×N chessboard such that no pair of queens commands each other. The maximum neuron model proposed by Takefuji et al. has been applied to two optimization problems where the optimization of objective functions is requested without constraints. This paper demonstrates the effectiveness of the maximum neuron model for constraint satisfaction problems through the N-queens problem. The performance is verified through simulations in up to 500-queens problems on the sequential mode, the N-parallel mode, and the N ²-parallel mode, where our maximum neural network shows the far better performance than the existing neural networks. Received: 4 June 1996/Accepted in revised form: 13 November 1996     

A flexible annealing chaotic neural network to maximum clique problem

Based on the analysis and comparison of several annealing strategies, we present a flexible annealing chaotic neural network which has flexible controlling ability and quick convergence rate to optimization problem. The proposed network has rich and adjustable chaotic dynamics at the beginning, and then can converge quickly to stable states. We test the network on the maximum clique problem by some graphs of the DIMACS clique instances, p-random and k random graphs. The simulations show that the flexible annealing chaotic neural network can get satisfactory solutions at very little time and few steps. The comparison between our proposed network and other chaotic neural networks denotes that the proposed network has superior executive efficiency and better ability to get optimal or near-optimal solution.     

An improved transiently chaotic neural network for the maximum independent set problem

By analyzing the dynamic behaviors of the transiently chaotic neural network and greedy heuristic for the maximum independent set (MIS) problem, we present an improved transiently chaotic neural network for the MIS problem in this paper. Extensive simulations are performed and the results show that this proposed transiently chaotic neural network can yield better solutions to p-random graphs than other existing algorithms. The efficiency of the new model is also confirmed by the results on the complement graphs of some DIMACS clique instances in the second DIMACS challenge. Moreover, the improved model uses fewer steps to converge to stable state in comparison with the original transiently chaotic neural network.     

A neural network algorithm for the multiple traveling salesmen problem

We developed an efficient neural network algorithm for solving the Multiple Traveling Salesmen Problem (MTSP). A new transformation of the N-city M-salesmen MTSP to the standard Traveling Salesmen Problem (TSP) is introduced. The transformed problem is represented by an expanded version of Hopfield-Tank's neuromorphic city-position map with (N + M-1)-cities and a single fictitious salesmen. The dynamic model associated with the problem is based on the Basic Differential Multiplier Method (BDMM) [26] which evaluates Lagrange multipliers simultaneously with the problem's state variables. The algorithm was successfully tested on many problems with up to 30 cities and five salesmen. In all test cases, the algorithm always converged to valid solutions. The great advantage of this kind of algorithm is that it can provide solutions to complex decision making problems directly by solving a system of ordinary differential equations. No learning steps, logical if statements or adjusting of parameters are required during the computation. The algorithm can therefore be implemented in hardware to solve complex constraint satisfaction problems such as the MTSP at the speed of analog silicon VLSI devices or possibly future optical neural computers.     

NETASA: neural network based prediction of solvent accessibility

MOTIVATION: Prediction of the tertiary structure of a protein from its amino acid sequence is one of the most important problems in molecular biology. The successful prediction of solvent accessibility will be very helpful to achieve this goal. In the present work, we have implemented a server, NETASA for predicting solvent accessibility of amino acids using our newly optimized neural network algorithm. Several new features in the neural network architecture and training method have been introduced, and the network learns faster to provide accuracy values, which are comparable or better than other methods of ASA prediction. RESULTS: Prediction in two and three state classification systems with several thresholds are provided. Our prediction method achieved the accuracy level upto 90% for training and 88% for test data sets. Three state prediction results provide a maximum 65% accuracy for training and 63% for the test data. Applicability of neural networks for ASA prediction has been confirmed with a larger data set and wider range of state thresholds. Salient differences between a linear and exponential network for ASA prediction have been analysed. AVAILABILITY: Online predictions are freely available at: http://www.netasa.org. Linux ix86 binaries of the program written for this work may be obtained by email from the corresponding author.     

On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

Solution of the direct and inverse kinematic problems by a common algorithm based on the mean of multiple computations

For the control of the movement of a multijoint manipulator a mental model which represents the geometrical properties of the arm may prove helpful. Using this model the direct and the inverse kinematic problem could be solved. Here we propose such a model which is based on a recurrent network. It is realized for the example of a three-joint manipulator working in a two-dimensional plane, i.e., for a manipulator with one extra degree of freedom. The system computes the complete set of variables, in our example the three joint angles and the two work-space coordinates of the endpoint of the manipulator. The system finds a stable state and a geometrically correct solution even if only a part of these state variables is given. Thus, the direct and the inverse kinematic problem as well as any mixed problem, including the underconstrained case, can be solved by the network.     

Derivative-free neural network for optimizing the scoring functions associated with dynamic programming of pairwise-profile alignment

Background

A profile-comparison method with position-specific scoring matrix (PSSM) is among the most accurate alignment methods. Currently, cosine similarity and correlation coefficients are used as scoring functions of dynamic programming to calculate similarity between PSSMs. However, it is unclear whether these functions are optimal for profile alignment methods. By definition, these functions cannot capture nonlinear relationships between profiles. Therefore, we attempted to discover a novel scoring function, which was more suitable for the profile-comparison method than existing functions, using neural networks.

Results

Although neural networks required derivative-of-cost functions, the problem being addressed in this study lacked them. Therefore, we implemented a novel derivative-free neural network by combining a conventional neural network with an evolutionary strategy optimization method used as a solver. Using this novel neural network system, we optimized the scoring function to align remote sequence pairs. Our results showed that the pairwise-profile aligner using the novel scoring function significantly improved both alignment sensitivity and precision relative to aligners using existing functions.

Conclusions

We developed and implemented a novel derivative-free neural network and aligner (Nepal) for optimizing sequence alignments. Nepal improved alignment quality by adapting to remote sequence alignments and increasing the expressiveness of similarity scores. Additionally, this novel scoring function can be realized using a simple matrix operation and easily incorporated into other aligners. Moreover our scoring function could potentially improve the performance of homology detection and/or multiple-sequence alignment of remote homologous sequences. The goal of the study was to provide a novel scoring function for profile alignment method and develop a novel learning system capable of addressing derivative-free problems. Our system is capable of optimizing the performance of other sophisticated methods and solving problems without derivative-of-cost functions, which do not always exist in practical problems. Our results demonstrated the usefulness of this optimization method for derivative-free problems.

     

Monte Carlo sampling can be used to determine the size and shape of the steady-state flux space

Constraint-based modeling results in a convex polytope that defines a solution space containing all possible steady-state flux distributions. The properties of this polytope have been studied extensively using linear programming to find the optimal flux distribution under various optimality conditions and convex analysis to define its extreme pathways (edges) and elementary modes. The work presented herein further studies the steady-state flux space by defining its hyper-volume. In low dimensions (i.e. for small sample networks), exact volume calculation algorithms were used. However, due to the #P-hard nature of the vertex enumeration and volume calculation problem in high dimensions, random Monte Carlo sampling was used to characterize the relative size of the solution space of the human red blood cell metabolic network. Distributions of the steady-state flux levels for each reaction in the metabolic network were generated to show the range of flux values for each reaction in the polytope. These results give insight into the shape of the high-dimensional solution space. The value of measuring uptake and secretion rates in shrinking the steady-state flux solution space is illustrated through singular value decomposition of the randomly sampled points. The V(max) of various reactions in the network are varied to determine the sensitivity of the solution space to the maximum capacity constraints. The methods developed in this study are suitable for testing the implication of additional constraints on a metabolic network system and can be used to explore the effects of single nucleotide polymorphisms (SNPs) on network capabilities.     

Face recognition from a single image per person using deep architecture neural networks

Implementing an accurate face recognition system requires images in different variations, and if our database is large, we suffer from problems such as storing cost and low speed in recognition algorithms. On the other hand, in some applications there is only one image available per person for training recognition model. In this article, we propose a neural network model inspired of bidirectional analysis and synthesis brain network which can learn nonlinear mapping between image space and components space. Using a deep neural network model, we have tried to separate pose components from person ones. After setting apart these components, we can use them to synthesis virtual images of test data in different pose and lighting conditions. These virtual images are used to train neural network classifier. The results showed that training neural classifier with virtual images gives better performance than training classifier with frontal view images.