Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting.     

Learning, memory, and the role of neural network architecture

The performance of information processing systems, from artificial neural networks to natural neuronal ensembles, depends heavily on the underlying system architecture. In this study, we compare the performance of parallel and layered network architectures during sequential tasks that require both acquisition and retention of information, thereby identifying tradeoffs between learning and memory processes. During the task of supervised, sequential function approximation, networks produce and adapt representations of external information. Performance is evaluated by statistically analyzing the error in these representations while varying the initial network state, the structure of the external information, and the time given to learn the information. We link performance to complexity in network architecture by characterizing local error landscape curvature. We find that variations in error landscape structure give rise to tradeoffs in performance; these include the ability of the network to maximize accuracy versus minimize inaccuracy and produce specific versus generalizable representations of information. Parallel networks generate smooth error landscapes with deep, narrow minima, enabling them to find highly specific representations given sufficient time. While accurate, however, these representations are difficult to generalize. In contrast, layered networks generate rough error landscapes with a variety of local minima, allowing them to quickly find coarse representations. Although less accurate, these representations are easily adaptable. The presence of measurable performance tradeoffs in both layered and parallel networks has implications for understanding the behavior of a wide variety of natural and artificial learning systems.     

On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

LMI-based asymptotic stability analysis of neural networks with time-varying delays

The problem of the global asymptotic stability for a class of neural networks with time-varying delays is investigated in this paper, where the activation functions are assumed to be neither monotonic, nor differentiable, nor bounded. By constructing suitable Lyapunov functionals and combining with linear matrix inequality (LMI) technique, new global asymptotic stability criteria about different types of time-varying delays are obtained. It is shown that the criteria can provide less conservative result than some existing ones. Numerical examples are given to demonstrate the applicability of the proposed approach.     

Exponential synchronization of memristive Cohen–Grossberg neural networks with mixed delays

This paper concerns the problem of global exponential synchronization for a class of memristor-based Cohen–Grossberg neural networks with time-varying discrete delays and unbounded distributed delays. The drive-response set is discussed. A novel controller is designed such that the response (slave) system can be controlled to synchronize with the drive (master) system. Through a nonlinear transformation, we get an alternative system from the considered memristor-based Cohen–Grossberg neural networks. By investigating the global exponential synchronization of the alternative system, we obtain the corresponding synchronization criteria of the considered memristor-based Cohen–Grossberg neural networks. Moreover, the conditions established in this paper are easy to be verified and improve the conditions derived in most of existing papers concerning stability and synchronization for memristor-based neural networks. Numerical simulations are given to show the effectiveness of the theoretical results.     

The concept of intuitive and logical in neurocomputing

A concept of the arrangement of a neural computer is proposed, which allows one to solve some problems of modern neural computing and to describe the effects of intuition, creation, as well as a number of effects that are not yet sufficiently understood. The concept is based on the idea that the activity of knowledge acquisition and of information storage are complementary and should be performed by two different subsystems, and the subsystem able to learn has to involve “noise,” i.e., an occasional element. The problem discussed to what extent the artificial intelligence could simulate the living one. Some methods are suggested in order to simulate the effects that traditionally were ascribed to the human organism only (the sense of humor, sleep and dreams, etc.).     

Bioinspiration as a method of problem‐based STEM education: A case study with a class structured around the COVID‐19 crisis

Bioinspiration is a promising lens for biology instruction as it allows the instructor to focus on current issues, such as the COVID‐19 pandemic. From social distancing to oxygen stress, organisms have been tackling pandemic‐related problems for millions of years. What can we learn from such diverse adaptations in our own applications? This review uses a seminar course on the COVID‐19 crisis to illustrate bioinspiration as an approach to teaching biology content. At the start of the class, students mind‐mapped the entire problem; this range of subproblems was used to structure the biology content throughout the entire class. Students came to individual classes with a brainstormed list of biological systems that could serve as inspiration for a particular problem (e.g., absorptive leaves in response to the problem of toilet paper shortages). After exploration of relevant biology content, discussion returned to the focal problem. Students dug deeper into the literature in a group project on mask design and biological systems relevant to filtration and transparency. This class structure was an engaging way for students to learn principles from ecology, evolution, behavior, and physiology. Challenges with this course design revolved around the interdisciplinary and creative nature of the structure; for instance, the knowledge of the participants was often stretched by engineering details. While the present class was focused on the COVID‐19 crisis, a course structured through a bioinspired approach can be applied to other focal problems, or subject areas, giving instructors a powerful method to deliver interdisciplinary content in an integrated and inquiry‐driven way.     

A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback

Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how behaviorally relevant adaptive changes in complex networks of spiking neurons could be achieved in a self-organizing manner through local synaptic plasticity. However, the capabilities and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allows us to predict under which conditions reward-modulated STDP will achieve a desired learning effect. These analytical results imply that neurons can learn through reward-modulated STDP to classify not only spatial but also temporal firing patterns of presynaptic neurons. They also can learn to respond to specific presynaptic firing patterns with particular spike patterns. Finally, the resulting learning theory predicts that even difficult credit-assignment problems, where it is very hard to tell which synaptic weights should be modified in order to increase the global reward for the system, can be solved in a self-organizing manner through reward-modulated STDP. This yields an explanation for a fundamental experimental result on biofeedback in monkeys by Fetz and Baker. In this experiment monkeys were rewarded for increasing the firing rate of a particular neuron in the cortex and were able to solve this extremely difficult credit assignment problem. Our model for this experiment relies on a combination of reward-modulated STDP with variable spontaneous firing activity. Hence it also provides a possible functional explanation for trial-to-trial variability, which is characteristic for cortical networks of neurons but has no analogue in currently existing artificial computing systems. In addition our model demonstrates that reward-modulated STDP can be applied to all synapses in a large recurrent neural network without endangering the stability of the network dynamics.     

Knowledge extraction from evolving spiking neural networks with rank order population coding

This paper demonstrates how knowledge can be extracted from evolving spiking neural networks with rank order population coding. Knowledge discovery is a very important feature of intelligent systems. Yet, a disproportionally small amount of research is centered on the issue of knowledge extraction from spiking neural networks which are considered to be the third generation of artificial neural networks. The lack of knowledge representation compatibility is becoming a major detriment to end users of these networks. We show that a high-level knowledge can be obtained from evolving spiking neural networks. More specifically, we propose a method for fuzzy rule extraction from an evolving spiking network with rank order population coding. The proposed method was used for knowledge discovery on two benchmark taste recognition problems where the knowledge learnt by an evolving spiking neural network was extracted in the form of zero-order Takagi-Sugeno fuzzy IF-THEN rules.     

“Neural” computation of decisions in optimization problems

Highly-interconnected networks of nonlinear analog neurons are shown to be extremely effective in computing. The networks can rapidly provide a collectively-computed solution (a digital output) to a problem on the basis of analog input information. The problems to be solved must be formulated in terms of desired optima, often subject to constraints. The general principles involved in constructing networks to solve specific problems are discussed. Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks. Good solutions to this problem are collectively computed within an elapsed time of only a few neural time constants. The effectiveness of the computation involves both the nonlinear analog response of the neurons and the large connectivity among them. Dedicated networks of biological or microelectronic neurons could provide the computational capabilities described for a wide class of problems having combinatorial complexity. The power and speed naturally displayed by such collective networks may contribute to the effectiveness of biological information processing.     

Non-linear data structure extraction using simple hebbian networks

. We present a class a neural networks algorithms based on simple hebbian learning which allow the finding of higher order structure in data. The neural networks use negative feedback of activation to self-organise; such networks have previously been shown to be capable of performing principal component analysis (PCA). In this paper, this is extended to exploratory projection pursuit (EPP), which is a statistical method for investigating structure in high-dimensional data sets. As opposed to previous proposals for networks which learn using hebbian learning, no explicit weight normalisation, decay or weight clipping is required. The results are extended to multiple units and related to both the statistical literature on EPP and the neural network literature on non-linear PCA. Received: 30 May 1994/Accepted in revised form: 18 November 1994     

Motion planning with complete knowledge using a colored SOM

The motion planning problem requires that a collision-free path be determined for a robot moving amidst a fixed set of obstacles. Most neural network approaches to this problem are for the situation in which only local knowledge about the configuration space is available. The main goal of the paper is to show that neural networks are also suitable tools in situations with complete knowledge of the configuration space. In this paper we present an approach that combines a neural network and deterministic techniques. We define a colored version of Kohonen's self-organizing map that consists of two different classes of nodes. The network is presented with random configurations of the robot and, from this information, it constructs a road map of possible motions in the work space. The map is a growing network, and different nodes are used to approximate boundaries of obstacles and the Voronoi diagram of the obstacles, respectively. In a second phase, the positions of the two kinds of nodes are combined to obtain the road map. In this way a number of typical problems with small obstacles and passages are avoided, and the required number of nodes for a given accuracy is within reasonable limits. This road map is searched to find a motion connecting the given source and goal configurations of the robot. The algorithm is simple and general; the only specific computation that is required is a check for intersection of two polygons. We implemented the algorithm for planar robots allowing both translation and rotation and experiments show that compared to conventional techniques it performs well, even for difficult motion planning scenes.     

Global and robust stability of interval Hopfield neural networks with time-varying delays

In this paper, we investigate the problem of global and robust stability of a class of interval Hopfield neural networks that have time-varying delays. Some criteria for the global and robust stability of such networks are derived, by means of constructing suitable Lyapunov functionals for the networks. As a by-product, for the conventional Hopfield neural networks with time-varying delays, we also obtain some new criteria for their global and asymptotic stability.     

Learning Universal Computations with Spikes

Providing the neurobiological basis of information processing in higher animals, spiking neural networks must be able to learn a variety of complicated computations, including the generation of appropriate, possibly delayed reactions to inputs and the self-sustained generation of complex activity patterns, e.g. for locomotion. Many such computations require previous building of intrinsic world models. Here we show how spiking neural networks may solve these different tasks. Firstly, we derive constraints under which classes of spiking neural networks lend themselves to substrates of powerful general purpose computing. The networks contain dendritic or synaptic nonlinearities and have a constrained connectivity. We then combine such networks with learning rules for outputs or recurrent connections. We show that this allows to learn even difficult benchmark tasks such as the self-sustained generation of desired low-dimensional chaotic dynamics or memory-dependent computations. Furthermore, we show how spiking networks can build models of external world systems and use the acquired knowledge to control them.     

Robust exponential stabilization of a class of delayed neural networks with reaction-diffusion terms

In this paper, the problem of global robust exponential stabilization for a class of neural networks with reaction-diffusion terms and time-varying delays which covers the Hopfield neural networks and cellular neural networks is investigated. A feedback control gain matrix is derived to achieve the global robust exponential stabilization of the neural networks by using the Lyapunov stability theory, and the stabilization condition can be verified if a certain Hamiltonian matrix with no eigenvalues on the imaginary axis. This condition can avoid solving an algebraic Riccati equation. Finally, a numerical simulation illustrates the effectiveness of the results.     

Learning paradigms for image interpretation

In this paper we discuss image understanding and object recognition as a class of processes which involve binding what is seen with what is known. It follows from this perspective that it is important to explicate how systems may learn about spatial information from images, how it is encoded, and, how, in turn, such knowledge is matched with new image data. As well as discussing very specific solutions to these problems, this paper questions the view that our scientific understanding of image understanding is sufficiently circumscribed by the study of how image features are extracted and matched, per se.     

Dynamical properties of min-max networks

In this paper we study the dynamical behavior of a class of neural networks where the local transition rules are max or min functions. We prove that sequential updates define dynamics which reach the equilibrium in O(n2) steps, where n is the size of the network. For synchronous updates the equilibrium is reached in O(n) steps. It is shown that the number of fixed points of the sequential update is at most n. Moreover, given a set of p < or = n vectors, we show how to build a network of size n such that all these vectors are fixed points.     

Learning Bayesian network classifiers using ant colony optimization

Bayesian networks are knowledge representation tools that model the (in)dependency relationships among variables for probabilistic reasoning. Classification with Bayesian networks aims to compute the class with the highest probability given a case. This special kind is referred to as Bayesian network classifiers. Since learning the Bayesian network structure from a dataset can be viewed as an optimization problem, heuristic search algorithms may be applied to build high-quality networks in medium- or large-scale problems, as exhaustive search is often feasible only for small problems. In this paper, we present our new algorithm, ABC-Miner, and propose several extensions to it. ABC-Miner uses ant colony optimization for learning the structure of Bayesian network classifiers. We report extended computational results comparing the performance of our algorithm with eight other classification algorithms, namely six variations of well-known Bayesian network classifiers, cAnt-Miner for discovering classification rules and a support vector machine algorithm.     

Learning the local landscape of protein structures with convolutional neural networks

One fundamental problem of protein biochemistry is to predict protein structure from amino acid sequence. The inverse problem, predicting either entire sequences or individual mutations that are consistent with a given protein structure, has received much less attention even though it has important applications in both protein engineering and evolutionary biology. Here, we ask whether 3D convolutional neural networks (3D CNNs) can learn the local fitness landscape of protein structure to reliably predict either the wild-type amino acid or the consensus in a multiple sequence alignment from the local structural context surrounding site of interest. We find that the network can predict wild type with good accuracy, and that network confidence is a reliable measure of whether a given prediction is likely going to be correct or not. Predictions of consensus are less accurate and are primarily driven by whether or not the consensus matches the wild type. Our work suggests that high-confidence mis-predictions of the wild type may identify sites that are primed for mutation and likely targets for protein engineering.

     

Passivity of memristor-based BAM neural networks with different memductance and uncertain delays

This paper addresses the passivity problem for a class of memristor-based bidirectional associate memory (BAM) neural networks with uncertain time-varying delays. In particular, the proposed memristive BAM neural networks is formulated with two different types of memductance functions. By constructing proper Lyapunov–Krasovskii functional and using differential inclusions theory, a new set of sufficient condition is obtained in terms of linear matrix inequalities which guarantee the passivity criteria for the considered neural networks. Finally, two numerical examples are given to illustrate the effectiveness of the proposed theoretical results.