Associative search network: A reinforcement learning associative memory

An associative memory system is presented which does not require a teacher to provide the desired associations. For each input key it conducts a search for the output pattern which optimizes an external payoff or reinforcement signal. The associative search network (ASN) combines pattern recognition and function optimization capabilities in a simple and effective way. We define the associative search problem, discuss conditions under which the associative search network is capable of solving it, and present results from computer simulations. The synthesis of sensory-motor control surfaces is discussed as an example of the associative search problem.     

Synthesis of nonlinear control surfaces by a layered associative search network

An approach to solving nonlinear control problems is illustrated by means of a layered associative network composed of adaptive elements capable of reinforcement learning. The first layer adaptively develops a representation in terms of which the second layer can solve the problem linearly. The adaptive elements comprising the network employ a novel type of learning rule whose properties, we argue, are essential to the adaptive behavior of the layered network. The behavior of the network is illustrated by means of a spatial learning problem that requires the formation of nonlinear associations. We argue that this approach to nonlinearity can be extended to a large class of nonlinear control problems.     

Unsupervised clustering and centroid estimation using dynamic competitive learning

In this paper, an unsupervised learning algorithm is developed. Two versions of an artificial neural network, termed a differentiator, are described. It is shown that our algorithm is a dynamic variation of the competitive learning found in most unsupervised learning systems. These systems are frequently used for solving certain pattern recognition tasks such as pattern classification and k-means clustering. Using computer simulation, it is shown that dynamic competitive learning outperforms simple competitive learning methods in solving cluster detection and centroid estimation problems. The simulation results demonstrate that high quality clusters are detected by our method in a short training time. Either a distortion function or the minimum spanning tree method of clustering is used to verify the clustering results. By taking full advantage of all the information presented in the course of training in the differentiator, we demonstrate a powerful adaptive system capable of learning continuously changing patterns.     

Artificial neural networks for blur identification and restoration of nonlinearly degraded images.

In this paper, an image restoration algorithm is proposed to identify noncausal blur function. Image degradation processes include both linear and nonlinear phenomena. A neural network model combining an adaptive auto-associative network with a random Gaussian process is proposed to restore the blurred image and blur function simultaneously. The noisy and blurred images are modeled as continuous associative networks, whereas auto-associative part determines the image model coefficients and the hetero-associative part determines the blur function of the system. The self-organization like structure provides the potential solution of the blind image restoration problem. The estimation and restoration are implemented by using an iterative gradient based algorithm to minimize the error function.     

An associative memory that can form hypotheses: a phase-coded neural network

 Nonlinear associative memories as realized, e.g., by Hopfield nets are characterized by attractor-type dynamics. When fed with a starting pattern, they converge to exactly one of the stored patterns which is supposed to be most similar. These systems cannot render hypotheses of classification, i.e., render several possible answers to a given classification problem. Inspired by von der Malsburg’s correlation theory of brain function, we extend conventional neural network architectures by introducing additional dynamical variables. Assuming an oscillatory time structure of neural firing, i.e., the existence of neural clocks, we assign a so-called phase to each formal neuron. The phases explicitly describe detailed correlations of neural activities neglected in conventional neural network architectures. Implementing this extension into a simple self-organizing network based on a feature map, we present an associative memory that actually is capable of forming hypotheses of classification. Received: 6 December 1993/Accepted in revised form: 14 July 1994     

Influence of noise on the function of a “physiological” neural network

A model neural network with stochastic elements in its millisecond dynamics is investigated. The network consists of neuronal units which are modelled in close analogy to physiological neurons. Dynamical variables of the network are the cellular potentials, axonic currents and synaptic efficacies. The dynamics of the synapses obeys a modified Hebbian rule and, as proposed by v. d. Malsburg (1981, 1985), develop on a time scale of a tenth of a second. In a previous publication (Buhmann and Schulten 1986) we have confirmed that the resulting noiseless autoassociative network is capable of the well-known computational tasks of formal associative networks (Cooper 1973; Kohonen et al. 1984, 1981; Hopfield 1982). In the present paper we demonstrate that random fluctuations of the membrane potential improve the performance of the network. In comparison to a deterministic network a noisy neural network can learn at lower input frequencies and with lower average neural firing rates. The electrical activity of a noisy network is very reminiscent of that observed by physiological recordings. We demonstrate furthermore that associative storage reduces the effective dimension of the phase space in which the electrical activity of the network develops.     

Associative sequential recall in a synaptic matrix

An adaptive neuronal network (synaptic matrix) was simulated. After learning a number of randomly generated scenes, the network was tested for associative sequential recall in a stimulus-bound mode and in an image-bound mode. It is demonstrated that neuronal mechanisms of this kind can exhibit orderliness or looseness of associative response suggestive of human recall behavior.     

Reinforcement Learning or Active Inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain.     

Spontaneous deadlock breaking on amoeba-based neurocomputer

Any artificial concurrent computing system involves a potential risk of “deadlock” that its multiple processes sharing common computational resources are stuck in starved conditions, if simultaneous accesses of the processes to the resources were unconditionally permitted. To avoid the deadlock, it is necessary to set up some form of central control protocol capable of appropriately regulating the resource allocation. On the other hand, many decentralized biological systems also perform concurrent computing based on interactions of components sharing limited amounts of available resources. Despite the absence of a central control unit, they appear to be free from the deadlock implying their death, as long as they are alive. Should we consider that biological computing paradigms are essentially different from artificial ones? Here we employ a photosensitive amoeboid cell known as a model organism for studying cellular information processing and construct an experimental system to explore how the amoeba copes with deadlock-like situations induced by optical feedback control. The feedback control is implemented by a recurrent neural network algorithm for leading the amoeba to solve a particular constraint satisfaction problem. We show that the amoeba is capable of breaking through the deadlock-like situations because its oscillating cellular membrane spontaneously produces a wide variety of spatiotemporal patterns. The result implies that our system can be developed to a neurocomputer that works as logical circuit, associative memory device, combinatorial optimization problem solver, and chaotic computer capable of spontaneous transition among multiple solutions.     

A Parallel Attractor Finding Algorithm Based on Boolean Satisfiability for Genetic Regulatory Networks

In biological systems, the dynamic analysis method has gained increasing attention in the past decade. The Boolean network is the most common model of a genetic regulatory network. The interactions of activation and inhibition in the genetic regulatory network are modeled as a set of functions of the Boolean network, while the state transitions in the Boolean network reflect the dynamic property of a genetic regulatory network. A difficult problem for state transition analysis is the finding of attractors. In this paper, we modeled the genetic regulatory network as a Boolean network and proposed a solving algorithm to tackle the attractor finding problem. In the proposed algorithm, we partitioned the Boolean network into several blocks consisting of the strongly connected components according to their gradients, and defined the connection between blocks as decision node. Based on the solutions calculated on the decision nodes and using a satisfiability solving algorithm, we identified the attractors in the state transition graph of each block. The proposed algorithm is benchmarked on a variety of genetic regulatory networks. Compared with existing algorithms, it achieved similar performance on small test cases, and outperformed it on larger and more complex ones, which happens to be the trend of the modern genetic regulatory network. Furthermore, while the existing satisfiability-based algorithms cannot be parallelized due to their inherent algorithm design, the proposed algorithm exhibits a good scalability on parallel computing architectures.     

Quartet-based phylogeny reconstruction with answer set programming

In this paper, a new representation is presented for the maximum quartet consistency (MQC) problem, where solving the MQC problem becomes searching for an ultrametric matrix that satisfies a maximum number of given quartet topologies. A number of structural properties of the MQC problem in this new representation are characterized through formulating into answer set programming, a recent powerful logic programming tool for modeling and solving search problems. Using these properties, a number of optimization techniques are proposed to speed up the search process. The experimental results on a number of simulated data sets suggest that the new representation, combined with answer set programming, presents a unique perspective to the MQC problem.     

Modelling how cleaner fish approach an ephemeral reward task demonstrates a role for ecologically tuned chunking in the evolution of advanced cognition

What makes cognition “advanced” is an open and not precisely defined question. One perspective involves increasing the complexity of associative learning, from conditioning to learning sequences of events (“chaining”) to representing various cue combinations as “chunks.” Here we develop a weighted graph model to study the mechanism enabling chunking ability and the conditions for its evolution and success, based on the ecology of the cleaner fish Labroides dimidiatus. In some environments, cleaners must learn to serve visitor clients before resident clients, because a visitor leaves if not attended while a resident waits for service. This challenge has been captured in various versions of the ephemeral reward task, which has been proven difficult for a range of cognitively capable species. We show that chaining is the minimal requirement for solving this task in its common simplified laboratory format that involves repeated simultaneous exposure to an ephemeral and permanent food source. Adding ephemeral–ephemeral and permanent–permanent combinations, as cleaners face in the wild, requires individuals to have chunking abilities to solve the task. Importantly, chunking parameters need to be calibrated to ecological conditions in order to produce adaptive decisions. Thus, it is the fine-tuning of this ability, which may be the major target of selection during the evolution of advanced associative learning.

What makes cognition ‘advanced’ is an open and not precisely defined question. In this study, a cognitive model of cleaner fish learning the ephemeral-reward task demonstrates how a critical step in cognitive evolution may be understood as the evolution of chunking and its tuning to fit ecological conditions.     

Neural network methods for identification and optimization of quantum mechanical features needed for bioactivity

This paper presents a new approach to the discovery and design of bioactive compounds. The focus of this application will be on the analysis of enzymatic inhibitors. At present the discovery of enzymatic inhibitors for therapeutic use is often accomplished through random searches. The first phase of discovery is a random search through a large pre-fabricated chemical library. Many molecules are tested with refined enzyme for signs of inhibition. Once a group of lead compounds have been discovered the chemical intuition of biochemists is used to find structurally related compounds that are more effective. This step requires new molecules to be conceived and synthesized, and it is the most time-consuming and expensive step. The development of computational and theoretical methods for prediction of the molecular structure that would bind most tightly prior to synthesis and testing, would facilitate the design of novel inhibitors. In the past, our work has focused on solving the problem of predicting the bioactivity of a molecule prior to synthesis. We used a neural network trained with the bioactivity of known compounds to predict the bioactivity of unknown compounds. In our current work, we use a separate neural network in conjunction with a trained neural network in an attempt to gain insight as to how to modify existing compounds and increase their bioactivity.     

A new MapReduce associative classifier based on a new storage format for large-scale imbalanced data

The process of knowledge discovery from big and high dimensional datasets has become a popular research topic. The classification problem is a key task in bioinformatics, business intelligence, decision science, astronomy, physics, etc. Building associative classifiers has been a notable research interest in recent years because of their superior accuracy. In associative classifiers, using under-sampling or over-sampling methods for imbalanced big datasets reduces accuracy or increases running time, respectively. Hence, there is a significant need to create efficient associative classifiers for imbalanced big data problems. These classifiers should be able to handle challenges such as memory usage, running time and efficiently exploring the search space. To this end, efficient calculation of measures is a primary objective for associative classifiers. In this paper, we propose a new efficient associative classifier for big imbalanced datasets. The proposed method is based on Rare-PEARs (a multi-objective evolutionary algorithm that efficiently discovers rare and reliable association rules) and is able to evaluate rules in a distributed manner by using a new storing data format. This format simplifies measures calculation and is fully compatible with the MapReduce programming model. We have applied the proposed method (RPII) on a well-known big dataset (ECBDL’14) and have compared our results with seven other learning methods. The experimental results show that RPII outperform other methods in sensitivity and final score measures (the values of sensitivity and final score measures were approximately 0.74 and 0.54 respectively). The results demonstrate that the proposed method is a good candidate for large-scale classification problems; furthermore, it achieves reasonable execution time when the target platform is a typical computer clusters.     

Evolution of Associative Learning in Chemical Networks

Organisms that can learn about their environment and modify their behaviour appropriately during their lifetime are more likely to survive and reproduce than organisms that do not. While associative learning – the ability to detect correlated features of the environment – has been studied extensively in nervous systems, where the underlying mechanisms are reasonably well understood, mechanisms within single cells that could allow associative learning have received little attention. Here, using in silico evolution of chemical networks, we show that there exists a diversity of remarkably simple and plausible chemical solutions to the associative learning problem, the simplest of which uses only one core chemical reaction. We then asked to what extent a linear combination of chemical concentrations in the network could approximate the ideal Bayesian posterior of an environment given the stimulus history so far? This Bayesian analysis revealed the ‘memory traces’ of the chemical network. The implication of this paper is that there is little reason to believe that a lack of suitable phenotypic variation would prevent associative learning from evolving in cell signalling, metabolic, gene regulatory, or a mixture of these networks in cells.     

A flexible crane scheduling methodology for container terminals

In this paper, we propose a flexible neighbourhood search strategy for quay crane scheduling problems based on the framework of tabu search (TS) algorithm. In the literature, the container workload of a ship is partitioned into a number of fixed jobs to deal with the complexity of the problem. In this paper, we propose flexible jobs which are dynamically changed by TS throughout the search process to eliminate the impact of fixed jobs on the generated schedules. Alternative job sequences are investigated for quay cranes and a new quay crane dispatching policy is developed to generate schedules. Computational experiments conducted with problem instances available in the literature showed that our algorithm is capable of generating quality schedules for quay crane handling operations at reasonable times.     

Polynomial-Time Algorithm for Controllability Test of a Class of Boolean Biological Networks

In recent years, Boolean-network-model-based approaches to dynamical analysis of complex biological networks such as gene regulatory networks have been extensively studied. One of the fundamental problems in control theory of such networks is the problem of determining whether a given substance quantity can be arbitrarily controlled by operating the other substance quantities, which we call the controllability problem. This paper proposes a polynomial-time algorithm for solving this problem. Although the algorithm is based on a sufficient condition for controllability, it is easily computable for a wider class of large-scale biological networks compared with the existing approaches. A key to this success in our approach is to give up computing Boolean operations in a rigorous way and to exploit an adjacency matrix of a directed graph induced by a Boolean network. By applying the proposed approach to a neurotransmitter signaling pathway, it is shown that it is effective.     

Abstract and associatively based representations in human sequence learning

We give an analysis of performance in an artificial neural network for which the claim had been made that it could learn abstract representations. Our argument is that this network is associative in nature, and cannot develop abstract representations. The network thus converges to a solution that is solely based on the statistical regularities of the training set. Inspired by human experiments that have shown that humans can engage in both associative (statistical) and abstract learning, we present a new, hybrid computational model that combines associative and more abstract, cognitive processes. To cross-validate the model we attempted to predict human behaviour in further experiments. One of these experiments reveals some evidence for the use of abstract representations, whereas the others provide evidence for associatively based performance. The predictions of the hybrid model stand in line with our empirical data.     

An integrated rule- and case-based approach to AIDS initial assessment

The traditional approach to the development of knowledge-based systems (KBS) has been rule-based, where heuristic knowledge is encoded in a set of production rules. A rule-based reasoning (RBR) system needs a well constructed domain theory as its reasoning basis, and it does not make substantial use of the knowledge embedded in previous cases. An RBR system performs relatively well in a knowledge-rich application environment. Although its capability may be limited when previous experiences are not a good representation of the whole population, a case-based reasoning (CBR) system is capable of using past experiences as problem solving tools, therefore, it is appropriate for an experience-rich domain. In recent years, both RBR and CBR have emerged as important and complementary reasoning methodologies in artificial intelligence. For problem solving in AIDS intervention and prevention, it is useful to integrate RBR and CBR. In this paper, a hybrid KBS which integrates a deductive RBR system and an inductive CRB system is proposed to assess AIDS-risky behaviors.