Landmark learning: An illustration of associative search

In a previous paper we defined the associative search problem and presented a system capable of solving it under certain conditions. In this paper we interpret a spatial learning problem as an associative search task and describe the behavior of an adaptive network capable of solving it. This example shows how naturally the associative search problem can arise and permits the search, association, and generalization properties of the adaptive network to bee clearly illustrated.     

Shaping embodied neural networks for adaptive goal-directed behavior

The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves.     

Cerebellar learning of accurate predictive control for fast-reaching movements

Long conduction delays in the nervous system prevent the accurate control of movements by feedback control alone. We present a new, biologically plausible cerebellar model to study how fast arm movements can be executed in spite of these delays. To provide a realistic test-bed of the cerebellar neural model, we embed the cerebellar network in a simulated biological motor system comprising a spinal cord model and a six-muscle two-dimensional arm model. We argue that if the trajectory errors are detected at the spinal cord level, memory traces in the cerebellum can solve the temporal mismatch problem between efferent motor commands and delayed error signals. Moreover, learning is made stable by the inclusion of the cerebello-nucleo-olivary loop in the model. It is shown that the cerebellar network implements a nonlinear predictive regulator by learning part of the inverse dynamics of the plant and spinal circuit. After learning, fast accurate reaching movements can be generated. Received: 8 February 1999 /Accepted in revised form: 7 August 1999     

Feedforward neural network with adaptive reference pattern layer.

A hybrid neural network architecture is investigated for modeling purposes. The proposed hybrid is based on the multilayer perceptron (MLP) network. In addition to the usual hidden layers, the first hidden layer is selected to be an adaptive reference pattern layer. Each unit in this new layer incorporates a reference pattern that is located somewhere in the space spanned by the input variables. The outputs of these units are the component wise-squared differences between the elements of a reference pattern and the inputs. The reference pattern layer has some resemblance to the hidden layer of the radial basis function (RBF) networks. Therefore the proposed design can be regarded as a sort of hybrid of MLP and RBF networks. The presented benchmark experiments show that the proposed hybrid can provide significant advantages over standard MLPs and RBFs in terms of fast and efficient learning, and compact network structure.     

Genetic algorithms with filters for optimal control problems in fed-batch bioreactors

When using a genetic algorithm (GA) to solve optimal control problems that can arise in a fed-batch bioreactor, the most obvious direct approach is to rely on a finite dimensional discretization of the optimal control problem into a nonlinear programming problem. Usually only the control function is discretized, and the continuous control function is approximated by a series of piecewise constant functions. Even though the piecewise discretized controls that the GA produces for the optimal control problem may give good performances, the control policies often show very high activity and differ considerably from those obtained using a continuous optimization strategy. The present study introduces a few filters into a real-coded genetic algorithm as additional operators and investigates the smoothing capabilities of the filters employed. It is observed that inclusion of a filter significantly smoothens the optimal control profile and often encourages the convergence of the algorithm. The applicability of the technique is illustrated by solving two previously reported optimal control problems in fed-batch bioreactors that are known to have singular arcs.     

Identification of alterations in the Jacobian of biochemical reaction networks from steady state covariance data at two conditions

Model building of biochemical reaction networks typically involves experiments in which changes in the behavior due to natural or experimental perturbations are observed. Computational models of reaction networks are also used in a systems biology approach to study how transitions from a healthy to a diseased state result from changes in genetic or environmental conditions. In this paper we consider the nonlinear inverse problem of inferring information about the Jacobian of a Langevin type network model from covariance data of steady state concentrations associated to two different experimental conditions. Under idealized assumptions on the Langevin fluctuation matrices we prove that relative alterations in the network Jacobian can be uniquely identified when comparing the two data sets. Based on this result and the premise that alteration is locally confined to separable parts due to network modularity we suggest a computational approach using hybrid stochastic-deterministic optimization for the detection of perturbations in the network Jacobian using the sparsity promoting effect of $\ell _p$ -penalization. Our approach is illustrated by means of published metabolomic and signaling reaction networks.     

Adaptive fuzzy control with output feedback for H infinity tracking of SISO nonlinear systems

Observer-based adaptive fuzzy H(infinity) control is proposed to achieve H(infinity) tracking performance for a class of nonlinear systems, which are subject to model uncertainty and external disturbances and in which only a measurement of the output is available. The key ideas in the design of the proposed controller are (i) to transform the nonlinear control problem into a regulation problem through suitable output feedback, (ii) to design a state observer for the estimation of the non-measurable elements of the system's state vector, (iii) to design neuro-fuzzy approximators that receive as inputs the parameters of the reconstructed state vector and give as output an estimation of the system's unknown dynamics, (iv) to use an H(infinity) control term for the compensation of external disturbances and modelling errors, (v) to use Lyapunov stability analysis in order to find the learning law for the neuro-fuzzy approximators, and a supervisory control term for disturbance and modelling error rejection. The control scheme is tested in the cart-pole balancing problem and in a DC-motor model.     

A parameter optimization approach for the optimal control of large-scale musculoskeletal systems.

This paper describes a computational method for solving optimal control problems involving large-scale, nonlinear, dynamical systems. Central to the approach is the idea that any optimal control problem can be converted into a standard nonlinear programming problem by parameterizing each control history using a set of nodal points, which then become the variables in the resulting parameter optimization problem. A key feature of the method is that it dispenses with the need to solve the two-point, boundary-value problem derived from the necessary conditions of optimal control theory. Gradient-based methods for solving such problems do not always converge due to computational errors introduced by the highly nonlinear characteristics of the costate variables. Instead, by converting the optimal control problem into a parameter optimization problem, any number of well-developed and proven nonlinear programming algorithms can be used to compute the near-optimal control trajectories. The utility of the parameter optimization approach for solving general optimal control problems for human movement is demonstrated by applying it to a detailed optimal control model for maximum-height human jumping. The validity of the near-optimal control solution is established by comparing it to a solution of the two-point, boundary-value problem derived on the basis of a bang-bang optimal control algorithm. Quantitative comparisons between model and experiment further show that the parameter optimization solution reproduces the major features of a maximum-height, countermovement jump (i.e., trajectories of body-segmental displacements, vertical and fore-aft ground reaction forces, displacement, velocity, and acceleration of the whole-body center of mass, pattern of lower-extremity muscular activity, jump height, and total ground contact time).     

A semi-parametric hybrid neural model for nonlinear blind signal separation

Nonlinear blind signal separation is an important but rather difficult problem. Any general nonlinear independent component analysis algorithm for such a problem should specify which solution it tries to find. Several recent neural networks for separating the post nonlinear blind mixtures are limited to the diagonal nonlinearity, where there is no cross-channel nonlinearity. In this paper, a new semi-parametric hybrid neural network is proposed to separate the post nonlinearly mixed blind signals where cross-channel disturbance is included. This hybrid network consists of two cascading modules, which are a neural nonlinear module for approximating the post nonlinearity and a linear module for separating the predicted linear blind mixtures. The nonlinear module is a semi-parametric expansion made up of two sub-networks, one of which is a linear model and the other of which is a three-layer perceptron. These two sub-networks together produce a "weak" nonlinear operator and can approach relatively strong nonlinearity by tuning parameters. A batch learning algorithm based on the entropy maximization and the gradient descent method is deduced. This model is successfully applied to a blind signal separation problem with two sources. Our simulation results indicate that this hybrid model can effectively approach the cross-channel post nonlinearity and achieve a good visual quality as well as a high signal-to-noise ratio in some cases.     

A novel efficient learning algorithm for self-generating fuzzy neural network with applications

In this paper, a novel efficient learning algorithm towards self-generating fuzzy neural network (SGFNN) is proposed based on ellipsoidal basis function (EBF) and is functionally equivalent to a Takagi-Sugeno-Kang (TSK) fuzzy system. The proposed algorithm is simple and efficient and is able to generate a fuzzy neural network with high accuracy and compact structure. The structure learning algorithm of the proposed SGFNN combines criteria of fuzzy-rule generation with a pruning technology. The Kalman filter (KF) algorithm is used to adjust the consequent parameters of the SGFNN. The SGFNN is employed in a wide range of applications ranging from function approximation and nonlinear system identification to chaotic time-series prediction problem and real-world fuel consumption prediction problem. Simulation results and comparative studies with other algorithms demonstrate that a more compact architecture with high performance can be obtained by the proposed algorithm. In particular, this paper presents an adaptive modeling and control scheme for drug delivery system based on the proposed SGFNN. Simulation study demonstrates the ability of the proposed approach for estimating the drug's effect and regulating blood pressure at a prescribed level.     

Computational modeling of evolutionary learning processes in the brain

The evolutionary selection circuits model of learning has been specified algorithmically. The basic structural components of the selection circuits model are enzymatic neurons, that is, neurons whose firing behavior is controlled by membrane-bound macromolecules called excitases. Learning involves changes in the excitase contents of neurons through a process of variation and selection. In this paper we report on the behavior of a basic version of the learning algorithm which has been developed through extensive interactive experiments with the model. This algorithm is effective in that it enables single neurons or networks of neurons to learn simple pattern classification tasks in a number of time steps which appears experimentally to be a linear function of problem size, as measured by the number of patterns of presynaptic input. The experimental behavior of the algorithm establishes that evolutionary mechanisms of learning are competent to serve as major mechanisms of neuronal adaptation. As an example, we show how the evolutionary learning algorithm can contribute to adaptive motor control processes in which the learning system develops the ability to reach a target in the presence of randomly imposed disturbances.     

Using Synchronous Boolean Networks to Model Several Phenomena of Collective Behavior

In this paper, we propose an approach for modeling and analysis of a number of phenomena of collective behavior. By collectives we mean multi-agent systems that transition from one state to another at discrete moments of time. The behavior of a member of a collective (agent) is called conforming if the opinion of this agent at current time moment conforms to the opinion of some other agents at the previous time moment. We presume that at each moment of time every agent makes a decision by choosing from the set (where 1-decision corresponds to action and 0-decision corresponds to inaction). In our approach we model collective behavior with synchronous Boolean networks. We presume that in a network there can be agents that act at every moment of time. Such agents are called instigators. Also there can be agents that never act. Such agents are called loyalists. Agents that are neither instigators nor loyalists are called simple agents. We study two combinatorial problems. The first problem is to find a disposition of instigators that in several time moments transforms a network from a state where the majority of simple agents are inactive to a state with the majority of active agents. The second problem is to find a disposition of loyalists that returns the network to a state with the majority of inactive agents. Similar problems are studied for networks in which simple agents demonstrate the contrary to conforming behavior that we call anticonforming. We obtained several theoretical results regarding the behavior of collectives of agents with conforming or anticonforming behavior. In computational experiments we solved the described problems for randomly generated networks with several hundred vertices. We reduced corresponding combinatorial problems to the Boolean satisfiability problem (SAT) and used modern SAT solvers to solve the instances obtained.     

瞳孔光反应系统的空间分布式神经网络模型

为模拟刺激光空间分布变化引起瞳孔反应的实验现象,本文建立了空间分布式神经网络瞳孔模型。它是在瞳孔双通道模型基础上,借鉴Ｃａｎｎｏｎ－Ｒｏｂｉｎｓｏｎ的Ｏｃｕｌｏｍｏｔｏｒ模型的双层网络结构和视网膜的镶嵌式特点,经空间延括而成。空间各部位信号经第一层神经元处理得到对应各部位的线性ＤＣ和非线性ＡＣ输出,在第二层神经元进行空间综合,再经第三层神经元复合去控制效应器官虹膜肌的反应。该分布式部位机制模型能解释多种瞳孔实验现象。     

Some examples of psychophysiology and the unconscious.

Different sources of the unconscious are discussed and illustrated, namely, failure to have learned an appropriate perception, repression motivated by an aversive emotion and reinforced by the reduction produced in it when the person stops the thoughts or other cue-producing responses eliciting that emotion, an interference with thoughts or perceptions by a distraction or stimulus overload, and failure to perceive a correct cause-and-effect relationship. The ways in which each of these forms of unconsciousness can reduce the adaptiveness of behavior are described and illustrated as well as how psychophysiological recording can facilitate therapy to improve consciousness and thus voluntary control and behavior that is more intelligent and adaptive.     

Data-Derived Modeling Characterizes Plasticity of MAPK Signaling in Melanoma

The majority of melanomas have been shown to harbor somatic mutations in the RAS-RAF-MEK-MAPK and PI3K-AKT pathways, which play a major role in regulation of proliferation and survival. The prevalence of these mutations makes these kinase signal transduction pathways an attractive target for cancer therapy. However, tumors have generally shown adaptive resistance to treatment. This adaptation is achieved in melanoma through its ability to undergo neovascularization, migration and rearrangement of signaling pathways. To understand the dynamic, nonlinear behavior of signaling pathways in cancer, several computational modeling approaches have been suggested. Most of those models require that the pathway topology remains constant over the entire observation period. However, changes in topology might underlie adaptive behavior to drug treatment. To study signaling rearrangements, here we present a new approach based on Fuzzy Logic (FL) that predicts changes in network architecture over time. This adaptive modeling approach was used to investigate pathway dynamics in a newly acquired experimental dataset describing total and phosphorylated protein signaling over four days in A375 melanoma cell line exposed to different kinase inhibitors. First, a generalized strategy was established to implement a parameter-reduced FL model encoding non-linear activity of a signaling network in response to perturbation. Next, a literature-based topology was generated and parameters of the FL model were derived from the full experimental dataset. Subsequently, the temporal evolution of model performance was evaluated by leaving time-defined data points out of training. Emerging discrepancies between model predictions and experimental data at specific time points allowed the characterization of potential network rearrangement. We demonstrate that this adaptive FL modeling approach helps to enhance our mechanistic understanding of the molecular plasticity of melanoma.     

Ein adaptives und lernfähiges System für diskrete stochastische Prozesse. II

Summary On the adaptive and learning control problems for unknown stochastic systems the estimation of its stochastic transition structure is one of the basic problems for the decision of control inputs to the system. The present paper following the preceding [1] which mainly concerned the decision problem provided of the known stochastic transition structure deals with the practical estimation of the possible supplemental prior information about it. Two examples of computer simulation for the adaptive and learning system controlling the objective system are shown.     

Some examples of psychophysiology and the unconscious

Different sources of the unconscious are discussed and illustrated, namely, failure to have learned an appropriate perception, repression motivated by an aversive emotion and reinforced by the reduction produced in it when the person stops the thoughts or other cue-producing responses eliciting that emotion, an interference with thoughts or perceptions by a distraction or stimulus overload, and failure to perceive a correct cause-and-effect relationship. The ways in which each of these forms of unconsciousness can reduce the adaptiveness of behavior are described and illustrated as well as how psychophysiological recording can facilitate therapy to improve consciousness and thus voluntary control and behavior that is more intelligent and adaptive.     

Behavioral,neural and cellular components underlying olfactory learning in the honeybee

A top-down approach as applied to learning and memory in honeybees provides the opportunity of relating different levels of complexity to each other, and of analyzing the rules and mechanisms from the viewpoint of the respective next higher level. Olfactory conditioning of harnessed bees exemplifies essential elements of associative learning and, in general, forms a bridge between the systems and the cellular levels of analysis. Intracellular recordings of identified neurons during olfactory conditioning play a key role in this effort. They allow testing of the assumptions made by modern behavioral theories of associative learning and provide access to cellular and molecular studies, owing to the identification of their transmitters and the peculiarities of the connectivities. Analysis at this intermediate level of complexity is particularly profitable in the bee, because essential neural elements of the associative network are known and can be tested during ongoing learning behavior. In this respect, the honeybee offers unique properties for the building of bridges between the molecular, cellular neuronal, network and behavioral levels of associative learning.     

Prediction of protein coarse contact maps

Prediction of topological representations of proteins that are geometrically invariants can contribute towards the solution of fundamental open problems in structural genomics like folding. In this paper we focus on coarse grained protein contact maps, a representation that describes the spatial neighborhood relation between secondary structure elements such as helices, beta sheets, and random coils. Our methodology is based on searching the graph space. The search algorithm is guided by an adaptive evaluation function computed by a specialized noncausal recursive connectionist architecture. The neural network is trained using candidate graphs generated during examples of successful searches. Our results demonstrate the viability of the approach for predicting coarse contact maps.