A multilayer neural network model for perception of rotational motion

A multilayer neural nerwork model for the perception of rotational motion has been developed usingReichardt's motion detector array of correlation type, Kohonen's self-organized feature map and Schuster-Wagner's oscillating neural network. It is shown that the unsupervised learning could make the neurons on the second layer of the network tend to be self-organized in a form resembling columnar organization of selective directions in area MT of the primate's visual cortex. The output layer can interpret rotation information and give the directions and velocities of rotational motion. The computer simulation results are in agreement with some psychophysical observations of rotation-al perception. It is demonstrated that the temporal correlation between the oscillating neurons would be powerful for solving the "binding problem" of shear components of rotational motion.     

Artificial neural network model for predicting membrane protein types

Membrane proteins can be classified among the following five types: (1) type I membrane protein. (2) type II membrane protein. (3) multipass transmembrane proteins. (4) lipid chain-anchored membrane proteins, and (5) GPI-anchored membrane proteins. T. Kohonen's self-organization model which is a typical neural network is applied for predicting the type of a given membrane protein based on its amino acid composition. As a result, the high rates of self-consistency (94.80%) and cross-validation (77.76%), and stronger fault-tolerant ability were obtained.     

A unified multitask architecture for predicting local protein properties

A variety of functionally important protein properties, such as secondary structure, transmembrane topology and solvent accessibility, can be encoded as a labeling of amino acids. Indeed, the prediction of such properties from the primary amino acid sequence is one of the core projects of computational biology. Accordingly, a panoply of approaches have been developed for predicting such properties; however, most such approaches focus on solving a single task at a time. Motivated by recent, successful work in natural language processing, we propose to use multitask learning to train a single, joint model that exploits the dependencies among these various labeling tasks. We describe a deep neural network architecture that, given a protein sequence, outputs a host of predicted local properties, including secondary structure, solvent accessibility, transmembrane topology, signal peptides and DNA-binding residues. The network is trained jointly on all these tasks in a supervised fashion, augmented with a novel form of semi-supervised learning in which the model is trained to distinguish between local patterns from natural and synthetic protein sequences. The task-independent architecture of the network obviates the need for task-specific feature engineering. We demonstrate that, for all of the tasks that we considered, our approach leads to statistically significant improvements in performance, relative to a single task neural network approach, and that the resulting model achieves state-of-the-art performance.     

Motion planning with complete knowledge using a colored SOM

The motion planning problem requires that a collision-free path be determined for a robot moving amidst a fixed set of obstacles. Most neural network approaches to this problem are for the situation in which only local knowledge about the configuration space is available. The main goal of the paper is to show that neural networks are also suitable tools in situations with complete knowledge of the configuration space. In this paper we present an approach that combines a neural network and deterministic techniques. We define a colored version of Kohonen's self-organizing map that consists of two different classes of nodes. The network is presented with random configurations of the robot and, from this information, it constructs a road map of possible motions in the work space. The map is a growing network, and different nodes are used to approximate boundaries of obstacles and the Voronoi diagram of the obstacles, respectively. In a second phase, the positions of the two kinds of nodes are combined to obtain the road map. In this way a number of typical problems with small obstacles and passages are avoided, and the required number of nodes for a given accuracy is within reasonable limits. This road map is searched to find a motion connecting the given source and goal configurations of the robot. The algorithm is simple and general; the only specific computation that is required is a check for intersection of two polygons. We implemented the algorithm for planar robots allowing both translation and rotation and experiments show that compared to conventional techniques it performs well, even for difficult motion planning scenes.     

Prediction of β-Turns

Kohonen's self-organization model, a neural network model, is applied to predict the -turns in proteins. There are 455 -turn tetrapeptides and 3807 non--turn tetrapeptides in the training database. The rates of correct prediction for the 110 -turn tetrapeptides and 30,229 non--turn tetrapeptides in the testing database are 81.8% and 90.7%, respectively. The high quality of prediction of neural network model implies that the residue-coupled effect along a polypeptide chain is important for the formation of reversal turns, such as -turns, during the process of protein folding.     

Learning in human neural networks on microelectrode arrays

This paper describes experiments involving the growth of human neural networks of stem cells on a MEA (microelectrode array) support. The microelectrode arrays (MEAs) are constituted by a glass support in which a set of tungsten electrodes are inserted. The artificial neural network (ANN) paradigm was used by stimulating the neurons in parallel with digital patterns distributed on eight channels, then by analyzing a parallel multichannel output. In particular, the microelectrodes were connected following two different architectures, one inspired by the Kohonen's SOM, the other by the Hopfield network. The output signals have been analyzed in order to evaluate the possibility of organized reactions by the natural neurons.f The results show that the network of human neurons reacts selectively to the subministered digital signals, i.e., it produces similar output signals referred to identical or similar patterns, and clearly differentiates the outputs coming from different stimulations. Analyses performed with a special artificial neural network called ITSOM show the possibility to codify the neural responses to different patterns, thus to interpret the signals coming from the network of biological neurons, assigning a code to each output. It is straightforward to verify that identical codes are generated by the neural reactions to similar patterns. Further experiments are to be designed that improve the hybrid neural networks' capabilities and to test the possibility of utilizing the organized answers of the neurons in several ways.     

Hybrid neural network modeling of a full-scale industrial wastewater treatment process

In recent years, hybrid neural network approaches, which combine mechanistic and neural network models, have received considerable attention. These approaches are potentially very efficient for obtaining more accurate predictions of process dynamics by combining mechanistic and neural network models in such a way that the neural network model properly accounts for unknown and nonlinear parts of the mechanistic model. In this work, a full-scale coke-plant wastewater treatment process was chosen as a model system. Initially, a process data analysis was performed on the actual operational data by using principal component analysis. Next, a simplified mechanistic model and a neural network model were developed based on the specific process knowledge and the operational data of the coke-plant wastewater treatment process, respectively. Finally, the neural network was incorporated into the mechanistic model in both parallel and serial configurations. Simulation results showed that the parallel hybrid modeling approach achieved much more accurate predictions with good extrapolation properties as compared with the other modeling approaches even in the case of process upset caused by, for example, shock loading of toxic compounds. These results indicate that the parallel hybrid neural modeling approach is a useful tool for accurate and cost-effective modeling of biochemical processes, in the absence of other reasonably accurate process models.     

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.     

Formal Models of the Network Co-occurrence Underlying Mental Operations

Systems neuroscience has identified a set of canonical large-scale networks in humans. These have predominantly been characterized by resting-state analyses of the task-unconstrained, mind-wandering brain. Their explicit relationship to defined task performance is largely unknown and remains challenging. The present work contributes a multivariate statistical learning approach that can extract the major brain networks and quantify their configuration during various psychological tasks. The method is validated in two extensive datasets (n = 500 and n = 81) by model-based generation of synthetic activity maps from recombination of shared network topographies. To study a use case, we formally revisited the poorly understood difference between neural activity underlying idling versus goal-directed behavior. We demonstrate that task-specific neural activity patterns can be explained by plausible combinations of resting-state networks. The possibility of decomposing a mental task into the relative contributions of major brain networks, the "network co-occurrence architecture" of a given task, opens an alternative access to the neural substrates of human cognition.     

A model for integrating elementary neural functions into delayed-response behavior

It is well established that various cortical regions can implement a wide array of neural processes, yet the mechanisms which integrate these processes into behavior-producing, brain-scale activity remain elusive. We propose that an important role in this respect might be played by executive structures controlling the traffic of information between the cortical regions involved. To illustrate this hypothesis, we present a neural network model comprising a set of interconnected structures harboring stimulus-related activity (visual representation, working memory, and planning), and a group of executive units with task-related activity patterns that manage the information flowing between them. The resulting dynamics allows the network to perform the dual task of either retaining an image during a delay (delayed-matching to sample task), or recalling from this image another one that has been associated with it during training (delayed-pair association task). The model reproduces behavioral and electrophysiological data gathered on the inferior temporal and prefrontal cortices of primates performing these same tasks. It also makes predictions on how neural activity coding for the recall of the image associated with the sample emerges and becomes prospective during the training phase. The network dynamics proves to be very stable against perturbations, and it exhibits signs of scale-invariant organization and cooperativity. The present network represents a possible neural implementation for active, top-down, prospective memory retrieval in primates. The model suggests that brain activity leading to performance of cognitive tasks might be organized in modular fashion, simple neural functions becoming integrated into more complex behavior by executive structures harbored in prefrontal cortex and/or basal ganglia.     

Predictive modeling of competitive biosorption equilibrium data

This paper compares regression and neural network modeling approaches to predict competitive biosorption equilibrium data. The regression approach is based on the fitting of modified Langmuir-type isotherm models to experimental data. Neural networks, on the other hand, are non-parametric statistical estimators capable of identifying patterns in data and correlations between input and output. Our results show that the neural network approach outperforms traditional regression-based modeling in correlating and predicting the simultaneous uptake of copper and cadmium by a microbial biosorbent. The neural network is capable of accurately predicting unseen data when provided with limited amounts of data for training. Because neural networks are purely data-driven models, they are more suitable for obtaining accurate predictions than for probing the physical nature of the biosorption process.     

Nonparametric density estimation and regression achieved with topographic maps maximizing the information-theoretic entropy of their outputs

We introduce an unsupervised competitive learning rule, called the extended Maximum Entropy learning Rule (eMER), for topographic map formation. Unlike Kohonen's Self-Organizing Map (SOM) algorithm, the presence of a neighborhood function is not a prerequisite for achieving topology-preserving mappings, but instead it is intended: (1) to speed up the learning process and (2) to perform nonparametric regression. We show that, when the neighborhood function vanishes, the neural weigh t density at convergence approaches a linear function of the input density so that the map can be regarded as a nonparametric model of the input density. We apply eMER to density estimation and compare its performance with that of the SOM algorithm and the variable kernel method. Finally, we apply the ‘batch’ version of eMER to nonparametric projection pursuit regression and compare its performance with that of back-propagation learning, projection pursuit learning, constrained topolog ical mapping, and the Heskes and Kappen approach. Received: 12 August 1996 / Accepted in revised form: 9 April 1997     

Implementation of chemometrics,design of experiments,and neural network analysis for prior process knowledge assessment,failure modes and effect analysis,scale-down model development,and process characterization for a chromatographic purification of Teriparatide

Process understanding and characterization forms the foundation, ensuring consistent and robust biologics manufacturing process. Using appropriate modeling tools and machine learning approaches, the process data can be monitored in real time to avoid manufacturing risks. In this article, we have outlined an approach toward implementation of chemometrics and machine learning tools (neural network analysis) to model and predict the behavior of a mixed-mode chromatography step for a biosimilar (Teriparatide) as a case study. The process development data and process knowledge was assimilated into a prior process knowledge assessment using chemometrics tools to derive important parameters critical to performance indicators (i.e., potential quality and process attributes) and to establish the severity ranking for the FMEA analysis. The characterization data of the chromatographic operation are presented alongwith the determination of the critical, key and non- key process parameters, set points, operating, process acceptance and characterized ranges. The scale-down model establishment was assessed using traditional approaches and novel approaches like batch evolution model and neural network analysis. The batch evolution model was further used to demonstrate batch monitoring through direct chromatographic data, thus demonstrating its application for continuos process verification. Assimilation of process knowledge through a structured data acquisition approach, built-in from process development to continuous process verification was demonstrated to result in a data analytics driven model that can be coupled with machine learning tools for real time process monitoring. We recommend application of these approaches with the FDA guidance on stage wise process development and validation to reduce manufacturing risks.     

Representational switching by dynamical reorganization of attractor structure in a network model of the prefrontal cortex

The prefrontal cortex (PFC) plays a crucial role in flexible cognitive behavior by representing task relevant information with its working memory. The working memory with sustained neural activity is described as a neural dynamical system composed of multiple attractors, each attractor of which corresponds to an active state of a cell assembly, representing a fragment of information. Recent studies have revealed that the PFC not only represents multiple sets of information but also switches multiple representations and transforms a set of information to another set depending on a given task context. This representational switching between different sets of information is possibly generated endogenously by flexible network dynamics but details of underlying mechanisms are unclear. Here we propose a dynamically reorganizable attractor network model based on certain internal changes in synaptic connectivity, or short-term plasticity. We construct a network model based on a spiking neuron model with dynamical synapses, which can qualitatively reproduce experimentally demonstrated representational switching in the PFC when a monkey was performing a goal-oriented action-planning task. The model holds multiple sets of information that are required for action planning before and after representational switching by reconfiguration of functional cell assemblies. Furthermore, we analyzed population dynamics of this model with a mean field model and show that the changes in cell assemblies' configuration correspond to those in attractor structure that can be viewed as a bifurcation process of the dynamical system. This dynamical reorganization of a neural network could be a key to uncovering the mechanism of flexible information processing in the PFC.     

A study of early stopping and model selection applied to the papermaking industry

This paper addresses the issues of neural network model development and maintenance in the context of a complex task taken from the papermaking industry. In particular, it describes a comparison study of early stopping techniques and model selection, both to optimise neural network models for generalisation performance. The results presented here show that early stopping via use of a Bayesian model evidence measure is a viable way of optimising performance while also making maximum use of all the data. In addition, they show that ten-fold cross-validation performs well as a model selector and as an estimator of prediction accuracy. These results are important in that they show how neural network models may be optimally trained and selected for highly complex industrial tasks where the data are noisy and limited in number.     

Primer design and marker clustering for multiplex SNP-IT primer extension genotyping assay using statistical modeling

MOTIVATION: The optimization of the primer design is critical for the development of high-throughput SNP genotyping methods. Recently developed statistical models of the SNP-IT primer extension genotyping reaction allow further improvement of primer quality for the assay. RESULTS: Here we describe how the statistical models can be used to improve primer design for the assay. We also show how to optimize clustering of the SNP markers into multiplex panels using statistical model for multiplex SNP-IT. The primer set failure probability calculated by a model is used as a minimization function for both primer selection and primers clustering. Three clustering algorithms for the multiplex genotyping SNP-IT assay are described and their relative performance is evaluated. We also describe the approaches to improve the speed of primer design and clustering calculations when using the statistical models. Our clustering decreases the average failure probability of the marker set by 7-25%. The experimental marker failure rate in the multiplex reaction was reduced dramatically and success rate can be achieved as high as 96%. AVAILABILITY: The primer design using statistical models is freely available from www.autoprimer.com.     

基于鼠脑海马位置细胞与Q学习面向目标导航

生理学实验表明在鼠脑海马结构中存在的一种具有特异性放电特征的细胞,它在大鼠空间导航和环境认知中起着关键性作用,该特异性神经元被称之为位置细胞。本文将基于位置细胞、运动神经元来构建一种前馈神经网络模型,采用Q学习算法实现大鼠面向目标导航任务。实验结果表明该前馈神经网络模型能快速实现大鼠面向目标导航任务。     

Reverse Engineering of Gene Regulatory Networks: A Comparative Study

Reverse engineering of gene regulatory networks has been an intensively studied topic in bioinformatics since it constitutes an intermediate step from explorative to causative gene expression analysis. Many methods have been proposed through recent years leading to a wide range of mathematical approaches. In practice, different mathematical approaches will generate different resulting network structures, thus, it is very important for users to assess the performance of these algorithms. We have conducted a comparative study with six different reverse engineering methods, including relevance networks, neural networks, and Bayesian networks. Our approach consists of the generation of defined benchmark data, the analysis of these data with the different methods, and the assessment of algorithmic performances by statistical analyses. Performance was judged by network size and noise levels. The results of the comparative study highlight the neural network approach as best performing method among those under study.