Dynamic updating of distributed neural representations using forward models

In this paper, we present a continuous attractor network model that we hypothesize will give some suggestion of the mechanisms underlying several neural processes such as velocity tuning to visual stimulus, sensory discrimination, sensorimotor transformations, motor control, motor imagery, and imitation. All of these processes share the fundamental characteristic of having to deal with the dynamic integration of motor and sensory variables in order to achieve accurate sensory prediction and/or discrimination. Such principles have already been described in the literature by other high-level modeling studies (Decety and Sommerville in Trends Cogn Sci 7:527–533, 2003; Oztop et al. in Neural Netw 19(3):254–271, 2006; Wolpert et al. in Philos Trans R Soc 358:593–602, 2003). With respect to these studies, our work is more concerned with biologically plausible neural dynamics at a population level. Indeed, we show that a relatively simple extension of the classical neural field models can endow these networks with additional dynamic properties for updating their internal representation using external commands. Moreover, an analysis of the interactions between our model and external inputs also shows interesting properties, which we argue are relevant for a better understanding of the neural processes of the brain.     

Direct queries for discovering network resource properties in a distributed environment

The development and performance of networkaware applications depends on the availability of accurate predictions of network resource properties. Obtaining this information directly from the network is a scalable solution that provides the accurate performance predictions and topology information needed for planning and adapting application behavior across a variety of networks. The performance predictions obtained directly from the network are as accurate as applicationlevel benchmarks, but the networkbased technique provides the added advantages of scalability and topology discovery. We describe how to determine network properties directly from the network using SNMP. We provide an overview of SNMP and describe the features it provides that make it possible to extract both available bandwidth and network topology information from network devices. The available bandwidth predictions based on network queries using SNMP are compared with traditional predictions based on application history to demonstrate that they are equally useful. To demonstrate the feasibility of topology discovery, we present results for a large Ethernet LAN.     

A resource for discovering specific and universal biomarkers for distributed stem cells

Specific and universal biomarkers for distributed stem cells (DSCs) have been elusive. A major barrier to discovery of such ideal DSC biomarkers is difficulty in obtaining DSCs in sufficient quantity and purity. To solve this problem, we used cell lines genetically engineered for conditional asymmetric self-renewal, the defining DSC property. In gene microarray analyses, we identified 85 genes whose expression is tightly asymmetric self-renewal associated (ASRA). The ASRA gene signature prescribed DSCs to undergo asymmetric self-renewal to a greater extent than committed progenitor cells, embryonic stem cells, or induced pluripotent stem cells. This delineation has several significant implications. These include: 1) providing experimental evidence that DSCs in vivo undergo asymmetric self-renewal as individual cells; 2) providing an explanation why earlier attempts to define a common gene expression signature for DSCs were unsuccessful; and 3) predicting that some ASRA proteins may be ideal biomarkers for DSCs. Indeed, two ASRA proteins, CXCR6 and BTG2, and two other related self-renewal pattern associated (SRPA) proteins identified in this gene resource, LGR5 and H2A.Z, display unique asymmetric patterns of expression that have a high potential for universal and specific DSC identification.     

Forming sparse representations by local anti-Hebbian learning

How does the brain form a useful representation of its environment? It is shown here that a layer of simple Hebbian units connected by modifiable anti-Hebbian feed-back connections can learn to code a set of patterns in such a way that statistical dependency between the elements of the representation is reduced, while information is preserved. The resulting code is sparse, which is favourable if it is to be used as input to a subsequent supervised associative layer. The operation of the network is demonstrated on two simple problems.     

Molecular surface representations by sparse critical points

We have defined a molecular surface representation that describes precisely and concisely the complete molecular surface. The representation consists of a limited number of critical points disposed at key locations over the surface. These points adequately represent the shape and the important characteristics of the surface, despite the fact that they are modest in number. We expect the representation to be useful in areas such as molecular recognition and visualization. In particular, using this representation, we are able to achieve accurate and efficient protein–protein and protein–small molecule docking. © 1994 John Wiley & Sons, Inc.     

A sparse matrix method for renal models

A sparse matrix method for the numerical solution of nonlinear differential equations arising in modeling of the renal concentrating mechanism is given. The method involves a renumbering of the variables and equations such that the resulting Jacobian matrix has a block tridiagonal structure and the blocks above and below the main diagonal have a known set of complementary nonzero columns. The computer storage for the method is O(n). Results of some numerical experiments showing the stability of the method are given.     

Generative probabilistic models for protein-protein interaction networks--the biclique perspective

MOTIVATION: Much of the large-scale molecular data from living cells can be represented in terms of networks. Such networks occupy a central position in cellular systems biology. In the protein-protein interaction (PPI) network, nodes represent proteins and edges represent connections between them, based on experimental evidence. As PPI networks are rich and complex, a mathematical model is sought to capture their properties and shed light on PPI evolution. The mathematical literature contains various generative models of random graphs. It is a major, still largely open question, which of these models (if any) can properly reproduce various biologically interesting networks. Here, we consider this problem where the graph at hand is the PPI network of Saccharomyces cerevisiae. We are trying to distinguishing between a model family which performs a process of copying neighbors, represented by the duplication-divergence (DD) model, and models which do not copy neighbors, with the Barabási-Albert (BA) preferential attachment model as a leading example. RESULTS: The observed property of the network is the distribution of maximal bicliques in the graph. This is a novel criterion to distinguish between models in this area. It is particularly appropriate for this purpose, since it reflects the graph's growth pattern under either model. This test clearly favors the DD model. In particular, for the BA model, the vast majority (92.9%) of the bicliques with both sides ≥4 must be already embedded in the model's seed graph, whereas the corresponding figure for the DD model is only 5.1%. Our results, based on the biclique perspective, conclusively show that a na?ve unmodified DD model can capture a key aspect of PPI networks.     

Classification using sparse representations: a biologically plausible approach

Representing signals as linear combinations of basis vectors sparsely selected from an overcomplete dictionary has proven to be advantageous for many applications in pattern recognition, machine learning, signal processing, and computer vision. While this approach was originally inspired by insights into cortical information processing, biologically plausible approaches have been limited to exploring the functionality of early sensory processing in the brain, while more practical applications have employed non-biologically plausible sparse coding algorithms. Here, a biologically plausible algorithm is proposed that can be applied to practical problems. This algorithm is evaluated using standard benchmark tasks in the domain of pattern classification, and its performance is compared to a wide range of alternative algorithms that are widely used in signal and image processing. The results show that for the classification tasks performed here, the proposed method is competitive with the best of the alternative algorithms that have been evaluated. This demonstrates that classification using sparse representations can be performed in a neurally plausible manner, and hence, that this mechanism of classification might be exploited by the brain.     

Removal of correlated speckle noise using sparse and overcomplete representations

Recently, there has been a growing interest in the sparse representation of signals over learned and overcomplete dictionaries. Instead of using fixed transforms such as the wavelets and its variants, an alternative way is to train a redundant dictionary from the image itself. This paper presents a novel de-speckling scheme for medical ultrasound and speckle corrupted photographic images using the sparse representations over a learned overcomplete dictionary. It is shown that the proposed algorithm can be used effectively for the removal of speckle by combining an existing pre-processing stage before an adaptive dictionary could be learned for sparse representation. Extensive simulations are carried out to show the effectiveness of the proposed filter for the removal of speckle noise both visually and quantitatively.     

Novel approaches for discovering industrial enzymes.

New technologies for enzyme discovery are changing the rules of the game for industrial biocatalysis. More kinds of enzymes are available, their hardiness is increasing, and their costs are coming down. These changes are the key drivers for a rebirth of interest in industrial applications of enzymes. The major enabling discovery approaches include screening of biodiversity, genomic sequencing, directed evolution and phage display.     

Generative models versus underlying symmetries to explain biological pattern

Mathematical models play an increasingly important role in the interpretation of biological experiments. Studies often present a model that generates the observations, connecting hypothesized process to an observed pattern. Such generative models confirm the plausibility of an explanation and make testable hypotheses for further experiments. However, studies rarely consider the broad family of alternative models that match the same observed pattern. The symmetries that define the broad class of matching models are in fact the only aspects of information truly revealed by observed pattern. Commonly observed patterns derive from simple underlying symmetries. This article illustrates the problem by showing the symmetry associated with the observed rate of increase in fitness in a constant environment. That underlying symmetry reveals how each particular generative model defines a single example within the broad class of matching models. Further progress on the relation between pattern and process requires deeper consideration of the underlying symmetries.     

The emergence of Miller's magic number on a sparse distributed memory

Human memory is limited in the number of items held in one's mind--a limit known as "Miller's magic number". We study the emergence of such limits as a result of the statistics of large bitvectors used to represent items in memory, given two postulates: i) the Sparse Distributed Memory; and ii) chunking through averaging. Potential implications for theoretical neuroscience are discussed.     

Properties of basis functions generated by shift invariant sparse representations of natural images

 The idea that a sparse representation is the computational principle of visual systems has been supported by Olshausen and Field [Nature (1996) 381: 607–609] and many other studies. On the other hand neurons in the inferotemporal cortex respond to moderately complex features called icon alphabets, and such neurons respond invariantly to the stimulus position. To incorporate this property into sparse representation, an algorithm is proposed that trains basis functions using sparse representations with shift invariance. Shift invariance means that basis functions are allowed to move on image data and that coefficients are equipped with shift invariance. The algorithm is applied to natural images. It is ascertained that moderately complex graphical features emerge that are not as simple as Gabor filters and not as complex as real objects. Shift invariance and moderately complex features correspond to the property of icon alphabets. The results show that there is another connection between visual information processing and sparse representations. Received: 3 November 1999 / Accepted in revised form: 17 February 2000     

Allowing for mandatory covariates in boosting estimation of sparse high-dimensional survival models

Background  

When predictive survival models are built from high-dimensional data, there are often additional covariates, such as clinical scores, that by all means have to be included into the final model. While there are several techniques for the fitting of sparse high-dimensional survival models by penalized parameter estimation, none allows for explicit consideration of such mandatory covariates.     

Stochastic differential equation models for spatially distributed neurons and propagation of chaos for interacting systems.

Distribution or nuclear space-valued stochastic differential equations (SDEs) (diffusions as well as discontinuous equations) are discussed as stochastic models for the behavior of voltage potentials of spatially distributed neurons. A propagation of chaos result is obtained for an interacting system of Hilbert space-valued SDEs.     

Coding of distributed, topographic and non-specific representations within the brain

This article explores the theoretical basis of coding within topographic representations, where neurons encoding specific features such as locations, are arranged into maps. A novel type of representation, termed non-specific, where each neuron does not encode specific features is also postulated. In common with the previously described distributed representations [Rolls, E.T., Treves, A., 1998. Neural Networks and Brain Function. Oxford University Press, Oxford], topographic representations display an exponential relationship between stimuli encoded and both number of neurons and maximum firing rate of those neurons. The non-specific representations described here display a binomial expansion between the number of stimuli encoded and the sum of the number of neurons and the maximum firing rate; therefore groups of non-specific neurons usually encode less stimuli than equivalent topographic layers of neurons. Lower and higher order sensory regions of the brain use either topographic or distributed representations to encode information. It is proposed that non-specific representations may occur in regions of the brain where different types of information may be represented by the same neurons, as occurs in the prefrontal cortex.