Perspectives on object manipulation and action grammar for percussive actions in primates

The skill of object manipulation is a common feature of primates including humans, although there are species-typical patterns of manipulation. Object manipulation can be used as a comparative scale of cognitive development, focusing on its complexity. Nut cracking in chimpanzees has the highest hierarchical complexity of tool use reported in non-human primates. An analysis of the patterns of object manipulation in naive chimpanzees after nut-cracking demonstrations revealed the cause of difficulties in learning nut-cracking behaviour. Various types of behaviours exhibited within a nut-cracking context can be examined in terms of the application of problem-solving strategies, focusing on their basis in causal understanding or insightful intentionality. Captive chimpanzees also exhibit complex forms of combinatory manipulation, which is the precursor of tool use. A new notation system of object manipulation was invented to assess grammatical rules in manipulative actions. The notation system of action grammar enabled direct comparisons to be made between primates including humans in a variety of object-manipulation tasks, including percussive-tool use.     

Ecosystem assemble grammars: generative capacity and empirical adequacy

Using models based on generative grammars a theory of ecosystem assembly can be formulated that maps a set of species onto a set of environments (Haefner, 1977). Such a theory must incorporate a minimal set of ecological properties in order to correctly describe the adaptive strategies of species and the non-random collection of species comprising an ecosystem. These properties include (1) concordance between activities performed by the individuals of a species, (2) the elaboration of niches due to species invasion, (3) concordance between resources and the users of resources, and (4) the plasticity of species behavior.These properties are used to define the criteria for the weak and strong empirical adequacy of grammars. Weak empirical adequacy of a grammar is the ability of a grammar to generate the sentences of a language. Strong empirical adequacy of a grammar is the ability of a grammar to generate the correct relationships between the elements of the sentences of a language. The adequacy of the members of the Chomsky hierarchy (regular grammars, context-free phrase-structure grammars, context-sensitive phrase-structure grammars, and transformational grammars) is evaluated by comparing their generative capacities and the criteria for empirical adequacy. This analysis indicates that for the representations of the phenomena considered strong empirical adequacy requires at least the generative capacity of a transformational grammar.     

A continuous-time neural model for sequential action

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.     

Walking but not barking improves verb recovery: implications for action observation treatment in aphasia rehabilitation

Recent studies have shown that action observation treatment without concomitant verbal cue has a positive impact on the recovery of verb retrieval deficits in aphasic patients. In agreement with an embodied cognition viewpoint, a hypothesis has been advanced that gestures and language form a single communication system and words whose retrieval is facilitated by gestures are semantically represented through sensory-motor features. However, it is still an open question as to what extent this treatment approach works. Results from the recovery of motor deficits have suggested that action observation promotes motor recovery only for actions that are part of the motor repertoire of the observer. The aim of the present experiment was to further investigate the role of action observation treatment in verb recovery. In particular, we contrasted the effects induced by observing human actions (e.g. dancing, kicking, pointing, eating) versus non human actions (e.g. barking, printing). Seven chronic aphasic patients with a selective deficit in verb retrieval underwent an intensive rehabilitation training that included five daily sessions over two consecutive weeks. Each subject was asked to carefully observe 115 video-clips of actions, one at a time and, after observing them, they had to produce the corresponding verb. Two groups of actions were randomly presented: humans versus nonhuman actions. In all patients, significant improvement in verb retrieval was found only by observing video-clips of human actions. Moreover, follow-up testing revealed long-term verb recovery that was still present two months after the two treatments had ended. In support of the multimodal concept representation's proposal, we suggest that just the observation of actions pertaining to the human motor repertoire is an effective rehabilitation approach for verb recovery.     

The evolutionary dynamics of grammar acquisition

Grammar is the computational system of language. It is a set of rules that specifies how to construct sentences out of words. Grammar is the basis of the unlimited expressibility of human language. Children acquire the grammar of their native language without formal education simply by hearing a number of sample sentences. Children could not solve this learning task if they did not have some pre-formed expectations. In other words, children have to evaluate the sample sentences and choose one grammar out of a limited set of candidate grammars. The restricted search space and the mechanism which allows to evaluate the sample sentences is called universal grammar. Universal grammar cannot be learned; it must be in place when the learning process starts. In this paper, we design a mathematical theory that places the problem of language acquisition into an evolutionary context. We formulate equations for the population dynamics of communication and grammar learning. We ask how accurate children have to learn the grammar of their parents' language for a population of individuals to evolve and maintain a coherent grammatical system. It turns out that there is a maximum error tolerance for which a predominant grammar is stable. We calculate the maximum size of the search space that is compatible with coherent communication in a population. Thus, we specify the conditions for the evolution of universal grammar.     

A word in the hand: action, gesture and mental representation in humans and non-human primates

The movements we make with our hands both reflect our mental processes and help to shape them. Our actions and gestures can affect our mental representations of actions and objects. In this paper, we explore the relationship between action, gesture and thought in both humans and non-human primates and discuss its role in the evolution of language. Human gesture (specifically representational gesture) may provide a unique link between action and mental representation. It is kinaesthetically close to action and is, at the same time, symbolic. Non-human primates use gesture frequently to communicate, and do so flexibly. However, their gestures mainly resemble incomplete actions and lack the representational elements that characterize much of human gesture. Differences in the mirror neuron system provide a potential explanation for non-human primates' lack of representational gestures; the monkey mirror system does not respond to representational gestures, while the human system does. In humans, gesture grounds mental representation in action, but there is no evidence for this link in other primates. We argue that gesture played an important role in the transition to symbolic thought and language in human evolution, following a cognitive leap that allowed gesture to incorporate representational elements.     

An approach to multiscale modelling with graph grammars

Background and Aims

Functional–structural plant models (FSPMs) simulate biological processes at different spatial scales. Methods exist for multiscale data representation and modification, but the advantages of using multiple scales in the dynamic aspects of FSPMs remain unclear. Results from multiscale models in various other areas of science that share fundamental modelling issues with FSPMs suggest that potential advantages do exist, and this study therefore aims to introduce an approach to multiscale modelling in FSPMs.

Methods

A three-part graph data structure and grammar is revisited, and presented with a conceptual framework for multiscale modelling. The framework is used for identifying roles, categorizing and describing scale-to-scale interactions, thus allowing alternative approaches to model development as opposed to correlation-based modelling at a single scale. Reverse information flow (from macro- to micro-scale) is catered for in the framework. The methods are implemented within the programming language XL.

Key Results

Three example models are implemented using the proposed multiscale graph model and framework. The first illustrates the fundamental usage of the graph data structure and grammar, the second uses probabilistic modelling for organs at the fine scale in order to derive crown growth, and the third combines multiscale plant topology with ozone trends and metabolic network simulations in order to model juvenile beech stands under exposure to a toxic trace gas.

Conclusions

The graph data structure supports data representation and grammar operations at multiple scales. The results demonstrate that multiscale modelling is a viable method in FSPM and an alternative to correlation-based modelling. Advantages and disadvantages of multiscale modelling are illustrated by comparisons with single-scale implementations, leading to motivations for further research in sensitivity analysis and run-time efficiency for these models.     

Mirroring and the development of action understanding

The discovery of mirror neurons in the monkey motor cortex has inspired wide-ranging hypotheses about the potential relationship between action control and social cognition. In this paper, we consider the hypothesis that this relationship supports the early development of a critical aspect of social understanding, the ability to analyse others’ actions in terms of goals. Recent investigations of infant action understanding have revealed rich connections between motor development and the analysis of goals in others’ actions. In particular, infants’ own goal-directed actions influence their analysis of others’ goals. This evidence indicates that the cognitive systems that drive infants’ own actions contribute to their analysis of goals in others’ actions. These effects occur at a relatively abstract level of analysis both in terms of the structure infants perceive in others’ actions and relevant structure in infants’ own actions. Although the neural bases of these effects in infants are not yet well understood, current evidence indicates that connections between action production and action perception in infancy involve the interrelated neural systems at work in generating planned, intelligent action.     

Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy

A basic question, intimately tied to the problem of action selection, is that of how actions are assembled into organized sequences. Theories of routine sequential behaviour have long acknowledged that it must rely not only on environmental cues but also on some internal representation of temporal or task context. It is assumed, in most theories, that such internal representations must be organized into a strict hierarchy, mirroring the hierarchical structure of naturalistic sequential behaviour. This article reviews an alternative computational account, which asserts that the representations underlying naturalistic sequential behaviour need not, and arguably cannot, assume a strictly hierarchical form. One apparent liability of this theory is that it seems to contradict neuroscientific evidence indicating that different levels of sequential structure in behaviour are represented at different levels in a hierarchy of cortical areas. New simulations, reported here, show not only that the original computational account can be reconciled with this alignment between behavioural and neural organization, but also that it gives rise to a novel explanation for how this alignment might develop through learning.     

Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity

The principles by which networks of neurons compute, and how spike-timing dependent plasticity (STDP) of synaptic weights generates and maintains their computational function, are unknown. Preceding work has shown that soft winner-take-all (WTA) circuits, where pyramidal neurons inhibit each other via interneurons, are a common motif of cortical microcircuits. We show through theoretical analysis and computer simulations that Bayesian computation is induced in these network motifs through STDP in combination with activity-dependent changes in the excitability of neurons. The fundamental components of this emergent Bayesian computation are priors that result from adaptation of neuronal excitability and implicit generative models for hidden causes that are created in the synaptic weights through STDP. In fact, a surprising result is that STDP is able to approximate a powerful principle for fitting such implicit generative models to high-dimensional spike inputs: Expectation Maximization. Our results suggest that the experimentally observed spontaneous activity and trial-to-trial variability of cortical neurons are essential features of their information processing capability, since their functional role is to represent probability distributions rather than static neural codes. Furthermore it suggests networks of Bayesian computation modules as a new model for distributed information processing in the cortex.     

A transformational-grammar approach to the study of the regulation of gene expression

An important problem in biology is the lack of a set of common principles unifying biological knowledge. We propose generative grammar for constructing an integrative paradigm for the understanding of genome organization and the regulation of gene expression. Linguistic terms in molecular biology are defined. A genetic syntactic structure is defined as being equivalent to a sentence. The hypotheses for the grammar of genome structure are: (i) the "grammaticality" of the linguistic approach studies the "regulability" of genome structures; (ii) the "regulability" of genetic structures is independent from their specific biochemical meaning and (iii) the dynamics of regulation is implicit in the genome structure. A general structure is presented for the grammar; the application of phase-structure rules is justified by the existence of lexical categories. Transformational rules are utilized to represent loops of regulation. Negative inducible, positive repressible, positive inducible and negative repressible alternative mechanisms of regulation are represented, by four transformational rules, and the application of these rules is established by two principles. Finally, this approach is compared to other linguistic applications in molecular biology.     

A range of complex probabilistic models for RNA secondary structure prediction that includes the nearest-neighbor model and more

The standard approach for single-sequence RNA secondary structure prediction uses a nearest-neighbor thermodynamic model with several thousand experimentally determined energy parameters. An attractive alternative is to use statistical approaches with parameters estimated from growing databases of structural RNAs. Good results have been reported for discriminative statistical methods using complex nearest-neighbor models, including CONTRAfold, Simfold, and ContextFold. Little work has been reported on generative probabilistic models (stochastic context-free grammars [SCFGs]) of comparable complexity, although probabilistic models are generally easier to train and to use. To explore a range of probabilistic models of increasing complexity, and to directly compare probabilistic, thermodynamic, and discriminative approaches, we created TORNADO, a computational tool that can parse a wide spectrum of RNA grammar architectures (including the standard nearest-neighbor model and more) using a generalized super-grammar that can be parameterized with probabilities, energies, or arbitrary scores. By using TORNADO, we find that probabilistic nearest-neighbor models perform comparably to (but not significantly better than) discriminative methods. We find that complex statistical models are prone to overfitting RNA structure and that evaluations should use structurally nonhomologous training and test data sets. Overfitting has affected at least one published method (ContextFold). The most important barrier to improving statistical approaches for RNA secondary structure prediction is the lack of diversity of well-curated single-sequence RNA secondary structures in current RNA databases.     

Predictive motor activation during action observation in human infants

Certain regions of the human brain are activated both during action execution and action observation. This so-called ‘mirror neuron system’ has been proposed to enable an observer to understand an action through a process of internal motor simulation. Although there has been much speculation about the existence of such a system from early in life, to date there is little direct evidence that young infants recruit brain areas involved in action production during action observation. To address this question, we identified the individual frequency range in which sensorimotor alpha-band activity was attenuated in nine-month-old infants'' electroencephalographs (EEGs) during elicited reaching for objects, and measured whether activity in this frequency range was also modulated by observing others'' actions. We found that observing a grasping action resulted in motor activation in the infant brain, but that this activity began prior to observation of the action, once it could be anticipated. These results demonstrate not only that infants, like adults, display overlapping neural activity during execution and observation of actions, but that this activation, rather than being directly induced by the visual input, is driven by infants'' understanding of a forthcoming action. These results provide support for theories implicating the motor system in action prediction.     

Reliable short-term memory in the trion model: toward a cortical language and grammar

 It has previously been shown that Hebb learning in a single column in the trion model of cortical organization occurs by selection. Motivated by von Neumann's solution for obtaining reliability and by models of circulating cortical activity, we introduce Hebb intercolumnar couplings to achieve dramatic enhancements in reliability in the firing of connected columns. In order for these enhancements to occur, specific temporal phase differences must exist between the same inherent spatial-temporal memory patterns in connected columns. We then generalize the criteria of large enhancements in the global firing of the entire connected columnar network to investigate the case when different inherent memory patterns are in the columns. The spatial rotations as well as the temporal phases now are crucial. Only certain combinations of inherent memory patterns meet these criteria with the symmetry properties playing a major role. The columnar order of these memory patterns not in the same symmetry family can be extremely important. This yields the first higher-level architecture of a cortical language and grammar within the trion model. The implications of this result with regard to an innate human language and grammar are discussed. Received: 14 June 2000 / Accepted in revised form: 25 July 2000     

Modeling Inhibitory Interneurons in Efficient Sensory Coding Models

There is still much unknown regarding the computational role of inhibitory cells in the sensory cortex. While modeling studies could potentially shed light on the critical role played by inhibition in cortical computation, there is a gap between the simplicity of many models of sensory coding and the biological complexity of the inhibitory subpopulation. In particular, many models do not respect that inhibition must be implemented in a separate subpopulation, with those inhibitory interneurons having a diversity of tuning properties and characteristic E/I cell ratios. In this study we demonstrate a computational framework for implementing inhibition in dynamical systems models that better respects these biophysical observations about inhibitory interneurons. The main approach leverages recent work related to decomposing matrices into low-rank and sparse components via convex optimization, and explicitly exploits the fact that models and input statistics often have low-dimensional structure that can be exploited for efficient implementations. While this approach is applicable to a wide range of sensory coding models (including a family of models based on Bayesian inference in a linear generative model), for concreteness we demonstrate the approach on a network implementing sparse coding. We show that the resulting implementation stays faithful to the original coding goals while using inhibitory interneurons that are much more biophysically plausible.     

Deep generative modeling for protein design

Deep learning approaches have produced substantial breakthroughs in fields such as image classification and natural language processing and are making rapid inroads in the area of protein design. Many generative models of proteins have been developed that encompass all known protein sequences, model specific protein families, or extrapolate the dynamics of individual proteins. Those generative models can learn protein representations that are often more informative of protein structure and function than hand-engineered features. Furthermore, they can be used to quickly propose millions of novel proteins that resemble the native counterparts in terms of expression level, stability, or other attributes. The protein design process can further be guided by discriminative oracles to select candidates with the highest probability of having the desired properties. In this review, we discuss five classes of generative models that have been most successful at modeling proteins and provide a framework for model guided protein design.     

Cortical mechanisms of action selection: the affordance competition hypothesis

At every moment, the natural world presents animals with two fundamental pragmatic problems: selection between actions that are currently possible and specification of the parameters or metrics of those actions. It is commonly suggested that the brain addresses these by first constructing representations of the world on which to build knowledge and make a decision, and then by computing and executing an action plan. However, neurophysiological data argue against this serial viewpoint. In contrast, it is proposed here that the brain processes sensory information to specify, in parallel, several potential actions that are currently available. These potential actions compete against each other for further processing, while information is collected to bias this competition until a single response is selected. The hypothesis suggests that the dorsal visual system specifies actions which compete against each other within the fronto-parietal cortex, while a variety of biasing influences are provided by prefrontal regions and the basal ganglia. A computational model is described, which illustrates how this competition may take place in the cerebral cortex. Simulations of the model capture qualitative features of neurophysiological data and reproduce various behavioural phenomena.     

How hierarchical is language use?

It is generally assumed that hierarchical phrase structure plays a central role in human language. However, considerations of simplicity and evolutionary continuity suggest that hierarchical structure should not be invoked too hastily. Indeed, recent neurophysiological, behavioural and computational studies show that sequential sentence structure has considerable explanatory power and that hierarchical processing is often not involved. In this paper, we review evidence from the recent literature supporting the hypothesis that sequential structure may be fundamental to the comprehension, production and acquisition of human language. Moreover, we provide a preliminary sketch outlining a non-hierarchical model of language use and discuss its implications and testable predictions. If linguistic phenomena can be explained by sequential rather than hierarchical structure, this will have considerable impact in a wide range of fields, such as linguistics, ethology, cognitive neuroscience, psychology and computer science.     

Evolutionary Sequence Modeling for Discovery of Peptide Hormones

There are currently a large number of “orphan” G-protein-coupled receptors (GPCRs) whose endogenous ligands (peptide hormones) are unknown. Identification of these peptide hormones is a difficult and important problem. We describe a computational framework that models spatial structure along the genomic sequence simultaneously with the temporal evolutionary path structure across species and show how such models can be used to discover new functional molecules, in particular peptide hormones, via cross-genomic sequence comparisons. The computational framework incorporates a priori high-level knowledge of structural and evolutionary constraints into a hierarchical grammar of evolutionary probabilistic models. This computational method was used for identifying novel prohormones and the processed peptide sites by producing sequence alignments across many species at the functional-element level. Experimental results with an initial implementation of the algorithm were used to identify potential prohormones by comparing the human and non-human proteins in the Swiss-Prot database of known annotated proteins. In this proof of concept, we identified 45 out of 54 prohormones with only 44 false positives. The comparison of known and hypothetical human and mouse proteins resulted in the identification of a novel putative prohormone with at least four potential neuropeptides. Finally, in order to validate the computational methodology, we present the basic molecular biological characterization of the novel putative peptide hormone, including its identification and regional localization in the brain. This species comparison, HMM-based computational approach succeeded in identifying a previously undiscovered neuropeptide from whole genome protein sequences. This novel putative peptide hormone is found in discreet brain regions as well as other organs. The success of this approach will have a great impact on our understanding of GPCRs and associated pathways and help to identify new targets for drug development.