Actions,Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized

Behavioral evidence suggests that instrumental conditioning is governed by two forms of action control: a goal-directed and a habit learning process. Model-based reinforcement learning (RL) has been argued to underlie the goal-directed process; however, the way in which it interacts with habits and the structure of the habitual process has remained unclear. According to a flat architecture, the habitual process corresponds to model-free RL, and its interaction with the goal-directed process is coordinated by an external arbitration mechanism. Alternatively, the interaction between these systems has recently been argued to be hierarchical, such that the formation of action sequences underlies habit learning and a goal-directed process selects between goal-directed actions and habitual sequences of actions to reach the goal. Here we used a two-stage decision-making task to test predictions from these accounts. The hierarchical account predicts that, because they are tied to each other as an action sequence, selecting a habitual action in the first stage will be followed by a habitual action in the second stage, whereas the flat account predicts that the statuses of the first and second stage actions are independent of each other. We found, based on subjects'' choices and reaction times, that human subjects combined single actions to build action sequences and that the formation of such action sequences was sufficient to explain habitual actions. Furthermore, based on Bayesian model comparison, a family of hierarchical RL models, assuming a hierarchical interaction between habit and goal-directed processes, provided a better fit of the subjects'' behavior than a family of flat models. Although these findings do not rule out all possible model-free accounts of instrumental conditioning, they do show such accounts are not necessary to explain habitual actions and provide a new basis for understanding how goal-directed and habitual action control interact.     

Amino acid encoding schemes from protein structure alignments: multi-dimensional vectors to describe residue types

Bioinformatic software has used various numerical encoding schemes to describe amino acid sequences. Orthogonal encoding, employing 20 numbers to describe the amino acid type of one protein residue, is often used with artificial neural network (ANN) models. However, this can increase the model complexity, thus leading to difficulty in implementation and poor performance. Here, we use ANNs to derive encoding schemes for the amino acid types from protein three-dimensional structure alignments. Each of the 20 amino acid types is characterized with a few real numbers. Our schemes are tested on the simulation of amino acid substitution matrices. These simplified schemes outperform the orthogonal encoding on small data sets. Using one of these encoding schemes, we generate a colouring scheme for the amino acids in which comparable amino acids are in similar colours. We expect it to be useful for visual inspection and manual editing of protein multiple sequence alignments.     

Action anticipation in human infants reveals assumptions about anteroposterior body-structure and action

Animal actions are almost universally constrained by the bilateral body-plan. For example, the direction of travel tends to be constrained by the orientation of the animal''s anteroposterior axis. Hence, an animal''s behaviour can reliably guide the identification of its front and back, and its orientation can reliably guide action prediction. We examine the hypothesis that the evolutionarily ancient relation between anteroposterior body-structure and behaviour guides our cognitive processing of agents and their actions. In a series of studies, we demonstrate that, after limited exposure, human infants as young as six months of age spontaneously encode a novel agent as having a certain axial direction with respect to its actions and rely on it when anticipating the agent''s further behaviour. We found that such encoding is restricted to objects exhibiting cues of agency and does not depend on generalization from features of familiar animals. Our research offers a new tool for investigating the perception of animate agency and supports the proposal that the underlying cognitive mechanisms have been shaped by basic biological adaptations in humans.     

Decoding spike timing: the differential reverse-correlation method

It is widely acknowledged that detailed timing of action potentials is used to encode information, for example, in auditory pathways; however, the computational tools required to analyze encoding through timing are still in their infancy. We present a simple example of encoding, based on a recent model of time-frequency analysis, in which units fire action potentials when a certain condition is met, but the timing of the action potential depends also on other features of the stimulus. We show that, as a result, spike-triggered averages are smoothed so much that they do not represent the true features of the encoding. Inspired by this example, we present a simple method, differential reverse correlations, that can separate an analysis of what causes a neuron to spike, and what controls its timing. We analyze with this method the leaky integrate-and-fire neuron and show the method accurately reconstructs the model's kernel.     

Expanding our Understanding of Sequence-Function Relationships of Type II Polyketide Biosynthetic Gene Clusters: Bioinformatics-Guided Identification of Frankiamicin A from Frankia sp. EAN1pec

A large and rapidly increasing number of unstudied “orphan” natural product biosynthetic gene clusters are being uncovered in sequenced microbial genomes. An important goal of modern natural products research is to be able to accurately predict natural product structures and biosynthetic pathways from these gene cluster sequences. This requires both development of bioinformatic methods for global analysis of these gene clusters and experimental characterization of select products produced by gene clusters with divergent sequence characteristics. Here, we conduct global bioinformatic analysis of all available type II polyketide gene cluster sequences and identify a conserved set of gene clusters with unique ketosynthase α/β sequence characteristics in the genomes of Frankia species, a group of Actinobacteria with underexploited natural product biosynthetic potential. Through LC-MS profiling of extracts from several Frankia species grown under various conditions, we identified Frankia sp. EAN1pec as producing a compound with spectral characteristics consistent with the type II polyketide produced by this gene cluster. We isolated the compound, a pentangular polyketide which we named frankiamicin A, and elucidated its structure by NMR and labeled precursor feeding. We also propose biosynthetic and regulatory pathways for frankiamicin A based on comparative genomic analysis and literature precedent, and conduct bioactivity assays of the compound. Our findings provide new information linking this set of Frankia gene clusters with the compound they produce, and our approach has implications for accurate functional prediction of the many other type II polyketide clusters present in bacterial genomes.     

Emergence of good conduct, scaling and zipf laws in human behavioral sequences in an online world

We study behavioral action sequences of players in a massive multiplayer online game. In their virtual life players use eight basic actions which allow them to interact with each other. These actions are communication, trade, establishing or breaking friendships and enmities, attack, and punishment. We measure the probabilities for these actions conditional on previous taken and received actions and find a dramatic increase of negative behavior immediately after receiving negative actions. Similarly, positive behavior is intensified by receiving positive actions. We observe a tendency towards antipersistence in communication sequences. Classifying actions as positive (good) and negative (bad) allows us to define binary 'world lines' of lives of individuals. Positive and negative actions are persistent and occur in clusters, indicated by large scaling exponents α ~ 0.87 of the mean square displacement of the world lines. For all eight action types we find strong signs for high levels of repetitiveness, especially for negative actions. We partition behavioral sequences into segments of length n (behavioral 'words' and 'motifs') and study their statistical properties. We find two approximate power laws in the word ranking distribution, one with an exponent of κ ~ -1 for the ranks up to 100, and another with a lower exponent for higher ranks. The Shannon n-tuple redundancy yields large values and increases in terms of word length, further underscoring the non-trivial statistical properties of behavioral sequences. On the collective, societal level the timeseries of particular actions per day can be understood by a simple mean-reverting log-normal model.     

Transient RNA structure features are evolutionarily conserved and can be computationally predicted

Functional RNA structures tend to be conserved during evolution. This finding is, for example, exploited by comparative methods for RNA secondary structure prediction that currently provide the state-of-art in terms of prediction accuracy. We here provide strong evidence that homologous RNA genes not only fold into similar final RNA structures, but that their folding pathways also share common transient structural features that have been evolutionarily conserved. For this, we compile and investigate a non-redundant data set of 32 sequences with known transient and final RNA secondary structures and devise a dedicated computational analysis pipeline.     

Marine regime shifts: drivers and impacts on ecosystems services

Marine ecosystems can experience regime shifts, in which they shift from being organized around one set of mutually reinforcing structures and processes to another. Anthropogenic global change has broadly increased a wide variety of processes that can drive regime shifts. To assess the vulnerability of marine ecosystems to such shifts and their potential consequences, we reviewed the scientific literature for 13 types of marine regime shifts and used networks to conduct an analysis of co-occurrence of drivers and ecosystem service impacts. We found that regime shifts are caused by multiple drivers and have multiple consequences that co-occur in a non-random pattern. Drivers related to food production, climate change and coastal development are the most common co-occurring causes of regime shifts, while cultural services, biodiversity and primary production are the most common cluster of ecosystem services affected. These clusters prioritize sets of drivers for management and highlight the need for coordinated actions across multiple drivers and scales to reduce the risk of marine regime shifts. Managerial strategies are likely to fail if they only address well-understood or data-rich variables, and international cooperation and polycentric institutions will be critical to implement and coordinate action across the scales at which different drivers operate. By better understanding these underlying patterns, we hope to inform the development of managerial strategies to reduce the risk of high-impact marine regime shifts, especially for areas of the world where data are not available or monitoring programmes are not in place.     

Minimal functional sites allow a classification of zinc sites in proteins

Zinc is indispensable to all forms of life as it is an essential component of many different proteins involved in a wide range of biological processes. Not differently from other metals, zinc in proteins can play different roles that depend on the features of the metal-binding site. In this work, we describe zinc sites in proteins with known structure by means of three-dimensional templates that can be automatically extracted from PDB files and consist of the protein structure around the metal, including the zinc ligands and the residues in close spatial proximity to the ligands. This definition is devised to intrinsically capture the features of the local protein environment that can affect metal function, and corresponds to what we call a minimal functional site (MFS). We used MFSs to classify all zinc sites whose structures are available in the PDB and combined this classification with functional annotation as available in the literature. We classified 77% of zinc sites into ten clusters, each grouping zinc sites with structures that are highly similar, and an additional 16% into seven pseudo-clusters, each grouping zinc sites with structures that are only broadly similar. Sites where zinc plays a structural role are predominant in eight clusters and in two pseudo-clusters, while sites where zinc plays a catalytic role are predominant in two clusters and in five pseudo-clusters. We also analyzed the amino acid composition of the coordination sphere of zinc as a function of its role in the protein, highlighting trends and exceptions. In a period when the number of known zinc proteins is expected to grow further with the increasing awareness of the cellular mechanisms of zinc homeostasis, this classification represents a valuable basis for structure-function studies of zinc proteins, with broad applications in biochemistry, molecular pharmacology and de novo protein design.     

Evolution and Optimality of Similar Neural Mechanisms for Perception and Action during Search

A prevailing theory proposes that the brain''s two visual pathways, the ventral and dorsal, lead to differing visual processing and world representations for conscious perception than those for action. Others have claimed that perception and action share much of their visual processing. But which of these two neural architectures is favored by evolution? Successful visual search is life-critical and here we investigate the evolution and optimality of neural mechanisms mediating perception and eye movement actions for visual search in natural images. We implement an approximation to the ideal Bayesian searcher with two separate processing streams, one controlling the eye movements and the other stream determining the perceptual search decisions. We virtually evolved the neural mechanisms of the searchers'' two separate pathways built from linear combinations of primary visual cortex receptive fields (V1) by making the simulated individuals'' probability of survival depend on the perceptual accuracy finding targets in cluttered backgrounds. We find that for a variety of targets, backgrounds, and dependence of target detectability on retinal eccentricity, the mechanisms of the searchers'' two processing streams converge to similar representations showing that mismatches in the mechanisms for perception and eye movements lead to suboptimal search. Three exceptions which resulted in partial or no convergence were a case of an organism for which the targets are equally detectable across the retina, an organism with sufficient time to foveate all possible target locations, and a strict two-pathway model with no interconnections and differential pre-filtering based on parvocellular and magnocellular lateral geniculate cell properties. Thus, similar neural mechanisms for perception and eye movement actions during search are optimal and should be expected from the effects of natural selection on an organism with limited time to search for food that is not equi-detectable across its retina and interconnected perception and action neural pathways.     

Differential Beta-Band Event-Related Desynchronization during Categorical Action Sequence Planning

A primate study reported the existence of neurons from the dorso-lateral prefrontal cortex which fired prior to executing categorical action sequences. The authors suggested these activities may represent abstract level information. Here, we aimed to find the neurophysiological representation of planning categorical action sequences at the population level in healthy humans. Previous human studies have shown beta-band event-related desynchronization (ERD) during action planning in humans. Some of these studies showed different levels of ERD according to different types of action preparation. Especially, the literature suggests that variations in cognitive factors rather than physical factors (force, direction, etc) modulate the level of beta-ERD. We hypothesized that the level of beta-band power will differ according to planning of different categorical sequences. We measured magnetoencephalography (MEG) from 22 subjects performing 11 four-sequence actions - each consisting of one or two of three simple actions - in 3 categories; ‘Paired (ooxx)’, ‘Alternative (oxox)’ and ‘Repetitive (oooo)’ (‘o’ and ‘x’ each denoting one of three simple actions). Time-frequency representations were calculated for each category during the planning period, and the corresponding beta-power time-courses were compared. We found beta-ERD during the planning period for all subjects, mostly in the contralateral fronto-parietal areas shortly after visual cue onset. Power increase (transient rebound) followed ERD in 20 out of 22 subjects. Amplitudes differed among categories in 20 subjects for both ERD and transient rebound. In 18 out of 20 subjects ‘Repetitive’ category showed the largest ERD and rebound. The current result suggests that beta-ERD in the contralateral frontal/motor/parietal areas during planning is differentiated by the category of action sequences.     

Action selection and refinement in subcortical loops through basal ganglia and cerebellum

Subcortical loops through the basal ganglia and the cerebellum form computationally powerful distributed processing modules (DPMs). This paper relates the computational features of a DPM's loop through the basal ganglia to experimental results for two kinds of natural action selection. First, functional imaging during a serial order recall task was used to study human brain activity during the selection of sequential actions from working memory. Second, microelectrode recordings from monkeys trained in a step-tracking task were used to study the natural selection of corrective submovements. Our DPM-based model assisted in the interpretation of puzzling data from both of these experiments. We come to posit that the many loops through the basal ganglia each regulate the embodiment of pattern formation in a given area of cerebral cortex. This operation serves to instantiate different kinds of action (or thought) mediated by different areas of cerebral cortex. We then use our findings to formulate a model of the aetiology of schizophrenia.     

Evolution of alpha-amylases: architectural features and key residues in the stabilization of the (beta/alpha)(8) scaffold

We provide a comprehensive analysis of the current enzymes with alpha-amylase activity (AAMYs) that belong to family 13 glycoside hydrolase (GH-13; 144 Archaea, Bacteria, and Eukaryota sequences from 87 different species). This study aims to further knowledge of the evolutionary molecular relationships among the sequences of their A and B domains with special emphasis on the correlation between what is observed in the structures and protein evolution. Multialignments for the A domain distinguish two clusters for sequences from Archaea organisms, eight for sequences from Bacteria organisms, and three for sequences from Eukaryota organisms. The clusters for Bacteria do not follow any strict taxonomic pathway; in fact, they are rather scattered. When we compared the A domains of sequences belonging to different kingdoms, we found that various pairs of clusters were significantly similar. Using either sequence similarity with crystallized structures or secondary-structure prediction methods, we identified in all AAMYs the eight putative beta-strands that constitute the beta-sheet in the TIM barrel of the A domain and studied the packing in its interior. We also discovered a "hidden homology" in the TIM barrel, an invariant Gly located upstream in the sequence before the conserved Asp in beta-strand 3. This Gly precedes an alpha-helix and is actively involved in capping its N-terminal end with a capping box. In all cases, a Schellman motif caps the C-terminal end of this helix.     

A general edit distance between RNA structures.

Arc-annotated sequences are useful in representing the structural information of RNA sequences. In general, RNA secondary and tertiary structures can be represented as a set of nested arcs and a set of crossing arcs, respectively. Since RNA functions are largely determined by molecular confirmation and therefore secondary and tertiary structures, the comparison between RNA secondary and tertiary structures has received much attention recently. In this paper, we propose the notion of edit distance to measure the similarity between two RNA secondary and tertiary structures, by incorporating various edit operations performed on both bases and arcs (i.e., base-pairs). Several algorithms are presented to compute the edit distance between two RNA sequences with various arc structures and under various score schemes, either exactly or approximately, with provably good performance. Preliminary experimental tests confirm that our definition of edit distance and the computation model are among the most reasonable ones ever studied in the literature.     

A neural network model for the prediction of membrane-spanning amino acid sequences

The architecture and weights of an artificial neural network model that predicts putative transmembrane sequences have been developed and optimized by the algorithm of structure evolution. The resulting filter is able to classify membrane/nonmembrane transition regions in sequences of integral human membrane proteins with high accuracy. Similar results have been obtained for both training and test set data, indicating that the network has focused on general features of transmembrane sequences rather than specializing on the training data. Seven physicochemical amino acid properties have been used for sequence encoding. The predictions are compared to hydrophobicity plots.     

A neurodynamic account of spontaneous behaviour

The current article suggests that deterministic chaos self-organized in cortical dynamics could be responsible for the generation of spontaneous action sequences. Recently, various psychological observations have suggested that humans and primates can learn to extract statistical structures hidden in perceptual sequences experienced during active environmental interactions. Although it has been suggested that such statistical structures involve chunking or compositional primitives, their neuronal implementations in brains have not yet been clarified. Therefore, to reconstruct the phenomena, synthetic neuro-robotics experiments were conducted by using a neural network model, which is characterized by a generative model with intentional states and its multiple timescales dynamics. The experimental results showed that the robot successfully learned to imitate tutored behavioral sequence patterns by extracting the underlying transition probability among primitive actions. An analysis revealed that a set of primitive action patterns was embedded in the fast dynamics part, and the chaotic dynamics of spontaneously sequencing these action primitive patterns was structured in the slow dynamics part, provided that the timescale was adequately set for each part. It was also shown that self-organization of this type of functional hierarchy ensured robust action generation by the robot in its interactions with a noisy environment. This article discusses the correspondence of the synthetic experiments with the known hierarchy of the prefrontal cortex, the supplementary motor area, and the primary motor cortex for action generation. We speculate that deterministic dynamical structures organized in the prefrontal cortex could be essential because they can account for the generation of both intentional behaviors of fixed action sequences and spontaneous behaviors of pseudo-stochastic action sequences by the same mechanism.     

A top-down linguistic approach to the analysis of genomic sequences: The metabotropic glutamate receptors 1 and 5 in human and in mouse as a case study

This paper presents a top-down strategy to detect features in genomic sequences. The strategy's core is to exploit dictionary-based compression algorithms and analyse the content of the automatically generated dictionary. We classify the different over-represented segments and in the case study we correlate them to experimentally identified or theoretically forecasted biological features. A large spectrum analysis reveals that the only feature co-located with the a priori extracted segments is the torsional flexibility of DNA, while non-B DNA configurations are anti-localized and other features are mostly independent of the extracted sequences. This analysis unravels complex relationships between the linguistic structures investigated under our approach and some known biological features.     

MSClique: Multiple Structure Discovery through the Maximum Weighted Clique Problem

We present a novel approach for feature correspondence and multiple structure discovery in computer vision. In contrast to existing methods, we exploit the fact that point-sets on the same structure usually lie close to each other, thus forming clusters in the image. Given a pair of input images, we initially extract points of interest and extract hierarchical representations by agglomerative clustering. We use the maximum weighted clique problem to find the set of corresponding clusters with maximum number of inliers representing the multiple structures at the correct scales. Our method is parameter-free and only needs two sets of points along with their tentative correspondences, thus being extremely easy to use. We demonstrate the effectiveness of our method in multiple-structure fitting experiments in both publicly available and in-house datasets. As shown in the experiments, our approach finds a higher number of structures containing fewer outliers compared to state-of-the-art methods.