Incremental mutual information: a new method for characterizing the strength and dynamics of connections in neuronal circuits

Understanding the computations performed by neuronal circuits requires characterizing the strength and dynamics of the connections between individual neurons. This characterization is typically achieved by measuring the correlation in the activity of two neurons. We have developed a new measure for studying connectivity in neuronal circuits based on information theory, the incremental mutual information (IMI). By conditioning out the temporal dependencies in the responses of individual neurons before measuring the dependency between them, IMI improves on standard correlation-based measures in several important ways: 1) it has the potential to disambiguate statistical dependencies that reflect the connection between neurons from those caused by other sources (e.g. shared inputs or intrinsic cellular or network mechanisms) provided that the dependencies have appropriate timescales, 2) for the study of early sensory systems, it does not require responses to repeated trials of identical stimulation, and 3) it does not assume that the connection between neurons is linear. We describe the theory and implementation of IMI in detail and demonstrate its utility on experimental recordings from the primate visual system.     

Modeling Higher-Order Correlations within Cortical Microcolumns

We statistically characterize the population spiking activity obtained from simultaneous recordings of neurons across all layers of a cortical microcolumn. Three types of models are compared: an Ising model which captures pairwise correlations between units, a Restricted Boltzmann Machine (RBM) which allows for modeling of higher-order correlations, and a semi-Restricted Boltzmann Machine which is a combination of Ising and RBM models. Model parameters were estimated in a fast and efficient manner using minimum probability flow, and log likelihoods were compared using annealed importance sampling. The higher-order models reveal localized activity patterns which reflect the laminar organization of neurons within a cortical column. The higher-order models also outperformed the Ising model in log-likelihood: On populations of 20 cells, the RBM had 10% higher log-likelihood (relative to an independent model) than a pairwise model, increasing to 45% gain in a larger network with 100 spatiotemporal elements, consisting of 10 neurons over 10 time steps. We further removed the need to model stimulus-induced correlations by incorporating a peri-stimulus time histogram term, in which case the higher order models continued to perform best. These results demonstrate the importance of higher-order interactions to describe the structure of correlated activity in cortical networks. Boltzmann Machines with hidden units provide a succinct and effective way to capture these dependencies without increasing the difficulty of model estimation and evaluation.     

Analyzing Short-Term Noise Dependencies of Spike-Counts in Macaque Prefrontal Cortex Using Copulas and the Flashlight Transformation

Simultaneous spike-counts of neural populations are typically modeled by a Gaussian distribution. On short time scales, however, this distribution is too restrictive to describe and analyze multivariate distributions of discrete spike-counts. We present an alternative that is based on copulas and can account for arbitrary marginal distributions, including Poisson and negative binomial distributions as well as second and higher-order interactions. We describe maximum likelihood-based procedures for fitting copula-based models to spike-count data, and we derive a so-called flashlight transformation which makes it possible to move the tail dependence of an arbitrary copula into an arbitrary orthant of the multivariate probability distribution. Mixtures of copulas that combine different dependence structures and thereby model different driving processes simultaneously are also introduced. First, we apply copula-based models to populations of integrate-and-fire neurons receiving partially correlated input and show that the best fitting copulas provide information about the functional connectivity of coupled neurons which can be extracted using the flashlight transformation. We then apply the new method to data which were recorded from macaque prefrontal cortex using a multi-tetrode array. We find that copula-based distributions with negative binomial marginals provide an appropriate stochastic model for the multivariate spike-count distributions rather than the multivariate Poisson latent variables distribution and the often used multivariate normal distribution. The dependence structure of these distributions provides evidence for common inhibitory input to all recorded stimulus encoding neurons. Finally, we show that copula-based models can be successfully used to evaluate neural codes, e.g., to characterize stimulus-dependent spike-count distributions with information measures. This demonstrates that copula-based models are not only a versatile class of models for multivariate distributions of spike-counts, but that those models can be exploited to understand functional dependencies.     

Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains

The statistical analysis of two simultaneously observed trains of neuronal spikes is described, using as a conceptual framework the theory of stochastic point processes.The first statistical question that arises is whether the observed trains are independent; statistical techniques for testing independence are developed around the notion that, under the null hypothesis, the times of spike occurrence in one train represent random instants in time with respect to the other. If the null hypothesis is rejected—if dependence is attributed to the trains—the problem then becomes that of characterizing the nature and source of the observed dependencies. Statistical signs of various classes of dependencies, including direct interaction and shared input, are discussed and illustrated through computer simulations of interacting neurons. The effects of nonstationarities on the statistical measures for simultaneous spike trains are also discussed. For two-train comparisons of irregularly discharging nerve cells, moderate nonstationarities are shown to have little effect on the detection of interactions.Combining repetitive stimulation and simultaneous recording of spike trains from two (or more) neurons yields additional clues as to possible modes of interaction among the monitored neurons; the theory presented is illustrated by an application to experimentally obtained data from auditory neurons.A companion paper covers the analysis of single spike trains.     

Global discriminative learning for higher-accuracy computational gene prediction

Most ab initio gene predictors use a probabilistic sequence model, typically a hidden Markov model, to combine separately trained models of genomic signals and content. By combining separate models of relevant genomic features, such gene predictors can exploit small training sets and incomplete annotations, and can be trained fairly efficiently. However, that type of piecewise training does not optimize prediction accuracy and has difficulty in accounting for statistical dependencies among different parts of the gene model. With genomic information being created at an ever-increasing rate, it is worth investigating alternative approaches in which many different types of genomic evidence, with complex statistical dependencies, can be integrated by discriminative learning to maximize annotation accuracy. Among discriminative learning methods, large-margin classifiers have become prominent because of the success of support vector machines (SVM) in many classification tasks. We describe CRAIG, a new program for ab initio gene prediction based on a conditional random field model with semi-Markov structure that is trained with an online large-margin algorithm related to multiclass SVMs. Our experiments on benchmark vertebrate datasets and on regions from the ENCODE project show significant improvements in prediction accuracy over published gene predictors that use intrinsic features only, particularly at the gene level and on genes with long introns.     

A generalized linear model for estimating spectrotemporal receptive fields from responses to natural sounds

In the auditory system, the stimulus-response properties of single neurons are often described in terms of the spectrotemporal receptive field (STRF), a linear kernel relating the spectrogram of the sound stimulus to the instantaneous firing rate of the neuron. Several algorithms have been used to estimate STRFs from responses to natural stimuli; these algorithms differ in their functional models, cost functions, and regularization methods. Here, we characterize the stimulus-response function of auditory neurons using a generalized linear model (GLM). In this model, each cell's input is described by: 1) a stimulus filter (STRF); and 2) a post-spike filter, which captures dependencies on the neuron's spiking history. The output of the model is given by a series of spike trains rather than instantaneous firing rate, allowing the prediction of spike train responses to novel stimuli. We fit the model by maximum penalized likelihood to the spiking activity of zebra finch auditory midbrain neurons in response to conspecific vocalizations (songs) and modulation limited (ml) noise. We compare this model to normalized reverse correlation (NRC), the traditional method for STRF estimation, in terms of predictive power and the basic tuning properties of the estimated STRFs. We find that a GLM with a sparse prior predicts novel responses to both stimulus classes significantly better than NRC. Importantly, we find that STRFs from the two models derived from the same responses can differ substantially and that GLM STRFs are more consistent between stimulus classes than NRC STRFs. These results suggest that a GLM with a sparse prior provides a more accurate characterization of spectrotemporal tuning than does the NRC method when responses to complex sounds are studied in these neurons.     

Kinetic models of odor transduction implemented as artificial neural networks

We present a formal model of olfactory transduction corresponding to the biochemical reaction cascade found in chemosensory neurons. It assumes that odorants bind to receptor proteins which, in turn, activate transducer mechanisms corresponding to second messenger-mediated processes. The model is reformulated as a mathematically equivalent artificial neural network (ANN). To enable comparison of the computational power of our model, previously suggested models of chemosensory transduction are also presented in ANN versions. In ANNs, certain biological parameters, such as rate constants and affinities, are transformed into weights that can be fitted by training with a given experimental data set. After training, these weights do not necessarily equal the real biological parameters, but represent a set of values that is sufficient to simulate an experimental set of data. We used ANNs to simulate data recorded from bee subplacodes and compare the capacity of our model with ANN versions of other models. Receptor neurons of the nonpheromonal, general odor-processing subsystem of the honeybee are broadly tuned, have overlapping response spectra, and show highly nonlinear concentration dependencies and mixture interactions, i.e., synergistic and inhibitory effects. Our full model alone has the necessary complexity to simulate these complex response characteristics. To account for the complex response characteristics of honeybee receptor neurons, we suggest that several different receptor protein types and at least two second messenger systems are necessary that may interact at various levels of the transduction cascade and may eventually have opposing effects on receptor neuron excitability.     

Querying quantitative logic models (Q2LM) to study intracellular signaling networks and cell-cytokine interactions

Mathematical models have substantially improved our ability to predict the response of a complex biological system to perturbation, but their use is typically limited by difficulties in specifying model topology and parameter values. Additionally, incorporating entities across different biological scales ranging from molecular to organismal in the same model is not trivial. Here, we present a framework called "querying quantitative logic models" (Q2LM) for building and asking questions of constrained fuzzy logic (cFL) models. cFL is a recently developed modeling formalism that uses logic gates to describe influences among entities, with transfer functions to describe quantitative dependencies. Q2LM does not rely on dedicated data to train the parameters of the transfer functions, and it permits straight-forward incorporation of entities at multiple biological scales. The Q2LM framework can be employed to ask questions such as: Which therapeutic perturbations accomplish a designated goal, and under what environmental conditions will these perturbations be effective? We demonstrate the utility of this framework for generating testable hypotheses in two examples: (i) a intracellular signaling network model; and (ii) a model for pharmacokinetics and pharmacodynamics of cell-cytokine interactions; in the latter, we validate hypotheses concerning molecular design of granulocyte colony stimulating factor.     

Modelling the unpredictability of future biodiversity in ecological networks

We consider the question of how accurately we can hope to predict future biodiversity in a world in which many interacting species are at risk of extinction. Simple models assuming that species’ extinctions occur independently are easily analysed, but do not account for the fact that many species depend on or otherwise interact with each other. In this paper we evaluate the effect of explicitly incorporating ecological dependencies on the predictive ability of models of extinction. In particular, we compare a model in which species’ extinction rates increase because of the extinction of their prey to a model in which the same average rate increase takes place, but in which extinctions occur independently from species to species. One might expect that including this ecological information would make the prediction of future biodiversity more accurate, but instead we find that accounting for food web dependencies reveals greater uncertainty. The expected loss of biodiversity over time is similar between the two models, but the variance in future biodiversity is considerably higher in the model that includes species interactions. This increased uncertainty is because of the non-independence of species—the tendency of two species to respond similarly to the loss of a species on which both depend. We use simulations to show that this increase in variance is robust to many variations of the model, and that its magnitude should be largest in food webs that are highly dependent on a few basal species. Our results should hold whenever ecological dependencies cause most species’ extinction risks to covary positively, and illustrate how more information does not necessarily improve our ability to predict future biodiversity loss.     

A model-based approach to study nearest-neighbor influences reveals complex substitution patterns in non-coding sequences

In this article, we present a likelihood-based framework for modeling site dependencies. Our approach builds upon standard evolutionary models but incorporates site dependencies across the entire tree by letting the evolutionary parameters in these models depend upon the ancestral states at the neighboring sites. It thus avoids the need for introducing new and high-dimensional evolutionary models for site-dependent evolution. We propose a Markov chain Monte Carlo approach with data augmentation to infer the evolutionary parameters under our model. Although our approach allows for wide-ranging site dependencies, we illustrate its use, in two non-coding datasets, in the case of nearest-neighbor dependencies (i.e., evolution directly depending only upon the immediate flanking sites). The results reveal that the general time-reversible model with nearest-neighbor dependencies substantially improves the fit to the data as compared to the corresponding model with site independence. Using the parameter estimates from our model, we elaborate on the importance of the 5-methylcytosine deamination process (i.e., the CpG effect) and show that this process also depends upon the 5' neighboring base identity. We hint at the possibility of a so-called TpA effect and show that the observed substitution behavior is very complex in the light of dinucleotide estimates. We also discuss the presence of CpG effects in a nuclear small subunit dataset and find significant evidence that evolutionary models incorporating context-dependent effects perform substantially better than independent-site models and in some cases even outperform models that incorporate varying rates across sites.     

Decreases of action potential amplitudes, in sodium-free and calcium-free conditions, of identifiable giant neurons of an African giant snail (Achatina fulica Férussac)--I. The right parietal ganglion

Decreases of the action potential amplitude in sodium- and calcium-free states were observed with respect to the four giant neurons, PON (periodically oscillating neuron), Tan (tonically autoactive neuron), RAPN (right anterior pallial neuron) and d-RPLN (dorsal-right parietal large neuron), identified in the right parietal ganglion of the suboesophageal ganglia of an African giant snail (Achatina fulica Férussac). The decrease of the PON action potential amplitude, caused in the sodium-free state, was observed to be 25.4 +/- 2.1% (23.0 +/- 2.0 mV), expressed by M +/- SE, while that of the calcium-free state was 35.0 +/- 2.1% (30.9 +/- 1.7 mV). Then, the two ionic dependencies of the PON action potential were estimated to be about 40-50% on sodium and 50-60% on calcium. The decrease of the TAN action potential in the sodium-free state, was observed to be 20.7 +/- 1.2% (18.8 +/- 1.3 mV), whereas that of the calcium-free state was 42.2 +/- 2.7% (39.0 +/- 2.2 mV), indicating that the two ionic dependencies were 30-40% on sodium and 60-70% on calcium. The decrease of the RAPN action potential in the sodium-free state, was 45.8 +/- 3.7% (40.3 +/- 3.1 mV), whereas that of the calcium-free state was 21.7 +/- 2.5% (17.8 +/- 2.0 mV), indicating that the two ionic dependencies were about 70% on sodium and about 30% on calcium. The decrease of the d-RPLN action potential in the sodium-free state was found to be 17.6 +/- 2.4% (15.2 +/- 1.8 mV), whereas that of the calcium-free state was 23.1 +/- 1.4% (20.8 +/- 1.4 mV), indicating that the two ionic dependencies were 40-50% on sodium and 50-60% on calcium. The action potential amplitudes of all the neurons tested were decreased in both sodium-free and calcium-free states. However, their ionic dependencies were estimated to vary from 70% on sodium (30% on calcium) to 30% on the sodium (70% on the calcium), according to the neurons tested.     

A new serially correlated gamma-frailty process for longitudinal count data

We describe a new multivariate gamma distribution and discuss its implication in a Poisson-correlated gamma-frailty model. This model is introduced to account for between-subjects correlation occurring in longitudinal count data. For likelihood-based inference involving distributions in which high-dimensional dependencies are present, it may be useful to approximate likelihoods based on the univariate or bivariate marginal distributions. The merit of composite likelihood is to reduce the computational complexity of the full likelihood. A 2-stage composite-likelihood procedure is developed for estimating the model parameters. The suggested method is applied to a meta-analysis study for survival curves.     

Learning the selectivity of V2 and V4 neurons using non-linear multi-layer wavelet networks

We investigate a non-linear network with two processing stages optimized to reduce the statistical dependencies in natural images. This network serves as a model for the neural information processing in the higher visual areas of primates (visual cortices V2-V4). The resulting population is analyzed with regard to non-linear selectivity and invariance properties. We find units that are very selective with respect to the space spanned by all possible input signals and units that are invariant with respect to certain stimulus classes. In comparison to the measured distribution of selectivity in V2 neurons, the selectivity histogram of the network units shows an even more pronounced tendency towards higher selectivities. A special property of the system is the emergence of non-linear interactions between coefficients from different scales and orientations, which are necessary for the exploitation of higher-order statistical redundancies of natural images. We extend the concept to multi-layer systems and present some simulation results.     

Neuromodulatory octopaminergic neurons and their functions during insect motor behaviour. The Ernst Florey memory lecture

In this article we describe recent advances in functional studies on the role of octopamine released in the periphery by efferent dorsal or ventral unpaired median neurons. In addition to the previously described modulatory effects on the neuromuscular junction, we describe a metabolic regulatory role for these neurons. Due to their activity glycolytic rates in target tissues, such as muscles, are increased. In flight muscles that use carbohydrate catabolism only at take-off but have to switch to lipid oxidation during prolonged flight, these neurons are only active at rest but are inhibited as soon as flight motor patterns are selected.     

Algorithms for variable length Markov chain modeling

We present a general purpose implementation of variable length Markov models. Contrary to fixed order Markov models, these models are not restricted to a predefined uniform depth. Rather, by examining the training data, a model is constructed that fits higher order Markov dependencies where such contexts exist, while using lower order Markov dependencies elsewhere. As both theoretical and experimental results show, these models are capable of capturing rich signals from a modest amount of training data, without the use of hidden states. AVAILABILITY: The source code is freely available at http://www.soe.ucsc.edu/~jill/src/     

Quantitative flux coupling analysis

Journal of Mathematical Biology - Flux coupling analysis (FCA) aims to describe the functional dependencies among reactions in a metabolic network. Currently studied coupling relations are...     

Modeling the Spatial and Temporal Dependence in fMRI Data

Summary Functional magnetic resonance imaging (fMRI) data sets are large and characterized by complex dependence structures driven by highly sophisticated neurophysiology and aspects of the experimental designs. Typical analyses investigating task‐related changes in measured brain activity use a two‐stage procedure in which the first stage involves subject‐specific models and the second‐stage specifies group (or population) level parameters. Customarily, the first‐level accounts for temporal correlations between the serial scans acquired during one scanning session. Despite accounting for these correlations, fMRI studies often include multiple sessions and temporal dependencies may persist between the corresponding estimates of mean neural activity. Further, spatial correlations between brain activity measurements in different locations are often unaccounted for in statistical modeling and estimation. We propose a two‐stage, spatio‐temporal, autoregressive model that simultaneously accounts for spatial dependencies between voxels within the same anatomical region and for temporal dependencies between a subject's estimates from multiple sessions. We develop an algorithm that leverages the special structure of our covariance model, enabling relatively fast and efficient estimation. Using our proposed method, we analyze fMRI data from a study of inhibitory control in cocaine addicts.