Optimum signal in a simple neuronal model with signal-dependent noise

How does the information about a signal in neural threshold crossings depend on the noise acting upon it? Two models are explored, a binary McCulloch and Pitts (threshold exceedance) model and a model of waiting time to exceedance--a discrete-time version of interspike intervals. If noise grows linearly with the signal, we find the best identification of the signal in terms of the Fisher information is for signals that do not reach the threshold in the absence of noise. Identification attains the same precision under weak and strong signals, but the coding range decreases at both extremes of signal level. We compare the results obtained for Fisher information with those using related first and second moment measures. The maximum obtainable information is plotted as a function of the ratio of noise to signal.     

Fisher information as a metric of locally optimal processing and stochastic resonance

The origins of Fisher information are in its use as a performance measure for parametric estimation. We augment this and show that the Fisher information can characterize the performance in several other significant signal processing operations. For processing of a weak signal in additive white noise, we demonstrate that the Fisher information determines (i) the maximum output signal-to-noise ratio for a periodic signal; (ii) the optimum asymptotic efficacy for signal detection; (iii) the best cross-correlation coefficient for signal transmission; and (iv) the minimum mean square error of an unbiased estimator. This unifying picture, via inequalities on the Fisher information, is used to establish conditions where improvement by noise through stochastic resonance is feasible or not.     

The mouse skull as a source of morphometric data for phylogeny inference

Brownian motion has been a model widely used for describing phenotypic evolution of continuous characters under random drift. Evolution of traits evolving under weak stabilizing selection, together with drift, can also be modeled by the Ornstein-Uhlenbeck process, in which a population moves at random on an adaptive peak under the influence of drift with selection returning the population towards the optimum. Obviously, reliability of an evolutionary model stands or falls with the extent to which the underlying assumptions are supported or violated. Another potential problem of continuous characters as a source of data for phylogeny inference is the correlation between them. To assess whether the Brownian motion model or the Ornstein-Uhlenbeck model are suitable for modeling the evolution of continuous cranial and dental characters and to what extent these characters are correlated with one another, 11 measurements encompassing various aspects of the mouse skull morphology were collected and subjected to a comparative analysis using the generalized least squares method. It could be shown that only about one-half of the characters evolved according to the Brownian motion model or the Ornstein-Uhlenbeck model. Moreover, about 44% of the correlation coefficients exceeded 0.8, suggesting a need for removing at least phenotypic covariances from the data prior to a phylogenetic analysis. Finally, ancestral states of the characters under study were estimated with the generalized least square method. There has been a general trend towards enlarging the overall size of the skull and increasing the braincase volume in the species of the genus Mus.     

Experimental design criteria in phylogenetics: where to add taxa

Accurate phylogenetic inference is a topic of intensive research and debate and has been studied in response to many different factors: for example, differences in the method of reconstruction, the shape of the underlying tree, the substitution model, and varying quantities and types of data. Investigating whether the conditions used might lead to inaccurate inference has been attempted through elaborate data exploration but less attention has been given to creating a unified methodology to enable experimental designs in phylogenetic analysis to be improved and so avoid suboptimal conditions. Experimental design has been part of the field of statistics since the seminal work of Fisher in the early 20th century and a large body of literature exists on how to design optimum experiments. Here we investigate the use of the Fisher information matrix to decide between candidate positions for adding a taxon to a fixed topology, and introduce a parameter transformation that permits comparison of these different designs. This extension to Goldman (1998. Proc. R. Soc. Lond. B. 265: 1779-1786) thus allows investigation of "where to add taxa" in a phylogeny. We compare three different measures of the total information for selecting the position to add a taxon to a tree. Our methods are illustrated by investigating the behavior of the three criteria when adding a branch to model trees, and by applying the different criteria to two biological examples: a simplified taxon-sampling problem in the balsaminoid Ericales and the phylogeny of seed plants.     

Variable selection for logistic regression using a prediction-focused information criterion

In biostatistical practice, it is common to use information criteria as a guide for model selection. We propose new versions of the focused information criterion (FIC) for variable selection in logistic regression. The FIC gives, depending on the quantity to be estimated, possibly different sets of selected variables. The standard version of the FIC measures the mean squared error of the estimator of the quantity of interest in the selected model. In this article, we propose more general versions of the FIC, allowing other risk measures such as the one based on L(p) error. When prediction of an event is important, as is often the case in medical applications, we construct an FIC using the error rate as a natural risk measure. The advantages of using an information criterion which depends on both the quantity of interest and the selected risk measure are illustrated by means of a simulation study and application to a study on diabetic retinopathy.     

A common goodness-of-fit framework for neural population models using marked point process time-rescaling

The noisy threshold regime, where even a small set of presynaptic neurons can significantly affect postsynaptic spike-timing, is suggested as a key requisite for computation in neurons with high variability. It also has been proposed that signals under the noisy conditions are successfully transferred by a few strong synapses and/or by an assembly of nearly synchronous synaptic activities. We analytically investigate the impact of a transient signaling input on a leaky integrate-and-fire postsynaptic neuron that receives background noise near the threshold regime. The signaling input models a single strong synapse or a set of synchronous synapses, while the background noise represents a lot of weak synapses. We find an analytic solution that explains how the first-passage time (ISI) density is changed by transient signaling input. The analysis allows us to connect properties of the signaling input like spike timing and amplitude with postsynaptic first-passage time density in a noisy environment. Based on the analytic solution, we calculate the Fisher information with respect to the signaling input’s amplitude. For a wide range of amplitudes, we observe a non-monotonic behavior for the Fisher information as a function of background noise. Moreover, Fisher information non-trivially depends on the signaling input’s amplitude; changing the amplitude, we observe one maximum in the high level of the background noise. The single maximum splits into two maximums in the low noise regime. This finding demonstrates the benefit of the analytic solution in investigating signal transfer by neurons.     

Can Monkeys Choose Optimally When Faced with Noisy Stimuli and Unequal Rewards?

We review the leaky competing accumulator model for two-alternative forced-choice decisions with cued responses, and propose extensions to account for the influence of unequal rewards. Assuming that stimulus information is integrated until the cue to respond arrives and that firing rates of stimulus-selective neurons remain well within physiological bounds, the model reduces to an Ornstein-Uhlenbeck (OU) process that yields explicit expressions for the psychometric function that describes accuracy. From these we compute strategies that optimize the rewards expected over blocks of trials administered with mixed difficulty and reward contingencies. The psychometric function is characterized by two parameters: its midpoint slope, which quantifies a subject''s ability to extract signal from noise, and its shift, which measures the bias applied to account for unequal rewards. We fit these to data from two monkeys performing the moving dots task with mixed coherences and reward schedules. We find that their behaviors averaged over multiple sessions are close to optimal, with shifts erring in the direction of smaller penalties. We propose two methods for biasing the OU process to produce such shifts.     

The value of asymmetric signal processing in klinokinesis

Klinokinesis is a behavioral mechanism in which an organism moves toward or away from a stimulus source by altering its frequency of change of direction without biasing its turns with respect to the stimulus field. Previous studies of a variety of organisms have demonstrated that rates of adaptation (or other information processing features) for increases and decreases in stimulus intensity are often very different from one another. In order to determine if such asymmetric signal processing could improve the efficiency of klinokinesis, computer modeling studies were performed. The model involved a simple generic version of klinokinesis in 2 dimensions with the rate of adaptation for increasing intensity varied independently of the rate for decreasing intensity. The effects of three types of noise that limit the performance of the model were tested-intensity noise, motor noise, and developmental noise. The results demonstrated that, with all three types of noise, the two adaptation rates had quite different effects on efficiency. The overall pattern of effects was different for each type of noise. In the cases of intensity noise and motor noise, the optimum combination of adaptation rates had a 3-to 5-fold higher rate for decreases in attractant than for increases, which is similar to what has previously been found with bacteria and nematodes.     

The information content of a character under a Markov model of evolution

The rate of evolutionary change associated with a character determines its utility for the reconstruction of phylogenetic history. For a given age of lineage splits, we examine the information content of a character to assess the magnitude and range of an optimal rate of substitution. On the one hand an optimal transition rate must provide sufficiently many character changes to distinguish subclades, whereas on the other hand changes must be sufficiently rare that reversals on a single branch (and hence homoplasy) are uncommon. In this study, we evolve binary characters over three tree topologies with fixed branch lengths, while varying transition rate as a parameter. We use the character state distribution obtained to measure the "information content" of a character given a transition rate. This is done with respect to several criteria-the probability of obtaining the correct tree using parsimony, the probability of infering the correct ancestral state, and Shannon-Weaver and Fisher information measures on the configuration of probability distributions. All of the information measures suggest the intuitive result of the existence of optimal rates for phylogeny reconstruction. This nonzero optimum is less pronounced if one conditions on there having been a change, in which case the parsimony-based results of minimum change being the most informative tends to hold.     

Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices

We explore the connection between two problems that have arisen independently in the signal processing and related fields: the estimation of the geometric mean of a set of symmetric positive definite (SPD) matrices and their approximate joint diagonalization (AJD). Today there is a considerable interest in estimating the geometric mean of a SPD matrix set in the manifold of SPD matrices endowed with the Fisher information metric. The resulting mean has several important invariance properties and has proven very useful in diverse engineering applications such as biomedical and image data processing. While for two SPD matrices the mean has an algebraic closed form solution, for a set of more than two SPD matrices it can only be estimated by iterative algorithms. However, none of the existing iterative algorithms feature at the same time fast convergence, low computational complexity per iteration and guarantee of convergence. For this reason, recently other definitions of geometric mean based on symmetric divergence measures, such as the Bhattacharyya divergence, have been considered. The resulting means, although possibly useful in practice, do not satisfy all desirable invariance properties. In this paper we consider geometric means of covariance matrices estimated on high-dimensional time-series, assuming that the data is generated according to an instantaneous mixing model, which is very common in signal processing. We show that in these circumstances we can approximate the Fisher information geometric mean by employing an efficient AJD algorithm. Our approximation is in general much closer to the Fisher information geometric mean as compared to its competitors and verifies many invariance properties. Furthermore, convergence is guaranteed, the computational complexity is low and the convergence rate is quadratic. The accuracy of this new geometric mean approximation is demonstrated by means of simulations.     

Population Decoding in Rat Barrel Cortex: Optimizing the Linear Readout of Correlated Population Responses

Sensory information is encoded in the response of neuronal populations. How might this information be decoded by downstream neurons? Here we analyzed the responses of simultaneously recorded barrel cortex neurons to sinusoidal vibrations of varying amplitudes preceded by three adapting stimuli of 0, 6 and 12 µm in amplitude. Using the framework of signal detection theory, we quantified the performance of a linear decoder which sums the responses of neurons after applying an optimum set of weights. Optimum weights were found by the analytical solution that maximized the average signal-to-noise ratio based on Fisher linear discriminant analysis. This provided a biologically plausible decoder that took into account the neuronal variability, covariability, and signal correlations. The optimal decoder achieved consistent improvement in discrimination performance over simple pooling. Decorrelating neuronal responses by trial shuffling revealed that, unlike pooling, the performance of the optimal decoder was minimally affected by noise correlation. In the non-adapted state, noise correlation enhanced the performance of the optimal decoder for some populations. Under adaptation, however, noise correlation always degraded the performance of the optimal decoder. Nonetheless, sensory adaptation improved the performance of the optimal decoder mainly by increasing signal correlation more than noise correlation. Adaptation induced little systematic change in the relative direction of signal and noise. Thus, a decoder which was optimized under the non-adapted state generalized well across states of adaptation.     

Robust experiment design via stochastic approximation

A new methodology is proposed for robust experiment design. It allows uncertainly in the nominal parameters of the model under study to be taken into account by assuming that these parameters belong to some population with known statistics. The mathematical expectation of the determinant of the Fisher information matrix over this population is here taken as a measure of optimality, but the expectation of other nonrobust criteria could have been considered as well. Stochastic approximation techniques are advocated as the simplest tools for optimizing these robust criteria. The efficiency of the proposed algorithms is demonstrated on simple examples—for which an analytical solution exists—as well as on more complex ones. A comparison is made with Landaw's suboptimal approach, which supports an interesting conjecture about the robustness of replicate samples.     

FGX: a frequentist gene expression index for Affymetrix arrays

We consider a new frequentist gene expression index for Affymetrix oligonucleotide DNA arrays, using a similar probe intensity model as suggested by Hein and others (2005), called the Bayesian gene expression index (BGX). According to this model, the perfect match and mismatch values are assumed to be correlated as a result of sharing a common gene expression signal. Rather than a Bayesian approach, we develop a maximum likelihood algorithm for estimating the underlying common signal. In this way, estimation is explicit and much faster than the BGX implementation. The observed Fisher information matrix, rather than a posterior credibility interval, gives an idea of the accuracy of the estimators. We evaluate our method using benchmark spike-in data sets from Affymetrix and GeneLogic by analyzing the relationship between estimated signal and concentration, i.e. true signal, and compare our results with other commonly used methods.     

A qualitative comparison of some diffusion models for neural activity via stochastic ordering

A number of diffusion processes have been proposed as a continuous analog of Stein's model for the subthreshold membrane potential of a neuron. Interspike intervals are then described as the first-passage-time of the corresponding diffusion model through a suitable threshold. Various biological considerations suggest the use of more sophisticated models in lieu of the Ornstein-Uhlenbeck model. However, the advantages of the additional complexity are not always clear. Comparisons among different models generally use numerical methods in specific examples without a general sensitivity analysis on the role of the model parameters. Here, we compare the distribution of interspike intervals from different models using the method of stochastic ordering. The qualitative comparison of the role of each parameter extends the results obtained from numerical simulations. One result on neurons with high positive net excitation is that the reversal potential models considered do not greatly differ from the Ornstein-Uhlenbeck model. For neurons with increased inhibition, the models give greater differences among the interspike interval distributions. In particular, when the mean trajectories are matched, the Feller model gives shorter times than the Ornstein-Uhlenbeck model but longer times than our double reversal potential model. Received: 5 August 1999 / Accepted in revised form: 8 May 2000     

Information gain for genetic parameter estimation with incorporation of marker data

Genetic marker data has been increasingly incorporated into segregation analysis, as combined segregation and linkage analysis has been performed more frequently. In this article, we study the extent of information gains with incorporation of marker data in segregation analysis, a topic that has not been investigated rigorously. Specifically, the current study is to investigate the influence of marker data on genetic model parameter estimation. A variance matrix criterion (as the inverse of the Fisher information matrix) and a relative entropy criterion (a measure of flatness of expected log-likelihood surface) are used to quantify the information gains. Our results indicate that substantial information gain can be achieved with the incorporation of marker data. The amount of variance reduction increases as the heterozygosity of the linked marker increases and as the trait gets closer to the linked marker(s). Incorporation of marker data in larger pedigrees also yields greater information gains based on both criteria. The effect of pedigree structure is also studied.     

Quantum-like model of processing of information in the brain based on classical electromagnetic field

We propose a model of quantum-like (QL) processing of mental information. This model is based on quantum information theory. However, in contrast to models of "quantum physical brain" reducing mental activity (at least at the highest level) to quantum physical phenomena in the brain, our model matches well with the basic neuronal paradigm of the cognitive science. QL information processing is based (surprisingly) on classical electromagnetic signals induced by joint activity of neurons. This novel approach to quantum information is based on representation of quantum mechanics as a version of classical signal theory which was recently elaborated by the author. The brain uses the QL representation (QLR) for working with abstract concepts; concrete images are described by classical information theory. Two processes, classical and QL, are performed parallely. Moreover, information is actively transmitted from one representation to another. A QL concept given in our model by a density operator can generate a variety of concrete images given by temporal realizations of the corresponding (Gaussian) random signal. This signal has the covariance operator coinciding with the density operator encoding the abstract concept under consideration. The presence of various temporal scales in the brain plays the crucial role in creation of QLR in the brain. Moreover, in our model electromagnetic noise produced by neurons is a source of superstrong QL correlations between processes in different spatial domains in the brain; the binding problem is solved on the QL level, but with the aid of the classical background fluctuations.     

An Information-Based Approach to Change-Point Analysis with Applications to Biophysics and Cell Biology

This article describes the application of a change-point algorithm to the analysis of stochastic signals in biological systems whose underlying state dynamics consist of transitions between discrete states. Applications of this analysis include molecular-motor stepping, fluorophore bleaching, electrophysiology, particle and cell tracking, detection of copy number variation by sequencing, tethered-particle motion, etc. We present a unified approach to the analysis of processes whose noise can be modeled by Gaussian, Wiener, or Ornstein-Uhlenbeck processes. To fit the model, we exploit explicit, closed-form algebraic expressions for maximum-likelihood estimators of model parameters and estimated information loss of the generalized noise model, which can be computed extremely efficiently. We implement change-point detection using the frequentist information criterion (which, to our knowledge, is a new information criterion). The frequentist information criterion specifies a single, information-based statistical test that is free from ad hoc parameters and requires no prior probability distribution. We demonstrate this information-based approach in the analysis of simulated and experimental tethered-particle-motion data.     

Regime changes in ecological systems: an information theory approach

We present our efforts at developing an ecological system index using information theory. Specifically, we derive an expression for Fisher Information based on sampling of the system trajectory as it evolves in the space defined by the state variables of the system, i.e. its state space. The Fisher Information index, as we have derived it, is a measure of system order, and captures the characteristic variation in speed and acceleration along the system's periodic steady-state trajectories. When calculated repeatedly over the system period, this index tracks steady states and transient behavior. We believe that such an index could be useful in detecting system 'flips' associated with a regime change, i.e. determining when systems are in a transient between one steady state and another. We illustrate the concepts using model ecosystems.