The identification of nonlinear biological systems: Wiener and Hammerstein cascade models

Systems that can be represented by a cascade of a dynamic linear subsystem preceded (Hammerstein cascade model) or followed (Wiener cascade model) by a static nonlinearity are considered. Various identification schemes that have been proposed for the Hammerstein and Wiener systems are critically reviewed with reference to the special problems that arise in the identification of nonlinear biological systems. Examples of Wiener and Hammerstein systems are identified from limited duration input-output data using an iterative identification scheme.     

From spiking neuron models to linear-nonlinear models

Neurons transform time-varying inputs into action potentials emitted stochastically at a time dependent rate. The mapping from current input to output firing rate is often represented with the help of phenomenological models such as the linear-nonlinear (LN) cascade, in which the output firing rate is estimated by applying to the input successively a linear temporal filter and a static non-linear transformation. These simplified models leave out the biophysical details of action potential generation. It is not a priori clear to which extent the input-output mapping of biophysically more realistic, spiking neuron models can be reduced to a simple linear-nonlinear cascade. Here we investigate this question for the leaky integrate-and-fire (LIF), exponential integrate-and-fire (EIF) and conductance-based Wang-Buzsáki models in presence of background synaptic activity. We exploit available analytic results for these models to determine the corresponding linear filter and static non-linearity in a parameter-free form. We show that the obtained functions are identical to the linear filter and static non-linearity determined using standard reverse correlation analysis. We then quantitatively compare the output of the corresponding linear-nonlinear cascade with numerical simulations of spiking neurons, systematically varying the parameters of input signal and background noise. We find that the LN cascade provides accurate estimates of the firing rates of spiking neurons in most of parameter space. For the EIF and Wang-Buzsáki models, we show that the LN cascade can be reduced to a firing rate model, the timescale of which we determine analytically. Finally we introduce an adaptive timescale rate model in which the timescale of the linear filter depends on the instantaneous firing rate. This model leads to highly accurate estimates of instantaneous firing rates.     

Identification of nonlinear systems using random impulse train inputs

Nonlinear systems that require discrete inputs can be characterized by using random impulse train (Poisson process) inputs. The method is analagous to the Wiener method for continuous input systems, where Gaussian white-noise is the input. In place of the Wiener functional expansion for the output of a continuous input system, a new series for discrete input systems is created by making certain restrictions on the integrals in a Volterra series. The kernels in the new series differ from the Wiener kernels, but also serve to identify a system and are simpler to compute. For systems whose impulse responses vary in amplitude but maintain a similar shape, one argument may be held fixed in each kernel. This simplifies the identification problem. As a test of the theory presented, the output of a hypothetical second order nonlinear system in response to a random impulse train stimulus was computer simulated. Kernels calculated from the simulated data agreed with theoretical predictions. The Poisson impulse train method is applicable to any system whose input can be delivered in discrete pulses. It is particularly suited to neuronal synaptic systems when the pattern of input nerve impulses can be made random.     

Structural classification of multi-input nonlinear systems

We present new structural classification and parameter estimation results that are applicable to multi-input nonlinear systems. The mathematical relationships between the self- and cross-(Volterra and Wiener) kernels are derived for a basic two-input nonlinear structure. These results are then used to develop classification methods for more complicated two-input structures. Algorithms for estimating the parameters (linear and nonlinear subsystems) of these structures are also presented.     

Screening stochastic Life Cycle assessment inventory models

A screening methodology is presented that utilizes the linear structure within the deterministic life cycle inventory (LCI) model. The methodology ranks each input data element based upon the amount it contributes toward the final output. The identified data elements along with their position in the deterministic model are then sorted into descending order based upon their individual contributions. This enables practitioners and model users to identify those input data elements that contribute the most in the inventory stage. Percentages of the top ranked data elements are then selected, and their corresponding data quality index (DQI) value is upgraded in the stochastic LCI model. Monte Carlo computer simulations are obtained and used to compare the output variance of the original stochastic model with modified stochastic model. The methodology is applied to four real-world beverage delivery system LCA inventory models for verification. This research assists LCA practitioners by streamlining the conversion process when converting a deterministic LCI model to a stochastic model form. Model users and decision-makers can benefit from the reduction in output variance and the increase in ability to discriminate between product system alternatives.     

Linear and quadratic models of point process systems: Contributions of patterned input to output

In the 1880's Volterra characterised a nonlinear system using a functional series connecting continuous input and continuous output. Norbert Wiener, in the 1940's, circumvented problems associated with the application of Volterra series to physical problems by deriving from it a new series of terms that are mutually uncorrelated with respect to Gaussian processes. Subsequently, Brillinger, in the 1970's, introduced a point-process analogue of Volterra's series connecting point-process inputs to the instantaneous rate of point-process output. We derive here a new series from this analogue in which its terms are mutually uncorrelated with respect to Poisson processes. This new series expresses how patterned input in a spike train, represented by third-order cross-cumulants, is converted into the instantaneous rate of an output point-process. Given experimental records of suitable duration, the contribution of arbitrary patterned input to an output process can, in principle, be determined. Solutions for linear and quadratic point-process models with one and two inputs and a single output are investigated. Our theoretical results are applied to isolated muscle spindle data in which the spike trains from the primary and secondary endings from the same muscle spindle are recorded in response to stimulation of one and then two static fusimotor axons in the absence and presence of a random length change imposed on the parent muscle. For a fixed mean rate of input spikes, the analysis of the experimental data makes explicit which patterns of two input spikes contribute to an output spike.     

Statistical analysis of multipath neural systems

Peripheral sensory and motor systems may be characterized by models consisting of multiple parallel convergent pathways, each described by the same set of equations, but having different parameter values in each path. Such models, although deterministic, are best analyzed using a statistical approach, which is illustrated here by analysis of several simple multipath models composed of linear dynamic elements and static non-linear elements. Relationships between instantaneous means of signals at different points in such systems are used to show that a multipath system can exhibit behaviour which would not be expected from observations of individual pathways. Mechanisms for linearization of static non-linearities are briefly described. Important implications for neurophysiologists are discussed.     

A nonlinear cascade model for action potential encoding in an insect sensory neuron.

Action potential encoding in the cockroach tactile spine neuron can be represented as a single-input single-output nonlinear dynamic process. We have used a new functional expansion method to characterize the nonlinear behavior of the neural encoder. This method, which yields similar kernels to the Wiener method, is more accurate than the latter and is efficient enough to obtain reasonable kernels in less than 15 min using a personal computer. The input stimulus was band-limited white Gaussian noise and the output consisted of the resulting train of action potentials, which were unitized to give binary values. The kernels and the system input-output signals were used to identify a model for encoding comprising a cascade of dynamic linear, static nonlinear, and dynamic linear components. The two dynamic linear components had repeatable and distinctive forms with the first being low-pass and the second being high-pass. The static nonlinearity was fitted with a fifth-order polynomial function over several input amplitude ranges and had the form of a half-wave rectifier. The complete model gave a good approximation to the output of the neuron when both were subjected to the same novel white noise input signal.     

Adaptive Nachbildung eines linearen Systems bei gestörtem Ausgangssignal

As an example of identification of linear systems an echo canceller was studied. The stochastic properties of two algorithms which are based on the gradient method were tested under highly distorted output signal conditions.The theoretical calculations and computer simulations show that both algorithms are suitable for system identification.     

On-line identification of sensory systems using pseudorandom binary noise perturbations.

A technique of on-line identification of linear system characteristics from sensory systems with spike train or analog voltage outputs was developed and applied to the semicircular canal. A pseudorandom binary white noise input was cross-correlated with the system's output to produce estimates of linear system unit impulse responses (UIRs), which were then corrected for response errors of the input transducers. The effects of variability in the system response characteristics and sensitivity were studied by employing the technique with known linear analog circuits. First-order unit afferent responses from the guitarfish horizontal semicircular canal were cross-correlated with white noise rotational acceleration inputs to produce non-parametric UIR models. In addition, the UIRs were fitted by nonlinear regression to truncated exponential series to produce parametric models in the form of low-order linear system equations. The experimental responses to the white noise input were then compared with those predicted from the UIR models linear convolution, and the differences were expressed as a percent mean-square-error (%MSE). The average difference found from a population of 62 semicircular canal afferents was relatively low mean and standard deviation of 10.2 +/- 5.9 SD%MSE, respectively. This suggests that relatively accurate inferences can be made concerning the physiology of the semicircular canal from the linear characteristics of afferent responses.     

Testing demographic models of effective population size

Due to practical difficulties in obtaining direct genetic estimates of effective sizes, conservation biologists have to rely on so-called 'demographic models' which combine life-history and mating-system parameters with F-statistics in order to produce indirect estimates of effective sizes. However, for the same practical reasons that prevent direct genetic estimates, the accuracy of demographic models is difficult to evaluate. Here we use individual-based, genetically explicit computer simulations in order to investigate the accuracy of two such demographic models aimed at investigating the hierarchical structure of populations. We show that, by and large, these models provide good estimates under a wide range of mating systems and dispersal patterns. However, one of the models should be avoided whenever the focal species' breeding system approaches monogamy with no sex bias in dispersal or when a substructure within social groups is suspected because effective sizes may then be strongly overestimated. The timing during the life cycle at which F-statistics are evaluated is also of crucial importance and attention should be paid to it when designing field sampling since different demographic models assume different timings. Our study shows that individual-based, genetically explicit models provide a promising way of evaluating the accuracy of demographic models of effective size and delineate their field of applicability.     

Predicting the accuracy of multiple sequence alignment algorithms by using computational intelligent techniques

Multiple sequence alignments (MSAs) have become one of the most studied approaches in bioinformatics to perform other outstanding tasks such as structure prediction, biological function analysis or next-generation sequencing. However, current MSA algorithms do not always provide consistent solutions, since alignments become increasingly difficult when dealing with low similarity sequences. As widely known, these algorithms directly depend on specific features of the sequences, causing relevant influence on the alignment accuracy. Many MSA tools have been recently designed but it is not possible to know in advance which one is the most suitable for a particular set of sequences. In this work, we analyze some of the most used algorithms presented in the bibliography and their dependences on several features. A novel intelligent algorithm based on least square support vector machine is then developed to predict how accurate each alignment could be, depending on its analyzed features. This algorithm is performed with a dataset of 2180 MSAs. The proposed system first estimates the accuracy of possible alignments. The most promising methodologies are then selected in order to align each set of sequences. Since only one selected algorithm is run, the computational time is not excessively increased.     

Successful Reconstruction of a Physiological Circuit with Known Connectivity from Spiking Activity Alone

Identifying the structure and dynamics of synaptic interactions between neurons is the first step to understanding neural network dynamics. The presence of synaptic connections is traditionally inferred through the use of targeted stimulation and paired recordings or by post-hoc histology. More recently, causal network inference algorithms have been proposed to deduce connectivity directly from electrophysiological signals, such as extracellularly recorded spiking activity. Usually, these algorithms have not been validated on a neurophysiological data set for which the actual circuitry is known. Recent work has shown that traditional network inference algorithms based on linear models typically fail to identify the correct coupling of a small central pattern generating circuit in the stomatogastric ganglion of the crab Cancer borealis. In this work, we show that point process models of observed spike trains can guide inference of relative connectivity estimates that match the known physiological connectivity of the central pattern generator up to a choice of threshold. We elucidate the necessary steps to derive faithful connectivity estimates from a model that incorporates the spike train nature of the data. We then apply the model to measure changes in the effective connectivity pattern in response to two pharmacological interventions, which affect both intrinsic neural dynamics and synaptic transmission. Our results provide the first successful application of a network inference algorithm to a circuit for which the actual physiological synapses between neurons are known. The point process methodology presented here generalizes well to larger networks and can describe the statistics of neural populations. In general we show that advanced statistical models allow for the characterization of effective network structure, deciphering underlying network dynamics and estimating information-processing capabilities.     

Membrane potential changes of skeletomotor neurons in response to random stretches of the triceps surae muscles in decerebrate cats

 The properties of membrane potential changes of skeletomotor neurons (S, FR, and FF) innervating triceps surae muscles during pseudorandom stretching of these muscles were studied in decerebrate cats. Peak amplitudes of pseudorandom muscle stretches ranged from 119 μm to 4.15 mm peak-to-peak. Sequences of ten identical stretching periods were applied for averaging. Shapes of membrane potential changes and probability density distribution of amplitudes of the input and output signals and power spectra suggest that the skeletomotor neuron membrane has nonlinear properties. First- and second-order Wiener kernels were determined by applying the cross-correlation (Lee-Schetzen) method. The results suggest that the transfer function between muscle stretches and subthreshold membrane potentials is a Wiener-type cascade. This cascade is consistent with a linear, second-order, underdamped transfer function followed by a simple quadratic nonlinearity [linear (L) system followed by nonlinear (N) system, or LN cascade]. Including the nonlinear component calculated from the second-order Wiener kernel improved the model significantly over its linear counterpart, especially in S-type motoneurons. Qualitatively similar results were obtained with all types of motoneurons studied. Received: 1 April 1993/Accepted in revised form: 24 March 1994     

Evolutionary optimization with data collocation for reverse engineering of biological networks

MOTIVATION: Modern experimental biology is moving away from analyses of single elements to whole-organism measurements. Such measured time-course data contain a wealth of information about the structure and dynamic of the pathway or network. The dynamic modeling of the whole systems is formulated as a reverse problem that requires a well-suited mathematical model and a very efficient computational method to identify the model structure and parameters. Numerical integration for differential equations and finding global parameter values are still two major challenges in this field of the parameter estimation of nonlinear dynamic biological systems. RESULTS: We compare three techniques of parameter estimation for nonlinear dynamic biological systems. In the proposed scheme, the modified collocation method is applied to convert the differential equations to the system of algebraic equations. The observed time-course data are then substituted into the algebraic system equations to decouple system interactions in order to obtain the approximate model profiles. Hybrid differential evolution (HDE) with population size of five is able to find a global solution. The method is not only suited for parameter estimation but also can be applied for structure identification. The solution obtained by HDE is then used as the starting point for a local search method to yield the refined estimates.     

Identifiability and interval identifiability of mammillary and catenary compartmental models with some known rate constants

The identifiability problem is addressed for n-compartment linear mammillary and catenary models, for the common case of input and output in the first compartment and prior information about one or more model rate constants. We first define the concept of independent constraints and show that n-compartment linear mammillary or catenary models are uniquely identifiable under n-1 independent constraints. Closed-form algorithms for bounding the constrained parameter space are then developed algebraically, and their validity is confirmed using an independent approach, namely joint estimation of the parameters of all uniquely identifiable submodels of the original multicompartmental model. For the noise-free (deterministic) case, the major effects of additional parameter knowledge are to narrow the bounds of rate constants that remain unidentifiable, as well as to possibly render others identifiable. When noisy data are considered, the means of the bounds of rate constants that remain unidentifiable are also narrowed, but the variances of some of these bound estimates increase. This unexpected result was verified by Monte Carlo simulation of several different models, using both normally and lognormally distributed data assumptions. Extensions and some consequences of this analysis useful for model discrimination and experiment design applications are also noted.     

The Spectro-Temporal Receptive Field

The Spectro-Temporal Receptive Field (STRF) of an auditory neuron has been introduced experimentally on the base of the average spectrotemporal structure of the acoustic stimuli which precede the occurrence of action potentials (Aertsen et al., 1980, 1981). In the present paper the STRF is considered in the general framework of nonlinear system theory, especially in the form of the Volterra integral representation. The STRF is proposed to be formally identified with a linear functional of the second order Volterra kernel. The experimental determination of the STRF leads to a formulation in terms of the Wiener expansion where the kernels can be identified by evaluation of the system's input-output correlations. For a Gaussian stimulus ensemble and a nonlinear system with no even order contributions of order higher than two, it is shown that the second order cross correlation of stimulus and response, normalized with respect to the spectral contents of the stimulus ensemble, leads to the stimulus-invariant spectrotemporal receptive field. The investigation of stimulus-invariance of the STRF for more general nonlinear systems and for stimulus ensembles which can be generated by nonlinear transformations of Gaussian noise involve the evaluation of higher order stimulus-response correlation functions.