Linear and quadratic models of point process systems: Contributions of patterned input to output

In the 1880's Volterra characterised a nonlinear system using a functional series connecting continuous input and continuous output. Norbert Wiener, in the 1940's, circumvented problems associated with the application of Volterra series to physical problems by deriving from it a new series of terms that are mutually uncorrelated with respect to Gaussian processes. Subsequently, Brillinger, in the 1970's, introduced a point-process analogue of Volterra's series connecting point-process inputs to the instantaneous rate of point-process output. We derive here a new series from this analogue in which its terms are mutually uncorrelated with respect to Poisson processes. This new series expresses how patterned input in a spike train, represented by third-order cross-cumulants, is converted into the instantaneous rate of an output point-process. Given experimental records of suitable duration, the contribution of arbitrary patterned input to an output process can, in principle, be determined. Solutions for linear and quadratic point-process models with one and two inputs and a single output are investigated. Our theoretical results are applied to isolated muscle spindle data in which the spike trains from the primary and secondary endings from the same muscle spindle are recorded in response to stimulation of one and then two static fusimotor axons in the absence and presence of a random length change imposed on the parent muscle. For a fixed mean rate of input spikes, the analysis of the experimental data makes explicit which patterns of two input spikes contribute to an output spike.     

General methodology for nonlinear modeling of neural systems with Poisson point-process inputs

This paper presents a general methodological framework for the practical modeling of neural systems with point-process inputs (sequences of action potentials or, more broadly, identical events) based on the Volterra and Wiener theories of functional expansions and system identification. The paper clarifies the distinctions between Volterra and Wiener kernels obtained from Poisson point-process inputs. It shows that only the Wiener kernels can be estimated via cross-correlation, but must be defined as zero along the diagonals. The Volterra kernels can be estimated far more accurately (and from shorter data-records) by use of the Laguerre expansion technique adapted to point-process inputs, and they are independent of the mean rate of stimulation (unlike their P-W counterparts that depend on it). The Volterra kernels can also be estimated for broadband point-process inputs that are not Poisson. Useful applications of this modeling approach include cases where we seek to determine (model) the transfer characteristics between one neuronal axon (a point-process 'input') and another axon (a point-process 'output') or some other measure of neuronal activity (a continuous 'output', such as population activity) with which a causal link exists.     

Minimum-order Wiener modelling of spike-output systems

Systems that generate spike outputs in response to continuous inputs abound in neurophysiology. The study of their dynamics with the use of systems analysis methods has been complicated by the difference in modality of the input and output signals. When the problem is placed in the framework of Wiener's theory in discrete time, an infinite functional series is required for the formal representation of the input-output relation. This has given rise to the belief that a large number of Wiener functionals is needed in practice before a model of reasonable accuracy can be obtained. In this paper, we introduce the concept of minimum-order Wiener models for spike-output systems, and we show that a low-order Wiener model is adequate in many cases for predicting fully the timing of the output spikes.     

The analysis of nonlinear synaptic transmission

In order to characterize synaptic transmission at a unitary facilitating synapse in the lobster cardiac ganglion, a new nonlinear systems analysis technique for discrete-input systems was developed and applied. From the output of the postsynaptic cell in response to randomly occurring presynaptic nerve impulses, a set of kernels, analogous to Wiener kernels, was computed. The kernels up to third order served to characterize, with reasonable accuracy, the input-output properties of the synapse. A mathematical model of the synapse was also tested with a random impulse train and model predictions were compared with experimental synaptic output. Although the model proved to be even more accurate overall than the kernel characterization, there were slight but consistent errors in the model's performance. These were also reflected as differences between model and experimental kernels. It is concluded that a random train analysis provides a comprehensive and objective comparison between model and experiment and automatically provides an arbitrarily accurate characterization of a system's input-output behavior, even in complicated cases where other approaches are impractical.     

Band-limited white noise stimulation and reverse correlation analysis in the prediction of impulse responses of encoder models

A method is described to test the predictability of impulse responses from responses to Gaussiandistributed random stimulation by means of the reverse correlation analysis. In addition, this analysis is tested as to whether it can handle responses of nonlinear systems to random inputs of strongly limited frequency content, which is often the case in data from physiological experiments. The basis for all computation is a simple backward averaging (peri-spike averaging, Istorder PSA) of the noise input triggered from the output pulsatile events, which was extended to two-dimensional peri-spike averaging (2nd-order PSA). These functions were shown to represent the 1st- and 2nd-order Wiener kernel and were taken to calculate the 1st-and 2nd-order response predictions to a given short random test sequence. Different models of impulse-initiating mechanisms were tested for their expression of nonlinearities in these PSAs. Output impulse densities of test sequence (the observed response) could be fairly well approximated by the result of the computations (the predicted response). The difference between observation and prediction was evaluated and expressed as the mean-least squares error. In some of the data the 2nd-order kernel seems sufficient to account for the major nonlinear component, in others, kernels of orders higher than two.     

Poisson process stimulation of an excitable membrane cable model.

The convergence of multiple inputs within a single-neuronal substrate is a common design feature of both peripheral and central nervous systems. Typically, the result of such convergence impinges upon an intracellularly contiguous axon, where it is encoded into a train of action potentials. The simplest representation of the result of convergence of multiple inputs is a Poisson process; a general representation of axonal excitability is the Hodgkin-Huxley/cable theory formalism. The present work addressed multiple input convergence upon an axon by applying Poisson process stimulation to the Hodgkin-Huxley axonal cable. The results showed that both absolute and relative refractory periods yielded in the axonal output a random but non-Poisson process. While smaller amplitude stimuli elicited a type of short-interval conditioning, larger amplitude stimuli elicited impulse trains approaching Poisson criteria except for the effects of refractoriness. These results were obtained for stimulus trains consisting of pulses of constant amplitude and constant or variable durations. By contrast, with or without stimulus pulse shape variability, the post-impulse conditional probability for impulse initiation in the steady-state was a Poisson-like process. For stimulus variability consisting of randomly smaller amplitudes or randomly longer durations, mean impulse frequency was attenuated or potentiated, respectively. Limitations and implications of these computations are discussed.     

A nonlinear cascade model for action potential encoding in an insect sensory neuron.

Action potential encoding in the cockroach tactile spine neuron can be represented as a single-input single-output nonlinear dynamic process. We have used a new functional expansion method to characterize the nonlinear behavior of the neural encoder. This method, which yields similar kernels to the Wiener method, is more accurate than the latter and is efficient enough to obtain reasonable kernels in less than 15 min using a personal computer. The input stimulus was band-limited white Gaussian noise and the output consisted of the resulting train of action potentials, which were unitized to give binary values. The kernels and the system input-output signals were used to identify a model for encoding comprising a cascade of dynamic linear, static nonlinear, and dynamic linear components. The two dynamic linear components had repeatable and distinctive forms with the first being low-pass and the second being high-pass. The static nonlinearity was fitted with a fifth-order polynomial function over several input amplitude ranges and had the form of a half-wave rectifier. The complete model gave a good approximation to the output of the neuron when both were subjected to the same novel white noise input signal.     

Analysis of nonlinear physiological systems with single or multiple spike inputs and analogue or spike outputs

This paper outlines an extension of Krausz' (1975) approach to nonlinear analysis of physiological systems with random impulse train inputs. This extension concerns multiple (here two) inputs and spike train outputs in addition to analogue outputs. The respective theoretical formula are given, and the approach is illustrated by experimental data relating to the nonlinear interaction of cat motor units activated with random pulse trains. As outputs are considered muscle tension (as an example for an analogue signal) and discharge frequency of a Ia fibre (as an example for a spike output).     

Simulation and parameter estimation of dynamics of synaptic depression

Synaptic release was simulated using a Simulink sequential storage model with three vesicular pools. Modeling was modular and easily extendable to the systems with greater number of vesicular pools, parallel input, or time-varying parameters. Given an input (short or long tetanic trains, patterned or random stimulation) and the storage model, the vesicular release, the replenishment of various vesicular pools, and the vesicular content of all pools could be simulated for the time-invariant and time-varying storage systems. From the input stimuli and either a noiseless or a noisy output, the parameters of such storage systems could also be estimated using the optimization technique that minimizes in the least square sense the error between the observed release and the predicted release. All parameters of the storage model could be evaluated with sufficiently long input–output data pairs. Not surprisingly, the parameters characterizing the processes near the release locus, such as the fractional release and the size of the immediately available pool and its coupling to the small store, as well as the state variables associated with the immediately available pool, such as its vesicular content and replenishment, could be determined with fewer stimuli. The possibility of estimating parameters with random inputs extends the applicability of the method to in vivo synapses with the physiological inputs. The parameter estimation was also possible under the time-variant, but slowly changing, conditions as well as for open systems that are part of larger vesicular storage systems but whose parameters can either not be reliably determined or are of no interest. The quality of parameter estimation was monitored continuously by comparing the observed and predicted output and/or estimated parameters with the true values. Finally, the method was tested experimentally using the rat phrenic-diaphragm neuromuscular junction.     

Role of microstructure of nerve impulse train in relation to transmission of neuronal activity and coding mechanism of neural information

Input-output relation were of giant neurons of a marine mollusc, Onchidium verruculatum, and a computer-simulated neuron investigated in terms of microstructure of nerve impulse train. The microstructure of input impulse train, the size of a unitary EPSP, and the extent of spontaneous firing activity of a single neuron had an important influence upon the effective summation of arriving synaptic inputs, the elicitation of output spikes, and intervals between succeeding output spikes. The neuron responded differently to respective input trains with different time structures, i.e. it discriminated input time pattern to various degrees. The manner in discrimination of input time pattern was dependent on the size of the unitary EPSP and the extent of the spontaneous firing activity, if it had. Some discussions were made with regard to possible coding systems of neural signal, assuming a frequency code and/or a pattern code.     

A novel estimator for the rate of information transfer by continuous signals

The information transfer rate provides an objective and rigorous way to quantify how much information is being transmitted through a communications channel whose input and output consist of time-varying signals. However, current estimators of information content in continuous signals are typically based on assumptions about the system's linearity and signal statistics, or they require prohibitive amounts of data. Here we present a novel information rate estimator without these limitations that is also optimized for computational efficiency. We validate the method with a simulated Gaussian information channel and demonstrate its performance with two example applications. Information transfer between the input and output signals of a nonlinear system is analyzed using a sensory receptor neuron as the model system. Then, a climate data set is analyzed to demonstrate that the method can be applied to a system based on two outputs generated by interrelated random processes. These analyses also demonstrate that the new method offers consistent performance in situations where classical methods fail. In addition to these examples, the method is applicable to a wide range of continuous time series commonly observed in the natural sciences, economics and engineering.     

On-line identification of sensory systems using pseudorandom binary noise perturbations.

A technique of on-line identification of linear system characteristics from sensory systems with spike train or analog voltage outputs was developed and applied to the semicircular canal. A pseudorandom binary white noise input was cross-correlated with the system's output to produce estimates of linear system unit impulse responses (UIRs), which were then corrected for response errors of the input transducers. The effects of variability in the system response characteristics and sensitivity were studied by employing the technique with known linear analog circuits. First-order unit afferent responses from the guitarfish horizontal semicircular canal were cross-correlated with white noise rotational acceleration inputs to produce non-parametric UIR models. In addition, the UIRs were fitted by nonlinear regression to truncated exponential series to produce parametric models in the form of low-order linear system equations. The experimental responses to the white noise input were then compared with those predicted from the UIR models linear convolution, and the differences were expressed as a percent mean-square-error (%MSE). The average difference found from a population of 62 semicircular canal afferents was relatively low mean and standard deviation of 10.2 +/- 5.9 SD%MSE, respectively. This suggests that relatively accurate inferences can be made concerning the physiology of the semicircular canal from the linear characteristics of afferent responses.     

A new algorithm for the identification of multiple input Wiener systems

Multiple-input Wiener systems consist of two or more linear dynamic elements, whose outputs are transformed by a multiple-input static non-linearity. Korenberg (1985) demonstrated that the linear elements of these systems can be estimated using either a first order input-ouput cross-covariance or a slice of the second, or higher, order input-output cross-covariance function. Korenberg's work used a multiple input LNL structure, in which the output of the static nonlinearity was then filtered by a linear dynamic system. In this paper we show that by restricting our study to the slightly simpler Wiener structure, it is possible to improve the linear subsystem estimates obtained from the measured cross-covariance functions. Three algorithms, which taken together can identify any multiple-input Wiener system, have been developed. We present the theory underlying these algorithms and detail their implementation. Simulation results are then presented which demonstrate that the algorithms are robust in the presence of output noise, and provide good estimates of the system dynamics under a wide set of conditions.     

Linear models for biological transducers and impulse train spectra: general formulation and review

Starting with the assumption that the output of a biological transducer is the linear sum of discrete waveshapes, then under appropriate conditions, such a system may be modelled by a transfer function whose input is a train of delta functions. The transfer function is obtained by averaging over the population of possible waveshapes. The representation of the input as a train of delta functions facilitates the calculation of its frequency power spectrum. A number of examples of possible physiological interest are given.     

An integral equation approach to measuring turnover in nonsteady compartmental and distributed systems

Physiological systems are often modelled by a set of compartments. Alternatively they can be described by the diffusion-convection-reaction equations governing distributed systems. The problem considered here is that of identifying a continuously changing input of some metabolite )tracee), endogenous to the system and hence inaccessible, when a nonlinear or time-varying component is also introduced into the loss parameter, as for example through feedback mechanisms. A tracer is used to determine the steady-state impulse response under time-invariant, linear conditions. A known input of tracer is also administered when the system is driven out of steady state. The integral equations developed utilize the predetermined impulse response, the measured concentrations of both tracer and tracee (output) in some region of the system to estimate the changing loss parameter and the unknown input in a continuous fashion.     

System analysis of Phycomyces light-growth response: madC, madG, and madH mutants.

The light-growth response of Phycomyces has been studied further with the sum-of-sinusoids method in the framework of the Wiener theory of nonlinear system identification. The response was treated as a black box with the logarithm of light intensity as the input and elongation rate as the output. The nonlinear input-output relation of the light-growth response can be represented mathematically by a set of weighting functions called kernels, which appear in the Wiener intergral series. The linear (first-order) kernels of wild type, and of single and double mutants affected in genes madA to madG were determined previously with Gaussian white noise test stimuli, and were used to investigate the interactions among the products of these genes (R.C. Poe, P. Pratap, and E.D. Lipson. 1986. Biol. Cybern. 55:105.). We have used the more precise sum-of-sinusoids method to extend the interaction studies, including both the first- and second-order kernels. Specifically, we have investigated interactions of the madH ("hypertropic") gene product with the madC ("night blind") and madG ("stiff") gene products. Experiments were performed on the Phycomyces tracking machine. The log-mean intensity of the stimulus was 6 x 10(-2) W m-2 and the wavelength was 477 nm. The first- and second-order kernels were analyzed in terms of nonlinear kinetic models.(ABSTRACT TRUNCATED AT 250 WORDS)     

Structural identifiability of compartmental systems with respect to data obtained from constant-infusion tracer experiments

The problem of structural identifiability of compartmental systems receiving constant input rates of tracer material is studied, and the relationship between this steady-state problem and that of identification using the impulse response is sought. Input connectability of the compartmental system allows exogenous inputs to produce arbitrary steady-state values anywhere in state space, resulting in sufficient conditions for the structural identifiability of the system when direct measurements can be made for every compartment. Because of the steady-state nature of the problem, the systems concept of output connectability is shown to play no role in this identification scheme. The importance of constant-infusion tracer experiments is demonstrated for a compartment model describing volatile fatty acid production and conversion in ruminants.