Identification of nonlinear systems using random impulse train inputs

Nonlinear systems that require discrete inputs can be characterized by using random impulse train (Poisson process) inputs. The method is analagous to the Wiener method for continuous input systems, where Gaussian white-noise is the input. In place of the Wiener functional expansion for the output of a continuous input system, a new series for discrete input systems is created by making certain restrictions on the integrals in a Volterra series. The kernels in the new series differ from the Wiener kernels, but also serve to identify a system and are simpler to compute. For systems whose impulse responses vary in amplitude but maintain a similar shape, one argument may be held fixed in each kernel. This simplifies the identification problem. As a test of the theory presented, the output of a hypothetical second order nonlinear system in response to a random impulse train stimulus was computer simulated. Kernels calculated from the simulated data agreed with theoretical predictions. The Poisson impulse train method is applicable to any system whose input can be delivered in discrete pulses. It is particularly suited to neuronal synaptic systems when the pattern of input nerve impulses can be made random.     

General methodology for nonlinear modeling of neural systems with Poisson point-process inputs

This paper presents a general methodological framework for the practical modeling of neural systems with point-process inputs (sequences of action potentials or, more broadly, identical events) based on the Volterra and Wiener theories of functional expansions and system identification. The paper clarifies the distinctions between Volterra and Wiener kernels obtained from Poisson point-process inputs. It shows that only the Wiener kernels can be estimated via cross-correlation, but must be defined as zero along the diagonals. The Volterra kernels can be estimated far more accurately (and from shorter data-records) by use of the Laguerre expansion technique adapted to point-process inputs, and they are independent of the mean rate of stimulation (unlike their P-W counterparts that depend on it). The Volterra kernels can also be estimated for broadband point-process inputs that are not Poisson. Useful applications of this modeling approach include cases where we seek to determine (model) the transfer characteristics between one neuronal axon (a point-process 'input') and another axon (a point-process 'output') or some other measure of neuronal activity (a continuous 'output', such as population activity) with which a causal link exists.     

The dynamic nonlinear behavior of fly photoreceptors evoked by a wide range of light intensities.

Fly photoreceptor cells were stimulated with steps of light over a wide intensity range. First- and second-order Volterra kernels were then computed from sequences of combined step responses. Diagonal values of the second-order Volterra kernels were much greater than the off-diagonal values, and the diagonal values were roughly proportional to the corresponding first-order kernels, suggesting that the response could be approximated by a static nonlinearity followed by a dynamic linear component (Hammerstein model). The amplitudes of the second-order kernels were much smaller in light-adapted than in dark-adapted photoreceptors. Hammerstein models constructed from the step input/output measurements gave reasonable approximations to the actual photoreceptor responses, with light-adapted responses being relatively better fitted. However, Hammerstein models could not account for several features of the photoreceptor behavior, including the dependence of the step response shape on step amplitude. A model containing an additional static nonlinearity after the dynamic linear component gave significantly better fits to the data. These results indicate that blowfly photoreceptors have a strong early gain control nonlinearity acting before the processes that create the characteristic time course of the response, in addition to the nonlinearities caused by membrane conductances.     

System analysis of Phycomyces light-growth response with gaussian white-noise test stimuli

The light-growth response of the Phycomyces sporangiophore is a transient change of elongation rate in response to changes in ambient blue-light intensity. The white-noise method of nonlinear system identification (Wiener-Lee-Schetzen theory) has been applied to this response, and the results have been interpreted by system analysis methods in the frequency domain. Experiments were performed on the Phycomyces tracking machine. Gaussian white-noise stimulus patterns were applied to the logarithm of the light intensity. The log-mean intensity of the broadband blue illumination was 0.1 W m^-2 and the standard deviation of the Gaussian white-noise was 0.58 decades. The results, in the form of temporal functions called Wiener kernels, represent the input-output relation of the light-growth response system. The transfer function, which was obtained as the Fourier transform of the first-order kernel, was analyzed in the frequency domain in terms of a dynamic model that consisted of a first-order high-pass filter, two secondorder low-pass filters, a delay element, and a gain factor. Parameters in the model (cutoff frequencies, damping coefficients, latency, and gain constant) were evaluated by nonlinear least-squares methods applied to the complex-valued transfer function. Analysis of the second-order kernel in the frequency domain suggests that the residual nonlinearity of the system lies close to the input.     

Membrane potential changes of skeletomotor neurons in response to random stretches of the triceps surae muscles in decerebrate cats

 The properties of membrane potential changes of skeletomotor neurons (S, FR, and FF) innervating triceps surae muscles during pseudorandom stretching of these muscles were studied in decerebrate cats. Peak amplitudes of pseudorandom muscle stretches ranged from 119 μm to 4.15 mm peak-to-peak. Sequences of ten identical stretching periods were applied for averaging. Shapes of membrane potential changes and probability density distribution of amplitudes of the input and output signals and power spectra suggest that the skeletomotor neuron membrane has nonlinear properties. First- and second-order Wiener kernels were determined by applying the cross-correlation (Lee-Schetzen) method. The results suggest that the transfer function between muscle stretches and subthreshold membrane potentials is a Wiener-type cascade. This cascade is consistent with a linear, second-order, underdamped transfer function followed by a simple quadratic nonlinearity [linear (L) system followed by nonlinear (N) system, or LN cascade]. Including the nonlinear component calculated from the second-order Wiener kernel improved the model significantly over its linear counterpart, especially in S-type motoneurons. Qualitatively similar results were obtained with all types of motoneurons studied. Received: 1 April 1993/Accepted in revised form: 24 March 1994     

Spike discharges of skeletomotor neurons during random noise modulated transmembrane current stimulation and muscle stretch

 Spike discharges of skeletomotor neurons innervating triceps surae muscles elicited by white noise modulated transmembrane current stimulation and muscle stretch were studied in decerebrated cats. The white noise modulated current intensity ranged from 4.3 to 63.2 nA peak-to-peak, while muscle stretches ranged from 100 μm to 4.26 mm peak-to-peak. The neuronal responses were studied by averaging the muscle length records centered at the skeletomotor action potentials (peri-spike average, PSA) and by Wiener analysis. Skeletomotor spikes appeared after a sharp peak in PSA of the injected current, preceded by a longer-lasting smaller wavelet of either depolarizing or hyperpolarizing direction. The PSA amplitude was not related to the injected current amplitude nor showed any differences related to the motor unit type. The PSA amplitudes were virtually independent of the stretching amplitude σ, after an initial increase with stretching amplitudes in the range of 15–40 μm (S.D.), or 100–270 μm peak-to-peak.Analyses of cross-spectra indicated a small or absent increase in gain with frequency in response to injected current, but about 20 dB/decade in the range 10–100 Hz in response to muscle stretch. The peaks of both Wiener kernels in response to current injection appear to decrease with the amplitude of injected current, but this decrease was not statistically significant. The narrow first-order kernels suggest that the transfer function between the current input and spike discharge is lowpass with a wide passband, i.e. there is very little change in dynamics. The values of the second-order kernels appear to be nonzero only along the main diagonal. This is characteristic of a simple Hammerstein type cascade, i.e. a zero memory nonlinearity followed by a linear system. Small values of second-order kernels away from the origin and narrow first-order kernels suggest that the linear cascade contributes very little to the overall dynamic response.In contrast to Wiener kernels found in response to current injection, the Wiener kernels in response to stretch showed a decreasing trend with stretch amplitude. The size of the second-order kernels decreased to a somewhat larger extent with input amplitude than that of the first-order kernels, indicating an amplitude-dependent nonlinearity. Overall, the transformation between length and spike output was described as an LNNL cascade with second-order nonlinearities. Received: 1 April 1993/Accepted in revised form: 24 March 1994     

Retinal responses to white-noise modulated current stimuli

Electric current was injected into a rabbit's eye with white-noise modulations of the current amplitude. A variable D.C. bias was added to the whitenoise stimulus to study the effects of stimulus bias. For each bias level, the ERG response to the electrical stimulus was cross-correlated with the random stimulus to estimate first-and second-order Wiener kernels. The kernels indicated both linear and nonlinear characteristics of the Electrical ERG. The results for the zero biased stimulus are particularly relevant for clinical testing because the root mean square (RMS) level of the stimulus was less than 0.2 mA.This work was supported by NIH Grant Number EY03022     

System analysis of Phycomyces light-growth response: Single mutants

The white-noise method of system identification has been applied to the transient light-growth response of a set of seven mutants of Phycomyces with abnormal phototropism, affected in genes madA to madG. The Wiener kernels, which represent the input-output relation of the light-growth response, have been evaluated for each of these mutants and the wild-type strain at a log-mean blue-light intensity of 0.1 W m^-2. Additional experiments were done at 3x10^-4 and 10 W m^-2 on the madA strain C21 and wild-type. In the normal intensity range (0.1 W m^-2) the madA mutant behaves similarly to wild-type, but, at high intensity, the madA response is about twice as strong as that of wild-type. Except for C21 (madA), the first-order kernels of all mutants were smaller than the wild-type kernel. The first-order kernels for C111 (madB) and L15 (madC) show a prolonged time course, and C111 has a longer latency. The kernels for C110 (madE), C316 (madF), and C307 (madG) have a shallow and extended negative phase. For C68 (madD), the latency and time course are shorter than in the wild-type. These features are also reflected in the parameters estimated from fits of the anlytical model introduced in the previous paper to the experimental transfer functions (Fourier transforms of the kernels). The kernel for L15 (madC) is described better by a model that lacks one of the two second-order low-pass filters, because its response kinetics are dynamically of lower order.     

A retinal circuit model accounting for wide-field amacrine cells

In previous experimental studies on the visual processing in vertebrates, higher-order visual functions such as the object segregation from background were found even in the retinal stage. Previously, the “linear–nonlinear” (LN) cascade models have been applied to the retinal circuit, and succeeded to describe the input-output dynamics for certain parts of the circuit, e.g., the receptive field of the outer retinal neurons. And recently, some abstract models composed of LN cascades as the circuit elements could explain the higher-order retinal functions. However, in such a model, each class of retinal neurons is mostly omitted and thus, how those neurons play roles in the visual computations cannot be explored. Here, we present a spatio-temporal computational model of the vertebrate retina, based on the response function for each class of retinal neurons and on the anatomical inter-cellular connections. This model was capable of not only reproducing the spatio-temporal filtering properties of the outer retinal neurons, but also realizing the object segregation mechanism in the inner retinal circuit involving the “wide-field” amacrine cells. Moreover, the first-order Wiener kernels calculated for the neurons in our model showed a reasonable fit to the kernels previously measured in the real retinal neuron in situ.     

Structural classification of multi-input nonlinear systems

We present new structural classification and parameter estimation results that are applicable to multi-input nonlinear systems. The mathematical relationships between the self- and cross-(Volterra and Wiener) kernels are derived for a basic two-input nonlinear structure. These results are then used to develop classification methods for more complicated two-input structures. Algorithms for estimating the parameters (linear and nonlinear subsystems) of these structures are also presented.     

Dynamics of turtle horizontal cell response

The small- and large-field (cone) horizontal cells produce similar dynamic responses to a stimulus whose mean luminance is modulated by a white-noise signal. Nonlinear components increase with an increase in the mean luminance and may produce a mean square error (MSE) of up to 15%. Increases in the mean luminance of the field stimulus bring about three major changes: the incremental sensitivity defined by the amplitude of the kernels decreases in a Weber-Fechner fashion; the waveforms of the kernels are transformed from monophasic (integrating) to biphasic (differentiating); the peak response time of the kernels becomes shorter and the cells respond to much higher-frequency inputs. The dynamics of the horizontal cell response also depend on the area of the retina stimulated. Smaller spots of light produce monophasic kernels of a longer peak response time. The presence of a steady background produces three major changes in the spot kernels: the kernel's amplitude becomes larger (incremental sensitivity increases); the peak response times become shorter; the waveform of the kernels changes in a fashion similar to that observed with an increase in the mean luminance of the field stimulus. A similar enhancement in the incremental sensitivity by a steady background has also been observed in catfish, which shows that this phenomenon is a common feature of the horizontal cells in the lower vertebrate retina.     

Calculation of the Volterra kernels of non-linear dynamic systems using an artificial neural network

The Volterra series is a well-known method of describing non-linear dynamic systems. A major limitation of this technique is the difficulty involved in the calculation of the kernels. More recently, artificial neural networks have been used to produce black box models of non-linear dynamic systems. In this paper we show how a certain class of artificial neural networks are equivalent to Volterra series and give the equation for the nth order Volterra kernel in terms of the internal parameters of the network. The technique is then illustrated using a specific non-linear system. The kernels obtained by the method described in the paper are compared with those obtained by a Toeplitz matrix inversion technique. Received: 4 June 1993/Accepted in revised form: 2 March 1994     

Dynamics of the ganglion cell response in the catfish and frog retinas

Responses were evoked from ganglion cells in catfish and frog retinas by a Gaussian modulation of the mean luminance. An algorithm was devised to decompose intracellularly recorded responses into the slow and spike components and to extract the time of occurrence of a spike discharge. The dynamics of both signals were analyzed in terms of a series of first-through third-order kernels obtained by cross-correlating the slow (analog) or spike (discrete or point process) signals against the white-noise input. We found that, in the catfish, (a) the slow signals were composed mostly of postsynaptic potentials, (b) their linear components reflected the dynamics found in bipolar cells or in the linear response component of type-N (sustained) amacrine cells, and (c) their nonlinear components were similar to those found in either type-N or type-C (transient) amacrine cells. A comparison of the dynamics of slow and spike signals showed that the characteristic linear and nonlinear dynamics of slow signals were encoded into a spike train, which could be recovered through the cross-correlation between the white-noise input and the spike (point process signals. In addition, well-defined spike correlates could predict the observed slow potentials. In the spike discharges from frog ganglion cells, the linear (or first-order) kernels were all inhibitory, whereas the second-order kernels had characteristics of on-off transient excitation. The transient and sustained amacrine cells similar to those found in catfish retina were the sources of the nonlinear excitation. We conclude that bipolar cells and possibly the linear part of the type-N cell response are the source of linear, either excitatory or inhibitory, components of the ganglion cell responses, whereas amacrine cells are the source of the cells' static nonlinearity.     

A spatiotemporal white noise analysis of photoreceptor responses to UV and green light in the dragonfly median ocellus

Adult dragonflies augment their compound eyes with three simple eyes known as the dorsal ocelli. While the ocellar system is known to mediate stabilizing head reflexes during flight, the ability of the ocellar retina to dynamically resolve the environment is unknown. For the first time, we directly measured the angular sensitivities of the photoreceptors of the dragonfly median (middle) ocellus. We performed a second-order Wiener Kernel analysis of intracellular recordings of light-adapted photoreceptors. These were stimulated with one-dimensional horizontal or vertical patterns of concurrent UV and green light with different contrast levels and at different ambient temperatures. The photoreceptors were found to have anisotropic receptive fields with vertical and horizontal acceptance angles of 15 degrees and 28 degrees, respectively. The first-order (linear) temporal kernels contained significant undershoots whose amplitudes are invariant under changes in the contrast of the stimulus but significantly reduced at higher temperatures. The second-order kernels showed evidence of two distinct nonlinear components: a fast acting self-facilitation, which is dominant in the UV, followed by delayed self- and cross-inhibition of UV and green light responses. No facilitatory interactions between the UV and green light were found, indicating that facilitation of the green and UV responses occurs in isolated compartments. Inhibition between UV and green stimuli was present, indicating that inhibition occurs at a common point in the UV and green response pathways. We present a nonlinear cascade model (NLN) with initial stages consisting of separate UV and green pathways. Each pathway contains a fast facilitating nonlinearity coupled to a linear response. The linear response is described by an extended log-normal model, accounting for the phasic component. The final nonlinearity is composed of self-inhibition in the UV and green pathways and inhibition between these pathways. The model can largely predict the response of the photoreceptors to UV and green light.     

The transducer and encoder of frog muscle spindles are essentially nonlinear. Physiological conclusions from a white-noise analysis

1. Nonlinear second order white-noise analysis has been applied to the isolated frog muscle spindle. Power (²) of the Gaussian white noise (GWN) and the average prestretch level L were varied and the response of both the isolated receptor potential (transducer) and the action potential (encoder) level were analysed. 2. The standard white-noise method is briefly presented. Particular emphasis, however, is put on the limitations in the range of validity of the method and, consequently, on the use and interpretation of the kernels as a Wiener model. Conclusions in the present paper are within this frame and are mainly of qualitative nature. 3. The analysis reveals that the nonlinear contributions of the model are essential for approximating physiological results, thus ruling out purely linear modelling for this receptor organ. 4. The dependence of the transducer kernels on are compatible with the behaviour of a rectifier. Rectification is represented by the lack of hyperpolarization within the isolated receptor potential and is enhanced by the substantial memory in the linear and nonlinear kernels as demonstrated by their extent in time. This is equivalent to low power in high frequencies of the response. Obviously, the hyperpolarizing potentials following each spike counteract the long transducer memory. 5. At the encoder level the memory of the system is strongly reduced. This is achieved by using predominantly high frequency components of the receptor potential for triggering the process of impulse generation, and by the precise coupling and high frequency content of the impulses. This coupling precision is possible because of the sensitivity of the spike-generating mechanism to steep rising transients of the receptor potential and also owing to the reduction in transducer memory by the hyperpolarizing afferpotentials. 6. The preference given to the high frequency components is also read from the structure of the second order transducer kernel and from both the linear and the second order encoder kernels, which allows the most effective input waveform for triggering action potentials to be determined. 6. When the operating point is changed to higher prestretch values, kernel heights increase strongly implying higher response strength of the muscle spindle. The kernel structure is changed as well in the direction of reducing the effective memory already at the level of the receptor potentials, probably a means to prevent too high depolarization values.     

White noise analysis of Phycomyces light growth response system. II. Extended intensity ranges.

By means of white gaussian noise stimulation, the Wiener kernels are derived for the Phycomyces light growth response for a variety of intensity conditions. In one experiment the intensity I, rather than log I, is used as the input variable. Under the very limited dynamic range of that experiment, the response is fairly linear. To examine the dependence of the kernels on dynamic range, a series of experiments were performed in which the range of log I was halved and doubled relative to normal. The amplitude of the kernels, but not the time course, is affected strongly by the choice of dynamic range, and the dependence reveals large-scale nonlinearities not evident in the kernels themselves. In addition kernels are evaluated for experiments at a number of absolute intensity levels ranging from 10(-12) to 10(-3) W/cm2. The kernel amplitudes are maximal at about 10(-6) W/cm2. At 10(-12) W/cm2, just above the absolute threshold, the respond is very small. The falloff at high intensity, attributable to inactivation of the photoreceptor, is analyzed in the framework of a first-order pigment kinetics model, yielding estimates for the partial extinction coefficient for inactivation epsilonI455 = (1.5 +/- 0.2) X 10(4) liter/mol-cm and a regeneration time constant of tau = (2.7 +/- 0.6) min. A model is introduced which associates the processes of adaptation and photoreceptor inactivation. The model predicts that the time constants for adaptation and pigment should be identical. This prediction is consistent with values in this and the preceding paper. The effects of pigment inactivation are simulated by a linear electronic analog circuit element, which may be cascaded with the linear simulator circuit in the preceding paper.     

A nonlinear cascade model for action potential encoding in an insect sensory neuron.

Action potential encoding in the cockroach tactile spine neuron can be represented as a single-input single-output nonlinear dynamic process. We have used a new functional expansion method to characterize the nonlinear behavior of the neural encoder. This method, which yields similar kernels to the Wiener method, is more accurate than the latter and is efficient enough to obtain reasonable kernels in less than 15 min using a personal computer. The input stimulus was band-limited white Gaussian noise and the output consisted of the resulting train of action potentials, which were unitized to give binary values. The kernels and the system input-output signals were used to identify a model for encoding comprising a cascade of dynamic linear, static nonlinear, and dynamic linear components. The two dynamic linear components had repeatable and distinctive forms with the first being low-pass and the second being high-pass. The static nonlinearity was fitted with a fifth-order polynomial function over several input amplitude ranges and had the form of a half-wave rectifier. The complete model gave a good approximation to the output of the neuron when both were subjected to the same novel white noise input signal.