Application Fields of the Empirical Regression - A Case Study

Empirical regression is defined as conditional expected value based on an estimation of a twodimensional density. It is a modelfree mathematical means for a first evaluation of measured data of an unknown stochastical relation between two quantities. The numerical procedures may be applied for calculation of the mean course of an unknown relation hidden in the measured data. Disregarding the statistical background an other aspect of application is the analyzing of time series, especially smoothing of time series and modelfree recording of the trend component in non-stationary time series. The calculated regression curve provides an objective basis for comparing of different measured courses as well as for a further evaluation, e. g. in respect of a suitable choice of an analytical expression. The possibility of interpolation and the smoothing properties of empirical regression give essential advantages for internal regression as one step in the process of model construction.     

Model-free model elimination: A new step in the model-free dynamic analysis of NMR relaxation data

Model-free analysis is a technique commonly used within the field of NMR spectroscopy to extract atomic resolution, interpretable dynamic information on multiple timescales from the R ₁, R ₂, and steady state NOE. Model-free approaches employ two disparate areas of data analysis, the discipline of mathematical optimisation, specifically the minimisation of a χ² function, and the statistical field of model selection. By searching through a large number of model-free minimisations, which were setup using synthetic relaxation data whereby the true underlying dynamics is known, certain model-free models have been identified to, at times, fail. This has been characterised as either the internal correlation times, τ _e , τ _f , or τ _s , or the global correlation time parameter, local τ _m , heading towards infinity, the result being that the final parameter values are far from the true values. In a number of cases the minimised χ² value of the failed model is significantly lower than that of all other models and, hence, will be the model which is chosen by model selection techniques. If these models are not removed prior to model selection the final model-free results could be far from the truth. By implementing a series of empirical rules involving inequalities these models can be specifically isolated and removed. Model-free analysis should therefore consist of three distinct steps: model-free minimisation, model-free model elimination, and finally model-free model selection. Failure has also been identified to affect the individual Monte Carlo simulations used within error analysis. Each simulation involves an independent randomised relaxation data set and model-free minimisation, thus simulations suffer from exactly the same types of failure as model-free models. Therefore, to prevent these outliers from causing a significant overestimation of the errors the failed Monte Carlo simulations need to be culled prior to calculating the parameter standard deviations.     

Model-free analysis for large proteins at high magnetic field strengths

Protein backbone dynamics is often characterized using model-free analysis of three sets of ¹⁵N relaxation data: longitudinal relaxation rate (R ₁), transverse relaxation rate (R ₂), and ¹⁵N–{H} NOE values. Since the experimental data is limited, a simplified model-free spectral density function is often used that contains one Lorentzian describing overall rotational correlation but not one describing internal motion. The simplified spectral density function may be also used in estimating the overall rotational correlation time, by making the R ₂/R ₁ largely insensitive to internal motions, as well as used as one of the choices in the model selection protocol. However, such approximation may not be valid for analysis of relaxation data of large proteins recorded at high magnetic field strengths since the contribution to longitudinal relaxation from the Lorentzian describing the overall rotational diffusion of the molecule is comparably small relative to that describing internal motion. Here, we quantitatively estimate the errors introduced by the use of the simplified spectral density in model-free analysis for large proteins at high magnetic field strength.     

Early warning signals for critical transitions: a generalized modeling approach

Critical transitions are sudden, often irreversible, changes that can occur in a large variety of complex systems; signals that warn of critical transitions are therefore highly desirable. We propose a new method for early warning signals that integrates multiple sources of information and data about the system through the framework of a generalized model. We demonstrate our proposed approach through several examples, including a previously published fisheries model. We regard our method as complementary to existing early warning signals, taking an approach of intermediate complexity between model-free approaches and fully parameterized simulations. One potential advantage of our approach is that, under appropriate conditions, it may reduce the amount of time series data required for a robust early warning signal.     

Insulin granule trafficking in beta-cells: mathematical model of glucose-induced insulin secretion

A mathematical model that represents the dynamics of intracellular insulin granules in beta-cells is proposed. Granule translocation and exocytosis are controlled by signals assumed to be essentially related to ATP-to-ADP ratio and cytosolic Ca(2+) concentration. The model provides an interpretation of the roles of the triggering and amplifying pathways of glucose-stimulated insulin secretion. Values of most of the model parameters were inferred from available experimental data. The numerical simulations represent a variety of experimental conditions, such as the stimulation by high K(+) and by different time courses of extracellular glucose, and the predicted responses agree with published experimental data. Model capacity to represent data measured in a hyperglycemic clamp was also tested. Model parameter changes that may reflect alterations of beta-cell function present in type 2 diabetes are investigated, and the action of pharmacological agents that bind to sulfonylurea receptors is simulated.     

The dynamics of velocity adaptation in human vision

Since Barlow and Hill's classic study of the adaptation of the rabbit ganglion cell to movement [1], there have been several reports that motion adaptation is accompanied by an exponential reduction in spike rate, and similar estimates of the time course of velocity adaptation have been found across species [2-4]. Psychophysical studies in humans have shown that perceived velocity may reduce exponentially with adaptation [5,6]. It has been suggested that the reduction in firing of single cells may constitute the neural substrate of the reduction in perceived speed in humans [1,5-7]. Although a model of velocity coding in which the firing rate directly encodes speed may have the advantage of simplicity, it is not supported by psychophysical research. Furthermore, psychophysical estimates of the time course of perceived speed adaptation are not entirely consistent with physiological estimates. This discrepancy between psychophysical and physiological estimates may be due to the unrealistic assumption that speed is coded in the gross spike rate of neurons in the primary visual cortex. The psychophysical data on motion processing are, however, generally consistent with a model in which perceived velocity is derived from the ratio of two temporal channels [8-14]. We have examined the time course of speed adaptation and recovery to determine whether the observed rates can be better related to the established physiology if a ratio model of velocity processing is assumed. Our results indicate that such a model describes the data well and can accommodate the observed difference in the time courses of physiological and psychophysical processes.     

Progressive hypoxia until brain electrical silence: a useful model for studying protective interventions

Anesthetized spontaneously breathing rats, fitted with epicortical electrodes and catheters for sampling arterial, venous, and cerebral venous blood, were exposed to standardized progressive hypoxia. Three minutes of hypoxia sequentially caused hyperpnea, hypopnea, apnea, and cessation of electrocorticogram "spiking," of synchronization, and of background in electroencephalogram (EEG). Blood data and cerebral blood flow and metabolism were measured throughout and at "insults," i.e., at apnea and cessation events, to clarify their interdependence. Arterial and brain venous PO2 fell linearly with inspired oxygen (final value of 2% at 280 s). Hyperpnea induced arterial alkalosis; subsequent hypopnea led to near-normal PCO2 and pH when EEG ceased. Hypercapnia was more pronounced in cerebral than in systemic venous blood; time courses of pH changes were similar. Sagittal sinus blood pressure and outflow were linearly related and resembled the time course of local cerebral blood flow. Blood flow increased by 25% at apnea and only 60% at EEG silence. Cerebral metabolic rate of O2 rose during the hyperpnea phase and fell exponentially thereafter. Cerebral glucose uptake and lactate release increased within the first 3 min but fell abruptly when cortico-electric spiking ceased. Time courses of cerebral O2 consumption and spike rate were linearly related; both showed inverse linear relations to cerebral perfusion. The hypoxic insults were well defined by blood data; critical PO2 values were lower than previously assumed. This model is proving to be a useful, controlled method by which mechanisms of cerebral hypoxia tolerance may be studied in vivo.     

A ratio model of perceived speed in the human visual system

The perceived speed of moving images changes over time. Prolonged viewing of a pattern (adaptation) leads to an exponential decrease in its perceived speed. Similarly, responses of neurones tuned to motion reduce exponentially over time. It is tempting to link these phenomena. However, under certain conditions, perceived speed increases after adaptation and the time course of these perceptual effects varies widely. We propose a model that comprises two temporally tuned mechanisms whose sensitivities reduce exponentially over time. Perceived speed is taken as the ratio of these filters' outputs. The model captures increases and decreases in perceived speed following adaptation and describes our data well with just four free parameters. Whilst the model captures perceptual time courses that vary widely, parameter estimates for the time constants of the underlying filters are in good agreement with estimates of the time course of adaptation of direction selective neurones in the mammalian visual system.     

Slow-wave sleep in daytime and nocturnal sleep: an estimate of the time course of "Process S"

Daan et al. (1984) have proposed that sleep and wakefulness are regulated, in part, by a "Process S" that increases during wakefulness and declines during sleep. Data derived from several studies were taken to determine the time course of Process S during both wakefulness and sleep. As required by the model, slow-wave-sleep (SWS; an index of Process S) was found to increase exponentially as a function of prior wake time (equation 1) and to decline exponentially as a function of time asleep (equation 2). The equations accounted for 91% and 96% of the variance, respectively. In addition, equation 1 accurately predicted the amount the amount of SWS in the first hour of nocturnal sleep.     

Model-free Analysis of Protein Dynamics: Assessment of Accuracy and Model Selection Protocols Based on Molecular Dynamics Simulation

The popular model-free approach to analyze NMR relaxation measurements has been examined using artificial amide (15)N relaxation data sets generated from a 10 nanosecond molecular dynamics trajectory of a dihydrofolate reductase ternary complex in explicit water. With access to a detailed picture of the underlying internal motions, the efficacy of model-free analysis and impact of model selection protocols on the interpretation of NMR data can be studied. In the limit of uncorrelated global tumbling and internal motions, fitting the relaxation data to the model-free models can recover a significant amount of quantitative information on the internal dynamics. Despite a slight overestimation, the generalized order parameter is quite accurately determined. However, the model-free analysis appears to be insensitive to the presence of nanosecond time scale motions with relatively small magnitude. For such cases, the effective correlation time can be significantly underestimated. As a result, proteins appear to be more rigid than they really are. The model selection protocols have a major impact on the information one can reliably obtain. The commonly employed protocol based on step-up hypothesis testing has severe drawbacks of oversimplification and underfitting. The consequences are that the order parameter is more severely overestimated and the correlation time more severely underestimated. Instead, model selection based on Bayesian Information Criteria (BIC), recently introduced to the model-free analysis by d'Auvergne and Gooley (2003), provides a better balance between bias and variance. More appropriate models can be selected, leading to improved estimate of both the order parameter and correlation time. In addition, the computational cost is significantly reduced and subjective parameters such as the significance level are unnecessary.     

Repopulation kinetics in irradiated mouse lip mucosa: the relative importance of treatment protraction and time distribution of irradiations

The repopulation kinetics of the irradiated lip mucosa of mice has been investigated. Split-dose experiments showed that, in this tissue, repopulation starts within 3 days after the first irradiation and increases exponentially within 10 days. To assess the relative importance of protraction and distribution of irradiations as a function of time, 10 fractions were given in (1) 3 days (three irradiations per day with a 4-hr interval), (2) 11 days (daily fractions), or (3) two short courses, each consisting of five fractions given in 1.5 days separated by a rest period of 8 days, with an overall time of 11 days. The results show that by protracting the treatment from 3 to 11 days (with daily irradiations) repopulation accounts for recovery of approximately 13 Gy. Delivering the radiation in two short courses separated by a rest period leads to an additional recovery of approximately 5 Gy. The most plausible explanation for this observation is that repopulation is much more efficient during the rest period between the two courses than during continuous daily irradiation. Although the regimen of two short courses with a rest period spares the acute reaction, it will not enhance the late tolerance. Before thorough knowledge about the repopulation kinetics of the tumors can be gained, caution should be observed for indiscriminate use of split-course multiple-fraction-per-day (MFD) regimens for treating various tumors.     

A differentiable,periodic function for pulsatile cardiac output based on heart rate and stroke volume

Many mathematical models of human hemodynamics, particularly those which describe pressure and flow pulses throughout the circulatory system, require as specified input a modeling function which describes cardiac output in terms of volume per unit time. To be realistic, this cardiac output function should capture, to the greatest extent possible, all relevant features observed in measured physical data. For model analysis purposes, it is also highly desirable to have a model function that is continuous, differentiable, and periodic. This paper addresses both classes of needs by developing such a function. Physically, the present function provides an accurate model for flow into the ascending aorta. It is completely specified by a minimal number of standard input parameters associated with left ventricle dynamics, including heart rate, mean cardiac output, and an estimation of the peak-to-mean flow ratio. Analytically, it can be expressed as a product of two continuous, differentiable and periodic factors. Further, the Fourier expansion of this model function is shown to be a finite Fourier series, and explicit closed-form expressions are given for the non-zero coefficients in this series.     

Stopped-flow kinetic analysis of the bacterial luciferase reaction.

The kinetics of the reaction catalyzed by bacterial luciferase have been measured by stopped-flow spectrophotometry at pH 7 and 25 degrees C. Luciferase catalyzes the formation of visible light, FMN, and a carboxylic acid from FMNH2, O2, and the corresponding aldehyde. The time courses for the formation and decay of the various intermediates have been followed by monitoring the absorbance changes at 380 and 445 nm along with the emission of visible light using n-decanal as the alkyl aldehyde. The synthesis of the 4a-hydroperoxyflavin intermediate (FMNOOH) was monitored at 380 nm after various concentrations of luciferase, O2, and FMNH2 were mixed. The second-order rate constant for the formation of FMNOOH from the luciferase-FMNH2 complex was found to be 2.4 x 10(6) M-1 s-1. In the absence of n-decanal, this complex decays to FMN and H2O2 with a rate constant of 0.10 s-1. The enzyme-FMNH2 complex was found to isomerize prior to reaction with oxygen. The production of visible light reaches a maximum intensity within 1 s and then decays exponentially over the next 10 s. The formation of FMN from the intermediate pseudobase (FMNOH) was monitored at 445 nm. This step of the reaction mechanism was inhibited by high levels of n-decanal which indicated that a dead-end luciferase-FMNOH-decanal could form. The time courses for these optical changes have been incorporated into a comprehensive kinetic model. Estimates for 15 individual rate constants have been obtained for this model by numeric simulations of the various time courses.     

Predicting germination response to temperature. II. Three-dimensional regression, statistical gridding and iterative-probit optimization using measured and interpolated-subpopulation data

BACKGROUND AND AIMS: Most current thermal-germination models are parameterized with subpopulation-specific rate data, interpolated from cumulative-germination-response curves. The purpose of this study was to evaluate the relative accuracy of three-dimensional models for predicting cumulative germination response to temperature. Three-dimensional models are relatively more efficient to implement than two-dimensional models and can be parameterized directly with measured data. METHODS: Seeds of four rangeland grass species were germinated over the constant-temperature range of 3 to 38 degrees C and monitored for subpopulation variability in germination-rate response. Models for estimating subpopulation germination rate were generated as a function of temperature using three-dimensional regression, statistical gridding and iterative-probit optimization using both measured and interpolated-subpopulation data as model inputs. KEY RESULTS: Statistical gridding is more accurate than three-dimensional regression and iterative-probit optimization for modelling germination rate and germination time as a function of temperature and subpopulation. Optimization of the iterative-probit model lowers base-temperature estimates, relative to two-dimensional cardinal-temperature models, and results in an inability to resolve optimal-temperature coefficients as a function of subpopulation. Residual model error for the three-dimensional model was extremely high when parameterized with measured-subpopulation data. Use of measured data for model evaluation provided a more realistic estimate of predictive error than did evaluation of the larger set of interpolated-subpopulation data. CONCLUSIONS: Statistical-gridding techniques may provide a relatively efficient method for estimating germination response in situations where the primary objective is to estimate germination time. This methodology allows for direct use of germination data for model parameterization and automates the significant computational requirements of a two-dimensional piece-wise-linear model, previously shown to produce the most accurate estimates of germination time.     

Optimisation of NMR dynamic models II. A new methodology for the dual optimisation of the model-free parameters and the Brownian rotational diffusion tensor

Finding the dynamics of an entire macromolecule is a complex problem as the model-free parameter values are intricately linked to the Brownian rotational diffusion of the molecule, mathematically through the autocorrelation function of the motion and statistically through model selection. The solution to this problem was formulated using set theory as an element of the universal set —the union of all model-free spaces (d’Auvergne EJ and Gooley PR (2007) Mol BioSyst 3(7), 483–494). The current procedure commonly used to find the universal solution is to initially estimate the diffusion tensor parameters, to optimise the model-free parameters of numerous models, and then to choose the best model via model selection. The global model is then optimised and the procedure repeated until convergence. In this paper a new methodology is presented which takes a different approach to this diffusion seeded model-free paradigm. Rather than starting with the diffusion tensor this iterative protocol begins by optimising the model-free parameters in the absence of any global model parameters, selecting between all the model-free models, and finally optimising the diffusion tensor. The new model-free optimisation protocol will be validated using synthetic data from Schurr JM et al. (1994) J Magn Reson B 105(3), 211–224 and the relaxation data of the bacteriorhodopsin (1–36)BR fragment from Orekhov VY (1999) J Biomol NMR 14(4), 345–356. To demonstrate the importance of this new procedure the NMR relaxation data of the Olfactory Marker Protein (OMP) of Gitti R et al. (2005) Biochem 44(28), 9673–9679 is reanalysed. The result is that the dynamics for certain secondary structural elements is very different from those originally reported. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Analysis of a two-stage, flexible production system with unreliable machines, finite buffers and non-negligible setups

We analyze a Markov model of a two-stage production system capable of producing two part types. Each stage consists of an unreliable machine and the different stages are decoupled by two intermediate buffers of finite capacity, one for each part type. Unlike previous work, we specifically consider non-negligible machine setup times during changeovers and also assume that machine failure probabilities are dependent on the part type being produced. We assume that machine processing times, repair/failure times and setup times are exponentially distributed and may have different mean rates for each machine and for each part-type. We describe a solution method to evaluate the system performance that reduces the total number of equations to be solved from a multiplicative function to an additive function of buffer sizes. This model may then be integrated with a new decomposition method for analyzing longer lines. The results show the relative influence of different factors on system performance and thus provide guidance to the optimal choice of system parameters such as buffer sizes.     

Nonlinear time series analysis of food intake in the dab and the rainbow trout

Evidence suggests that dab and rainbow trout are able to quickly adjust their food intake to an appropriate level when offered novel diets. In addition day-to-day and meal-to-meal food intake varies greatly and meal timing is plastic. Why this is the case is not clear: Food intake in fish is influenced by many factors, however the hierarchy and mechanisms by which these interact is not yet fully understood. A model of food intake may be helpful to understand these phenomena; to determine model type it is necessary to understand the qualitative nature of food intake. Food intake can be regarded as an autoregressive (AR) time series, as the amount of food eaten at time t will be influenced by previous meals, and this allows food intake to be considered using time series analyses. Here, time series data were analysed using nonlinear techniques to obtain qualitative information from which evidence for the hierarchy of mechanisms controlling food intake may be drawn. Time series were obtained for a group of dab and individuals and a group of rainbow trout for analysis. Surrogate data sets were generated to test several null hypotheses describing linear processes and all proved significantly different to the real data, suggesting nonlinear dynamics. Examination of topography and recurrence diagrams suggested that all series were deterministic and non-stationary. The point correlation dimension (PD2i) suggested low-dimensional dynamics. Our findings suggest therefore that any model of appetite should create output that is deterministic, non-stationary, low-dimensional and having nonlinear dynamics.     

Computable visually observed phenotype ontological framework for plants

Background

Gene regulatory networks have an essential role in every process of life. In this regard, the amount of genome-wide time series data is becoming increasingly available, providing the opportunity to discover the time-delayed gene regulatory networks that govern the majority of these molecular processes.

Results

This paper aims at reconstructing gene regulatory networks from multiple genome-wide microarray time series datasets. In this sense, a new model-free algorithm called GRNCOP2 (Gene Regulatory Network inference by Combinatorial OPtimization 2), which is a significant evolution of the GRNCOP algorithm, was developed using combinatorial optimization of gene profile classifiers. The method is capable of inferring potential time-delay relationships with any span of time between genes from various time series datasets given as input. The proposed algorithm was applied to time series data composed of twenty yeast genes that are highly relevant for the cell-cycle study, and the results were compared against several related approaches. The outcomes have shown that GRNCOP2 outperforms the contrasted methods in terms of the proposed metrics, and that the results are consistent with previous biological knowledge. Additionally, a genome-wide study on multiple publicly available time series data was performed. In this case, the experimentation has exhibited the soundness and scalability of the new method which inferred highly-related statistically-significant gene associations.

Conclusions

A novel method for inferring time-delayed gene regulatory networks from genome-wide time series datasets is proposed in this paper. The method was carefully validated with several publicly available data sets. The results have demonstrated that the algorithm constitutes a usable model-free approach capable of predicting meaningful relationships between genes, revealing the time-trends of gene regulation.