Regularized estimation in the accelerated failure time model with high-dimensional covariates

We consider two regularization approaches, the LASSO and the threshold-gradient-directed regularization, for estimation and variable selection in the accelerated failure time model with multiple covariates based on Stute's weighted least squares method. The Stute estimator uses Kaplan-Meier weights to account for censoring in the least squares criterion. The weighted least squares objective function makes the adaptation of this approach to multiple covariate settings computationally feasible. We use V-fold cross-validation and a modified Akaike's Information Criterion for tuning parameter selection, and a bootstrap approach for variance estimation. The proposed method is evaluated using simulations and demonstrated on a real data example.     

On parameter estimation for biaxial mechanical behavior of arteries

This article considers the parameter estimation of multi-fiber family models for biaxial mechanical behavior of passive arteries in the presence of the measurement errors. First, the uncertainty propagation due to the errors in variables has been carefully characterized using the constitutive model. Then, the parameter estimation of the artery model has been formulated into nonlinear least squares optimization with an appropriately chosen weight from the uncertainty model. The proposed technique is evaluated using multiple sets of synthesized data with fictitious measurement noises. The results of the estimation are compared with those of the conventional nonlinear least squares optimization without a proper weight factor. The proposed method significantly improves the quality of parameter estimation as the amplitude of the errors in variables becomes larger. We also investigate model selection criteria to decide the optimal number of fiber families in the multi-fiber family model with respect to the experimental data balancing between variance and bias errors.     

Rate estimation for a simple movement model

This paper introduces a simple stochastic model for waterfowl movement. After outlining the properties of the model, we focus on parameter estimation. We compare three standard least squares estimation procedures with maximum likelihood (ML) estimates using Monte Carlo simulations. For our model, little is gained by incorporating information about the covariance structure of the process into least squares estimation. In fact, misspecifying the covariance produces worse estimates than ignoring heteroscedasticity and autocorrelation. We also develop a modified least squares procedure that performs as well as ML. We then apply the five estimators to field data and show that differences in the statistical properties of the estimators can greatly affect our interpretation of the data. We conclude by highlighting the effects of density on per capita movement rates.     

Automatic kernel regression modelling using combined leave-one-out test score and regularised orthogonal least squares

This paper introduces an automatic robust nonlinear identification algorithm using the leave-one-out test score also known as the PRESS (Predicted REsidual Sums of Squares) statistic and regularised orthogonal least squares. The proposed algorithm aims to achieve maximised model robustness via two effective and complementary approaches, parameter regularisation via ridge regression and model optimal generalisation structure selection. The major contributions are to derive the PRESS error in a regularised orthogonal weight model, develop an efficient recursive computation formula for PRESS errors in the regularised orthogonal least squares forward regression framework and hence construct a model with a good generalisation property. Based on the properties of the PRESS statistic the proposed algorithm can achieve a fully automated model construction procedure without resort to any other validation data set for model evaluation.     

Model identification for DNA sequence-structure relationships

We investigate the use of algebraic state-space models for the sequence dependent properties of DNA. By considering the DNA sequence as an input signal, rather than using an all atom physical model, computational efficiency is achieved. A challenge in deriving this type of model is obtaining its structure and estimating its parameters. Here we present two candidate model structures for the sequence dependent structural property Slide and a method of encoding the models so that a recursive least squares algorithm can be applied for parameter estimation. These models are based on the assumption that the value of Slide at a base-step is determined by the surrounding tetranucleotide sequence. The first model takes the four bases individually as inputs and has a median root mean square deviation of 0.90 A. The second model takes the four bases pairwise and has a median root mean square deviation of 0.88 A. These values indicate that the accuracy of these models is within the useful range for structure prediction. Performance is comparable to published predictions of a more physically derived model, at significantly less computational cost.     

Analysis of genetic effects of major genes and polygenes on quantitative traits. II. Genetic models for seed traits of crops

Genetic models for quantitative seed traits with effects of several major genes and polygenes, as well as their GE interaction, were proposed. Mixed linear model approaches were suggested for analyzing the genetic models. Monte Carlo simulations were conducted to evaluate unbiasedness and efficiency for estimating fixed effects and variance components of the embryo and the endosperm models, including effects of a major gene from an unbalanced modified diallel mating design with nine parents, respectively. Simulation results showed that estimates of generalized least squares (GLS) were unbiased and efficient, while those of ordinary least squares (OLS) were almost as good as GLS. Minimum norm quadratic unbiased estimation (MINQUE) could obtain unbiased estimates of the variance components. It was also suggested that precision of MINQUE estimation would be improved with augmentation of experimental size. Data from a modified diallel design in upland cotton ( Gossypium hirsutum L.) were used as a worked example to illustrate the parameter estimation.     

Non-linear heteroscedastic regression model for determination of methotrexate in human plasma by high-performance liquid chromatography

Generalized least squares regression with variance function estimation was used to derive the calibration function for measurement of methotrexate plasma concentration and its results were compared with weighted least squares regression by usual weight factors and also with that of ordinary least squares method. In the calibration curve range of 0.05 to 100 microM, both heteroscedasticity and non-linearity were present therefore ordinary least squares linear regression methods could result in large errors in the calculation of methotrexate concentration. Generalized least squares regression with variance function estimation worked better than both the weighted regression with the usual weight factors and ordinary least squares regression and gave better estimates for methotrexate concentration.     

Bias compensated least squares estimate of the center of rotation

It is shown that a recently published least squares method for the estimation of the average center of rotation is biased. Consequently, a correction term is proposed, and an iterative algorithm is derived for finding a bias compensated solution to the least squares problem.The accuracy of the proposed bias compensated least squares method is compared to the previously proposed least squares method by Monte-Carlo simulations. The tests show that the new method gives a substantial improvement in accuracy.     

Exact Results on the Inadmissibility of the Feasible Generalized Least Squares Estimator in Regression Models with Non‐Spherical Disturbances

In various guises, feasible generalized least squares (FGLS) estimation has occupied an important place in regression analysis for more than 35 years. Past studies on the characteristics of the FGLS estimators are largely based on large sample evaluations, and the important issue of admissibility remains unexplored in the case of the FGLS estimator. In this paper, an exact sufficient condition for the dominance of a Stein‐type shrinkage estimator over the FGLS estimator in finite samples based on squared error loss is given. In deriving the condition, we assume that the model's disturbance covariance matrix is unknown except for a scalar multiple. Further, for models with AR(1) disturbances, it is observed that the dominance condition reduces to one that involves no unknown parameter. In other words, in the case of AR(1) disturbances and where the condition for risk dominance is met, the FGLS estimator is rendered inadmissible under squared error loss.     

Analysis of genetic effects of major genes and polygenes on quantitative traits I. Genetic model for diploid plants and animals

A genetic model was proposed to simultaneously investigate genetic effects of both polygenes and several single genes for quantitative traits of diploid plants and animals. Mixed linear model approaches were employed for statistical analysis. Based on two mating designs, a full diallel cross and a modified diallel cross including F₂, Monte Carlo simulations were conducted to evaluate the unbiasedness and efficiency of the estimation of generalized least squares (GLS) and ordinary least squares (OLS) for fixed effects and of minimum norm quadratic unbiased estimation (MINQUE) and Henderson III for variance components. Estimates of MINQUE (1) were unbiased and efficient in both reduced and full genetic models. Henderson III could have a large bias when used to analyze the full genetic model. Simulation results also showed that GLS and OLS were good methods to estimate fixed effects in the genetic models. Data on Drosophila melanogaster from Gilbert were used as a worked example to demonstrate the parameter estimation. Received: 11 November 2000 / Accepted: 2 May 2001     

Nonparametric estimation of 1-dimensional continuous distribution density functions using the continuous LOLINREG approximation

A nonparametric method for estimation of one-dimensional continuous probability distribution functions is presented. Procedures for calculation of estimation of the unknown distribution function and the distribution density will be discussed in their application. 2 items are what type of weight function may be chosen for the proposed local-linear continuous approximation of the empirical distribution function by the least squares method (LOLINREG), and upon what value of bandwidth- or smoothing parameter one optimally should settle. The latter problem is practically very important with respect to the quality of the estimation results. Examples of simulated measurements which come from a standardized normal distribution as random numbers serve to demonstrate the mode of working, the advantages as well as the limits of the presented continuous LOLINREG-approximation.     

Estimation of Individual Growth Curves by Empirically Weighted Least Squares

The simultaneous estimation of individual growth curves and a mean growth curve is accomplished by weighted least squares. A polynomial curve is fitted for each individual and the polynomial parameters are linear functions of parameters corresponding to covariates. A simple, computationally efficient variance-covariance estimator is derived. The resultant estimate is used in the weighted least squares estimation. The results are compared to empirical Bayes estimation.     

Evaluation of several methods for estimating phylogenetic trees when substitution rates differ over nucleotide sites

Several maximum likelihood and distance matrix methods for estimating phylogenetic trees from homologous DNA sequences were compared when substitution rates at sites were assumed to follow a gamma distribution. Computer simulations were performed to estimate the probabilities that various tree estimation methods recover the true tree topology. The case of four species was considered, and a few combinations of parameters were examined. Attention was applied to discriminating among different sources of error in tree reconstruction, i.e., the inconsistency of the tree estimation method, the sampling error in the estimated tree due to limited sequence length, and the sampling error in the estimated probability due to the number of simulations being limited. Compared to the least squares method based on pairwise distance estimates, the joint likelihood analysis is found to be more robust when rate variation over sites is present but ignored and an assumption is thus violated. With limited data, the likelihood method has a much higher probability of recovering the true tree and is therefore more efficient than the least squares method. The concept of statistical consistency of a tree estimation method and its implications were explored, and it is suggested that, while the efficiency (or sampling error) of a tree estimation method is a very important property, statistical consistency of the method over a wide range of, if not all, parameter values is prerequisite.     

Fitting of one ARMA model to multiple trials increases the time resolution of instantaneous coherence

This study presents a least mean squares (LMS) algorithm for the ensemble modeling of a multivariate ARMA process. Generally, an LMS algorithm makes possible the tracking of parameters for nonstationary time series. Our estimation incorporates multiple process observations that improve the accuracy of the parameter estimation. As a consequence, the estimation sequences come close to the true model parameters with a fast adaptation speed. This advantage also holds true of spectral quantities (e.g., the momentary coherence), which are derived from the model parameters. Thus the extension of the ARMA fitting from one to multiple trajectories allows the investigation of nonstationary biological signals with an increased time resolution. The applicability of the algorithm is demonstrated for event-related EEG coherence analysis of the Sternberg task. The changing interaction between posterior association cortex and anterior brain area was shown for verbal and nonverbal stimuli by means of the time-variant theta coherence.     

A comparison of methods to estimate cross-environment genetic correlations

Advanced techniques for quantitative genetic parameter estimation may not always be necessary to answer broad genetic questions. However, simpler methods are often biased, and the extent of this determines their usefulness. In this study we compare family mean correlations to least squares and restricted error maximum likelihood (REML) variance component approaches to estimating cross-environment genetic correlations. We analysed empirical data from studies where both types of estimates were made, and from studies in our own laboratories. We found that the agreement between estimates was better when full-sib rather than half-sib estimates of cross-environment genetic correlations were used and when mean family size increased. We also note biases in REML estimation that may be especially important when testing to see if correlations differ from 0 or 1. We conclude that correlations calculated from family means can be used to test for the presence of genetic correlations across environments, which is sufficient for some research questions. Variance component approaches should be used when parameter estimation is the objective, or if the goal is anything other than determining broad patterns.     

Effect of interbreath fluctuations on characterizing exercise gas exchange kinetics

Breathing has inherent irregularities that produce breath-to-breath fluctuations ("noise") in pulmonary gas exchange. These impair the precision of characterizing nonsteady-state gas exchange kinetics during exercise. We quantified the effects of this noise on the confidence of estimating kinetic parameters of the underlying physiological responses and hence of model discrimination. Five subjects each performed eight transitions from 0 to 100 W on a cycle ergometer. Ventilation, CO2 output, and O2 uptake were computed breath by breath. The eight responses were interpolated uniformly, time aligned, and averaged for each subject; and the kinetic parameters of a first-order model (i.e., the time constant and time delay) were then estimated using three methods: linear least squares, nonlinear least squares, and maximum likelihood. The breath-by-breath noise approximated an uncorrelated Gaussian stochastic process, with a standard deviation that was largely independent of metabolic rate. An expression has therefore been derived for the number of square-wave repetitions required for a specified parameter confidence using methods b and c; method a being less appropriate for parameter estimation of noisy gas exchange kinetics.     

Modulation of dynamic parameters of muscle reflex by selective activation of its gamma system

The ability of the gamma system to modify the dynamics of the muscle reflex control system is investigated. The dynamics of the triceps surae muscle in decerebrate cat preparations with intact reflex loops were modulated by contralateral tibial and peroneal nerve stimulation. The parameters of the mathematical models representing the muscle reflex system were estimated by the least squares method. The behaviour of the mathematical models of the system, under various degrees of gamma system stimulation, was then studied in response to disturbance inputs. It is shown that the gamma motor system is an efficient agent for carrying out modifications in the dynamics of the muscle reflex system. The possible functional significance of this phenomenon within the framework of operation of the muscular control system is discussed with reference to optimal adaptive system concepts.     

On the application of compartmental models to radioactive tracer kinetic studies of in vivo protein turnover in animals

A mathematical framework is presented for unifying and extending the various compartmental models and formulae used to calculate fractional protein synthesis and degradation rates in animals from data obtained by infusing labelled amino acids. It is shown how the various schemes can be derived as special cases of the product-precursor model or some three-pool variant. Three-compartment representations, which circumvent the need to measure the specific radioactivity of the precursor pool, are proposed. The mathematical solutions are generally presented in a form that is amenable to parameter estimation by non-linear least squares. The problems of measuring the true precursor pool for protein synthesis are addressed, and theoretical consideration is given to assaying aminoacyl-tRNA.     

An application of multivariate ratio methods for the analysis of a longitudinal clinical trial with missing data.

This paper presents an analysis of a longitudinal multi-center clinical trial with missing data. It illustrates the application, the appropriateness, and the limitations of a straightforward ratio estimation procedure for dealing with multivariate situations in which missing data occur at random and with small probability. The parameter estimates are computed via matrix operators such as those used for the generalized least squares analysis of catetorical data. Thus, the estimates may be conveniently analyzed by asymptotic regression methods within the same computer program which computes the estimates, provided that the sample size is sufficiently computer program which computes the estimates, provided that the sample size is sufficiently large.     

Quantifying the relative importance of experimental data points in parameter estimation

Background

Ordinary differential equations (ODEs) are often used to understand biological processes. Since ODE-based models usually contain many unknown parameters, parameter estimation is an important step toward deeper understanding of the process. Parameter estimation is often formulated as a least squares optimization problem, where all experimental data points are considered as equally important. However, this equal-weight formulation ignores the possibility of existence of relative importance among different data points, and may lead to misleading parameter estimation results. Therefore, we propose to introduce weights to account for the relative importance of different data points when formulating the least squares optimization problem. Each weight is defined by the uncertainty of one data point given the other data points. If one data point can be accurately inferred given the other data, the uncertainty of this data point is low and the importance of this data point is low. Whereas, if inferring one data point from the other data is almost impossible, it contains a huge uncertainty and carries more information for estimating parameters.

Results

G1/S transition model with 6 parameters and 12 parameters, and MAPK module with 14 parameters were used to test the weighted formulation. In each case, evenly spaced experimental data points were used. Weights calculated in these models showed similar patterns: high weights for data points in dynamic regions and low weights for data points in flat regions. We developed a sampling algorithm to evaluate the weighted formulation, and demonstrated that the weighted formulation reduced the redundancy in the data. For G1/S transition model with 12 parameters, we examined unevenly spaced experimental data points, strategically sampled to have more measurement points where the weights were relatively high, and fewer measurement points where the weights were relatively low. This analysis showed that the proposed weights can be used for designing measurement time points.

Conclusions

Giving a different weight to each data point according to its relative importance compared to other data points is an effective method for improving robustness of parameter estimation by reducing the redundancy in the experimental data.