Individual and population penalized regression splines for accelerated longitudinal designs

In an accelerated longitudinal design (ALD), individuals enter the study at different points of their growth trajectory and are observed over a short time span relative to the entire time span of interest. ALD data are combined across independent units to provide an estimate of an overall population curve and predictions of individual patterns of change. As a modest extension of the work of Ruppert et al. (2003, Semiparametric Regression, Cambridge University Press), we develop a computationally efficient procedure for the application of longitudinal semiparametric methods under ALD sampling schemes. We compare balanced and complete longitudinal designs to ALDs using the Berkeley Growth Study data and apply our method to longitudinal magnetic resonance imaging (MRI) brain structure size (volume) measurements from an ongoing developmental study. Potential applications extend beyond growth studies to many other fields in which cost and feasibility constraints impose restrictions on sample size and on the numbers and timings of repeated measurements across subjects.     

On the asymptotics of penalized splines

We study the asymptotic behaviour of penalized spline estimatorsin the univariate case. We use B-splines and a penalty is placedon mth-order differences of the coefficients. The number ofknots is assumed to converge to infinity as the sample sizeincreases. We show that penalized splines behave similarly toNadaraya--Watson kernel estimators with ‘equivalent’kernels depending upon m. The equivalent kernels we obtain forpenalized splines are the same as those found by Silverman forsmoothing splines. The asymptotic distribution of the penalizedspline estimator is Gaussian and we give simple expressionsfor the asymptotic mean and variance. Provided that it is fastenough, the rate at which the number of knots converges to infinitydoes not affect the asymptotic distribution. The optimal rateof convergence of the penalty parameter is given. Penalizedsplines are not design-adaptive.     

Overview of LASSO-related penalized regression methods for quantitative trait mapping and genomic selection

Quantitative trait loci (QTL)/association mapping aims at finding genomic loci associated with the phenotypes, whereas genomic selection focuses on breeding value prediction based on genomic data. Variable selection is a key to both of these tasks as it allows to (1) detect clear mapping signals of QTL activity, and (2) predict the genome-enhanced breeding values accurately. In this paper, we provide an overview of a statistical method called least absolute shrinkage and selection operator (LASSO) and two of its generalizations named elastic net and adaptive LASSO in the contexts of QTL mapping and genomic breeding value prediction in plants (or animals). We also briefly summarize the Bayesian interpretation of LASSO, and the inspired hierarchical Bayesian models. We illustrate the implementation and examine the performance of methods using three public data sets: (1) North American barley data with 127 individuals and 145 markers, (2) a simulated QTLMAS XII data with 5,865 individuals and 6,000 markers for both QTL mapping and genomic selection, and (3) a wheat data with 599 individuals and 1,279 markers only for genomic selection.     

A censored quantile regression approach for relative survival analysis: Relative survival quantile regression

We propose a censored quantile regression model for the analysis of relative survival data. We create a hybrid data set consisting of the study observations and counterpart randomly sampled pseudopopulation observations imputed from population life tables that adjust for expected mortality. We then fit a censored quantile regression model to the hybrid data incorporating demographic variables (e.g., age, biologic sex, calendar time) corresponding to the population life tables of demographically-similar individuals, a population versus study covariate, and its interactions with the variables of interest. These latter variables can be interpreted as relative survival parameters that depict the differences in failure quantiles between the study participants and their population counterparts.     

Hazard regression for interval-censored data with penalized spline

This article introduces a new approach for estimating the hazard function for possibly interval- and right-censored survival data. We weakly parameterize the log-hazard function with a piecewise-linear spline and provide a smoothed estimate of the hazard function by maximizing the penalized likelihood through a mixed model-based approach. We also provide a method to estimate the amount of smoothing from the data. We illustrate our approach with two well-known interval-censored data sets. Extensive numerical studies are conducted to evaluate the efficacy of the new procedure.     

Bayesian adaptive regression splines for hierarchical data

This article considers methodology for hierarchical functional data analysis, motivated by studies of reproductive hormone profiles in the menstrual cycle. Current methods standardize the cycle lengths and ignore the timing of ovulation within the cycle, both of which are biologically informative. Methods are needed that avoid standardization, while flexibly incorporating information on covariates and the timing of reference events, such as ovulation and onset of menses. In addition, it is necessary to account for within-woman dependency when data are collected for multiple cycles. We propose an approach based on a hierarchical generalization of Bayesian multivariate adaptive regression splines. Our formulation allows for an unknown set of basis functions characterizing the population-averaged and woman-specific trajectories in relation to covariates. A reversible jump Markov chain Monte Carlo algorithm is developed for posterior computation. Applying the methods to data from the North Carolina Early Pregnancy Study, we investigate differences in urinary progesterone profiles between conception and nonconception cycles.     

Jackknife model averaging for high-dimensional quantile regression

In this paper, we propose a frequentist model averaging method for quantile regression with high-dimensional covariates. Although research on these subjects has proliferated as separate approaches, no study has considered them in conjunction. Our method entails reducing the covariate dimensions through ranking the covariates based on marginal quantile utilities. The second step of our method implements model averaging on the models containing the covariates that survive the screening of the first step. We use a delete-one cross-validation method to select the model weights, and prove that the resultant estimator possesses an optimal asymptotic property uniformly over any compact (0,1) subset of the quantile indices. Our proof, which relies on empirical process theory, is arguably more challenging than proofs of similar results in other contexts owing to the high-dimensional nature of the problem and our relaxation of the conventional assumption of the weights summing to one. Our investigation of finite-sample performance demonstrates that the proposed method exhibits very favorable properties compared to the least absolute shrinkage and selection operator (LASSO) and smoothly clipped absolute deviation (SCAD) penalized regression methods. The method is applied to a microarray gene expression data set.     

Noncrossing quantile regression curve estimation

Since quantile regression curves are estimated individually, the quantile curves can cross, leading to an invalid distribution for the response. A simple constrained version of quantile regression is proposed to avoid the crossing problem for both linear and nonparametric quantile curves. A simulation study and a reanalysis of tropical cyclone intensity data shows the usefulness of the procedure. Asymptotic properties of the estimator are equivalent to the typical approach under standard conditions, and the proposed estimator reduces to the classical one if there is no crossing. The performance of the constrained estimator has shown significant improvement by adding smoothing and stability across the quantile levels.     

Bayesian nonparametric quantile process regression and estimation of marginal quantile effects

Flexible estimation of multiple conditional quantiles is of interest in numerous applications, such as studying the effect of pregnancy-related factors on low and high birth weight. We propose a Bayesian nonparametric method to simultaneously estimate noncrossing, nonlinear quantile curves. We expand the conditional distribution function of the response in I-spline basis functions where the covariate-dependent coefficients are modeled using neural networks. By leveraging the approximation power of splines and neural networks, our model can approximate any continuous quantile function. Compared to existing models, our model estimates all rather than a finite subset of quantiles, scales well to high dimensions, and accounts for estimation uncertainty. While the model is arbitrarily flexible, interpretable marginal quantile effects are estimated using accumulative local effect plots and variable importance measures. A simulation study shows that our model can better recover quantiles of the response distribution when the data are sparse, and an analysis of birth weight data is presented.     

Classification of gene microarrays by penalized logistic regression

Classification of patient samples is an important aspect of cancer diagnosis and treatment. The support vector machine (SVM) has been successfully applied to microarray cancer diagnosis problems. However, one weakness of the SVM is that given a tumor sample, it only predicts a cancer class label but does not provide any estimate of the underlying probability. We propose penalized logistic regression (PLR) as an alternative to the SVM for the microarray cancer diagnosis problem. We show that when using the same set of genes, PLR and the SVM perform similarly in cancer classification, but PLR has the advantage of additionally providing an estimate of the underlying probability. Often a primary goal in microarray cancer diagnosis is to identify the genes responsible for the classification, rather than class prediction. We consider two gene selection methods in this paper, univariate ranking (UR) and recursive feature elimination (RFE). Empirical results indicate that PLR combined with RFE tends to select fewer genes than other methods and also performs well in both cross-validation and test samples. A fast algorithm for solving PLR is also described.     

Dimension reduction-based penalized logistic regression for cancer classification using microarray data

The use of penalized logistic regression for cancer classification using microarray expression data is presented. Two dimension reduction methods are respectively combined with the penalized logistic regression so that both the classification accuracy and computational speed are enhanced. Two other machine-learning methods, support vector machines and least-squares regression, have been chosen for comparison. It is shown that our methods have achieved at least equal or better results. They also have the advantage that the output probability can be explicitly given and the regression coefficients are easier to interpret. Several other aspects, such as the selection of penalty parameters and components, pertinent to the application of our methods for cancer classification are also discussed.     

Smoothed quantile regression analysis of competing risks

Censored quantile regression models, which offer great flexibility in assessing covariate effects on event times, have attracted considerable research interest. In this study, we consider flexible estimation and inference procedures for competing risks quantile regression, which not only provides meaningful interpretations by using cumulative incidence quantiles but also extends the conventional accelerated failure time model by relaxing some of the stringent model assumptions, such as global linearity and unconditional independence. Current method for censored quantile regressions often involves the minimization of the L₁‐type convex function or solving the nonsmoothed estimating equations. This approach could lead to multiple roots in practical settings, particularly with multiple covariates. Moreover, variance estimation involves an unknown error distribution and most methods rely on computationally intensive resampling techniques such as bootstrapping. We consider the induced smoothing procedure for censored quantile regressions to the competing risks setting. The proposed procedure permits the fast and accurate computation of quantile regression parameter estimates and standard variances by using conventional numerical methods such as the Newton–Raphson algorithm. Numerical studies show that the proposed estimators perform well and the resulting inference is reliable in practical settings. The method is finally applied to data from a soft tissue sarcoma study.     

Adaptive regression splines in the Cox model

We develop a method for constructing adaptive regression spline models for the exploration of survival data. The method combines Cox's (1972, Journal of the Royal Statistical Society, Series B 34, 187-200) regression model with a weighted least-squares version of the multivariate adaptive regressi on spline (MARS) technique of Friedman (1991, Annals of Statistics 19, 1-141) to adaptively select the knots and covariates. The new technique can automatically fit models with terms that represent nonlinear effects and interactions among covariates. Applications based on simulated data and data from a clinical trial for myeloma are presented. Results from the myeloma application identified several important prognostic variables, including a possible nonmonotone relationship with survival in one laboratory variable. Results are compared to those from the adaptive hazard regression (HARE) method of Kooperberg, Stone, and Truong (1995, Journal of the American Statistical Association 90, 78-94).     

Semiparametric regression splines in matched case-control studies

We develop semiparametric methods for matched case-control studies using regression splines. Three methods are developed: 1) an approximate cross-validation scheme to estimate the smoothing parameter inherent in regression splines, as well as 2) Monte Carlo expectation maximization (MCEM) and 3) Bayesian methods to fit the regression spline model. We compare the approximate cross-validation approach, MCEM, and Bayesian approaches using simulation, showing that they appear approximately equally efficient; the approximate cross-validation method is computationally the most convenient. An example from equine epidemiology that motivated the work is used to demonstrate our approaches.     

Panel semiparametric quantile regression neural network for electricity consumption forecasting

Addressing the forecasting issues is one of the core objectives of developing and restructuring of electric power industry in China. However, there are not enough efforts that have been made to develop an accurate electricity consumption forecasting procedure. In this paper, a panel semiparametric quantile regression neural network (PSQRNN) is developed by combining an artificial neural network and semiparametric quantile regression for panel data. By embedding penalized quantile regression with least absolute shrinkage and selection operator (LASSO), ridge regression and backpropagation, PSQRNN keeps the flexibility of nonparametric models and the interpretability of parametric models simultaneously. The prediction accuracy is evaluated based on China's electricity consumption data set, and the results indicate that PSQRNN performs better compared with three benchmark methods including BP neural network (BP), Support Vector Machine (SVM) and Quantile Regression Neural Network (QRNN).     

Semiparametric distributed lag quantile regression for modeling time-dependent exposure mixtures

Studying time-dependent exposure mixtures has gained increasing attentions in environmental health research. When a scalar outcome is of interest, distributed lag (DL) models have been employed to characterize the exposures effects distributed over time on the mean of final outcome. However, there is a methodological gap on investigating time-dependent exposure mixtures with different quantiles of outcome. In this paper, we introduce semiparametric partial-linear single-index (PLSI) DL quantile regression, which can describe the DL effects of time-dependent exposure mixtures on different quantiles of outcome and identify susceptible periods of exposures. We consider two time-dependent exposure settings: discrete and functional, when exposures are measured in a small number of time points and at dense time grids, respectively. Spline techniques are used to approximate the nonparametric DL function and single-index link function, and a profile estimation algorithm is proposed. Through extensive simulations, we demonstrate the performance and value of our proposed models and inference procedures. We further apply the proposed methods to study the effects of maternal exposures to ambient air pollutants of fine particulate and nitrogen dioxide on birth weight in New York University Children's Health and Environment Study (NYU CHES).