Sliced inverse regression with regularizations

Summary .   In high-dimensional data analysis, sliced inverse regression (SIR) has proven to be an effective dimension reduction tool and has enjoyed wide applications. The usual SIR, however, cannot work with problems where the number of predictors, p , exceeds the sample size, n , and can suffer when there is high collinearity among the predictors. In addition, the reduced dimensional space consists of linear combinations of all the original predictors and no variable selection is achieved. In this article, we propose a regularized SIR approach based on the least-squares formulation of SIR. The L ₂ regularization is introduced, and an alternating least-squares algorithm is developed, to enable SIR to work with   n < p   and highly correlated predictors. The L ₁ regularization is further introduced to achieve simultaneous reduction estimation and predictor selection. Both simulations and the analysis of a microarray expression data set demonstrate the usefulness of the proposed method.     

Sufficient dimension reduction for censored regressions

Methodology of sufficient dimension reduction (SDR) has offered an effective means to facilitate regression analysis of high-dimensional data. When the response is censored, however, most existing SDR estimators cannot be applied, or require some restrictive conditions. In this article, we propose a new class of inverse censoring probability weighted SDR estimators for censored regressions. Moreover, regularization is introduced to achieve simultaneous variable selection and dimension reduction. Asymptotic properties and empirical performance of the proposed methods are examined.     

A note on sliced inverse regression with regularizations

Summary .   In Li and Yin (2008, Biometrics 64, 124–131), a ridge SIR estimator is introduced as the solution of a minimization problem and computed thanks to an alternating least-squares algorithm. This methodology reveals good performance in practice. In this note, we focus on the theoretical properties of the estimator. It is shown that the minimization problem is degenerated in the sense that only two situations can occur: Either the ridge SIR estimator does not exist or it is zero.     

Determining the dimension of the central subspace and central mean subspace

The central subspace and central mean subspace are two importanttargets of sufficient dimension reduction. We propose a weightedchi-squared test to determine their dimensions based on matriceswhose column spaces are exactly equal to the central subspaceor the central mean subspace. The asymptotic distribution ofthe test statistic is obtained. Simulation examples are usedto demonstrate the performance of this test.     

Dimension reduction strategies for analyzing global gene expression data with a response

The analysis of global gene expression data from microarrays is breaking new ground in genetics research, while confronting modelers and statisticians with many critical issues. In this paper, we consider data sets in which a categorical or continuous response is recorded, along with gene expression, on a given number of experimental samples. Data of this type are usually employed to create a prediction mechanism for the response based on gene expression, and to identify a subset of relevant genes. This defines a regression setting characterized by a dramatic under-resolution with respect to the predictors (genes), whose number exceeds by orders of magnitude the number of available observations (samples). We present a dimension reduction strategy that, under appropriate assumptions, allows us to restrict attention to a few linear combinations of the original expression profiles, and thus to overcome under-resolution. These linear combinations can then be used to build and validate a regression model with standard techniques. Moreover, they can be used to rank original predictors, and ultimately to select a subset of them through comparison with a background 'chance scenario' based on a number of independent randomizations. We apply this strategy to publicly available data on leukemia classification.     

Sufficient dimension reduction via bayesian mixture modeling

Dimension reduction is central to an analysis of data with many predictors. Sufficient dimension reduction aims to identify the smallest possible number of linear combinations of the predictors, called the sufficient predictors, that retain all of the information in the predictors about the response distribution. In this article, we propose a Bayesian solution for sufficient dimension reduction. We directly model the response density in terms of the sufficient predictors using a finite mixture model. This approach is computationally efficient and offers a unified framework to handle categorical predictors, missing predictors, and Bayesian variable selection. We illustrate the method using both a simulation study and an analysis of an HIV data set.     

Implications of influence function analysis for sliced inverse regression and sliced average variance estimation

Sliced inverse regression, sliced inverse regression II andsliced average variance estimation are three related dimension-reductionmethods that require relatively mild model assumptions. As anapproximation for the relative influence of single observationsfrom large samples, the influence function is used to comparethe sensitivity of the three methods to particular observationaltypes. The analysis carried out here helps to explain why thereis a lack of agreement concerning the preferability of thesedimension-reduction procedures in general. An efficient sampleversion of the influence function is also developed and evaluated.     

Inference for the dimension of a regression relationship using pseudo-covariates

In data analysis using dimension reduction methods, the main goal is to summarize how the response is related to the covariates through a few linear combinations. One key issue is to determine the number of independent, relevant covariate combinations, which is the dimension of the sufficient dimension reduction (SDR) subspace. In this work, we propose an easily-applied approach to conduct inference for the dimension of the SDR subspace, based on augmentation of the covariate set with simulated pseudo-covariates. Applying the partitioning principal to the possible dimensions, we use rigorous sequential testing to select the dimensionality, by comparing the strength of the signal arising from the actual covariates to that appearing to arise from the pseudo-covariates. We show that under a “uniform direction” condition, our approach can be used in conjunction with several popular SDR methods, including sliced inverse regression. In these settings, the test statistic asymptotically follows a beta distribution and therefore is easily calibrated. Moreover, the family-wise type I error rate of our sequential testing is rigorously controlled. Simulation studies and an analysis of newborn anthropometric data demonstrate the robustness of the proposed approach, and indicate that the power is comparable to or greater than the alternatives.     

Optimal sufficient dimension reduction for the conditional mean in multivariate regression

The aim of this article is to develop optimal sufficient dimensionreduction methodology for the conditional mean in multivariateregression. The context is roughly the same as that of a relatedmethod by Cook & Setodji (2003), but the new method hasseveral advantages. It is asymptotically optimal in the sensedescribed herein and its test statistic for dimension alwayshas a chi-squared distribution asymptotically under the nullhypothesis. Additionally, the optimal method allows tests ofpredictor effects. A comparison of the two methods is provided.     

Technical note: Regression analysis in adult age estimation

Accurate estimation of human adult age has always been a problem for anthropologists, archaeologists and forensic scientists. The main factor contributing to the difficulties is the high variability of physiological age indicators. However, confounding this variability in many age estimation applications is a systematic tendency for age estimates, regardless of physiological indicator employed, to assign ages which are too high for young individuals, and too low for older individuals. This paper shows that at least part of this error is the inevitable consequence of the statistical procedures used to extract an estimate of age from age indicators, and that the magnitude of the error is inversely related to how well an age indicator is correlated with age. The use of classical calibration over inverse calibration is recommended for age estimation. Am J Phys Anthropol 104:259–265, 1997. © 1997 Wiley-Liss, Inc.