On Lasso-type estimators under a moving-parameter framework

Abstract

Limiting distributions and residual bootstrap methods for Lasso-type regression estimators under a moving parameter framework are investigated in this thesis.

In the first part, we study the asymptotic behaviour of Lasso-type regression estimators under a general moving-parameter framework and minimal assumptions on design covariates. Our results elucidate how distributional properties of the estimators are affected by the values of individual regression coefficients. In particular, we show that the convergence rates of estimators of large coefficients depend critically on the magnitudes of coefficients that are close to zero, even when the estimators possess the acclaimed oracle property under a fixed-parameter asymptotic framework. We demonstrate in the case of SCAD estimation how our theoretical results motivate a solution path-based procedure for diagnosing viability of the oracle property in practice.

In the second part, we consider moving-parameter asymptotic properties of residual bootstrap methods recently proposed for making statistical inference based on Lasso-type estimators. We show that consistency of the natural residual bootstrap Lasso-type distribution depends critically on whether orders of individual regression coefficients can be correctly restored. In particular, if there exist small coefficients close, but not equal, to zero, the natural residual bootstrap may be inconsistent even when it is applied to estimate the distribution of a large coefficient estimator. To help alleviate this problem, we propose a hybrid residual bootstrap method, which uses a hybrid estimator based on an oracle and ordinary least squares estimators, to restore more correctly the orders of the small coefficients. In hypothesis testing problems, we show by simulations that the natural residual bootstrap of the conventional Studentized test statistic often suffers from over-rejection in the presence of small coefficients. As a remedy we propose two modified test statistics and a constrained residual bootstrap method for the hypothesis test. Our simulation results suggest that our new testing procedures improve upon existing methods.