Group variable selection using Bootstrap and group LASSO method

Abstract

This work considers using Bootstrap method for group variable selection via group LASSO procedure. Much literature discusses the implementation of LASSO and LASSO type estimates to solve the problem in either the coefficient estimate precision or index selection accuracy. Group LASSO is one of them, which is extremely helpful in group variable selection. In this work, Bootstrap samples will be generated to enhance group LASSO’s performance via constructing a series of active sets and taking intersection of them. Both excessive exclusion ( selection rule is too strict to exclude some significant groups) and undue inclusion ( selection rule is too loose to include some insignificant groups) will be studied and exact probability bounds will be derived under normal and sub-gaussian errors. Based on sub-gradient solution, the probability of selecting correct groups of variables will be presented under conditions to ensure the consistency property. The efficiency of the Bootstrapped group LASSO under high-dimensional group number p and large within-group dimension d_max will be illustrated where such high-dimensional case can be as large as log p = o(n), Simulation results will be presented for both simple linear model and nonparametric additive model with comparison to original group LASSO and Bootstrapped LASSO. The result shows that Bootstrapped group LASSO is more stable and reliable in variable selection.