A Comparison of Hamming Errors of Representative Variable Selection Methods

Tracy Ke · Longlin Wang

[ Abstract ]
[ Visit Poster at Spot H3 in Virtual World ] [ Slides [ OpenReview
Wed 27 Apr 10:30 a.m. PDT — 12:30 p.m. PDT

Abstract: Lasso is a celebrated method for variable selection in linear models, but it faces challenges when the covariates are moderately or strongly correlated. This motivates alternative approaches such as using a non-convex penalty, adding a ridge regularization, or conducting a post-Lasso thresholding. In this paper, we compare Lasso with 5 other methods: Elastic net, SCAD, forward selection, thresholded Lasso, and forward backward selection. We measure their performances theoretically by the expected Hamming error, assuming that the regression coefficients are ${\it iid}$ drawn from a two-point mixture and that the Gram matrix is block-wise diagonal. By deriving the rates of convergence of Hamming errors and the phase diagrams, we obtain useful conclusions about the pros and cons of different methods.

Chat is not available.