#StackBounty: #lasso #model #linear #weights How to solve an adaptive lasso model?

Bounty: 50

Assuming we are working with a linear regression model, lasso penalization solves:

begin{equation}
min_{beta}left{leftlVert y-XbetarightrVert_2^2+lambdasum_{j=1}^p leftvert beta_jrightvertright}
end{equation}

While adaptive lasso penalization solves:

begin{equation}
min_{beta}left{leftlVert y-XbetarightrVert_2^2+lambdasum_{j =1}^p w_jleftvert beta_jrightvertright}
end{equation}

where $w$ defines a vector of weights previusly defined by the researcher.

This adaptive idea was initially proposed in “The adaptive Lasso and its Oracle Properties” (Journal of the American Statistical Association 101.476 (2006): 1418-1429.), and in this paper, in section 3.5, the authors state that it is possible to solve the adaptive lasso penalization using any algorithm for solving lasso penalization, just taking into account the following steps:

  1. Define $x_j^{**}=x_j/hat{w_j}, j=1,ldots,p$
  2. Solve the lasso problem
    begin{equation}
    hat{beta}^{**}=argmin_{beta}left{leftlVert y- sum_{j=1}^px_j^{**}beta_jrightrVert^2+lambdasum_{j=1}^pleftvert beta_jrightvertright}
    end{equation}
  3. Output $hat{beta_j}^*=hat{beta}_j^{**}/w_j$

So here they state that just by dividing each predictor column by the weight associated to that predictor, solving the lasso model and dividing the solution obtained here by the weights, we get the adaptive lasso solution. They say that the demonstration of this fact is very simple and it is therefore omitted, but I have been unable to mathematically check this. I would appreciate any hint on how to solve this doubt.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.