#StackBounty: #regression #lasso #convergence #high-dimensional High-dimensional regression: why is $log p/n$ special?

Bounty: 100

I am trying to read up on the research in the area of high-dimensional regression; when $p >> n$. It seems like the term $log p/n$ appears often in terms of rate of convergence for regression estimators.

For example, here, equation (17) says that the lasso fit, $hat{beta}$ satisfies
$$ dfrac{1}{n}|Xhat{beta} – X beta|_2^2 = O_P left(sigma sqrt{dfrac{log p}{n} } |beta|_1right),.$$

Usually, this also implies that $log p$ should be smaller than $n$.

  1. Is there any intuition as to why this ratio of $log p/n$ is so prominent?
  2. Also, it seems from the literature the high-dimensional regression problem gets complicated when $log p geq n$. Why is it so?
  3. Is there a good reference that discusses the issues with how fast $p$ and $n$ should grow compared to each other?

Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.