*Bounty: 100*

*Bounty: 100*

I am trying to read up on the research in the area of high-dimensional regression; when $p >> n$. It seems like the term $log p/n$ appears often in terms of rate of convergence for regression estimators.

For example, here, equation (17) says that the lasso fit, $hat{beta}$ satisfies

$$ dfrac{1}{n}|Xhat{beta} – X beta|_2^2 = O_P left(sigma sqrt{dfrac{log p}{n} } |beta|_1right),.$$

Usually, this also implies that $log p$ should be smaller than $n$.

- Is there any intuition as to why this ratio of $log p/n$ is so prominent?
- Also, it seems from the literature the high-dimensional regression problem gets complicated when $log p geq n$. Why is it so?
- Is there a good reference that discusses the issues with how fast $p$ and $n$ should grow compared to each other?