Lasso Limit Cases

What happens to the Lasso coefficients when $\lambda = 0$ and as $\lambda \to \infty$ ?

Answer

When $\lambda = 0$ , there is no penalty and the coefficients coincide with the OLS estimates. As $\lambda \to \infty$ , the $\ell_{1}$ penalty dominates and all coefficients shrink toward zero, with many becoming exactly zero.

The main point here is that Lasso both shrinks coefficient magnitudes and sets many exactly to zero, yielding a sparse model which performs variable selection.