Singular Covariance Matrix

When is a covariance matrix singular?

Answer

A covariance matrix $\Sigma$ is singular (not invertible) iff it is positive semidefinite but not positive definite.

Indeed, if $\Sigma$ is only positive semi-definite, then there exists a nonzero vector $$v$$ such that

v^{T}\Sigma v = 0.

But

v^{T}\Sigma v = \mathrm{Var}(v^{T}X),

so $\mathrm{Var}(v^{T}X)=0$ , which means $v^{T}X$ is a constant (almost surely). This implies there is one column/variable which can be written as a linear combination of the others. Hence $\Sigma$ is singular.