Multicollinearity

Multicollinearity arises when two or more predictors are nearly linear combinations of one another, so their column vectors in the design matrix are almost dependent. For example, in predicting house price \(Y\) from square footage \(X_1\) and number of rooms \(X_2\):

because larger houses tend to have more rooms.