Errors and Residuals
What is the difference between an error and a residual?
Answer
An error (unobservable) is the difference between the observed value \( Y \) and the true value \( f(X) \).
\[
\varepsilon = Y - f(X)
\]
A residual (observable) is the difference between the observed value \( Y \) and the predicted value \( \hat{f}(X) \) given by the model.
\[
r = Y - \hat{f}(X)
\]