Selecting \(k\)

How do we choose the value of \(k\) in k-NN?

Answer

Some heuristics are:

  • Perform cross-validation (most common): Choose the \(k\) that gives the highest performance with respect to some evaluation metric (e.g., accuracy, F1 score, RMSE, etc.).

  • Odd \(k\) values for classification: This is done to avoid ties in voting.

  • Simple rule of thumb: Start with \(k \approx \sqrt{n}\).

Back to collection