k-nearest neighbours, explained simply
k-nearest neighbours, or kNN, is one of the simplest machine learning methods, which makes it great for seeing how a model decides.
To classify something new, the model finds the k training examples most similar to it, its nearest neighbours. Each neighbour votes with its label, and the most popular label wins.
The value of k matters. With k = 1, one odd example can swing the answer. A larger k listens to more neighbours and smooths out oddities, but too large and it starts ignoring genuine local patterns.
Example: Who lives nearby?
- A mystery creature appears on the map.
- Its 5 nearest neighbours are 3 foxes and 2 cats.
- With k = 5, the vote is 3 to 2, so the model guesses "fox".
- With k = 1, it only asks the single closest neighbour, which happens to be a cat.
Try this
- Stand in a room of family members and ask: who are my three nearest neighbours, and what do most of them like best?
- Draw dots of two colours on paper, add a new dot, and colour it by its 3 nearest neighbours.
- Try k = 1 and k = 7 on the same dot. Does the answer change?