Data leakage, explained simply
Sometimes a model gets a brilliant score for the wrong reason: one of its features accidentally contains the answer. This is called data leakage.
Imagine training a model to spot ripe fruit, but every ripe fruit photo was taken in the kitchen and every unripe one in the garden. The model may learn "kitchen means ripe" instead of anything about the fruit.
In the real world that shortcut disappears, and the model fails. Good model detectives ask of every feature: would we really know this at the moment we need to make the prediction?
Example: A suspiciously perfect score
- A model predicts whether a library book will be returned late.
- One feature is "late fee paid". It gets 100% accuracy.
- But you only know a fee was paid after the book was late. The answer leaked into the input.
Try this
- Look at a quiz where the question gives away the answer. How is that like leakage?
- For a "will it rain tomorrow?" model, list features you would know today, and one that would be cheating.
- When something scores 100%, practise asking "what shortcut could it be using?"