Bias & fairness, explained simply
Every AI model makes mistakes. Many of them come from the training data. If a model only saw red berries, it may not recognise a blue one. It is not being silly; it simply never saw that kind of example.
When the gaps in the data line up with groups of people, the mistakes become unfair. A system trained mostly on one kind of voice or face can work well for some people and badly for others. This is called bias.
One overall score can hide this. A fair check measures how well the model works for each group separately. Fixing bias usually means collecting better, more varied data and testing with everyone in mind.
Example: A scanner that is 90% right
- A plant scanner gets 90 out of 100 test leaves right. Sounds good.
- Look closer: it gets 85 of 85 garden leaves right, but only 5 of 15 forest leaves.
- The training photos were mostly garden leaves. Adding varied forest photos is the fix.
Try this
- Imagine a model trained only on pictures of cats taken indoors. Which cats might it miss?
- Look at the pictures in a book or app and ask: who or what is missing?
- Choose a group score to check, not just the overall score, next time you test a model.