7 · Overfitting: memorizing vs. learning
Here's a sneaky problem that trips up even grown-up engineers. Imagine a student who, instead of understanding math, just memorizes the exact answers to the practice test. They ace the practice test — then bomb the real exam, because the questions are a little different. The model version of this has a name:
Overfitting is when a model memorizes its training examples instead of learning the general pattern. It looks amazing on the data it trained with, but flops on new, real examples.
That gap is the tell: great on training data, bad on new data = overfitting. It's the opposite of what you want. A good model generalizes — it handles things it never saw before, because it learned the idea ("this is a thumbs up") rather than memorizing specific photos.
Why does overfitting happen? Usually because the training data was too small or too samey:
- Too few examples. With only 5 photos, the easiest "pattern" to find is just those 5 photos, not the real idea.
- Too little variety. If every thumbs-up photo is you, same shirt, same wall, the model can memorize that exact scene and call it a day.
- A lucky shortcut. Remember the "couch = cat" trick? That's overfitting to an accidental clue.
How makers fight overfitting — and you can do all of these in Teachable Machine:
| Fix | How it helps |
|---|---|
| More examples | Harder to memorize a big, varied pile |
| More variety | Different angles, light, backgrounds force it to learn the real idea |
| Test on fresh examples | The only way to catch overfitting (Lesson 6) |
| Don't over-train tiny data | Cramming hard on a few examples just memorizes them |
The opposite mistake is underfitting — too few or too sloppy examples so the model never even learns the pattern (it's bad at everything, training data included). The sweet spot is in the middle: enough good, varied examples to learn the real idea and generalize.
Think about it. A model is 100% right on its training photos but only 40% right on new ones. What's the one-word diagnosis — and what's the first thing you'd change about the training data to fix it?
Sources
- MIT RAISE. Day of AI — training, testing, and how models generalize. https://raise.mit.edu/
- Code.org. How AI Works (machine learning concepts). https://code.org/curriculum/how-ai-works