3 · How a robot "sees" (computer vision basics)

A camera by itself doesn't understand anything — it just records a grid of colored dots called pixels. Turning that grid of dots into "that's a stop sign" or "that's a person stepping off the curb" is a job called computer vision: teaching a computer to make sense of images and video. Code.org's grades-6-12 program has a whole computer-vision unit where students explore "image and video analysis" (Code.org, n.d.).

How does it work? With the pattern-learning you already know:

Examples. Show the model thousands of labeled pictures — "this is a stop sign," "this is not."
Find the pattern. The model learns what shapes, edges, and colors usually mean "stop sign."
Guess on something new. Point a camera at a new scene; the model predicts what's there.

Two honest truths about computer vision, both of which matter for safety later:

It's a guess, not a fact. The model isn't "looking it up." It's matching patterns, so it can be confidently wrong — a smudge of snow on a sign, or a shadow, can fool it. (Same idea as a hallucination in a chatbot: it can be sure and still wrong.)
It's only as good as its examples. If the training pictures left some situations out — night, rain, an unusual sign — the model may not recognize them. This is the same bias-from-missing-examples problem you met in Young Makers, except now a robot might act on the mistake by moving in the real world.

You don't need a self-driving car to see computer vision in action. Free tools let students train an image model from their own example pictures (sorting recycling from trash, say) and watch it guess — the exact loop above, at a classroom scale (Code.org, n.d.; MIT RAISE, n.d.).

Think about it. A delivery robot's camera was trained mostly on sunny daytime sidewalks. Name one situation where its computer vision might guess wrong, and explain why, using "examples."

Sources

Code.org. (n.d.). How AI works (computer vision unit, grades 6-12). https://code.org/ai MIT RAISE. (n.d.). Day of AI — free K-12 AI literacy curriculum. https://raise.mit.edu/