8 · Safety: when a robot should ask a human
Everything so far points to one careful conclusion: an AI-powered robot is making guesses, and it acts in the real world — so we must design, on purpose, when it's allowed to act alone and when it must stop and ask a human. This isn't being timid; it's how the best engineers build.
Why guesses aren't enough on their own:
- A learned model can be confidently wrong (a sign in shadow, an accent it never heard).
- The world throws situations the examples missed — and that's exactly when a wrong guess is most likely.
- When a robot moves, a wrong guess isn't just an odd chatbot answer; it can bump, drop, or collide with something real.
So engineers build in a "keep a human in the loop" rule for anything risky. The clearest example you already met: NASA's rovers drive themselves for routine paths, but big or risky decisions still go back to the team on Earth — the cameras "help ensure a safe path," and the rover makes its own decisions only "without consulting on every move," not every move (NASA JPL, n.d.). Routine driving: the robot decides. Risky calls: a human is consulted.
A simple way to think about it — sort actions into three buckets:
| Bucket | What the robot should do | Example |
|---|---|---|
| Low stakes | act on its own | sort recycling; show a smiley when shaken |
| Medium | act, but flag it / let a human override | suggest a route; pause and ask "is this right?" |
| High stakes | require a human to confirm | anything that could hurt a person or do real damage |
And remember the trick from Lesson 2: for the highest-stakes action, engineers often don't leave the decision to a learned guess at all — they use a plain, readable rule ("if a person is detected within 1 meter, stop"), so the safety behavior is predictable. The learned AI does the clever recognizing; a simple, checkable rule does the protecting.
Think about it. Design the rule for a delivery robot at a crosswalk. What can it decide alone, and what should make it stop and wait for — or ask — a human? Why put the "stop" as a hard rule rather than a learned guess?
Sources
NASA JPL. (n.d.). Mars 2020 Perseverance rover components (autonomy with team oversight). https://science.nasa.gov/mission/mars-2020-perseverance/rover-components/ NASA. (n.d.). Robotics (autonomous systems operating independently). https://www.nasa.gov/robotics/