A few weeks ago, in a high school auditorium in North Charleston, South Carolina, a group of teachers watched an AI tool try to draw a map of the world.
It did not go well.
Mali was labeled “Mail.” Egypt came back as “Sopth.” Libya vanished entirely, replaced by a country simply called “Africa.” The room reportedly erupted in laughter and disbelief.
Then something interesting happened. The instructor running the training, Amanda Bickerstaff of AI for Education, didn’t apologize for the tool or move past the failure. She used it as the entire point of the exercise. “If you ever want kids not to over trust these tools, try the map demo,” she said.
That single moment captures a shift happening quietly across American schools this year. After an initial wave of AI bans, a growing number of districts are trying something different: deliberate, structured exposure to AI’s failures, so students learn early that fluent-sounding output is not the same as correct output.
Knowing when not to use it
Rebecca Winthrop, who directs the Center for Universal Education at Brookings, offered the line that I think best summarizes the whole movement: good AI literacy includes knowing when not to use it.
That is a strikingly different goal from where most AI training, in schools or in workplaces, tends to start. Most training begins and ends with capability: how to write a better prompt, how to get a cleaner output, how to move faster. Almost none of it starts with the harder and more important question, which is when a person should simply not delegate the thinking to the machine at all.
Charleston County School District, the state’s second-largest, built its own answer to that question over the past year, since South Carolina hadn’t published statewide guidance the way 37 other states have. Their approach unfolded in phases. First, a policy built with input from teachers, students, and parents. Then focus groups, which revealed something predictable: students were already using AI constantly, without guidance, and both students and teachers wanted actual instruction rather than a blanket rule.
Phase two, now underway as the new school year starts, is training. Students walk through familiar scenarios, like asking a chatbot for help on a research paper, and are shown exactly how confidently a model will invent a source, a statistic, or a study that doesn’t exist. The instruction is blunt: verify everything, especially for schoolwork.
This is the thesis of the book, aimed at a different audience
I wrote The Amplified Mind around a single distinction: the machine generates, the thinker decides. Generation is not authorship. Fluency is not accuracy. A tool producing an answer instantly does not relieve the person using it of the responsibility to verify that answer before acting on it.
Chapter 5 of the book goes into this in more detail, using incident command as the model. In fire and emergency response, verification before action isn’t optional or aspirational. It’s procedural. You confirm before you commit resources, every time, because the cost of acting on a wrong assumption is measured in outcomes that can’t be undone.
What’s happening in Charleston, and in Utah, which was the first state to appoint a full-time AI education specialist, is essentially an attempt to build that same procedural discipline into how kids grow up using these tools. Utah’s Matt Winters has trained over 7,000 teachers, nearly a third of the state’s public school instructors, on the basics of how AI works, where its bias and hallucination risks come from, and what that means for classroom use. Other states, including Maine, West Virginia, and Georgia, have since copied the model.
Where the comparison gets uncomfortable
Here’s the part I keep sitting with. If a Utah eighth grader is now getting structured training on hallucination detection before they graduate high school, and a growing number of states are treating that training as a basic civic responsibility, what does it say that most enterprise AI governance conversations are still stuck at the adoption stage?
I’ve written before, including in a piece for Solutions Review, about what I call the verification gap: the space between how confidently an AI system presents an answer and how rigorously anyone actually checks that answer before it gets used. Incident command teaches you to close that gap by procedure. Most corporate AI rollouts I’ve seen don’t have an equivalent procedure at all. They have usage dashboards.
Schools, for all their resource constraints and uneven access to funding, appear to be getting something right that many well-funded organizations are still missing: the recognition that the skill worth building isn’t prompting. It’s judgment.
For educators
If you’re a teacher, administrator, or district leader working through your own AI literacy approach this year, I built a resource specifically for this. It’s on the book’s site under For Educators, and it applies the same thinking-versus-generating framework from the book to a classroom context.
The machine generates. The thinker decides. It turns out a growing number of school districts figured that out before most boardrooms did.
Note on process: AI tools may assist with editing or structure in this post. The ideas, judgment, and conclusions are mine. The machine generates. The thinker decides.
