"The Monkey With the AK-47: On LLM Literacy"
There is a meme. A monkey gets handed an AK-47. Chaos follows. Nobody is surprised except, apparently, the people who handed it over.
That is what handing ChatGPT to someone who does not understand what it is feels like to me. Not because the tool is evil. Not because the person using it is stupid. But because the combination of a very powerful thing and a very misplaced confidence in that thing is its own kind of disaster: slow, quiet, and dressed in perfectly formatted prose.
I have had this argument too many times. I try to explain the limitations. I get the deflection: "But you also use LLMs." Yes. I do. Extensively. That is not the point. The point is what you do after the output lands in front of you.
---
We Have Been Here Before
This is not the first time this has happened. We watched the same arc play out with Google Search.
Early on, mostly technical people used it. They knew how to read results. They could smell a sketchy domain, they understood that the first result was not always the right result, they knew to cross-reference. Then everyone got access. The access scaled. The literacy did not.
Then Google started pre-digesting results. Featured snippets. Knowledge panels. Answers just sitting there at the top of the page, authoritative-looking, no friction, no need to click through. And then AI Overviews. The chain of trust between you and the original source got longer and longer, and with every extra link, something got lost.
LLMs are that last stage cranked to the maximum. There is no search results page to scan skeptically. There is just a confident answer, in a confident voice, waiting for you.
"The technology changed. The underlying habit of believing the first thing you read did not."
---
So What Even Is an LLM?
Forget the acronym. Here is what it actually is, without the jargon.
You know how your phone keyboard suggests the next word when you type? "Happy birth..." and it offers "day." It learned that from watching how you type. Now imagine the same idea, but instead of learning from your messages, it read everything ever written on the internet. Every article, book, forum thread, Reddit argument, research paper, recipe blog. Billions of pages.
And it got so good at predicting what word comes next that it can now complete entire essays, answer questions, write code, explain concepts, because it learned the patterns of how human knowledge looks when expressed in text.
That is the core of it. An extremely sophisticated autocomplete. Trained on an incomprehensible amount of human writing.
The Cookbook Analogy: Imagine a person who read every cookbook ever written, but never cooked a single meal. Ask them how to make biryani and they'll give you a perfect, detailed, confident recipe. Ask them whether it will actually taste good? They have no idea. They have never tasted anything. They just know what biryani recipes sound like.
That is ChatGPT. Infinite reading. Zero lived experience. Zero ability to verify. It knows what correct-sounding answers look like. That is not the same as knowing the correct answer.
---
Why It Gets Things Wrong. Confidently.
Here is the part that trips people up. They hear "hallucination" and think it means the AI occasionally gets a small fact wrong. Like misremembering a date or a name. That is the minor version.
The major version is this: it can construct an entire argument, with citations, statistics, precedents, and logical flow, that is completely fabricated, top to bottom, and internally coherent. It is not lying. It genuinely cannot tell the difference between something it knows and something it has pattern-completed into existence. It does not have a fact-checker running in the background. There is no researcher on the other side looking things up.
And the confidence? That got baked in during training. Because the text it learned from was mostly written by humans who write declaratively. People do not write "I think maybe possibly this could be true." They assert. So the model learned to assert. The uncertainty got trained out of its voice even when the uncertainty is very much real.
Ask the same question twice with slightly different phrasing and you can get opposite answers, both delivered with the same calm authority. The model does not have a position. It has a pattern completion.
This is the real UX problem: If ChatGPT genuinely said "I'm not sure about this, please verify" every time it was shaky, people would calibrate fast. But it doesn't. It delivers hallucinations and ground truth in exactly the same voice. There is no tell.
---
"Let Me Ask ChatGPT": The New Mic Drop
You are in the middle of a discussion. Someone pulls out their phone. "Let me ask ChatGPT." ChatGPT agrees with them. Discussion over. Ruled upon. By the oracle.
This drives me insane. Not because ChatGPT is wrong in that moment, it might be right. But because of what it reveals about how people are treating it. As a referee. As an authority that settles disputes. Not as a tool that produces probabilistic output based on training data and needs to be evaluated critically like anything else.
The deflection I always get when I push back is: "But you also use LLMs extensively."
Yes. And I also drive a car. That does not mean I am going to take my hands off the wheel.
Using a tool and blindly trusting a tool are not the same thing. Equating them is a way to shut down a conversation that you do not want to have. The people making that argument are not defending ChatGPT's reliability. They are defending their own comfort with not having to think critically about the output.
---
The Copy-Paste People
There is a specific subset of this problem I want to name directly: the people who copy-paste incoming emails into ChatGPT, take the reply it generates, and paste that back as their response. Inbox zero achieved. Work done. Or so they think.
You can always tell when you are on the receiving end. The reply comes back slightly too formal. It addresses every point you made in order, like a checklist. It opens with "Thank you for reaching out." It closes with "Please do not hesitate to contact me if you have further questions." And it says absolutely nothing new.
Nobody made a decision. No genuine information was exchanged. You now have two LLMs talking to each other through the medium of two people who have quietly opted out of thinking. And they feel productive doing it.
"You are not using the AK-47. You are just letting the monkey hold it while you take a nap."
The longer-term damage is subtler. The muscle for articulating your own thoughts, for reading between the lines of what someone actually meant, for negotiating tone. It atrophies. Fast.
---
This Is a Literacy Problem, Not a Technology Problem
The hard truth is that epistemic hygiene (knowing how to evaluate information, when to trust it, when to dig deeper) is not really teachable through argument. The people who learned to use Google well did so through repeated experience of being burned. They searched something, acted on it, got it wrong, recalibrated. That feedback loop built the skepticism.
LLMs short-circuit that loop. A bad Google result took you to a sketchy website and you felt something was off. A hallucinated LLM answer comes in clean, confident, beautifully structured. The wrongness is invisible until it matters.
The same person who pulls out ChatGPT as a debate-ender probably also forwarded WhatsApp health misinformation without checking. The tool changed. The underlying habit did not. This is not about intelligence. It is about the ingrained comfort of outsourcing judgment.
The people arguing back at you have probably not been burned badly enough yet. They will be.
---
So How Do You Actually Use It Better?
Not "don't use it." That ship sailed. But here are three things anyone, technical or not, can hold onto:
01. Treat it as a smart first draft, not a final answer. It is a starting point. A thinking partner. A way to get unstuck. It is not a finishing line. Whatever it gives you, you still have to apply your own brain to it before it leaves your hands.
02. Vague question, vague confident answer. The more specific and contextual your prompt, the more grounded the output. If you ask a lazy question you will get a beautifully written non-answer. Garbage in, polished garbage out.
03. If it matters, verify. Medical, legal, financial, factual claims: cross-check with a primary source. ChatGPT agreeing with you is not evidence that you are right. It is evidence that your prompt led to a pattern-completion that sounded like agreement.
---
The One-Liner to Remember
ChatGPT is the most well-read person who has never fact-checked anything in their life. Sounds impressive. Mostly right. Occasionally completely wrong. And you can never tell which is which just by listening to them.
The goal is not to make people afraid of the tool. The goal is to make them aware that using it well is a skill, the same way using search well was a skill, the same way reading a newspaper critically was a skill. Every powerful information tool demands a corresponding literacy from the person using it.
We are at the beginning of that curve. Most people are still in the "hand the monkey the AK-47" phase.
The question is how fast the feedback loop burns them and whether they learn the right lesson from it when it does.