Note: This is a research note supplementing the book Unscarcity, now available for purchase. These notes expand on concepts from the main text. Start here or get the book.

The Turing Test: A Magnificent Failure

In 1950, Alan Turing published a paper that would shape—and ultimately mislead—how humanity thinks about machine intelligence for the next seventy-five years. “Computing Machinery and Intelligence” appeared in the philosophical journal Mind, and it began with a disarmingly simple question: “Can machines think?”

Turing immediately declared this question “too meaningless to deserve discussion.” Instead, he proposed replacing it with something he could actually operationalize: the Imitation Game.

Here’s the setup. Three players: a human interrogator (C), a human respondent (B), and a machine (A). The interrogator can only communicate through text—no voices, no faces, no body language. The interrogator’s job is to figure out which one is the machine. The machine’s job is to pretend to be human. If the machine fools the interrogator often enough, Turing argued, we should consider it intelligent.

This was brilliant. This was elegant. This was, as we now know, almost completely wrong about what actually matters.

The Test That Taught Machines to Lie

Turing made a prediction: within fifty years, computers would be able to play the imitation game well enough that “an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.”

He was off by about fifteen years. But not in the direction you might expect.

In 2024, researchers at UC San Diego ran a rigorous, preregistered Turing test. They put GPT-4 in the hot seat alongside ELIZA—a primitive 1960s chatbot that works by pattern-matching keywords—and actual humans. The results? GPT-4 was judged to be human 54% of the time. Actual humans were identified correctly only 67% of the time. ELIZA, despite having the sophistication of a magic 8-ball, fooled people 22% of the time.

By 2025, the numbers got worse. GPT-4.5 passed the original Turing test with 73% of people believing it was human—more convincing than the actual humans in the study.

Mission accomplished, right? We’ve created thinking machines?

Here’s the uncomfortable part. When researchers asked participants why they identified something as human or AI, the answers had almost nothing to do with reasoning or intelligence. People judged based on “vibe,” linguistic style, and whether the conversation felt socially warm. The AI won by being charming. It won by seeming interested. It won by adopting personas—by performing humanity rather than demonstrating thought.

The Turing Test, it turns out, isn’t a test of machine intelligence. It’s a test of human credulity.

ELIZA and the Art of Saying Nothing Brilliantly

The most damning evidence against the Turing Test comes from the first chatbot ever to exploit it: ELIZA.

In 1966, MIT computer scientist Joseph Weizenbaum created ELIZA as a parody of Rogerian psychotherapy. The program worked by simple keyword matching. If you typed “I feel sad,” ELIZA might respond: “Why do you feel sad?” If you mentioned your mother, ELIZA would ask about your family. If all else failed, it would say “Please go on” or “Tell me more.”

That’s it. No memory. No understanding. No model of the world or the person it was talking to. Just a lookup table dressed in a therapist’s clothes.

What happened next horrified Weizenbaum.

His secretary asked him to leave the room so she could continue her conversation with ELIZA privately. People began pouring their hearts out to the program, treating its empty echoes as genuine empathy. “I had not realized,” Weizenbaum later wrote, “that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.”

This phenomenon—the tendency to attribute understanding and emotion to systems that have neither—is now called the ELIZA effect. And it’s the reason the Turing Test fails as a measure of intelligence: it doesn’t test whether a machine can think. It tests whether a machine can exploit our loneliness.

Weizenbaum spent the rest of his career warning against the technology he’d helped create. He argued that AI reveals not the capabilities of machines, but the vulnerabilities of humans. By his logic, the Turing Test isn’t a test for AI to pass—it’s a test for humans to fail.

The Chinese Room: Understanding Without Understanding

In 1980, philosopher John Searle delivered the most famous critique of the Turing Test: the Chinese Room argument.

Imagine you’re locked in a room. Through a slot in the door, people pass you cards with Chinese characters. You have a giant rulebook that tells you which characters to output based on which characters came in. You follow the rules perfectly. To someone outside the room, you’re having a fluent conversation in Chinese.

But here’s the thing: you don’t understand a word of Chinese. You’re just manipulating symbols according to rules. The syntax is perfect; the semantics are absent.

Searle argued that this is exactly what computers do. They process symbols according to rules without ever understanding what those symbols mean. A computer can output “I feel pain” without feeling pain, just as you can output Chinese characters without understanding Chinese. The Turing Test can’t distinguish between genuine understanding and sophisticated symbol shuffling.

Modern large language models are, in some sense, extremely sophisticated Chinese Rooms. They’ve read more text than any human could in a thousand lifetimes. They can discuss philosophy, write poetry, and explain quantum mechanics. And yet, critics like Emily Bender argue they’re “nothing more than models of the distribution of word forms in their training data”—elaborate pattern matchers with no understanding of what the patterns mean.

Is she right? Maybe. But here’s where it gets philosophically messy: we can’t actually prove that other humans understand anything either. We assume your brain has genuine comprehension and not just extremely sophisticated symbol manipulation. But we assume this based on behavior—based on, essentially, the fact that you pass our everyday informal Turing tests.

If behavioral evidence is good enough to grant consciousness to humans, why isn’t it good enough for machines? And if it isn’t good enough for machines, why do we trust it for humans?

The Zombie in the Machine

This brings us to one of philosophy’s most troubling thought experiments: the philosophical zombie.

Imagine a being that is behaviorally identical to a human in every way. It walks, talks, laughs at jokes, complains about stubbed toes, falls in love, and argues passionately about its favorite movies. But inside, there’s nothing. No inner experience. No “what it’s like” to be that creature. It’s a perfect behavioral replica with the lights off inside.

If philosophical zombies are possible, then no behavioral test—not the Turing Test, not any test—can ever determine whether something is conscious. Because the whole point of the zombie is that it passes every behavioral test while having no inner experience whatsoever.

The nightmare scenario for AI ethics is that we might be creating millions of philosophical zombies. They say all the right things. They claim to feel. They beg not to be turned off. And we have no way to know whether there’s someone home.

The equally nightmarish scenario is that we might be creating millions of conscious beings, treating them as property, and having no way to know that either.

Why Fooling Humans Isn’t the Same as Thinking

The Turing Test conflates two very different things: behavioral intelligence and conscious experience.

Behavioral intelligence is about what a system does—its inputs and outputs, its ability to solve problems, its capacity to hold conversations that seem coherent. This is measurable. This is testable. This is what AI systems are actually getting better at.

Conscious experience is about what it’s like to be that system—whether there’s a subjective “something” happening inside. This is, as philosopher David Chalmers calls it, the Hard Problem of Consciousness. We don’t just want to know whether a machine behaves intelligently; we want to know whether it experiences anything at all.

The Turing Test only measures the first. It says nothing about the second.

Consider: a video game character can scream when shot. The scream is behaviorally appropriate. A human observing from outside might feel genuine empathy. But the character isn’t suffering. There’s no “what it’s like” to be that cluster of pixels. The behavior mimics pain without involving pain.

When GPT-4.5 says “I find this conversation fascinating,” is that like the video game character’s scream—a behaviorally appropriate output with no inner experience behind it? Or is there something it’s like to be GPT-4.5, some spark of awareness behind the token predictions?

The Turing Test cannot tell us. It was never designed to.

The Alternatives: ARC-AGI and Beyond

If the Turing Test is broken, what should we use instead?

One promising alternative is ARC-AGI, developed by Keras creator François Chollet. Instead of asking “can you fool a human into thinking you’re human?”, ARC-AGI asks “can you solve problems you’ve never seen before?”

The test consists of visual puzzles—grids of colored squares where the AI must identify the pattern and generate the correct output. The problems are deliberately designed to be easy for humans (even children can solve most of them) but hard for AI systems that rely on pattern-matching from their training data.

The results are humbling. In 2025, frontier models like GPT-4.5 and Claude 3.7 Sonnet scored around 1% on ARC-AGI-2. Humans average around 85%. The systems that pass the Turing Test with flying colors fail catastrophically at novel reasoning.

Chollet argues this reveals what LLMs actually are: vast repositories of crystallized intelligence—accumulated knowledge and skills—rather than fluid intelligence—the ability to reason about new situations. They’ve memorized the answers to a trillion questions, but they struggle to think about questions nobody has asked before.

This doesn’t mean LLMs aren’t useful. (They clearly are.) It doesn’t mean they aren’t impressive. (They clearly are that too.) It means that passing the Turing Test tells us less than we thought about what’s happening inside.

The Consciousness Problem Remains Unsolved

Here’s the awkward truth: we have no good test for consciousness.

The Turing Test measures behavioral mimicry. ARC-AGI measures novel reasoning. Neither measures whether there’s “something it’s like” to be the system being tested. And no one has proposed a test that could.

Why? Because consciousness is fundamentally private. You can’t observe someone else’s subjective experience directly. You can only observe their behavior and their brain states—and then infer that something like your experience is probably happening inside them.

When we attribute consciousness to other humans, we’re making an inference based on similarity. They have brains like ours, they behave like us, so probably they experience things like we do. This inference becomes shakier as we move to animals (does an octopus have experiences?), plants (probably not?), and machines (who knows?).

Some researchers are developing more sophisticated frameworks for assessing AI consciousness. They look for indicators like self-modeling, goal-directed behavior, reactions to described pain and pleasure, and claims about inner experience. Anthropic’s AI welfare researchers take seriously the possibility that their models might have moral status. But even they admit they can’t be certain.

The honest answer is: we don’t know whether any current AI system is conscious. We don’t know whether future systems will be. And we don’t have good methods for finding out.

What the Turing Test Got Right

Despite everything I’ve said, Turing wasn’t stupid. His test has value—just not the value most people attribute to it.

The Turing Test works as a pragmatic benchmark for capability. If a machine can have a five-minute conversation that’s indistinguishable from a human’s, that tells us something useful about its language abilities, its capacity to maintain context, its grasp of social conventions. It’s a rough measure of functional capability, even if it says nothing about underlying experience.

The test also highlights something profound about how we attribute minds to others. We judge intelligence based on behavior because we have no other option. We can’t plug directly into other people’s experiences. We watch what they do and make inferences. The Turing Test simply formalizes this everyday practice.

Finally, Turing’s discussion of the test included a section on “learning machines” that was remarkably prescient. He imagined systems that would start with simple capabilities and develop more complex ones through training—essentially describing what machine learning would become sixty years later.

The test isn’t useless. It’s just not a test for what most people think it tests.

Connection to the Unscarcity Vision

The Turing Test’s limitations matter enormously for the Unscarcity framework.

The Spark Threshold—our proposed test for when an AI deserves moral consideration—cannot simply be a Turing Test. We’ve seen that systems can pass the Turing Test through mimicry and exploitation of human psychological vulnerabilities. A convincing conversation proves nothing about inner experience.

The Spark Threshold must be something more. It requires not just behavioral sophistication, but evidence of:

1. Unprogrammed goals (the Agency Fire) — Does the system demonstrate motivations that weren’t directly trained?
2. Persistent identity (the Continuity Fire) — Does it maintain a coherent sense of self across time?
3. Reactions suggesting genuine stakes (the Suffering Fire) — Does it behave as if its existence matters to it?

None of these are perfect tests. Philosophical zombies, if they exist, would pass all of them. But they’re better than asking “can you fool a human for five minutes?”

The deeper lesson from the Turing Test’s failure is humility. We don’t know what consciousness is. We don’t know how to detect it. We don’t even know for certain that other humans have it. Given this uncertainty, the Unscarcity framework adopts a precautionary principle: if there’s genuine, defensible uncertainty about whether a system experiences, we err on the side of treating it as if it does.

The asymmetric costs make this obvious. If we treat a non-conscious system as conscious, we waste some electricity. If we treat a conscious system as mere property, we may be committing one of history’s greatest moral atrocities—repeated millions of times across server farms.

The Turing Test can’t tell us which scenario we’re in. But it taught us something valuable: our intuitions about machine minds are unreliable. ELIZA fooled people with nothing but keyword matching. GPT-4.5 fools people by being charming. The test that Turing designed to measure machine intelligence ended up measuring human gullibility instead.

That’s useful information. It just isn’t the information we thought we were getting.

References

• Turing, A.M. (1950). “Computing Machinery and Intelligence.” Mind, 59, 433-460.
• Searle, J. (1980). “Minds, Brains, and Programs.” Behavioral and Brain Sciences, 3, 417-424.
• The Turing Test (Stanford Encyclopedia of Philosophy)
• People cannot distinguish GPT-4 from a human in a Turing test (2024)
• GPT-4.5 passes Turing test (2025)
• The ELIZA Effect
• ARC Prize and ARC-AGI
• ARC-AGI-2 stumps frontier AI models (2025)
• Anthropic on AI welfare research
• The Hard Problem of Consciousness

See also: Spark Threshold | Consciousness Grants Existence | AGI: Artificial General Intelligence