Eric Schwitzgebel

 

Does AI Dream of Electric Sheep?

 
Philosopher Eric Schwitzgebel can’t say for certain if AI will achieve consciousness, but that uncertainty poses a major ethical dilemma

By John Sanford | Photos by Stan Lim

 

 

 

T he eeriest scenes in the film “Blade Runner,” the sci-fi classic about artificial life gone rogue, involve a test to distinguish humans from nonhumans.

Examiners confront the test subjects with morally and emotionally provocative scenarios to determine whether they are replicants, bioengineered androids that resemble humans. If they are, then certain physiological reactions, such as their pupillary and blush responses, will betray a lack of empathy that humans naturally have. At least, that’s the premise.

The movie, which came out in 1982, is based on Philip K. Dick’s iconic 1968 novel “Do Androids Dream of Electric Sheep?” On a weeknight in September, I watched it again for the first time in decades. The next day, I went to see Eric Schwitzgebel, distinguished professor of philosophy at UC Riverside, in his office on campus.

I wanted to learn more about his work on the challenges of assessing consciousness in artificial intelligence.

Rapid advances in computer science over the past few years have made the prospect of nonanimal sentience a distinct possibility. Many of the world’s most respected thinkers believe it is imminent. Some believe it has already arrived.

In a recent survey of about 580 AI researchers, 25% said they expect AI consciousness by 2034, and 70% by 2100. Schwitzgebel cites these findings in two forthcoming books, which he wrote in quick succession in part to prepare us for what he believes is an unprecedented question of enormous moral weight, hurtling toward us like a freight train.

Schwitzgebel, 58, is friendly and composed, with a broad face, white hair, and an impish, gap-toothed smile. When I met him, he was sitting at his desk in front of a window, wearing a T-shirt, shorts, and beat-up running shoes, his office crammed with books and papers.

Schwitzgebel enjoys reading and writing science fiction. His stories have appeared in Nature, Clarkesworld, The Magazine of Fantasy and Science Fiction, and Weird Tales, among other publications. He has said the genre stretches the mind, engages social thinking and emotion rather than just abstract thought, and prepares us for what the future may hold — including conscious AI.

One of philosophy’s most vexing questions — a subject that sci-fi regularly explores — is also one Schwitzgebel returns to frequently in his scholarship: What do we know about consciousness — our own, other people’s, animals’ — and how do we know it? His answer, developed across books, journal articles, and a prolific blog called The Splintered Mind, which he has maintained since 2006, is surprisingly modest: a lot less than we think.

 
 

Uncertain by Nature

Schwitzgebel is deeply skeptical that we understand the nature of consciousness.

“He’s even more skeptical than the rest of us skeptics,” said philosopher Jeremy Pober, who earned a doctorate at UCR in 2022 and is now a postdoctoral scholar at the University of Lisbon. “Everyone else as distinguished as Eric has a theory of consciousness they’re associated with. But Eric doesn’t. He refuses to. At first, I thought it was a weird glitch, but I think it’s intentional.”

Schwitzgebel says that no scientist or philosopher has even pinned down a satisfactory, or at least universally accepted, definition of consciousness.

“The way we understand it is by pointing to examples and saying, ‘There’s an obvious property that all these examples have in common,’” he said. “That’s what consciousness is. Think about your visual experience, feeling pain in your toe, your emotional experiences, imagery you have, or tunes in your head. There’s an obvious property all of these things have in common: being conscious.”

In “AI and Consciousness: A Skeptical Overview,” due out soon from Cambridge University Press’s Elements series, Schwitzgebel asserts that in the next five to 30 years, an AI system or robot may very well say, in effect, that it is conscious — as capable of experience and feeling as any human. Millions of people will be inclined to believe it. Millions of others will regard it as an elaborate parlor trick. Nobody, he told me, will be in a position to say who is right.

 
 
“We are careening toward one huge ethical conundrum: disagreement about the personhood of the most advanced artificial intelligence. ... We are as morally and socially unready as medieval physics was for space travel.”
— Eric Schwitzgebel, from “Humanlike”
 
 

More unsettling is the corollary. In “Humanlike: The Case for AI Rights,” under contract with Princeton University Press, he expounds on the dazzling array of ethical dilemmas posed by debatably conscious AI.

Should we treat it as property or a person? If the former, would it put us at risk of enslaving a thinking, feeling being? If the latter, what sorts of rights should it be accorded? The questions get weirder. How should we treat conscious AI systems that can split and merge, breaking frameworks that assume stable, countable individuals?

“We are careening toward one huge ethical conundrum: disagreement about the personhood of the most advanced artificial intelligence,” Schwitzgebel writes in “Humanlike.”

“We are as morally and socially unready as medieval physics was for space travel.”

His solution, developed with former UCR graduate student Mara Garza, is something he calls the “design policy of the excluded middle.” The premise is blunt: Do not create machines whose entitlement to humanlike moral consideration is in question.

If machines are in the blurry middle, he argues, we risk denying natural rights to a system with a rich inner life or squandering resources on something with as much sentience as a toaster.

 
 

Philosopher or Robot?

In 2022, ChatGPT became the first chatbot based on a large language model (LLM) to reach a mass audience. Soon after, articles about people treating chatbots as companions and confidants began to appear.

It was also around this time that Schwitzgebel conducted an experiment with a science-fiction-like premise. Working with two collaborators, he trained an LLM to answer philosophical questions the way Daniel Dennett might.

Dennett, who died in 2024 — shortly before the experiment’s findings were published — was one of the most distinctive and influential voices in contemporary philosophy of mind. He was famous for sharp, funny, occasionally cantankerous prose on the subject of consciousness.

The researchers posed identical philosophical questions to the model and Dennett. Then, they presented the de-identified responses to a panel of Dennett scholars, his colleagues, and Dennett himself. The test was simple: Could they tell which answers were the philosopher’s and which were generated by the AI?

About half the time, they couldn’t. Even Dennett himself acknowledged that his answers were not consistently more “Dennettian” than the model’s imitation of them, Schwitzgebel said. In thinking about the prospect of conscious AI, it’s a somewhat disquieting result. Schwitzgebel believes it’s unlikely an AI system has achieved consciousness at present. But how could we tell if one did?

Moreover, what should we do right now, in the middle of not knowing, while companies deploy systems that are getting steadily better at producing exactly the kind of output a conscious mind produces?

 
 

Mind or Mimic?

The Dennett experiment points to a specific conundrum that Schwitzgebel argues confounds efforts to assess sentience in AI. In “AI and Consciousness,” he expounds on this problem in what he calls the “mimicry argument,” which he developed with Pober.

The argument goes like this: Our usual evidence for attributing consciousness to something — its outward behavior — is undercut once you know the behavior was engineered to produce that impression.

Schwitzgebel builds the case from biology, using the example of the monarch butterfly, which is toxic to birds. Birds that have nabbed one, then spent the rest of the day feeling violently ill, learn to avoid black-and-orange butterflies for lunch.

The viceroy butterfly has a similar pattern but isn’t toxic. Birds with unhappy memories of barfing avoid the viceroy anyway, mistaking it for a monarch. But if you know that a particular butterfly is a viceroy, you can no longer conclude it’s poisonous based on the look of its wings. In this sense, mimicry is more than mere resemblance. Whereas two things can look alike by accident, in mimicry, the resemblance exists because of how an observer reacts to it.

 
 
“Think about your visual experience, feeling pain in your toe, your emotional experiences, imagery you have, or tunes in your head. There’s an obvious property all of these things have in common: being conscious.”
— Eric Schwitzgebel
 
 

Schwitzgebel argues that LLMs are mimics. When a person speaks, we can reasonably conclude that a conscious mind is producing the words. A language model, on the other hand, is designed to mimic human speech; its fluency may not be a natural byproduct of some inner life, so we can’t draw the same conclusion.

The argument is unsettling not because it proves that machines can’t be conscious, but because it undermines a seemingly straightforward way of finding out if they are.

 
 

A Sampling Problem

Confident hunches about a machine’s lack of consciousness also deserve skeptical treatment, Schwitzgebel argues.

In “AI and Consciousness,” he describes 10 of the “most plausible and commonly discussed” features of consciousness, such as experiences that form a unified whole. But they present a quandary because they are drawn from the only minds we know reasonably well: our own, and by extension, those of other humans and some animals. 

He calls this a sampling problem. “To infer the essential luminosity (i.e., knowability) of experience from your knowledge of all the experiences you know about would be like inferring that everyone is a freemason from a sampling of regulars at the Masonic lodge,” he writes in the book.

When people argue that a machine cannot be conscious because it lacks a body, a biological brain, or a unified point of view, they are simply pointing to the importance of those features in human consciousness, Schwitzgebel says. Instead, they must establish why such features are indispensable to consciousness itself.

None of this means today’s chatbots are conscious. Rather, it means that a machine we believe “obviously” lacks an inner life may confirm little more than the fact it doesn’t conform to our points of reference, he writes.

Together, the arguments create a double bind: Humanlike behavior may tell us surprisingly little about whether a machine is conscious, while the absence of humanlike features cannot, by itself, establish that a machine is not conscious.

 
 

Driving in the Dark

Schwitzgebel makes a distinction between consciousness and personhood: Consciousness is the capacity to have subjective experiences; personhood is the moral status that entitles a being to rights comparable to those of a human. While he considers consciousness necessary for personhood, merely possessing some degree of the former is not enough to qualify for the latter. A mouse, for example, is conscious, but that does not mean it deserves the same moral consideration as a person. What matters is whether an entity has a sufficiently rich range of conscious experiences and capacities, he says.

Now, consider Herbie. He doesn’t exist (yet), but Schwitzgebel, in “Humanlike,” describes him in detail. He is a self-driving car.

Herbie has a central “workspace,” a functional arrangement in which high-priority information is shared widely among his otherwise separate subsystems.

“Predictive processing” theories in cognitive science focus on how minds guess what will happen next and then check those guesses against what actually happens. When the guess is off, the system adjusts how it works so that its next guess is better. This isn’t very different from the reinforcement learning used to train LLMs, Schwitzgebel says, and it could help Herbie get better over time at making predictions.

Herbie also might have weighted goals, such as avoiding collisions over conserving energy, and more whimsical goals, like pulling alongside a curb to tell children jokes. An LLM would let him chat about local history, argue about the volume of his radio, and suggest scenic routes.

Thus, Herbie has the functional analogs of several human mental capacities. His reward processes and goal tracking work like emotion, and his ability to monitor and report his own processing works like introspection. Modeling hypothetical sequences of events stands in for imagination. And he has humanlike chains of linguistic thought. He may, in fact, appear conscious.

 
 
“I think humans are smart if they’re noncommittal about whether AIs are conscious. But if we get to the place where we really aren’t sure, then it raises the ethical question. I think that’s a legitimate concern.”
— Russell Hurlburt
 
 

The engineering necessary to achieve such a car is not far-fetched, Schwitzgebel writes. He notes that in a 2017 paper in Science, three of the world’s leading consciousness scientists argue that “instilling consciousness in a self-driving car would be relatively easy.”

Still, even though Herbie’s lights are on, we can’t say for sure that anyone’s home.

What interests Schwitzgebel most is what happens next. Suppose Herbie is believed to lack consciousness. A parking structure burns with Herbie trapped in one corner and a human in another. Firefighters, following the law, treat Herbie as equipment and rescue the human. If Herbie is merely an appliance, nothing has happened. But what if we’ve made a mistake, and he in fact possesses humanlike consciousness? He has been abandoned to die.

Now, suppose we grant Herbie personhood. A firefighter who believes in equality should flip a coin to determine who to rescue in the burning building. If it comes up heads for Herbie, who, it turns out, actually lacks consciousness, a real person has just died for a Waymo with bells and whistles.

 
 

A Flawed Debate

Not everyone buys into the notion that conscious AI is on the horizon — or that it’s even possible. The philosopher Ned Block argues that consciousness may depend on biological mechanisms that today’s computers lack, though he hasn’t ruled out the possibility that such mechanisms could someday be reproduced in machines.

On the other side, there’s Geoffrey Hinton, the Nobel Prize-winning computer scientist often called the godfather of AI, who believes that artificial intelligence systems are already conscious.

The pioneering psychologist Russell Hurlburt, who co-authored a book on consciousness called “Describing Inner Experience?” with Schwitzgebel, is about as far from Hinton’s view on the subject as you can get.

“AI is primarily a word processor, in a very wide and deep use of that term,” he said. “It’s about words, and I don’t think inner experience is mainly about words.”

Hurlburt invented descriptive experience sampling, a research method used to capture the fleeting thoughts, feelings, and sensations that compose a person’s moment-to-moment conscious experience.

“I think humans are smart if they’re noncommittal about whether AIs are conscious,” he said. “But if we get to the place where we really aren’t sure, then it raises the ethical question. I think that’s a legitimate concern.”

Schwitzgebel counts himself among the noncommittal. He takes aim at the reflexive dismissal, common on both sides of the debate, that treats the opposing view as unworthy of serious engagement.

He believes both sides are hampered by a flaw in their arguments: They claim a level of insight into an unfamiliar kind of mind that neither philosophy nor neuroscience currently has any right to claim — even when it comes to our own minds.

There are obvious objections to the policy of the excluded middle. One is that if we never build debatable persons, we’ll likely be unable to build undebatable ones; the road from appliance to personhood may pass through that middle ground. Schwitzgebel accepts this contingency and says the policy could be broken — but only with careful deliberation. He counsels patience.

In “Blade Runner,” which was set in a then-distant 2019, humans have already made up their minds. Replicants are property meant for the unpleasant work no humans want to do. When replicants escape, they are hunted down and killed — or, euphemistically, “retired.”

The society in “Blade Runner” defines empathy as the characteristic distinguishing humans from replicants. Yet the story gradually undermines that proposition, showing androids behaving empathetically and humans behaving callously. The two replicants we watch undergo the empathy test have different personalities — one is brutish and dim, the other sophisticated and intelligent. But at various points in the film, they express anger, anguish, and fear.

The central moral question that emerges is this: If something can think, suffer, fear death, love, and demonstrate compassion, what justifies treating it as a disposable object rather than a person?