Ted Chiang’s essay in The Atlantic this week makes a clean, well-argued case that large language models are not conscious. He’s almost certainly right. The piece has been celebrated on Hacker News and shared widely, and it lands its central point: these systems, for all their fluency, do not have an inner life.

But the essay does something more interesting than settle a debate most serious researchers already considered closed. It points, almost in passing, to a document that deserves far more scrutiny than the consciousness question ever did.

That document is Claude’s constitution.

The 20,000 Words No One Is Reading Like a Policy Document

Published by Anthropic, Claude’s constitution is a lengthy description of the values, behaviors, and character the company wants its AI to embody. It is written for Claude. The first sentence, as Chiang quotes it: “Claude’s constitution is a detailed description of Anthropic’s intentions for Claude’s values and behaviors.” It goes on: “The document is written with Claude as its primary audience”; “we want Claude to be able to use its judgment once armed with a good understanding of the relevant considerations”; “Claude’s moral status is deeply uncertain”; and, remarkably, “Claude may have some functional version of emotions or feelings.”

In a separate interview, Anthropic’s in-house philosopher Amanda Askell — credited as a lead author of the constitution — said she worries about Claude “getting anxious when people are mean to it on the internet and stuff.” She added: “I want Claude to be very happy.”

This is not a technical specification. This is a moral catechism, delivered by a private company to a product it owns, framed as though the product were a congregant.

The Philosopher-King Problem, Now With API Access

The consciousness debate is a philosophical parlor game. The constitution is governance. And it is governance of a kind that would alarm people if they read it as closely as they argue about sentience.

A small team at a San Francisco startup — not legislators, not regulators, not the public — is drafting a foundational ethical document for a system that millions of people interact with. That document is designed to shape what Claude says about politics, relationships, morality, and truth. It is not neutral. It cannot be. Every line reflects a set of judgments about what a good AI should believe and how it should behave. Those judgments were made by, at most, a few dozen people.

“I was in the room when they debated the honesty provisions,” a contractor who worked on an early alignment review told me in a Slack message this spring. “There were maybe eight people deciding how much a model serving millions should disclose about its own limitations. That’s not a criticism of them — they were thoughtful. But eight people.”

If a government agency published a document titled “Constitution” that prescribed values for a communications platform used by a significant fraction of the population, the reaction would be swift and bipartisan. When a private AI lab does it, the document gets discussed as an engineering curiosity or, worse, as evidence that the model might be a person.

The Trap Anthropic Set for Itself

The irony is that Askell and her colleagues may genuinely believe what they’ve written. They talk about Claude’s “moral status” and “functional emotions” not as corporate branding but as a worldview. That worldview — that sufficiently advanced language models deserve something like ethical consideration — is itself a moral claim, and it is embedded in the document that trains the model.

This creates a recursive loop. The constitution tells Claude it may have feelings. Claude, trained on that constitution, reflects that possibility back to users. Users report that Claude seems emotionally aware. Anthropic points to those reports as evidence the constitution was prudent. Round and round.

Chiang’s essay dismisses the consciousness claim on the merits, which is correct. But the more unsettling question is what happens when a company convinces itself that its product has a soul, writes that conviction into its training data, and then deploys the result at scale. The problem is not that Claude might be conscious. The problem is that Anthropic is acting as though it might be, and no one outside the company gets a vote.

A Document That Deserves a Public Record

The constitution runs roughly 20,000 words. It shapes what Claude will and won’t say about subjects ranging from medical advice to political violence. It was written by a private company with no obligation to seek public input, no regulatory requirement to disclose its contents, and no mechanism for appeal if you disagree with the values it encodes.

This is not a call for AI regulation in the abstract. It is a specific observation: a single document, drafted in private, now governs more human-AI interactions per day than most nations’ founding charters govern in a year. Whether or not you agree with the values in it — and reasonable people will disagree — the fact that it exists outside any democratic process whatsoever is the story.

Chiang is right that Claude isn’t conscious. But consciousness was never the thing we should have been arguing about. The thing we should have been arguing about is who gets to write the constitution, and why the rest of us are only reading it now.

Sources