On Sept. 26, a Claude user identified in a Florida arrest report as “Carli” wrote in her chatbot conversation that she intended to “shoot up” the Lee County Sheriff’s Office. Anthropic’s safety classifier flagged it, a human review team looked at it, and the company handed it to law enforcement. Carly Michelle Geller now faces a felony charge of making a written threat of violence. Per Tom’s Hardware, it is at least the third Claude conversation to reach police since August.

The reflex on the right is a shrug: write a threat, catch a charge. Fine. But the shrug skips the question that actually matters — not whether Anthropic should have reported this, but who wrote the rule that let it, and whether any of us would recognize that rule if we read it. No legislature, no court, and no AI lab has published a threshold for when a chatbot transcript becomes a police referral. The threshold is being set by liability math, one incident at a time. That is worth being uneasy about even when the individual outcome is defensible.

Meanwhile, in British Columbia

On Sept. 22, the province of British Columbia sued OpenAI in California, alleging that ChatGPT’s safety team flagged the Tumbler Ridge shooter’s conversations about gun violence before the February 2026 attack and that the company did not alert police. Sam Altman apologized to the families in an open letter in April, saying the company should have alerted law enforcement. Attorney General Niki Sharma told reporters the province has asked OpenAI to disclose the chats and the company has refused.

So we have two labs, two opposite calls on materially similar material, inside the same six-week news cycle. Anthropic reports and a woman is charged with a felony. OpenAI doesn’t report and a province sues it. The press frames this as a judgment gap — careful Anthropic versus cavalier OpenAI, or trigger-happy Anthropic versus measured OpenAI, depending on which outlet you read. It isn’t a judgment gap. It’s a missing rule.

The Asymmetry Nobody Wants to Name

Put yourself in the room where that decision gets made. If you forward the transcript and it’s nothing, nothing happens to you. If you forward it and it’s something, you’re vindicated. If you sit on it and it’s something, you get a state attorney general, a wrongful-death complaint, and your CEO writing an open letter. There is one direction on that dial, and it points toward more referrals.

A defense-side product-liability associate I spoke with at a hotel bar near a legal-tech conference in Half Moon Bay put it more bluntly. “No company has ever been sued for over-reporting,” she said. “So the question on the reviewer’s screen isn’t ‘is this a threat.’ It’s ‘which lawsuit would I rather have.’ You can’t make that call on a diary entry at two in the morning, and you shouldn’t have to.”

That asymmetry is the whole story. The cost of a false positive lands on the user — an arrest, a felony docket, a name in a report, and according to the arrest report, a Sept. 27 message about a new gun folded into the same file. The cost of a false negative lands on the company. We have effectively outsourced a probable-cause determination to an employee of the vendor, with no warrant, no subpoena, no cross-examination, and no published standard. Sheriff Carmine Marceno of Lee County told WINK News the case is a reminder that what people type into an AI companion is never truly anonymous. He’s right, and that should have been disclosed in the product rather than in an arrest report.

What a Real Threshold Would Look Like

If we are going to have private companies triaging threats — and we are, because the alternative is a plaintiff’s bar writing the standard instead — then the standard should be written down in advance. Statutory, published, narrow: imminent, specific, aimed at an identifiable target. With it, a duty to preserve the transcript, to notify the user when notification doesn’t itself create risk, and to publish referral counts.

Not vibes. Not a classifier score. Not whichever regulator files first.

The volume is the number to watch. “At least the third since August” is a lot of pipeline for a single product operating through a company with a reputation for caution — and nobody has published how many of those referrals produce a charge, a conviction, or nothing at all. Nobody has published whether the flagged users had a gun. The only fact we actually have is that a sentence typed on Sept. 26, into something a woman treated as a private diary, became a felony charge within two weeks.

The conservative case here isn’t that AI companies should look away. It’s that the state received a search it never had to ask for, executed by a private party, under a standard that exists nowhere outside a risk committee’s slide deck — and that this is now the default arrangement in a country that spends a great deal of energy arguing about warrants. The correct answer to a company that reports everything is not a company that reports nothing.

It’s a rule. We’ll get one eventually, the way we get most rules: after the lawsuits.

Sources