On July 30, the Internet Archive announced it was turning to crowdfunding to keep its servers running. The organization that has preserved one trillion web pages — a civilization-scale backup of the open internet — is now passing the hat to individual donors because its institutional support is drying up.

The same week, The New York Times and The Guardian were quietly updating their robots.txt files to restrict the Archive’s crawlers from indexing their content, worried that the Wayback Machine was becoming a backdoor for AI training data.

You can see where this is going.

The internet is currently in the middle of one of its periodic fits of self-mourning. A Walrus essay titled “Google Search Is Dying” has been making the rounds on Hacker News, arguing that AI-generated summaries are replacing the link-based web and, with it, our collective memory. Google’s May 6 rollout of AI Mode — which tucks citations behind a conversational interface and nudges users toward follow-up questions rather than actual websites — has given the lament a convenient villain. The machines are eating the archive.

But the archive was already being starved. And the people starving it are, in many cases, the same people writing the elegies.

The Newspapers Want It Both Ways

The New York Times has run no shortage of pieces worrying over link rot, digital decay, and the fragility of online journalism. The Guardian has built a brand around being the paper of record for the progressive internet. Both outlets have sued AI companies for training on their content without permission — a legally coherent position, whatever you think of the merits.

But blocking the Internet Archive is a different category of decision. The Archive is not a commercial AI trainer. It is a nonprofit library, recognized as such by the state of California, operating under a controlled digital lending framework that courts have scrutinized and, in key respects, upheld. When the Times restricts the Archive’s crawler, it is not protecting its IP from a competitor. It is ensuring that a 2024 investigation into, say, nursing home deaths will be unretrievable in 2034 if the Times ever decides to take down the page or restructure its CMS — which, as anyone who has worked in digital publishing knows, happens constantly.

One archivist who works on the Wayback Machine’s crawl infrastructure put it in a Slack channel for digital preservation professionals: “Every major paper has killed at least one article I’ve tried to retrieve this year. Not paywalled — gone. Redirected to a section front. And we’re the bad guys?”

What Actually Disappears

The AI-overviews panic has a way of making the problem sound abstract and futuristic. It is neither. The United States has lost nearly 3,500 newspapers since 2005. Two hundred and thirteen counties have no local news outlet at all. When a small-town paper folds, its digital archive — decades of city council meetings, obituaries, high school sports scores, zoning decisions — typically vanishes within months. The hosting bill stops getting paid. The domain lapses. The Wayback Machine is often the only copy left.

That is not an AI problem. That is an incentives problem. And the incentives are not mysterious: preserving old content costs money and generates no revenue. News organizations, even the prestige ones, are businesses. They optimize for the present. The past is a liability — something that might contain an embarrassing correction request, an unflattering quote, a story that complicates the current editorial line.

AI didn’t create that dynamic. It just gave everyone a more satisfying thing to blame than their own indifference.

The Real Memory Crisis Is Boring

Google’s AI Overviews may well reduce traffic to informational websites by 34 to 61 percent, as multiple independent studies now suggest. That is a genuine business problem for publishers who built their models on search-referred ad revenue. But traffic is not memory. A page that gets fewer visitors still exists. A page blocked from the Internet Archive does not.

The distinction matters because the solutions are different. Fixing the traffic problem requires publishers to produce content that AI cannot satisfactorily summarize — first-hand experience, proprietary data, strong opinion, decision-stage depth, as Google’s own guidance now recommends. Fixing the memory problem requires publishers to stop treating the Internet Archive like a threat and start treating it like the public utility it actually is.

That is a less glamorous argument than “AI is eating our collective soul.” It does not generate 137 points on Hacker News. It does not make for a compelling magazine feature with a moody illustration of a search bar crumbling into pixels. But it has the advantage of being true, and it has the advantage of being something we could actually fix — tomorrow, with a robots.txt edit, at no cost to anyone’s bottom line.

The internet’s memory is not disappearing because AI is eating it. It is disappearing because the people who own it have decided, quietly and rationally, that remembering is not worth the trouble. The machines are just the excuse.

Sources