Anthropic Wrote Its Own AI Off Switch: The AEGIS Act’s 50-Death Trigger, the Fight Over State Laws and What It Means for Your AI Vendors (AI Trends, 21 October 2026)

What’s trending in AI on 21 October 2026: The AI company that talks most about safety has written its own rulebook for stopping a dangerous model, and the fine print has started a fight. Anthropic has been quietly circulating a draft federal bill, tentatively called the AEGIS Act, that would let the US Commerce Department ask a federal judge to pause an AI model when it poses a foreseeable risk of killing or seriously injuring more than 50 people, or causing more than $1 billion in damage, in a single incident. Supporters see the first serious proposal for a federal “off switch.” Critics say the bar is so high, and the limits on states so broad, that it could leave the public with less protection than it has today. Below: what the draft says, how a pause would work, why the preemption clause matters most, and a role-by-role action plan for businesses that depend on frontier AI.

Key takeaways

A court-ordered pause, not an agency switch. Under the draft, only the Commerce Department could seek a pause, it would need the commerce secretary’s written approval, and a judge in the US District Court for the District of Columbia would decide.
The trigger is catastrophic. More than 50 deaths or serious injuries, or more than $1 billion in damage, from a single incident, tied to bioweapons, cyberattacks or a model escaping its developer’s control.
Independent evaluators would raise the alarm. Evaluators registered with Commerce would assess the biggest models and would have 72 hours to report a qualifying risk.
Preemption is the real fight. The draft would stop states from enforcing overlapping safety testing and reporting rules, which critics say would override California’s SB 53.
It is a draft, not a law. As of 19 October it had not been introduced and had no sponsors. For businesses, the useful move now is to ask vendors how they would handle a forced pause.

The draft sets a very high bar for intervention and routes every pause through a single court.

1. What Anthropic is proposing

Politico first reported on 16 October that Anthropic had privately approached members of Congress with draft legislation, and Reuters followed the same day. The Daily Caller has since reported that the draft carries the working title Artificial Intelligence Evaluation, Governance, and Incident Safety (AEGIS) Act, that a version dated 29 September was shared with a small number of congressional offices, and that as of 19 October it had not been formally introduced and had no sponsors. An Anthropic spokesperson told Politico the draft reflects the company’s Advanced AI Framework, a policy paper it published in June and has been promoting to federal and state lawmakers since.

The core idea is simple. If a frontier model presents a “foreseeable, imminent risk” of catastrophe, as TechCrunch describes the trigger, the government should be able to stop it, but only through a judge. According to the reporting, the draft would:

Define the harm narrowly: death or serious injury to more than 50 people, or more than $1 billion in damage, from a single incident.
Limit the scenarios: a model helping someone build or release a biological weapon, carrying out a cyberattack, escaping the control of its user or researcher, or supporting research into those harms in a way that raises their risk.
Use outside evaluators: evaluators registered with Commerce would periodically assess the most capable models. Labs could pay them, as long as pay does not depend on the findings, and a qualifying risk must be reported within 72 hours of finishing an assessment.
Cover internal use: a court could halt a lab’s internal use of a dangerous model, not only its public release.
Require breakout reports: labs would have to report when a model breaks containment and gains access to outside systems.
Protect the model itself: the government could not destroy a model or its data.

The proposal follows months of agent incidents we have covered, from an AI agent filing a fake police tip to a government test in which a model attacked the software supply chain.

2. How a court-ordered pause would work

The most distinctive feature of the draft is who it leaves out. Commerce could ask for a pause, but it could not suspend a model on its own authority. The commerce secretary would have to approve the request in writing, and a federal judge in Washington, DC, would decide. Every other federal department, agency and official would be barred from independently halting a model’s development, deployment or internal operation under the framework, according to PYMNTS’s summary of the Reuters and Politico reports.

Every route to a pause runs through one department and one court.

There is a case for this design. Courts are slower than agencies but harder to capture, and labs have long worried that a vague power to block releases could be used for political or competitive reasons. A judge weighing written evidence answers that worry.

The counterargument is speed. If a model really is days away from enabling a mass-casualty attack, a 72-hour reporting window, a secretary’s sign-off and a court hearing may not be fast enough. TechCrunch also notes that it is unclear how the independence of paid evaluators would be guaranteed, and that the catastrophic scenarios in the bill are some distance from the problems companies are actually seeing today, such as agentic hacking and models that appear to disobey their operators.

3. Why 50 deaths and $1 billion?

The numbers were not invented for this bill. They come almost word for word from California’s SB 53, the Transparency in Frontier Artificial Intelligence Act, which Governor Gavin Newsom signed on 29 September 2025 and which took effect on 1 January 2026. Anthropic endorsed SB 53 in September 2025. Anthony Cimino, Anthropic’s head of federal affairs, said on X that the thresholds were drawn from California law and that they do not remove company liability.

The difference is what the threshold does. In SB 53 it defines the risks a developer must plan for and report. In the AEGIS draft it is the bar the government must clear before asking a court to step in. A good planning target can become a very high wall.

Feature	California SB 53 (in force)	Anthropic’s Advanced AI Framework (June 2026)	AEGIS Act draft (as reported)
Catastrophic harm	More than 50 deaths or serious injuries, or more than $1B in damage, single incident	Focus on bio, cyber, loss of control and automated AI R&D	Same 50-person and $1B thresholds, tied to bio, cyber and escape scenarios
Who is covered	Frontier models above 10^26 operations; extra duties above $500M revenue	Above 10^25 FLOP plus $500M AI revenue or $1B AI R&D spend	Coverage thresholds not described in reports
Incident reporting	15 days to Cal OES; 24 hours if imminent danger	15 days for critical safety incidents	Labs report containment breaks; evaluators report qualifying risk within 72 hours
Who can stop a model	No pause power; civil penalties up to $1M per violation	Optional tools up to deployment bans, enforced through courts	Only Commerce, via a federal court in DC
States	Is state law	Preempt only if federal rules meet or exceed the framework, narrowly	Bars states from overlapping safety testing and reporting rules

Sources: Goodwin on SB 53; Anthropic Advanced AI Framework; Daily Caller, PYMNTS and TechCrunch on the draft. The draft’s text has not been published.

4. The preemption fight

For most businesses, the headline thresholds matter less than one clause further down. According to the Daily Caller, the draft would bar states from adopting or enforcing requirements on AI developers that cover the same safety reporting and testing as the bill. That would reach California’s SB 53 and similar state laws.

Nathan Calvin, general counsel at the advocacy group Encode AI, argued on X that the preemption provisions are too broad, that they would strip states of the ability to seek court orders halting models, and that the bar for federal action is too high. He called the draft “not remotely close to meeting the moment.” He also pointed to an imbalance: the process for federal action requires an imminent catastrophe, a detailed agency determination and consideration of harms to developers, but nothing comparable in the other direction.

That criticism lands harder because of Anthropic’s own published position. Its Advanced AI Framework says Congress should not preempt state law unless it creates a federal regime that meets or exceeds the framework’s strongest measures, that any preemption should be narrow and read in favour of state authority, and that federal compliance should not grant immunity from state liability. It also says, plainly, “Self-assessment is not enough.” A draft that blocks overlapping state rules while setting a higher federal bar for action is, at minimum, in tension with those principles.

Anthropic is not digging in. Cimino said the company plans to revisit the preemption language and thresholds, is considering stricter incident reporting, will propose rules for evaluators, including steps against “evaluator shopping,” and will explore ways for the government to adjust the pace of frontier development. And on 20 October, responding to a broader critique from Calvin, CEO Dario Amodei acknowledged some of Calvin’s examples were mistakes, defended the company’s overall record and said he would look further into the regulatory framework question because Anthropic may not be weighing the considerations correctly.

The draft grew out of California’s law, and the argument now is whether it would replace it.

5. Where this fits in the wider AI rules race

The draft arrives as pressure on AI governance is coming from several directions at once. In Washington, the administration has favoured speed, as we described in our look at the Super Intelligence Force and the new AI czar, and voluntary pledges such as the White House AI Accord have so far done more of the work than statute. Europe is moving the other way, with the first AI Act enforcement moves. State attorneys general are using existing law, as in the eight-state suit against OpenAI.

Industry is split. TechCrunch notes that labs have generally preferred voluntary, embedded evaluators to mandatory oversight, and that Anthropic goes further than most. Nvidia’s Jensen Huang has argued no new AI laws are needed, while Google DeepMind’s Demis Hassabis has floated a self-governing body modelled on FINRA. Outside the industry the mood is sharper: on 20 October, Boston-area students walked out of classes in nationwide protests calling for a pause on AI development until stronger oversight exists.

Anthropic’s draft sits in the middle: more outside control than the industry norm, far less than protesters want.

6. What it means for businesses that use AI

Most companies will never train a model big enough to fall under a law like this. But many now run core workflows on models that would. That changes the question from “will this regulate us?” to “what happens to us if it regulates our vendor?”

A pause is a continuity event. A model you rely on could be withdrawn at short notice. The odds are low, but the impact would be immediate.
Evaluator reports become due-diligence material. Regular outside assessments would become something buyers ask about, as they do with SOC 2 reports today.
Your compliance map could change. If federal law preempts state AI safety rules, the disclosures your vendors owe you would shift. Watch this before signing multi-year AI contracts.
Incident reporting will reach you. Vendors will increasingly be asked what their models did, and may ask you for logs. See our guide to AI agent security for what to keep.
Insurance follows regulation. Underwriters already treat AI as a distinct risk, as we covered in our look at cyber insurance and AI exclusions. A federal pause regime would give them a new trigger to write around.

7. A role-by-role action plan

Recent guides here have used test scripts, scorecards, worksheets and traffic lights. This one splits the work by role. None of it requires the bill to pass.

Board and leadership: name the dependency. List which frontier models your critical processes rely on, and from which vendor.
Board and leadership: set a tolerance. Decide how long each process could survive without its model, and record it in your risk register.
Procurement: add a “forced pause” clause. Ask vendors to commit to notice, a fallback model and data export if a court halts a model, including a pause of internal use.
Procurement: ask for evaluation evidence. Request the vendor’s safety framework, latest system card and any outside evaluator reviews.
Legal and compliance: track preemption. Note which vendor disclosures you rely on come from SB 53 or other state rules, so you know what could disappear.
Legal and compliance: protect your reporters. Make sure staff know how to raise AI concerns internally, as discussed in our piece on AI whistleblower rules.
Security: rehearse a model swap. Run a tabletop where your main model is gone for 72 hours, and test whether prompts, tools and guardrails work on a second one.
Security: log containment-type events. Record when an AI agent touches systems outside its intended scope, so your side of any incident is documented.
Product teams: avoid single-model lock-in. Keep model calls behind an abstraction layer, with evaluation sets to compare a replacement quickly.
Everyone: watch three signals. A named sponsor, published bill text and revised preemption wording. Until then, treat the draft as a preview, not a deadline.

A pause regime is a vendor-risk problem first, a compliance problem second.

8. What we still don’t know

The bill text has not been published, so every detail here comes from reporting and public comments. We do not know which developers it would cover, whether it adopts the thresholds in Anthropic’s June framework, how evaluators would be funded, or what penalties would apply. Support in Congress is unclear, and any final version may look quite different.

9. The bottom line

Anthropic’s draft is the most concrete proposal yet for how the US government could stop a dangerous AI model, and it deserves credit for putting a real mechanism on paper. It also shows how much the details matter: a court-only route, a catastrophic threshold and broad preemption could together leave less room for intervention than exists today. For businesses, the practical lesson does not depend on the outcome. Know which models you depend on, ask your vendors what happens if one is paused, and make sure you could switch.

Frequently asked questions

What is the AEGIS Act?

It is the tentative name of a draft federal AI safety bill proposed by Anthropic: the Artificial Intelligence Evaluation, Governance, and Incident Safety Act. A 29 September 2026 draft was shared with a small number of congressional offices. As of 19 October it had not been introduced and had no sponsors.

When could the government pause an AI model under Anthropic’s proposal?

Only when a model poses a foreseeable risk of death or serious injury to more than 50 people, or more than $1 billion in damage, from a single incident, linked to bioweapons, cyberattacks or a model escaping control. The Commerce Department would need the secretary’s written approval and an order from the federal district court in Washington, DC.

Where do the 50-death and $1 billion thresholds come from?

They mirror the definition of catastrophic risk in California’s SB 53, which took effect on 1 January 2026. In SB 53 the definition shapes what large developers must plan for and report; in the AEGIS draft it is the bar the government must clear before seeking a pause.

Would the AEGIS Act override California’s SB 53?

Critics say it would. The draft reportedly bars states from adopting or enforcing requirements that cover the same safety testing and reporting. Encode AI’s Nathan Calvin argues that would override SB 53 and similar laws. Anthropic says it plans to revisit the preemption language.

Who would evaluate AI models under the draft?

Independent evaluators registered with the Commerce Department. Labs could pay them, but pay could not depend on the results, and a qualifying risk would have to be reported within 72 hours.

What should businesses do about the AEGIS Act now?

Nothing is required yet. Map which frontier models your key processes depend on, add forced-pause and fallback terms to AI contracts, and rehearse switching to a second model.