Security

Humans Caught Only 13.6% of Dangerous Commands. From Aug 14, AI Blocks Them Instead.

Starting August 14, Claude Code Auto Mode becomes the default for Pro, Max, and Team. The automated classifier caught 89% of dangerous commands versus 13.6% for manual approval. Here is the background behind the surprising numbers and how to configure it.

· Huuush

Humans Caught Only 13.6% of Dangerous Commands. From Aug 14, AI Blocks Them Instead

Key takeaways

  1. 1Starting August 14, Auto Mode becomes the default for Claude Code Pro, Max, and Team users with no setup required.
  2. 2In a blind controlled experiment with 1,053 paid developers, humans caught 13.6% of dangerous commands and a separate classifier (Auto Mode) caught 89%, and the harm in real sessions was 6.3% and 2.4%, respectively.
  3. 3Human blocking rates fall from about 17% early in a session to about 5% after more than 50 prior prompts, while Auto Mode stays steady regardless of session length.
Image of a red glass panel facing a black keyboard keycap, symbolizing an auto-approval gate
Generated image symbolizing a command passing through an approval gate

Starting August 14, Auto Mode becomes the default for Claude Code Pro, Max, and Team users without them turning it on. The evidence ClaudeDevs shared is a surprise: humans caught only 13.6% of dangerous commands, while a separate classifier caught 89%.

An Experiment With 1,053 People: A Sixfold Difference

According to Anthropic's official announcement, these figures came from a controlled experiment with 1,053 paid developers. Participants were blind to which actions were being tested. The results are as follows.

GroupDangerous command detection rateHarm in real sessions (production severity)
Humans (manual approval)13.6% (143 of 1,053)6.3%
Auto Mode (classifier)89% (937 of 1,053)2.4%

Why People Miss Them: Fatigue, Not Skill

More striking is the gap that opens as sessions get longer. The human blocking rate falls from about 17% early in a session to about 5% once there are more than 50 prior prompts. People get used to repeated approvals and their attention drops, while Auto Mode's blocking rate stays steady regardless of session length.

Not Automation Without Brakes

Safeguards came along with it. Commands that could lead to data exfiltration are hard-blocked unless the user overrides, git state is checked before destructive actions, and content coming in from outside is screened for prompt injection. After 3 blocks in a row or 20 blocks in a session, it switches automatically to manual mode.

What to Check Now: Who's Affected and How to Turn It Off

The default switch applies only to Pro, Max, and Team. Enterprise, API, AWS Bedrock, Google Cloud Agent Platform, and Microsoft Foundry remain opt-in for now, giving admins time to review. In the CLI you can switch with Shift+Tab or the mode dropdown, and admins can set organization-wide policy with defaultMode or the disableAutoMode flag in managed settings. There is no extra cost for using the classifier.

Sources: ClaudeDevs's X post · Anthropic official announcement

Frequently asked questions

When does Auto Mode become the default, and for whom?
It applies to Claude Code Pro, Max, and Team users starting August 14. Enterprise, API, AWS Bedrock, Google Cloud Agent Platform, and Microsoft Foundry remain opt-in for now.
How big is the gap between humans and the classifier at catching dangerous commands?
In the experiment with 1,053 participants, humans caught 13.6% (143 cases) and the Auto Mode classifier caught 89% (937 cases), about a sixfold difference.
Why can't people catch dangerous commands?
It is fatigue, not skill. As people get used to repeated approvals, their attention drops, and the blocking rate falls from about 17% early in a session to about 5% after more than 50 prior prompts.
Does Auto Mode have safeguards?
Commands that could lead to data exfiltration are hard-blocked with no user override, git state is checked before destructive actions, external content is screened for prompt injection, and the session switches automatically to manual mode after 3 blocks in a row or 20 blocks in a session.
Can I turn Auto Mode off or manage it at the organization level?
In the CLI you can switch with Shift+Tab or the mode dropdown, and admins can set organization policy with defaultMode or the disableAutoMode flag in managed settings. There is no extra cost for using the classifier.

Related articles