What happened
Claude Opus 5's new effort settings — low, medium, high, xhigh, and max — were supposed to be a simple dial: more effort, better output. That assumption is now falling apart. Data from FrontierCode, a coding benchmark, shows scores actually decline once you push past the "high" setting. Anthropic's own migration guide, buried in the model's system card, admits the reason: at higher effort levels, the model starts making unnecessary refactors and edits outside the scope it was given. In other words, it overthinks simple tasks and rewrites code nobody asked it to touch.
This isn't an isolated finding. On the closed-book AA-Omniscience benchmark, Opus 5 is roughly 11% more accurate than its predecessor, Opus 4.8. But its hallucination rate is also about 6% higher. More reasoning steps mean more opportunities to sound confident while being wrong.
CodeRabbit ran its own test, using Opus 5 at the xhigh setting against its production code-review baseline. Precision on actionable comments improved, climbing from 35.2% to 39.3%. But the model caught fewer of the benchmark's known issues — 55.2% versus 61.1% previously — and it generated roughly four times as many nitpick comments. That's a worse signal-to-noise ratio for anyone actually trying to ship code.
The flip side matters just as much. On Zapier's AutomationBench, Opus 5 running at its lowest effort setting still outperforms every other model tested, at any setting. For a large share of everyday tasks, the cheapest option on the dial is already the best one.
Why it matters
The assumption that "more compute equals better results" is baked into how most people use reasoning models. Effort sliders, extended thinking modes, and reasoning tokens all get marketed the same way — turn it up when the problem is hard. But this data suggests that past a certain point, the model isn't reasoning more carefully. It's inventing scope. It second-guesses instructions, drifts into refactors nobody requested, and pads its own output with speculative changes.
For teams paying by the token, this is a direct cost problem: you can be paying premium prices for a demonstrably worse result. For teams relying on AI code review or fact-based outputs, it's a quality problem — more hallucinations, more nitpicks, fewer actual bugs caught. And because there's no published map of where each model's ceiling sits, most people find out the hard way, after burning budget on "max" settings by default.
There's also a quieter issue worth flagging: when a safety classifier flags a request inside Claude.ai, Claude Code, or Cowork, the system silently falls back to Opus 4.8 instead of the model you selected. Anthropic's own Frontier-Bench comparisons were run under this same fallback behavior, according to a footnote on their published chart. Nobody has disclosed what percentage of real-world requests get quietly downgraded this way, which makes benchmark numbers harder to trust at face value.
How to use it today
If you're using Opus 5 for coding tasks, start at "high" and only escalate if you have a specific, well-defined reason to believe the task needs deeper reasoning — not as a default. Test both directions: try a task at "medium" before assuming you need "xhigh" or "max". For code review workflows specifically, expect xhigh to raise precision but also flood you with nitpicks — budget time to filter them out, or set stricter prompts about scope.
For lighter workloads — drafting copy, formatting data, simple automations — the lowest effort setting is frequently sufficient, per the AutomationBench results. If you want to experiment with lightweight AI tasks without committing to a paid tier or complex setup, a free tool like the ones at mykreatool.com is a reasonable place to test where your own ceiling sits before scaling up to heavier, more expensive settings.
Track your own benchmark. Anthropic hasn't published a universal map of where each effort level stops improving results, and it likely varies by codebase, prompt style, and task type. Run the same task across two or three effort levels on your actual work before deciding on a default.
Who benefits
Developers and engineering teams running Opus 5 for code generation or review stand to save the most, both in token spend and in review overhead, once they stop defaulting to max effort. Solo founders and indie developers using AI coding assistants for day-to-day feature work will see the clearest gains, since unscoped refactors are exactly the kind of noise that slows down small teams without dedicated code review processes.
Product teams building on top of Claude's API also benefit from knowing this, since it changes how you'd configure effort levels per endpoint or per feature rather than hardcoding one setting across an entire product. Anyone running automation pipelines — the Zapier AutomationBench use case — benefits from confirming that low effort settings are enough, cutting costs without losing task completion.
Risks
The most direct risk is financial: paying premium per-token rates for max effort, only to get lower benchmark scores and messier code changes. The second risk is silent quality drift — higher hallucination rates on knowledge tasks at higher effort settings mean confident-sounding wrong answers can slip into production content or decisions more easily.
There's also the fallback risk: if a request gets flagged and silently rerouted to Opus 4.8, your output quality changes without any visible warning, and you have no way of knowing how often this happens. Finally, relying on a single vendor benchmark (FrontierCode, AA-Omniscience, CodeRabbit's internal test) to make settings decisions is itself a risk — these numbers are directionally useful, but nobody has published a comprehensive map for how effort settings interact with your specific codebase or industry.
Conclusion
Opus 5's effort dial isn't a simple "more is better" slider — on coding tasks, performance actually peaks around the "high" setting and drops off at xhigh and max, largely because the model starts making unscoped edits and refactors. Combined with a higher hallucination rate at greater reasoning depth and Zapier's finding that even the lowest effort setting wins on automation tasks, the practical takeaway is clear: test your own ceiling before defaulting to max, since the wrong setting can mean paying more for a worse result.
Comments 0