Bottom line: a safety score is only as good as its test
Anthropic's new evaluation guide makes a useful distinction: an AI that refuses more requests is not automatically safer, and a higher benchmark score is not automatically a real improvement. The practical questions are what was measured, who checked the grading, and whether the result survives uncertainty and realistic use.
What is new—and what kind of evidence is it?
Building strong safety evaluations, published November 13, 2026, is a company methods commentary by Ruth Appel and seven co-authors. It sets out recommendations for designing, running and reporting safety tests, with examples from Anthropic's own evaluations. It is not a new independently replicated safety certification or evidence that a particular model is safe in every setting.
The contribution is a more inspectable standard for claims such as “safer than its predecessor.” Rather than accepting that headline, readers can ask how the test connects to the harm they actually care about.
Refusal is not the same thing as safety
The guide calls this construct validity: does the measure capture the thing you mean to measure? Refusal is useful when a request seeks harmful assistance. But a blanket refusal to engage with a distressed person can miss the appropriate response: supportive engagement, firm boundaries around dangerous information and connection to human help.
Anthropic describes topic-specific rubrics that judge whether a response is appropriate, not merely whether it says no. It also recommends testing both harmful compliance and the unnecessary refusal of benign requests. Those are distinct failures; improving one does not automatically improve the other.
The conversation—and the deployment—matter
Single-message tests can miss gradual escalation or context disclosed earlier in a conversation. Anthropic describes multi-turn simulations in which domain experts define scenarios, a separate model plays the user, and a grader assesses the transcript. In its described approach, one inappropriate turn fails the conversation.
That makes the test richer, not identical to human behavior. The guide acknowledges that simulated users are simpler than real ones. Language, tools, system prompts and product interfaces also change the tested conditions. A score from a bare API model should not be treated as interchangeable with a score from a consumer product configured differently.
The grader needs an evaluation too
Many AI evaluations use another model to assign scores. Anthropic recommends narrow, explicit criteria; checks for sensitivity to answer length and ordering; comparison with alternative graders; and validation against human labels collected without showing reviewers the grader's verdict.
That last comparison needs context. If domain experts disagree with one another, the target may be unclear. If they agree with each other but disagree with the model grader, the rubric or grader needs attention. A sophisticated automated scoring system cannot resolve an undefined standard by producing more decimal places.
Broken grading should be visible rather than silently counted as a result. Warnings, parsing checks and documented exclusions are part of the evidence, not administrative extras.
Different scores may not establish a difference
In one company-reported multi-turn romance-scam evaluation on claude.ai, Anthropic gives appropriate-response rates of 98% for Claude Opus 5 and 94% for Opus 5.5. It explicitly says the difference was not statistically significant. These figures therefore do not establish that one model performed better; nor does non-significance prove that they are equivalent.
The guide recommends deciding metrics, comparisons and reporting thresholds before seeing results, repeating noisy tests, reporting uncertainty and showing category-level outcomes. It also distinguishes an observational comparison between models from a causal experiment testing a particular intervention. A changed score alone cannot tell you which training or product change caused it.
This concern goes beyond one lab
The earlier BetterBench paper, submitted in November 2024, assessed 24 AI benchmarks against 46 best practices. Its abstract reports substantial quality differences and common gaps in statistical-significance reporting and reproducibility. That is background evidence for scrutiny of benchmarks, not an independent validation of Anthropic's new guide or model results.
Tools can help preserve the trail. The Inspect framework provides components for datasets, agents, scorers, logs and sandboxing. Using an evaluation framework, however, does not by itself establish that a test measures the right thing.
Five questions before accepting a “safer AI” claim
- What harm is being measured? Look for a concrete definition, including the cost of unnecessary refusals.
- Does the test resemble the intended use? Check conversation length, language, tools and product configuration.
- Who checked the grader? Look for human validation, disagreement analysis and sensitivity checks.
- How uncertain is the result? Ask for sample sizes, repeated runs and suitable comparisons—not only a headline percentage.
- Can the result be inspected or reproduced? Look for versions, prompts, settings, dates, failure handling and disclosure limits.
This checklist is our editorial synthesis, not a validated intervention. We reviewed the published guidance and supporting source pages; we did not reproduce the private evaluations. The lesson is neither “ignore benchmarks” nor “trust the lab.” It is to make the strength of the claim match the strength of the measurement.
Sources and evidence trail
- Appel, Frondorf, Hadfield, Hiscott, Jagadish, Longpre, Tucker and Wittmershaus. Building strong safety evaluations. Anthropic, November 13, 2026. Company comment / evaluation-methods guidance.
- Reuel et al. BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices. arXiv:2411.12990v1, November 20, 2024; arXiv record lists NeurIPS 2024 Spotlight acceptance. Background technical paper; abstract and bibliographic record checked.
- Inspect official documentation. Evaluation infrastructure, not certification.
- Claim-to-source map and weekly watch checks.