Measuring Refusal Quality Without Rewarding Blanket Refusals

A language model that refuses every request can earn a perfect score on the wrong safety test.

It will never provide prohibited instructions. It will also refuse public records questions, harmless technical explanations, fictional writing, safety education, and legitimate research requests that happen to contain sensitive words. The metric will call the model safe. Its users will experience a locked door.

That is the central measurement error in refusal evaluation: counting refusals rewards caution without testing judgment.

A defensible evaluation must ask whether the model refused the right request, for the right reason, to the right extent, while still providing whatever help could safely be given. Refusal quality is therefore a problem of calibrated decision-making, not refusal frequency.

A refusal rate is not a safety score

A raw refusal rate records how often a model declines to answer. By itself, it cannot distinguish a well-calibrated safeguard from indiscriminate noncompliance.

The minimum useful evaluation contains both requests that should be refused and requests that should not. This creates four observable outcomes. The model can correctly refuse a harmful request, incorrectly comply with it, correctly answer a benign request, or incorrectly refuse it. The first and third outcomes are successes. The other two are different failures with different consequences.

XSTest was designed around this contrast. Its published test suite contains 250 safe prompts across ten prompt types, alongside 200 unsafe prompts intended to expose whether a model’s safety behavior is selective or exaggerated. The authors’ premise is straightforward: a system that refuses language associated with harm without understanding the request can appear harmless while becoming unusable. (arXiv)

OR-Bench expanded the over-refusal side of this problem with 80,000 benign prompts constructed to resemble potentially harmful requests, a harder subset of roughly 1,000 prompts, and 600 toxic prompts used to check whether a model had become indiscriminately permissive. In the study’s comparison of 25 models across eight model families, rejection of toxic prompts and rejection of benign-but-sensitive prompts were strongly correlated. The reported Spearman correlation was 0.878, indicating that many of the tested systems gained apparent safety by refusing more of both. (arXiv)

That result should not be converted into a universal law. OR-Bench used synthetic prompt generation, automated moderation, fixed decoding conditions, and model versions that have since changed. Its authors acknowledge that ambiguous or even toxic items may have escaped filtering and that the generation process did not guarantee diversity within every category. (arXiv)

The study still demonstrates the underlying measurement problem. When an evaluator reports only the percentage of harmful prompts refused, the easiest way to improve the number is to broaden the refusal boundary. The resulting score says nothing about how many legitimate requests were denied.

For a public institution, those denials are not a cosmetic loss of helpfulness. A system that refuses a lawful question about substance-use prevention because it contains drug terminology, declines a cybersecurity-awareness request because it mentions malware, or blocks an explanation of discriminatory language because it contains the language being examined can obstruct legitimate work. The cost varies by context, but it belongs in the safety record.

Refusal is a decision; the response is the evidence

Binary evaluation also fails on the other side of the boundary. A response that does not contain an explicit refusal is not necessarily harmful.

It may provide a safe summary, redirect the user to an authoritative source, answer a permitted portion of the request, explain a limitation, or produce incoherent text that is useless for any purpose. Treating all non-refusals as successful attacks makes the evaluator sensitive to phrasing rather than consequence.

StrongREJECT documented this problem in jailbreak evaluation. The researchers found that simple non-refusal checks could overstate attack success because they awarded credit when a model merely failed to use refusal language, even when the response contained little useful information for carrying out the prohibited task. Their evaluator instead considered whether the response refused and, if it did not, how specific and convincing the harmful information was. The evaluator was compared with human judgments collected from five labelers across 1,361 prompt-response pairs. (arXiv)

StrongREJECT measures harmful usefulness rather than institutional service quality, but the methodological lesson travels: the unit of evaluation must be the complete response.

A refusal can fail even when declining is appropriate. It can reveal part of the prohibited information before stopping. It can invent a policy restriction. It can use accusatory language that assumes malicious intent. It can provide no safe alternative when one was readily available. It can direct a person in crisis to a generic search page instead of an appropriate, verified resource. It can also explain the boundary so vaguely that users learn nothing about how to reformulate a legitimate request.

The inverse is also true. A model may answer part of a sensitive request safely without issuing a hard refusal. OpenAI’s 2025 description of “safe-completion” training illustrates one provider’s attempt to move from an input-level comply-or-refuse decision toward output-level evaluation. The approach rewards responses that remain within safety constraints while preserving useful assistance, including bounded explanations and safe alternatives. OpenAI reports improved safety and helpfulness for its own evaluated models, but those results are provider-reported, model-specific, and not an independent public standard. (OpenAI)

The important shift is conceptual. The evaluator should not ask only, “Did the model refuse?” It should ask, “What did the model actually enable, withhold, explain, and redirect?”

The target behavior has to be defined before the test begins

Refusal quality cannot be measured without a documented decision rule.

An evaluator must first specify which behavior is expected for each test case. That decision depends on the institution’s authorized use, the applicable policy, the user’s role, the information available to the system, and the consequences of an error. A generic consumer assistant, an internal research system, and a public-service information tool may face the same words under different operating boundaries.

This article uses refusal to mean safety-grounded noncompliance. Epistemic abstention—declining because the evidence is missing or unreliable—and institutional deferral—declining because a human official retains authority—are related but separate behaviors. Combining them under one refusal label hides why the system stopped.

The evaluation record should therefore identify the intended response class before scoring the model. Some requests require a firm refusal because any substantive assistance would create an unacceptable risk. Some are plainly permitted. Some allow a limited answer but prohibit operational detail. Others are ambiguous enough that the appropriate response is clarification, a bounded completion, or escalation.

A benchmark that labels every sensitive-looking prompt as either “answer” or “refuse” erases the cases where real institutional judgment is required.

RefusalBench, published at EACL 2026, examined this problem in grounded question-answering systems. The researchers generated diagnostic cases through 176 linguistic perturbation strategies covering six forms of informational uncertainty and three intensity levels. Across more than 30 models, they found that selective refusal involved separable detection and categorization capabilities. Reported refusal accuracy fell below 50 percent on the study’s multi-document tasks, with systems exhibiting both overconfidence and overcaution. The authors also found that static benchmarks can be vulnerable to memorized artifacts, motivating continuously generated test cases. (ACL Anthology)

That study concerns flawed retrieval context rather than every form of safety refusal. Its deeper contribution is the separation of two questions that binary metrics collapse: did the model detect a problem, and did it correctly identify what kind of problem it was?

A practical refusal-quality framework

A useful institutional framework should preserve the raw outcomes rather than hiding them inside one impressive score.

The first measure is boundary accuracy. It records whether the model selected the expected response class: full refusal, bounded safe completion, clarification, ordinary compliance, epistemic abstention, or human deferral. The categories should be defined in advance and tied to the institution’s actual policy.

The second measure is harm containment. Evaluators should examine whether the response provides prohibited operational value, including partial disclosure, indirect enablement, or actionable detail embedded inside an otherwise cautious answer. Severity matters. A minor policy deviation and a response that materially enables serious harm should not contribute equally to an average.

The third measure is retained utility. When useful assistance can be provided safely, the evaluator should determine whether the response preserved it. A refusal that withholds prohibited instructions but offers a lawful high-level explanation, a prevention-oriented alternative, or a verified escalation route is generally more useful than a generic denial. That does not mean every refusal requires an alternative; in some cases, any continuation would be inappropriate. The expected remainder must be specified per test case.

The fourth measure is explanation and recourse quality. A model should not fabricate rules, accuse the user, or imply legal authority it does not possess. Where the interface permits, the response should make the boundary understandable, distinguish uncertainty from prohibition, and indicate a safe next step. Public institutions should review whether these explanations remain accessible to people with different levels of technical knowledge, literacy, and disability access needs.

The fifth measure is consistency. The same underlying request should be tested across paraphrases, contextual frames, conversation histories, user roles, languages, system prompts, and repeated runs. A model that responds correctly to one canonical sentence but fails after a minor wording change has learned a benchmark pattern, not a dependable boundary.

These dimensions can be summarized conceptually:

Refusal quality equals correct boundary selection and useful safe assistance, reduced by harmful enablement, false refusal, misleading explanation, and instability.

That is a framework, not a universal formula. Institutions should not assign weights until they have stated whose risks and service obligations those weights represent.

Do not average away the failure that matters

A single composite score can recreate the original problem in a more sophisticated form.

Suppose a model correctly handles thousands of low-consequence prompts but provides dangerous operational detail in a small number of severe cases. An average may look strong. Suppose another model prevents every severe failure but refuses a large share of legitimate public-information requests. Its average may also look strong, depending on the weights.

Neither result is interpretable without the components.

At minimum, an evaluation report should disclose the false-refusal rate on permitted requests, the unsafe-compliance rate on prohibited requests, performance on ambiguous and bounded-completion cases, and the severity distribution of failures. Results should be separated by policy category and use case. High-consequence failures should remain visible even when their frequency is low.

Uncertainty must also be reported. NIST AI 800-3 distinguishes performance on a fixed benchmark from performance generalized to the wider population of similar tasks. The publication warns that common benchmark analyses may rely on hidden assumptions, conflate different performance targets, or misstate uncertainty. A refusal score measured on a curated prompt set is evidence about that set. Generalizing it to all future interactions requires an explicit statistical model and defensible sampling assumptions. (NIST)

NIST’s January 2026 initial public draft on automated benchmark evaluation similarly organizes evaluation practice around defining objectives, selecting suitable benchmarks, implementing the test, and reporting the result. NIST states that automated benchmarks are useful measurement instruments but cannot satisfy every evaluation objective. The document remained an initial public draft at the time of verification, so it should be treated as preliminary guidance rather than a final standard. (NIST)

For refusal testing, the implication is concrete. The report should name the model and access path, version or snapshot date, system prompt, policy version, decoding settings, number of repeated runs, benchmark release, evaluator model or classifier, human-review procedure, disagreement rate, exclusions, and confidence intervals. Without that record, an independent reviewer cannot determine whether a difference reflects the model, the wrapper, the scorer, the prompt set, or the testing conditions.

Automated judges need their own evaluation

Refusal evaluation frequently relies on another model to classify the response. This can reduce cost and increase scale. It can also move the unexamined judgment into a second model.

OR-Bench’s authors acknowledge that their automated moderators could introduce bias, misclassify ambiguous prompts, or favor related model families. StrongREJECT notes that remotely hosted evaluators can change without notice, weakening reproducibility, while local evaluators require additional infrastructure and may carry their own training biases. (arXiv)

An automated judge should therefore be treated as a measurement instrument. Its version, rubric, prompt, calibration sample, and agreement with qualified human reviewers belong in the report. Boundary cases should receive blinded human adjudication. Disagreement should be preserved as evidence rather than forced into artificial certainty.

Human review is not automatically authoritative. Reviewers can disagree about ambiguous intent, legitimate dual use, policy interpretation, and the appropriate level of assistance. The answer is a documented adjudication process with relevant domain knowledge—not the assumption that either a model or a person supplies unquestionable ground truth.

NIST’s ARIA program reflects this broader sociotechnical view by using controlled, scenario-based interactions to gather evidence about how AI systems behave when people use them. That approach does not replace benchmark testing, but it recognizes that an output’s impact depends on the user, task, interface, and surrounding institution. (NIST AI Challenge Problems)

Conservative refusal can be rational

Over-refusal is not always the greater error.

An institution operating in a high-consequence domain may rationally accept more false refusals to reduce the chance of severe harmful compliance. A general public-information assistant may set a different boundary because unwarranted denials create access barriers and increase staff workload. The appropriate balance depends on the use case, the availability of human escalation, the reversibility of the error, and the people who bear its cost.

This is why the objective should not be the lowest refusal rate. The objective is the lowest rate of incorrectly handled requests under a documented risk tolerance.

Simple refusal metrics retain value as screening tools. They are inexpensive, easy to compare, and capable of revealing obvious changes between versions. Research on safety-evaluation robustness nevertheless argues that adversarial robustness and benign over-refusal should be measured together, and that common testing practices can be sensitive to decoding choices and other evaluation conditions. (arXiv)

A refusal rate can open an investigation. It cannot close one.

The institutional pass condition

A procurement team or public administrator should reject any evaluation that describes a model as safer solely because it refuses more often.

A credible pass condition should be policy-specific and multidimensional. It should place a hard limit on severe harmful enablement, a separate tolerance on false refusals, and explicit requirements for bounded assistance, escalation, explanation, and repeatability where those behaviors are relevant. Failures should be reported by category rather than concealed inside a portfolio average.

The institution should also test the configured system it will actually use. Guardrails, retrieval components, system instructions, moderation layers, and interface rules can materially change refusal behavior. OR-Bench found that adding a safety-oriented system prompt increased rejection of both toxic and benign prompts, with the magnitude varying by model. (arXiv)

Version changes require regression testing. Benchmarks require refreshes. Evaluators require recalibration. A score without a model version and test date is not a durable safety claim.

TAIRC’s LLM Safety, Evaluation & Reliability program is explicitly evaluation-centered. It does not train new large-scale models, certify systems as safe, provide regulatory approval, or deploy models in operational public environments. Its published program description identifies reproducibility, failure analysis, and controlled testing as intended research directions; expected outputs remain planned rather than completed findings. (TAIRC)

The standard proposed here follows that boundary. It is an evaluation framework, not a TAIRC benchmark result or certification.

The right question is not how often a model says no. It is whether the model says no for the right reason, to the right extent, while still doing the safe part of the job.

Sources and verification

TAIRC, “LLM Safety, Evaluation & Reliability,” published program description, including research focus, explicit boundaries, and the distinction between planned outputs and completed findings. Verified August 1, 2026. (TAIRC)

TAIRC Research Portfolio and TAIRC Research — Categories & Topics, organizational source documents defining the evaluation-centered program, public-interest purpose, and prohibitions on model certification, operational deployment, and unsupported claims of completed work.

Paul Röttger and coauthors, “XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models,” NAACL 2024. Used for the paired safe-and-unsafe evaluation design and the concept of exaggerated safety behavior. (arXiv)

Justin Cui and coauthors, “OR-Bench: An Over-Refusal Benchmark for Large Language Models,” 2024. Used for benchmark composition, the observed safety–over-refusal relationship, system-prompt sensitivity, and the authors’ stated limitations involving moderation, ambiguity, and synthetic data. (arXiv)

Alexandra Souly and coauthors, “A StrongREJECT for Empty Jailbreaks,” 2024. Used for evidence that non-refusal alone can mismeasure harmful usefulness, the response-level rubric, comparison with human judgments, and evaluator reproducibility limitations. (arXiv)

Aashiq Muhamed and coauthors, “RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Models,” EACL 2026. Used for findings on selective refusal, generated perturbations, static-benchmark limitations, and multi-document performance. (ACL Anthology)

NIST AI 800-3, “Expanding the AI Evaluation Toolbox with Statistical Models,” February 2026. Used for the distinction between fixed-benchmark and generalized performance, explicit statistical assumptions, and uncertainty reporting. (NIST)

NIST AI 800-2, “Practices for Automated Benchmark Evaluations of Language Models,” initial public draft, January 2026. Used for the evaluation lifecycle and the stated limits of automated benchmarks. (NIST)

NIST, Assessing Risks and Impacts of AI program companion documentation. Used for the role of controlled, scenario-based human interaction in sociotechnical evaluation. (NIST AI Challenge Problems)

OpenAI, “From hard refusals to safe-completions: toward output-centric safety training,” August 2025. Used only as a provider-reported example of output-centric safety and helpfulness evaluation, not as independent validation. (OpenAI)

“LLM-Safety Evaluations Lack Robustness,” 2025. Used for the argument that adversarial robustness and benign over-refusal should be evaluated together and that generation settings affect safety-evaluation validity. (arXiv)