How Repeated Runs Expose Instability Hidden by a Single Demo
The cleanest AI demo is often the least informative one: the run that worked.
A language model receives a prompt, produces a fluent answer, follows the requested format, and appears to solve the task. The result is copied into a presentation or evaluation report. By the time anyone asks whether the same system would produce the same quality again, the successful output has already become evidence of “capability.”
It is evidence of something narrower. The model produced that answer once.
A single run can establish possibility. It cannot establish reliability. Reliability concerns the distribution of outcomes: how often the system succeeds, how widely its answers vary, which inputs produce unstable behavior, and whether a different run would reverse the conclusion that an evaluator drew from the first one.
A demo is a witness, not a distribution.
Repeated-run evaluation exposes what a polished example conceals. It can reveal output disagreement under identical inputs, benchmark scores that move enough to alter comparisons, failures concentrated on particular test items, and infrastructure effects that persist even when decoding is configured to minimize randomness. The practical consequence is direct: any consequential claim about a generative model should be based on replicated behavior, not its most convenient output.
The same prompt does not guarantee the same result
Variation is expected when a model samples from several plausible continuations. Temperature, top-p, and related controls affect how that sampling occurs. But reducing temperature does not necessarily create identical behavior.
Anthropic’s current documentation states that identical API requests may produce different outputs even when temperature is set to zero. Google Cloud describes a fixed seed as a best-effort mechanism rather than a guarantee of deterministic output. These are operational disclosures from model providers, not theoretical edge cases.
A 2025 preprint by Berk Atil and colleagues tested five language models on eight tasks, using ten runs for each condition while holding inputs and nominally deterministic settings constant. The authors reported accuracy variation of up to 15 percent across runs. Their item-level comparison between the best outcome obtainable across the runs and the worst reached a gap of up to 70 percent. None of the models produced consistently repeatable accuracy across every task they examined.
Those figures should not be universalized. The study covered a limited group of models, selected benchmark tasks, and particular access environments. It does establish the narrower point needed here: settings commonly treated as deterministic did not make a single result representative of every repeated execution.
The instability can enter before the model generates its first visible word. A NeurIPS 2025 study traced output divergence to numerical precision and runtime configuration. Under greedy decoding, changing the evaluation batch size, GPU count, or GPU generation could alter outputs because floating-point operations do not always produce identical results when their execution order changes. In one reported configuration, the authors observed accuracy differences of up to 9 percent and response-length differences of roughly 9,000 tokens despite evaluating the same model and task.
The implication is uncomfortable for benchmark culture. Recording the prompt and temperature may still leave the experiment underspecified.
Repetition reveals more than wording differences
Two responses can use different words while reaching the same correct answer. They can also look similar while quietly changing a date, citation, classification, recommendation, or statistical conclusion. An evaluation that measures only exact string agreement will confuse harmless variation with substantive instability. An evaluation that records only average accuracy can miss the opposite problem: stable-looking aggregate performance built from inconsistent item-level outcomes.
This distinction matters because aggregate scores can cancel out failures. Suppose a model answers 80 of 100 questions correctly on every run. The score appears perfectly stable. Yet the 80 correct answers need not be the same 80 questions. One run may fail on items that another run answers correctly. The headline score remains 80 percent while the system’s behavior on individual cases moves underneath it.
A 2026 preprint examining deterministic programming tasks makes this problem concrete. The researchers evaluated 16 models on 100 problems, with two prompt templates and five runs per problem, producing 16,000 judged instances. They distinguished average run-level success from “perfect stability,” meaning the share of problems solved correctly in every run. Run-level success exceeded this retry-free measure by as much as 17.8 percentage points, and the stability metric reversed some rankings between closely matched models.
The study is limited to a relatively small code benchmark, and the authors caution that provider-side changes, contamination, orchestration, retrieval, and multi-turn workflows were outside its scope. The finding should therefore be read as evidence about the evaluated conditions, not a permanent ranking of provider families. Its measurement insight is broader: average success and dependable success answer different questions.
The first asks, “How frequently did an invocation succeed?”
The second asks, “Which tasks can the system be trusted to complete without needing a favorable retry?”
A product that automatically retries until one response passes a test may benefit from the first measure. A researcher, public administrator, or analyst who receives one answer and acts on it may care far more about the second.
A successful retry can hide an unreliable system
Repeated sampling is sometimes presented as a solution to model weakness. Generate several candidates, select the best, and report pass-at-k performance: the probability that at least one of several attempts succeeds. That metric is useful when multiple attempts are genuinely part of the intended system.
It becomes misleading when the evaluated deployment gives a person only one answer.
A model that solves a task once in ten attempts has demonstrated a recoverable capability. It has not demonstrated dependable first-attempt performance. Reporting only the successful sample converts variability into apparent competence.
This is especially risky when no deterministic checker exists. Code can often be compiled and tested. A factual synthesis, policy summary, research analysis, or explanation may have no automatic mechanism that reliably selects the sound answer from several plausible outputs.
A 2026 preprint on LLM-generated data analysis ran six models under two prompting approaches and four temperature settings, with ten independent executions per configuration. Across 480 attempts, identical configurations sometimes produced materially different analytical results. The authors reported that some temperature-zero executions led to different conclusions about the same research question. They traced variation to choices such as file exclusion, year parsing, record ordering, tie-breaking, and missing-data handling.
The study examined one dataset and one analytical task, and its authors explicitly warn that the findings may not generalize to other domains. Human analysts can also make different defensible choices. That limitation sharpens the lesson rather than erasing it. Repeated execution can expose a hidden analytical branch: the model made an unstated choice that changed the result.
The danger is not always a wildly irrational answer. It may be two reasonable-looking pipelines that produce incompatible findings.
Three kinds of instability require three different responses
Repeated runs are useful because they help separate problems that are often collapsed into the word “randomness.”
The first is generation variability. The model samples a different continuation or follows a different reasoning path. Lower temperatures, fixed seeds, constrained output formats, or deterministic decoding may reduce this form of variation.
The second is evaluation-design variability. The result changes because the evaluator selected different few-shot examples, task formulations, prompt formats, answer parsers, scoring rules, or test subsets. Repeating the exact same flawed setup will not discover all of these effects; the evaluation must deliberately vary the factors that represent plausible use conditions.
A Findings of EMNLP 2025 study demonstrated this with few-shot evaluation. The researchers ran eight evaluations for GPT-4o and DeepSeek-R1 on WinoGrande, HellaSwag, and MMLU-Pro using different sets of few-shot examples. Across the tested conditions, the distance between the best and worst runs reached 12 percentage points, and changing examples could alter model comparisons. Their proposed instance-level randomization method was intended to reduce dependence on one fixed set of examples.
The third is execution variability. Hardware, parallelism, precision, software versions, batching, provider infrastructure, or model updates can change the output even when the visible evaluation configuration appears fixed. The NeurIPS findings on numerical nondeterminism belong here. So does vLLM’s documentation, which states that reproducibility is not guaranteed by default and, even with its reproducibility controls enabled, is limited to the same hardware and vLLM version.
These categories call for different remedies. More samples can estimate generation variability. Randomized or counterbalanced evaluation designs can test prompt and example sensitivity. Environment pinning, batch invariance, precision controls, version records, and provider fingerprints can reduce or explain execution variability.
Repeating an experiment without documenting what was held constant leaves the source of disagreement unknown. Holding everything constant without testing realistic variations can produce a narrow form of reproducibility that says little about actual use.
What a serious repeated-run evaluation should report
There is no defensible universal rule that every model must be run exactly five, ten, or one hundred times. The needed sample depends on the consequence of failure, the expected variability, the size of the test set, the metric, and the decision the evidence will support.
A screening experiment may begin with a modest number of runs to detect obvious instability. A comparison between closely matched systems requires enough repetition to determine whether the apparent gap survives sampling variation. A high-consequence use case needs coverage of rare but material failures rather than confidence built only around the mean.
The protocol should be specified before the results are inspected. Otherwise, the evaluator can stop when a preferred answer appears or continue until an inconvenient result is diluted.
Each run should preserve the complete evaluative condition: exact model identifier, access date, system and user prompts, decoding settings, seed where available, tool configuration, retrieval corpus and snapshot, response schema, benchmark version, scoring logic, software dependencies, runtime environment, and relevant hardware details. OLMES, a language-model evaluation standard published in Findings of NAACL 2025, shows why this level of specification matters. Its authors documented how task formulation and normalization choices can produce sharply different scores under labels that appear to describe the same benchmark.
The result should then be reported as a distribution. Depending on the task, that may include mean or median performance, dispersion, confidence intervals, the range across runs, per-item agreement, the proportion of items that always pass, the proportion that ever pass, rank stability, semantic disagreement, format-failure rates, and the severity of the worst observed output.
The most useful metric is often the one that matches the user’s exposure. A person who receives one unreviewed answer needs first-run reliability. A system that samples several candidates and validates them needs pipeline-level success. A research benchmark needs uncertainty around the score and evidence that rankings are not artifacts of one configuration. A creative-writing system may tolerate lexical diversity while still requiring stable compliance with safety and factual constraints.
“Consistency” is therefore not a demand that every output use identical sentences. It is a demand that the evaluation define which variations are acceptable before declaring the system reliable.
Repeated runs do not solve every evaluation problem
Replication can expose instability, but it cannot repair an invalid benchmark.
Running a model fifty times on contaminated test data yields a more precise estimate of the wrong thing. Repeating a task that does not resemble the intended deployment does not establish real-world validity. A stable system can still be consistently biased, factually wrong, unsafe, or poorly calibrated.
NIST’s AI Risk Management Framework calls for rigorous measurement with uncertainty, documented test methods, conditions comparable to intended use, and repeatable testing processes. Its Generative AI Profile advises organizations not to extrapolate broad capability claims from narrow, nonsystematic, or anecdotal assessments. It also recommends measuring and documenting statistical variance in evaluation metrics.
That guidance sets the proper hierarchy. Repetition is one component of valid test, evaluation, verification, and validation. It must sit alongside construct validity, representative test conditions, domain review, error analysis, human-factors evidence, and post-deployment monitoring where deployment is appropriate.
There is also a resource cost. Repeated inference consumes time, money, and energy. Exhaustive replication would be wasteful for many low-risk exploratory tasks. The answer is not to default back to one run. It is to scale the evaluation to the claim.
A minor wording assistant may need only lightweight stability testing. A model comparison presented to a procurement board needs uncertainty estimates and configuration records. A system whose output could affect rights, benefits, safety, health, employment, or public administration requires stronger evidence and professional review appropriate to the domain. Repetition does not authorize automated use in any of those settings.
The standard should change from “show me” to “show me again”
The single-demo habit survives because it is efficient. One strong output is easy to display, easy to understand, and easy to remember. A distribution is slower. It contains inconvenient information: middling runs, malformed answers, reversals, and cases where the model appears capable only when chance or infrastructure breaks in its favor.
That inconvenience is the evidence.
A credible evaluation should make it difficult to confuse the best observed run with typical behavior. It should show whether conclusions survive repetition, whether failures cluster around particular inputs, whether rankings remain stable, and whether the evaluator can reproduce the conditions that generated the result.
Repeated runs do not prove that a model is safe. They do something more modest and indispensable: they reveal how much confidence a single output did not deserve.
Within TAIRC’s research architecture, this work remains controlled, evaluation-centered, and documentation-focused. It does not certify a model as safe, grant regulatory approval, train proprietary large-scale models, or deploy systems in operational public environments.
The next time an AI system succeeds on command, the right response is not disbelief. It is another run.
Sources and verification
The article’s scope, evidence requirements, editorial boundaries, and prohibition against invented organizational claims were derived from the TAIRC 400-Article Editorial Authority Project.
TAIRC’s program description and explicit boundaries for LLM Safety, Evaluation & Reliability were verified against the TAIRC research-category document and research portfolio.
Atil et al., “Non-Determinism of ‘Deterministic’ LLM Settings,” version 5, 2025. This preprint supplied the repeated-run results involving five models, eight tasks, ten runs, and nominally deterministic settings.
Yuan et al., “Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference,” NeurIPS 2025. This peer-reviewed paper supported the discussion of numerical precision, GPU configuration, batch size, greedy decoding, and execution-level reproducibility.
The Findings of EMNLP 2025 paper “Instance-Level Randomization: Toward More Stable LLM Evaluation” supported the analysis of few-shot-example selection, score variance, and unstable model comparisons.
Zhou et al., “Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Tasks,” 2026. This preprint supported the distinction between run-level success and retry-free stability; its domain and sample limitations are stated in the article.
Cui and Alexander, “Same Prompt, Different Outcomes: Evaluating the Reproducibility of Data Analysis by LLMs,” 2026. This preprint supplied the repeated data-analysis experiment and its stated limitations.
Gu et al., “OLMES: A Standard for Language Model Evaluations,” Findings of NAACL 2025. This peer-reviewed source supported the discussion of task formulation, normalization, and complete evaluation specification.
The NIST AI Risk Management Framework and NIST AI 600-1 Generative AI Profile supported the recommendations on repeatable testing, uncertainty measurement, empirically validated capability claims, and avoiding extrapolation from anecdotal demonstrations.
Anthropic’s Claude documentation and Google Cloud’s Gemini documentation were used to verify current provider disclosures concerning temperature-zero nondeterminism and the limitations of fixed seeds.
The vLLM reproducibility documentation was used to verify that deterministic controls require explicit configuration and remain bounded by hardware and software-version conditions.