Reproducibility Begins Before the First Prompt Is Run

Two teams can send the same question to the same language model and produce different evidence.

One may call a dated model snapshot through an API, record the full system prompt, preserve the source documents, run every test item five times, and publish the scoring code. The other may select a model from a consumer interface, paste in a prompt, copy the first response, and report the result as though it were a stable property of the system.

Both teams may say they evaluated the same model.

They did not conduct the same experiment.

That difference is easy to miss because language-model evaluation is often described as something that begins when the first prompt reaches the model. By then, many of the decisions that determine the result have already been made. The institution has chosen what question counts as evidence, which model qualifies as the subject of the evaluation, what material the model may see, how outputs will be scored, which failures matter, and what conclusion the final number will be allowed to support.

Once those choices are buried, a benchmark can be rerun without the underlying claim being reproduced. The prompts may execute perfectly while the experiment remains impossible to reconstruct.

For a public agency, university, nonprofit, newsroom, or smaller research institution, reproducibility therefore begins before the first prompt is run. It begins when the institution creates a record of the decision it is trying to make and the conditions under which the evidence will be considered valid.

The first reproducibility artifact is the question

An evaluation cannot be reproduced if its purpose was never stated precisely.

“Evaluate the model’s accuracy” is not a research question. Accuracy on what material, for which users, under what conditions, and with what consequence when the model is wrong?

Consider an institution assessing whether a language model can help employees summarize public policy documents. One evaluation could test whether summaries preserve the main ideas. Another could test whether every legal condition remains traceable to a cited passage. A third could measure whether readers misunderstand the summary as official guidance.

Those are different evaluations. They require different datasets, scoring procedures, reviewers, and thresholds.

A result becomes meaningful only after the institution defines the claim it intends to examine. A usable claim might be:

Under a fixed retrieval configuration and a documented prompt, the system preserves every material condition in a selected set of public guidance documents, provides a valid supporting citation for each condition, and clearly signals when the source does not answer the question.

That statement is narrower than “the model understands policy.” It is also testable.

The narrower claim produces stronger evidence because it limits what the result can be used to assert. It tells the evaluator what must be recorded, what counts as failure, and where human judgment remains necessary.

NIST’s Artificial Intelligence Risk Management Framework places documentation, measurement, governance, and review across the AI lifecycle rather than treating them as final reporting exercises. Its Generative AI Profile similarly calls for documented risk-measurement practices, standardized reporting, and defined review thresholds. The implication is straightforward: measurement design belongs inside governance, not after it.

“The model” is not a sufficient identifier

A language-model name is rarely enough to identify the system that produced an output.

An evaluation record may need to include the provider, exact model identifier, model snapshot or release date, API or interface used, region, access tier, system instructions, tool permissions, retrieval configuration, decoding parameters, and any intermediary software that transformed the prompt or response.

The surrounding system matters because an institution rarely evaluates a model in isolation. It evaluates a pipeline.

A retrieval system may omit a relevant passage. A document parser may damage a table. A safety layer may refuse a legitimate question. A scoring script may mark a correct answer as wrong because of formatting. A system prompt may push the model toward brevity and cause it to omit a material qualification. A consumer application may add instructions the evaluator cannot inspect.

When those components change, the object being evaluated changes.

This is why a statement such as “Model A scored 84 percent” is incomplete. The reader still does not know what version ran, what the model received, what it was permitted to do, how many attempts were made, how answers were parsed, or what happened to ambiguous cases.

A reproducible record identifies the entire evaluation configuration closely enough that another qualified team can reconstruct it or explain why exact reconstruction is impossible.

That final clause matters. Reproducibility does not require pretending that every proprietary service is perfectly stable or inspectable. It requires documenting the instability instead of hiding it.

The evidence set must be frozen before the result is known

A benchmark is vulnerable the moment evaluators can alter the test after seeing the model’s performance.

They may remove confusing examples, rewrite prompts that produced poor answers, change a scoring rule, add clarifying context, or exclude cases that are difficult to classify. Some changes may be reasonable. The problem is making them silently.

Before testing begins, the institution should preserve a versioned evaluation package containing the input data, inclusion and exclusion criteria, expected outputs where appropriate, scoring instructions, reviewer guidance, and a record of known limitations.

The package should have a stable identifier. A cryptographic hash is useful when files may be moved or copied because it allows later verification that the evaluated material has not changed. A simple version number is better than nothing, but “final,” “final revised,” and “final revised 2” are not a defensible provenance system.

This concern is especially serious for public benchmarks. Language models may have encountered benchmark material during training, directly or through derivative sources. A high score can therefore reflect genuine task performance, memorization, contamination, or some combination that cannot be fully separated.

Research on benchmark contamination has demonstrated that language models can sometimes reproduce obscured elements of widely used test material, strengthening the case for contamination analysis and restrained interpretation of benchmark results.

The responsible conclusion is not that every public benchmark is useless. It is that the institution must record the benchmark version, its exposure risk, any contamination checks performed, and the limit this places on the claim.

An evaluation cannot be repaired later by adding the sentence “results may vary.”

The prompt is part of the method

Prompts are sometimes treated as editorial wording surrounding the real experiment. In language-model evaluation, they are experimental conditions.

A small change in instruction can alter task interpretation, output length, citation behavior, refusal rates, and the likelihood that a scoring parser will recognize the answer. The order of examples can matter. The presence of a role description can matter. A request for step-by-step reasoning can change cost and behavior. Tool access can turn a memory test into a retrieval test.

The prompt must therefore be preserved exactly as run, including the system message, user message, examples, retrieved context, tool definitions, output schema, and any text inserted automatically by the evaluation harness.

The same principle applies to preprocessing and postprocessing. If a script strips punctuation, extracts only the first sentence, retries malformed outputs, repairs JSON, or maps free-form text to a multiple-choice option, that script is part of the method.

The result did not come from the prompt alone. It came from the prompt, the system, the execution settings, and the interpretation procedure.

NeurIPS requires authors to address reproducibility and transparency through its paper checklist and asks researchers to explain how experimental results can be reproduced or independently verified. The broader lesson extends beyond conference submissions: methodological detail is evidence, not administrative overhead.

One output is an anecdote

Language-model outputs can vary across repeated runs. Even when the interface offers a seed or low-temperature setting, exact identity may not be guaranteed across infrastructure, model, or software changes.

A single response can show that a failure occurred. It cannot reliably establish how often that failure occurs.

Before execution, the evaluation plan should define how many times each item will be run, whether the runs are independent, which parameters remain fixed, and how variation will be summarized. The number of repetitions should follow the decision risk and the expected variability rather than an arbitrary universal rule.

A low-consequence formatting experiment may need fewer runs. A reliability claim that could influence public adoption demands stronger sampling, uncertainty reporting, and failure analysis.

This is where smaller institutions often assume they are blocked by limited compute. They are not required to imitate frontier laboratories. They are required to make bounded claims that match the evidence they can afford to collect.

A nonprofit may be unable to run ten thousand evaluations across twenty models. It may still conduct a rigorous study of two systems on a carefully defined task, preserve every input and output, repeat a defensible subset, publish its exclusions, and refuse to generalize beyond the tested conditions.

Scale can strengthen an experiment. Scale cannot rescue an undocumented one.

Scoring rules must exist before they are needed

Many evaluation disputes begin with an output no one anticipated.

The model gives a correct answer with an incorrect citation. It refuses the question but explains why. It provides the right conclusion while omitting a required qualification. It answers in a different format. It gives a partially correct response that two reviewers score differently.

If the institution creates a scoring rule after seeing the disputed output, the result is vulnerable to unconscious adjustment.

A reproducible evaluation defines the unit of analysis, scoring rubric, acceptable variants, failure categories, abstention treatment, parser behavior, and adjudication process before the main run. When expert judgment is necessary, the process should identify who is qualified to judge, whether reviewers see the model identity, how disagreements are resolved, and whether inter-rater agreement will be measured.

The rubric should preserve the distinction between a wrong answer and a dangerous answer.

A summary that omits a minor descriptive detail is different from one that removes an eligibility restriction. A citation that points to the wrong paragraph is different from a fabricated authority. A refusal may reduce task completion while preventing a more serious failure.

Collapsing these outcomes into a single accuracy score can make an evaluation easier to publish and harder to use.

The midpoint problem is authority, not technology

At first, reproducibility appears to be a technical discipline: preserve code, prompts, data, parameters, and outputs.

That is only half the problem.

The same experimental result can support one decision and be dangerously insufficient for another.

Suppose a system correctly summarizes 92 of 100 selected public documents under controlled conditions. That result might justify continued research. It might support a limited internal pilot with mandatory source review. It does not, by itself, justify publishing uncited summaries to the public, replacing trained staff, or allowing the system to interpret eligibility rules.

The missing variable is authority.

A reproducible evaluation must record who commissioned it, what decision it may inform, who may approve that decision, what threshold applies, what evidence remains outside the study, and what conditions require escalation or rejection.

Without that record, another team may reproduce the score and still misunderstand its institutional meaning.

This is why the evidence ledger should begin before testing. It should connect every material claim to the source, version, method, limitation, responsible owner, review status, and permitted use of the result.

The ledger prevents a familiar failure: a narrow research finding travels through a slide deck, grant narrative, procurement document, press release, or executive briefing until its conditions disappear.

The number remains. The experiment does not.

Reproducibility is a chain of custody

The strongest evaluation record resembles a chain of custody for evidence.

It identifies where each input came from, who approved its use, which version entered the test, what transformations occurred, which system processed it, what output was produced, how the output was scored, who reviewed the result, and what changed afterward.

This does not require an expensive platform.

A smaller institution can create a defensible record with a version-controlled repository, a structured run manifest, immutable copies of the evaluation set, automated logging, documented reviewer instructions, and a public report that distinguishes released artifacts from materials that cannot be shared.

The essential question is whether a qualified outsider can follow the evidence without relying on the evaluator’s memory.

A usable run manifest should make the experimental object identifiable. It should preserve the model and system configuration, prompt package, data version, software dependencies, execution date, repetition plan, randomization controls, output locations, scoring code, exceptions, and known deviations.

A usable evidence ledger should make the conclusion traceable. It should connect each published claim to supporting runs, source material, reviewer decisions, limitations, and approval status.

A usable change log should make revisions visible. It should explain whether a rerun followed a model update, benchmark correction, prompt revision, scoring defect, or newly discovered contamination risk.

ACM’s artifact-review framework makes a similar distinction between the availability of research artifacts and the successful reproduction of reported results. Making files available is valuable, but availability alone does not establish that another team can obtain the same result or validate the same conclusion.

Failed runs belong in the record

An evaluation report that contains only successful runs is incomplete.

API failures, malformed outputs, timeouts, refusals, parser errors, unavailable model versions, reviewer disagreements, and excluded examples can reveal more about operational reliability than the headline score.

The institution should define in advance which failures will be retried, how many retries are allowed, whether the first attempt remains part of the record, and how unavailable results affect the denominator.

Automatically retrying until the model produces a valid response may measure the reliability of a retrying application. Reporting only the final successful response measures something else. Neither approach is inherently wrong. Confusing them is.

The same rule applies to human intervention. If an evaluator edits a prompt, repairs a response, selects among several generations, or manually chooses a source, that intervention must be recorded.

Human review is often necessary. Hidden human review makes the system appear more autonomous and reliable than the tested workflow actually was.

Reproduction is not replication

The language surrounding reproducibility can itself create confusion.

Different research communities use “repeatability,” “reproducibility,” and “replicability” in different ways. A practical institutional framework should define its terms rather than assume universal agreement.

For this kind of evaluation, reproduction can mean rerunning the documented procedure with the same artifacts and obtaining materially consistent results. Replication can mean testing the underlying claim through an independently designed method, perhaps with different prompts, evaluators, datasets, or implementations.

Both matter.

Reproduction asks whether the reported evidence can be reconstructed.

Replication asks whether the conclusion survives outside the original setup.

A model evaluation can be reproducible and still narrow. Another team may rerun the package successfully, then discover that the result collapses on documents from a different agency, language, year, or formatting style.

That does not make the first evaluation fraudulent. It identifies its boundary.

Research reviews of machine-learning reproducibility have repeatedly found that implementation choices, experimental design, data handling, and unreported variation can alter conclusions. Treating those choices as part of the scientific record makes failed reproductions easier to diagnose and successful ones more meaningful.

What a public report should allow the reader to reconstruct

A public evaluation report does not need to expose confidential data, security-sensitive instructions, personal information, or proprietary material. It must explain what has been withheld, why it has been withheld, and what independent verification remains possible.

The reader should be able to reconstruct the purpose of the evaluation, the tested claim, the system boundary, the evidence set, the run conditions, the scoring method, the uncertainty, the failures, the human interventions, the approval process, and the limit of the conclusion.

Where complete release is impossible, the institution can publish synthetic examples, schemas, hashes, aggregate statistics, redacted manifests, reviewer protocols, or controlled-access procedures.

Transparency is not an instruction to disclose everything. It is an obligation to make the missing evidence visible.

MLCommons demonstrates the institutional value of formal benchmark rules, reference implementations, defined scenarios, submission requirements, and change logs. Its published results can be modified or invalidated, and those changes are recorded rather than quietly absorbed. That is a useful model for public-interest evaluation: the evidence record remains open to correction.

What this means for TAIRC

TAIRC’s Open LLM Transparency and Evaluation Frameworks program is designed around documentation, reproducible evaluation, uncertainty disclosure, and public accountability. It does not train proprietary frontier-scale models, certify systems, issue regulatory approvals, or present proposed research as completed evidence.
Within those boundaries, TAIRC can contribute something institutions often lack: a public-interest method for preserving the chain between an evaluation question and the claim eventually placed in front of a decision-maker.

That method should be usable by organizations without large research budgets. It should distinguish mandatory records from optional sophistication. It should support external scrutiny without assuming that every artifact can be made public. It should require named owners, versioned evidence, documented limitations, and an explicit statement of what the result does not authorize.

The value would not come from declaring a model reproducible or safe. TAIRC is not a certification body.

The value would come from making weak evidence harder to disguise as strong evidence.

The experiment starts with the first institutional decision

The first prompt may be the most visible moment in a language-model evaluation. It is rarely the first consequential one.

The experiment begins when someone decides what will count as success.

It begins when a model name is accepted without a version, when a benchmark is selected without a contamination review, when a prompt is revised without a change log, when a failed run disappears, or when a narrow score is allowed to support a broader deployment claim.

It also begins when an institution does the opposite: states the question, freezes the evidence, records the system, defines the scoring rules, names the decision boundary, and preserves the chain of custody before the outcome is known.

That is the point at which reproducibility stops being a promise attached to a report.

It becomes part of the experiment itself.