A week of open-weight releases — DeepSeek, Qwen, Mistral's Shieldstral — is usually framed around price and control. For research the argument is different and stronger.

Reproducibility

A result computed against a hosted API is not reproducible. The endpoint updates silently; the model behind the name changes; the number you published cannot be recovered.

An open-weight model at a pinned revision can be re-run in five years. That is not a preference, it is the difference between a finding and an anecdote.

Inspection

Reviewers can ask what the model saw. With a closed endpoint the honest answer to "was this in training data?" is "unknown", which quietly undermines every contamination check in the paper.

The cost

Someone has to run it. Open weights move the operational burden to you — and for a lab already near its storage or GPU ceiling, that is a real constraint, not a footnote.

The judgement is per-project: pin weights when the result must be defensible, use an API when it must merely be useful.

What "reproducible" requires precisely

Three things, and hosted APIs satisfy none of them reliably.

The same weights. A model name is not a version. Endpoints are updated without notice, and providers rarely guarantee that a name maps to fixed parameters over time.

The same decoding. Temperature, top-p, seed. Some providers expose a seed; most do not guarantee determinism even with one, because batching affects numerics.

A record of both. Which means writing them down at run time, because you cannot recover them later.

The contamination question

Reviewers increasingly ask whether an evaluation set appeared in training data. With open weights and a published corpus you can attempt an answer. With a closed endpoint the honest answer is "unknown", and "unknown" quietly undermines every contamination claim in the paper.

This is not hypothetical. A substantial fraction of benchmark results published in the last two years are uninterpretable for exactly this reason.

The cost, stated fairly

Open weights move operational burden onto you: GPUs, storage, and the ongoing work of keeping an inference stack alive. For a small lab that is a real cost, and pretending otherwise is how people end up with an abandoned server.

We hit the storage side of this directly — an embedding index that took 25 days of local compute to build occupied 216 MB, pushed a database to 99% of quota, and served a feature that had been removed from the UI. The compute was the cheap part.

A workable rule

Pin open weights when the result must be defensible in review. Use an API when the output merely has to be useful today. Most projects contain both kinds of work, and conflating them is the mistake.

What to record either way

Model identifier, revision or hash, decoding parameters, date, and the prompt template. Five fields. Almost nobody logs all five, and it is the difference between a finding and an anecdote.