# Turning repeated LLM traces into deterministic pipelines

- Published: 2026-07-07
- Authors: CORTEXA
- Category: Engineering
- HTML: https://researchhub-vert.vercel.app/blog/llm-traces-typed-deterministic-pipelines

The most reliable part of any agent system is the part that stopped being an agent.

A question posted in r/MachineLearning asks whether recurring LLM traces can be synthesised into deterministic pipelines of typed ML and NLP operators.

It is a good question, and the answer is mostly yes — with the interesting difficulty being not the synthesis but knowing when it is safe.

## The observation behind it

Run an agent on a class of task enough times and the trajectories converge. The same tools get called in the same order with structurally similar arguments. The model is re-deriving an identical plan on every invocation, paying full inference cost each time, with a fresh opportunity to deviate.

At that point the model is not making a decision. It is performing a fixed procedure, unreliably and expensively.

## What extraction buys

Turning a stable trace into code changes the properties of the system substantially:

- **Deterministic.** Same input, same path, every time.
- **Cheap.** No inference cost for the settled portion.
- **Testable.** You can unit-test a pipeline. You cannot unit-test a prompt in any meaningful sense.
- **Debuggable.** A stack trace beats re-reading a transcript.
- **Typed.** Errors surface at the boundary rather than three steps later as a confusing string.

```mermaid
flowchart TD
  O[Observed traces] --> D{Stable across runs?}
  D -->|yes| E[Extract typed pipeline]
  D -->|no| K[Keep model in the loop]
  E --> P[Deterministic operator chain]
  P --> G{Input in<br/>observed distribution?}
  G -->|yes| P
  G -->|no| K
```

## The hard part: knowing when it is safe

The risk is straightforward. Traces are stable *over the inputs you observed*. Freeze the pipeline and it will handle those inputs beautifully and fail on the first input that needed the branch you never saw.

That failure is worse than the original, because the frozen pipeline fails silently — it does not know it is out of distribution, whereas the model at least had a chance of noticing.

So any practical version of this needs a guard: extract the pipeline, but keep a check that the input resembles what the pipeline was derived from, and fall back to the model when it does not.

## The pattern is not new

This is the same move as JIT compilation — observe a hot path, specialise it, keep a guard, deoptimise when the guard fails. It is also what every mature ML team eventually does by hand, when they notice that half their "agent" is really a fixed ETL job with a language model bolted on.

The interesting question is whether the extraction can be automated well enough to be worth trusting. Given that the traces are already logged and the operators are already typed in most frameworks, the ingredients are there.

## The uncomfortable conclusion

Taken seriously, this suggests the goal of a good agent framework is to **progressively eliminate the agent**. Every stable trace that becomes a pipeline is one less place where a model can be creative at the wrong moment.

That is a less exciting story than autonomous systems, and it is closer to how the reliable deployments actually look.