On November 19, an enterprise search and observability vendor announced a planned acquisition that would bring agent-specific tracing and evaluation into a broader observability platform. The deal has not closed. For teams that run AI agents, the two jobs in that description are worth separating: recording what an agent did, and judging whether it was right.
Agent observability is the practice of recording what an AI agent did in each run, including its model calls, tool calls, retrieved data and final action. Agent evaluation then checks runs against criteria the team has agreed on.
A production trace becomes a reliable test case once it carries the expected outcome, the data available at the time, and the rule that makes that outcome correct; the team adds whichever of these the trace does not already record. Choose traces for what they reveal, such as corrected failures, rare exceptions and runs under a rule that later changed, and keep a representative sample of common cases next to them.
Key takeaways
- A trace records what happened in one run. A test case also needs the expected result and the reason it is correct, which a trace may not record.
- Keep a representative sample of common cases, then add traces that a person corrected, that escalated, that hit a rare exception, or that ran under a rule that has since changed.
- Pin the records the agent read and the date the test treats as today, or a later rerun can fail for reasons unrelated to the agent.
- Traces cover only situations the agent has already encountered. Cases written from work documents can cover the ones it has not encountered yet.
What is the difference between agent observability and agent evaluation?
Observability shows what happened in a run; evaluation decides whether the result was acceptable. A trace shows the steps of one run: the prompt, each tool the agent called and its arguments, the rows or documents it pulled, and the action it took. An evaluation runs a set of cases against the agent and scores each result against expected outcomes or pass criteria. The CUBIG glossary defines both terms: observability for AI agents and agent evaluation.
The two meet when a team turns production runs into evaluation data. An engineer finds a wrong refund in the traces, saves that run, writes down what the agent should have done, and adds the case to the test set so the next release is checked against it. Observability and evaluation are distinct jobs, but some tools handle them together: OpenAI’s agent evals guide, for example, describes running graders against agent traces to find regressions and failure modes. Whichever tool holds the loop, the team still decides which runs deserve to become tests and what counts as correct.
Which agent traces should become test cases?
Keep the traces that show the test set something it does not already check, alongside a sample of common cases. A week in production can produce thousands of successful runs that look alike. Duplicate cases rarely uncover new failure modes, and they slow down every rerun.
| Trace type | Why keep it | What to add before it becomes a test |
|---|---|---|
| Runs a person corrected | The correction shows the expected outcome directly | The corrected result and who confirmed it |
| Runs that escalated or stopped | They mark the edge of what the agent should handle alone | Whether handing off was the right call, and the rule behind it |
| Rare cases that passed | A pass on a rare case can be luck; a test keeps checking it | The condition that makes it rare, such as a merged account or a disputed invoice |
| Runs under a rule that later changed | They show how the agent behaved under the old rule | The rule version the run used, and the expected outcome under the new rule |
| Common successful runs | A representative sample confirms that everyday work still succeeds | Coverage of each common case type; skip near-duplicates |
Routine work belongs in the set too. Keep representative common cases as a sample to confirm that everyday work still succeeds, and add human-intervention and rule-boundary cases separately to confirm that key exceptions are caught.
What might a trace leave out when you turn it into a test?
A trace may not contain everything needed to turn a run into a reliable test case. What it records depends on the tool and its setup; some setups link inputs, versions or evaluation results. Check whether it includes the expected outcome, the data available at the time, and the business rule behind the decision, and add what is missing.
- Expected outcome. The trace shows what the agent did. Someone who owns the process has to say what it should have done. For a refund agent, that may be a billing lead, not the engineer reading the trace.
- Data state. The trace may name the customer record and the policy document, but both keep changing after the run. If the test reads today’s record, a rerun can fail because the order shipped or the policy was updated.
- Business reason. In a hypothetical case, a refund was right because the order was 20 days old, inside a 30-day window, and unopened. Without that reason written next to the case, nobody can tell when the case stops being valid.
- Values to remove or replace before saving. Production traces can include customer names, account numbers or contract terms. Before a trace becomes a long-lived test case, the team decides under its own data policy which values to remove or replace, then checks that the case still tests the same situation.
- 01Saved run
A refund is approved: the order is 20 days old, inside a 30-day window, and unopened.
- 02Rerun later
Fifteen days later the team reruns the case, and the test calculates the order’s age from the current date.
- 03Result differs
The order now counts as 35 days old, outside the window, so the expected refund no longer holds.
- 04False failure
The rerun fails, and the team debugs an agent that did nothing wrong.
The fix is to pin the test’s reference date as well as the data version: store the values the agent saw, or a version reference to them, and the date the run treated as today. The broader version of this problem, an answer that changes while the model and prompt stay the same, is covered in LLM monitoring: what your traces show, and what the data side has to record.
How do you turn agent traces into test cases?
Turn a trace into a test case by fixing what the agent saw and writing down what it should have done, then rerun the case under the same conditions each time.
- Choose cases for what they reveal. Take corrected runs, escalations, rare exceptions and rule changes, plus a representative sample of common cases.
- Confirm the expected outcome with the owner. Ask the person who owns the process for the right result, and record who confirmed it and when.
- Pin the data and the reference date. Save a copy or version reference for the records, documents and tool responses the run used, along with the run’s reference date.
- Write the pass criterion. Decide whether the case checks the final state, the path of tool calls, or both, and what counts as a pass.
- Apply the data policy before sharing. Handle sensitive values before the case enters a shared test set, then confirm it still tests the same situation.
- Keep evaluation conditions fixed. When testing a change to the agent, model or prompt, keep the data and other evaluation conditions fixed. Change one component at a time when you need to understand its effect.
- Retire expired cases. When a business rule changes, update or remove the cases written under the old rule so they do not keep failing for the wrong reason.
The result is what many teams call a golden dataset. To keep it usable across tool changes, store it with the team rather than inside one tool; this guide to keeping a golden dataset when OpenAI Evals shuts down lists what to record.
How CUBIG approaches agent test cases
Traces can only become tests for situations the agent has already encountered. A new agent, or an agent given a new task, has no production history, and the rare exceptions that matter most may not show up for months. CUBIG’s view is that a test set needs cases written from the work itself alongside cases taken from traces. The related article on where test scenarios come from before you have failure history covers how to build cases before production history exists.
Syntitan, CUBIG’s AI-ready data platform, handles that part through Ready for Agent. From a team’s work documents for one task, Syntitan generates Tasks and Cases for review and provides a Case CSV, an agent input CSV and a run guide. The team runs those Cases in its own agent environment, with its own observability and evaluation tools, and can add the reviewed Cases to its golden dataset. Syntitan does not replace an observability or evaluation tool, and it does not certify an agent.
Whichever tools you use, a trace tells you what the agent did, and a test case also has to say what it should have done, with which data, and why. See how Syntitan creates agent test cases from work documents.
