What Should You Log When an AI Agent Runs?

What should you log when an AI agent runs?

Enough to answer three questions later: what did it decide, why did it decide that, and what did it change. In practice that means a span for the whole agent run, a child span for every model call, a child span for every tool call, token counts on each one, and a deliberate decision about whether to store the prompt text at all.

Most teams we talk to discover this the hard way. An automation does something wrong on a Tuesday, somebody notices on Friday, and there is no record of what the model was given or which tool it called. The run is gone. All that survives is the damage.

The good news is that this problem now has a standard shape, and you do not have to invent your own schema.

Why is logging an agent different from logging a script?

Because a script's behaviour is in the code and an agent's behaviour is in the run. With a script you can read the source and know what happened. With an agent, the same code, the same inputs and a different day can produce a different sequence of tool calls, and the only place that sequence ever existed is the trace.

The second difference is cost. A script that loops twice as many times costs you a little CPU. An agent that loops twice as many times costs you tokens, and the bill arrives at the end of the month with no explanation attached. We went through the money side of this in keeping AI costs under control on a web team.

The third is blame. When an agent calls a tool that writes to your CRM, the question after an incident is whether the model chose badly or the tool behaved badly. Without separate records for each, you cannot tell, and you will end up rewriting the wrong half.

Is there a standard for this yet?

There is a converging one. OpenTelemetry publishes semantic conventions for generative AI, which define the span and attribute names for exactly this. Using them means your traces are readable by ordinary observability tooling rather than by a bespoke dashboard somebody has to maintain.

One caveat worth knowing before you build on it. OpenTelemetry's own documentation site now states that the GenAI semantic conventions have moved to a dedicated repository and that the older specification page is no longer maintained. We could not confirm a stable 1.0 release for these conventions this run, so treat the attribute names as likely to shift and avoid hard-coding them in a hundred places.

That is still far better than inventing your own. A convention that changes is easier to follow than a schema only your team understands.

What does the span structure look like?

A tree, and the tree is the whole point. OpenTelemetry's guidance models an agent run as a top level invoke_agent span, with child chat spans for each model call and child execute_tool spans for each tool invocation.

Read a trace in that shape and the agent's reasoning becomes visible as structure. Six chat spans under one invoke_agent means the agent went around six times. Twelve execute_tool spans where you expected two means it retried something. A chat span with no execute_tool children after it means the model decided to stop, which is either correct or a silent failure depending on what you asked for.

You do not need to read the prompts to spot most problems. The shape of the tree gives them away.

Which attributes actually matter?

Start with the four that OpenTelemetry names and that you will use constantly. The attribute gen_ai.request.model records which model handled the call, which matters more than teams expect once you are running more than one. The attributes gen_ai.usage.input_tokens and gen_ai.usage.output_tokens carry the token counts for each call. And gen_ai.response.finish_reasons records why the model stopped generating, with values such as stop or tool_calls.

That last one is the most underrated field in the whole set. A run that ended because the model chose to stop and a run that ended because it hit a limit look identical in your application logs and mean completely different things.

On the metrics side, OpenTelemetry defines gen_ai.client.operation.duration and gen_ai.client.token.usage, which can be filtered by model and by token type. Those two give you a cost and latency dashboard without writing any custom instrumentation.

Should you log the prompts themselves?

Deliberately, and by default the answer is no. OpenTelemetry's position is explicit: by default, no prompt content or tool arguments are captured with GenAI telemetry, because these can contain sensitive data. Only metadata such as model names, token counts and durations are included.

Content capture is an opt-in. Enabling it populates span attributes with full prompt messages, system prompts, tool schemas, tool arguments and tool results, under attribute names including gen_ai.system_instructions, gen_ai.input.messages and gen_ai.output.messages.

The trade is real and it is not obvious. Without content you can see that a run went wrong but often not why. With content you have just copied every customer record the agent touched into your observability vendor, with a retention policy you probably did not choose. Our default is content capture off in production, on in staging, and on in production only for a named subset of low-sensitivity workflows.

What should you log that the conventions do not cover?

The side effects. Semantic conventions describe the conversation with the model and the calls to tools. They do not describe what those tools did to your systems, and that is usually the part you need after an incident.

So log, separately and in your own store, every write the agent caused. Which record changed, from what value to what value, under which run identifier. That last field is the one people forget, and it is the one that makes a trace and a database change reconcilable four days later.

Also log the decision not to act. An agent that considered sending an email and chose not to has made a decision worth seeing, and it leaves no trace anywhere else in your stack.

How long should you keep all this?

Longer than your incident discovery time, which is almost always longer than teams estimate. If problems typically surface a week after they happen, a three day retention on traces means you are systematically unable to investigate your own failures.

Our rule of thumb is thirty days for full traces and a year for the compact side-effect log, because the side-effect log is small and is the thing auditors and customers actually ask about. Token metrics can be aggregated after a week; you rarely need per-call cost data from two months ago, but you do need the monthly totals.

If you turned content capture on, retention becomes a data protection question rather than a budget question, and the answer should come from whoever owns that in your company rather than from your engineering team's default settings.

How does this connect to tool design?

Directly, because a well-named tool makes a trace readable and a badly named one makes it useless. An execute_tool span called run_query tells you nothing. One called update_crm_contact tells you what to worry about.

The same applies to tool arguments and errors. A tool that returns a generic failure gives the model nothing to recover with and gives you nothing to debug with. A tool that returns a specific, structured error does both jobs at once. We covered that design surface in what MCP servers mean for web teams.

The general principle: your observability is only as good as the vocabulary your tools expose. Naming is not cosmetic here.

Where should a team start this week?

Instrument one automation, not all of them. Pick the one that already worries you, wrap it in an invoke_agent span, emit chat and execute_tool child spans with the model and token attributes, and leave content capture off. That is a day of work and it will immediately answer questions you have been guessing at.

Then add the side-effect log with a run identifier, because that is the piece no standard gives you and the piece you will want first when something goes wrong. Everything after that is refinement.

The pattern to avoid is building a beautiful observability layer before you have anything running in production. Logs earn their keep by being read during an incident, and until you have had one you are guessing at what you will need. Instrument the risky thing, wait for the first surprise, and let it tell you what else to capture. That feedback loop is also how you build a real test set, which we wrote about in evaluating an AI automation in production.

If you are putting agents near a live site and want a second opinion on what to record before you need it, we are happy to walk through it. You can find us at phoenix.studio.