See what your agents are actually doing
Agent Insights records every conversation your agents have in production. Open any one as a trace: each step, each tool call and each model call, with what it cost and how long it took.
- Read any conversation turn by turn
- See which step was slow, failed or expensive
- Compare one version of an agent with the next
One conversation, read from the first step to the last.
A trace is the record of one conversation. It lays out what the agent did in order, so you can point at the step that went wrong instead of guessing from the reply.
Steps
The order things happened in, from the message that came in to the reply that went out, including the hand-offs between agents.
Tool calls
Which tool was called, with what input, what came back, and whether it failed or was retried.
Model calls
Each call to a model, the tokens it read and wrote, and which agent made it. Agents run on OpenAI, Anthropic and Google models.
Cost and time
What each step cost and how long it took, added up for the conversation and for the agent.
Every conversation knows which version of the agent handled it.
Agents change: a new prompt, a new tool, a new sub-agent. Agent Insights keeps the line of versions and records each conversation against the one that was live, so a change in the numbers can be traced to a change in the agent.
- See what changed between two versions
- Compare errors, reply time and cost before and after
- Open the conversations behind any figure
Three things a trace settles.
Three fictional conversations. On the left, what you start from; on the right, what the trace shows and the change that followed.
Customer: Can you move my boiler visit to Thursday morning?
- ModelScheduling agent picks a day
- Toolfind_slots, timed out at 2.4 s
- Toolfind_slots, retry, 0.6 s
Trace · 6.6 s
Where the time went
- find_slots: 3.0 s across a timeout and one retry
- Model calls: 3 calls, 2.9 s, $0.013
- Other tools: 2 calls, 0.7 s
What changed
- Version 13: Timeout raised from 2 s to 5 s
- Field-service dispatcher · conversation 0412
Illustrative · fictional agents and conversations
The reply took 6.6 seconds. One tool call took most of it.
A field-service agent is asked to move a repair visit. The answer is right, but slow. The trace shows a scheduling tool timing out and being retried, so the fix is in the tool settings, not the prompt.
Three jobs you do every week once agents are live.
A tracing tool can tell you which call returned an error. Agent Insights is built to answer the question you were actually asked: why did the customer get that answer?
- 01
Debug a bad answer
Open the conversation, find the turn, and read what the agent was given before it answered. Most wrong answers start with a tool result, and the trace shows it.
- 02
Compare versions
Put the new version of an agent beside the one it replaced: errors, reply time and cost over the same kind of traffic. You see whether a change helped.
- 03
Watch cost
See cost by agent, by conversation and by step. When one conversation costs far more than the rest, open it and see which step did it.
Build the agent, test the change, then watch it work.
Agent Insights is one part of Agent Builder. It reads the same agents and the same versions as the rest, so nothing has to be wired together.
Agent Builder
Where the agent is built and versioned. An agent built in Agentman Studio shows up in Agent Insights without tracing code.
See Agent BuilderAgent Testing
Checks a change before it ships. Agent Insights shows what happened after. Agent Testing is available on request.
See Agent TestingSecurity and governance
What is recorded, who can see it and where your data lives, for the security team that has to approve it.
See security
What agent developers ask
Which agents can I see in Agent Insights?
Agents built in Agentman Studio. They are instrumented when you build them, so there is no tracing code to add.
What does a trace show?
One conversation from start to finish: each step the agent took, each tool call with what went in and what came back, each model call with its tokens, and the cost and time of every step.
Which models does it cover?
Agents in Studio run on OpenAI, Anthropic and Google models. A trace shows the model calls an agent made, whichever of them it runs on.
Can I compare two versions of an agent?
Yes. Each conversation is recorded against the version of the agent that handled it, so you can set one version beside the next and compare errors, reply time and cost.
Do you need to be an engineer to use it?
No. The conversations view reads like a transcript, so a product or support lead can review what an agent said. The trace view is there when a developer needs to see why.
How does it work with Agent Testing?
Agent Testing checks a change before it ships, and Agent Insights shows how the agent behaved once it was live. Agent Testing is available on request. Agent Testing
What is your security position?
HIPAA, ISO 27001 and SOC 2 ready. Compliance audits underway. Security and governance
What does it cost?
Agent Insights is part of Agent Builder, and you can start free. The plans are on the Agent Builder pricing page. Agent Builder pricing
Open the next conversation that goes wrong, and see why.
Build an agent in Agentman Studio and its conversations are in Agent Insights from the first one.
HIPAA, ISO 27001 and SOC 2 ready. Compliance audits underway.