What does 'exact' mean for tool arguments?

Same key names and same values after JSON parsing. Number formatting differences like 89 vs 89.0 are treated as equal; string differences are not.

code · freeuipathdeterministicmirrors uipath.eval ToolCallArgsEvaluator

Tool call args (exact)

Tool call args (exact) computes the share of expected tool calls that appear in the output with the same name and exactly matching arguments. It mirrors UiPath's ToolCallArgsEvaluator and is free.

Run this example in the console runs in code · $0

how it works // frame by frame

What the evaluator does

For each expected call, look for an actual call with the same name whose arguments are structurally identical (JSON similarity ≥ 0.999). Score is matched / expected.

frame 1/4 · the two inputs

tool_calls

book_flight({flight_id:"UA-873", passenger:"Priya"})

expected_tool_calls

book_flight({flight_id:"UA-873", passenger:"Priya"})

1.match by name

2.compare args structurally

3.count matches / expected

1/1score 1

runs in code · $0 · ~0 ms · deterministic

when to use it

Reach for it when

Arguments that must be precise: recipients, amounts, IDs.
Deterministic CI gates.

Not the right tool when

Schemas or formats vary (use tool_call_semantic).

Watch out for

A renamed key fails the whole call; that is the point, but pair with the semantic version to see intent.

inputs and example

What to send

required fields

tool_callsexpected_tool_calls

optional fields

none

in suites

AgentEverything

result shape

score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "tool_call_args"
  ],
  "output": "…",
  "tool_calls": [
    {
      "name": "book_flight",
      "args": {
        "flight_id": "UA-873",
        "passenger": "Priya"
      }
    }
  ],
  "expected_tool_calls": [
    {
      "name": "book_flight",
      "args": {
        "flight_id": "UA-873",
        "passenger": "Priya"
      }
    }
  ]
}

expected

1/1 matched.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Precision, recall and F1

P = TP/(TP+FP) · R = TP/(TP+FN) · F1 = 2PR/(P+R)

Precision is the share of predicted positives that were really positive; recall is the share of real positives the evaluator caught; F1 is their harmonic mean, which is only high when both are.

Pass rate

passed / total

Pass rate is the share of items whose evaluator result met its threshold.

related evaluators

tool_call_semanticjev

Each expected tool call gets a probability that an equivalent call was made (same tool, arguments equal in meaning even if formatted differently). Exact-match evaluators fail on '2025-11-12' vs 'Nov 12 2025'; this does not.

tool_call_ordercode · free

Do the tool names appear in the expected order (as a subsequence)?

tool_call_countcode · free

Per-tool call counts match the expected counts.

questions people ask

What does 'exact' mean for tool arguments?: Same key names and same values after JSON parsing. Number formatting differences like 89 vs 89.0 are treated as equal; string differences are not.