JEVAL
code · freeautoevalsdeterministicmirrors autoevals.ValidJSON

Valid JSON

Valid JSON checks that the output parses as JSON. It mirrors autoevals ValidJSON and is the cheapest possible structural gate.

Run this example in the console runs in code · $0
how it works // frame by frame

What the evaluator does

JSON.parse; 1 if it succeeds, 0 otherwise.

frame 1/4 · the two inputs
output
{"status":"booked","pnr":"X7K9QF"}
check
JSON.parse(output)
1.attempt to parse
2.no exception → valid
validscore 1
runs in code · $0 · ~0 ms · deterministic
when to use it

Reach for it when

  • Any pipeline that feeds model output to JSON.parse.
  • Detecting markdown fences or trailing commentary around JSON.

Not the right tool when

  • You also need schema conformance (add json_similarity against an expected shape).

Watch out for

  • Valid JSON can still be the wrong shape.
inputs and example

What to send

required fields
output
optional fields
none
in suites
AgentEverything
result shape

score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.

POST /api/v1/evaluateopen in console 
{
  "evaluators": [
    "valid_json"
  ],
  "output": "{\"status\":\"booked\",\"pnr\":\"X7K9QF\"}"
}
expected

Parses: score 1.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Pass rate
passed / total

Pass rate is the share of items whose evaluator result met its threshold.

related evaluators
json_similaritycode · free
Recursive structural similarity between output JSON and expected JSON (fields, arrays, numbers, strings).
exact_matchcode · free
Output equals expected after whitespace and case normalisation.
questions people ask
Does Valid JSON accept JSON inside a code fence?
No. The whole output must be JSON. Strip fences upstream or treat them as a failure to fix.