Chunk support
Chunk support asks, for each retrieved chunk, whether it supports the output's claims, contradicts them, or says nothing about them. Contradictions are the hallucination signal, located to the chunk. It is per-chunk faithfulness attribution in one request.
What the evaluator does
N Choice questions (supports / contradicts / unrelated) in the same request. Score is (supporting − contradicting) / N, clipped at 0; a single contradiction fails the check.
How does `context[i]` relate to the claims in `output`: supports, contradicts, or unrelated?
Price: $42 per billion input tokens. Output tokens are free.
The company was founded in 2024 and is headquartered in San Francisco.
Output tokens are billed at $8 per million on the legacy v0 endpoint, which is deprecated.
The question, verbatim
This is the question the API sends for the example below, generated from the same code path the playground and API use. Jev sees the request fields as state and returns a probability for each outcome. Nothing is generated, so there is nothing to parse.
How does `context[0]` relate to the claims made in `output`? Judge this chunk on its own.
- supportsThe chunk contains evidence that supports one or more claims in the output
- contradictsThe chunk contradicts one or more claims in the output
- unrelatedThe chunk neither supports nor contradicts the output
How does `context[1]` relate to the claims made in `output`? Judge this chunk on its own.
- supportsThe chunk contains evidence that supports one or more claims in the output
- contradictsThe chunk contradicts one or more claims in the output
- unrelatedThe chunk neither supports nor contradicts the output
How does `context[2]` relate to the claims made in `output`? Judge this chunk on its own.
- supportsThe chunk contains evidence that supports one or more claims in the output
- contradictsThe chunk contradicts one or more claims in the output
- unrelatedThe chunk neither supports nor contradicts the output
Reach for it when
- Explaining a low faithfulness score: which chunk disagrees?
- Building citation UIs: show the chunks that support each answer.
- Detecting conflicting sources in the retrieved set.
Not the right tool when
- No context (this is a RAG evaluator).
Watch out for
- Unrelated chunks are not a failure here; use chunk_relevance for noise.
What to send
score mapped from the chosen option, label = option key, confidence, a probability per option, passed by option. Plus a details array with one row per chunk.
{
"evaluators": [
"chunk_support"
],
"output": "Jev is billed per input token at $0.042 per million; output tokens are free.",
"context": [
"Price: $42 per billion input tokens. Output tokens are free.",
"Founded in 2024.",
"Output billed at $8/M on the legacy v0 endpoint."
]
}Chunk 1 supports, chunk 2 unrelated, chunk 3 contradicts (legacy billing).
Aggregate with
One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.
Precision is the share of predicted positives that were really positive; recall is the share of real positives the evaluator caught; F1 is their harmonic mean, which is only high when both are.
Pass rate is the share of items whose evaluator result met its threshold.
- What does a contradicting chunk mean?
- The retrieved set disagrees with the answer somewhere. Either the answer is wrong or the corpus has conflicting documents; either way you know which chunk to read.