Context precision vs context recall?

Precision: how much of what was retrieved is relevant. Recall: how much of what was needed was retrieved. You want both.

jevragasscoremirrors ragas.context_precision / autoevals.ContextPrecision / uipath LegacyContextPrecisionEvaluator

Context precision

Context precision rates what share of the retrieved context is actually relevant to the question, as a 0–3 level. It mirrors RAGAS context_precision, autoevals ContextPrecision and UiPath's ContextPrecisionEvaluator, and measures retrieval quality rather than answer quality.

Run this example in the console 1 jev question · answered inside one request

how it works // frame by frame

What the evaluator does

Jev reads `input`, `context` (and `expected` if present) and rates the proportion of chunks that help answer the question. For a per-chunk breakdown use chunk_relevance, which asks one question per chunk in the same request.

frame 1/4 · the state

state

inputHow is Jev priced?

context5 chunks: price · rate limits · founding · enterprise · legacy billing

question → jev

Rate what proportion of the chunks in `context` are relevant to answering `input`.

ordered levels

none useful

a minority relevant; mostly noise

62%

most relevant; some noise

30%

every chunk relevant

1.3 / 3conf 62%fail · threshold 75%

exactly what jev is asked

The question, verbatim

This is the question the API sends for the example below, generated from the same code path the playground and API use. Jev sees the request fields as state and returns a probability for each outcome. Nothing is generated, so there is nothing to parse.

qscore · ordered rubric

Rate what proportion of the chunks in `context` are relevant to answering `input` (use `expected` as the target answer if present).

0None of the retrieved context is useful for answering the input
1A minority of the context chunks are relevant; most is noise
2Most chunks are relevant; some noise
3Every retrieved chunk is relevant to answering the input

when to use it

Reach for it when

Tuning retrieval: top-k, rerankers, chunk size.
Diagnosing whether a hallucination came from noisy context.
Comparing two retrievers on the same questions.

Not the right tool when

You need to know which chunk was noise (use chunk_relevance).
The context is a single document rather than retrieved chunks.

Watch out for

Precision ignores whether the right chunk was retrieved at all; pair with context_recall.
Very short contexts inflate precision; report it alongside the number of chunks.

inputs and example

What to send

required fields

inputcontext

optional fields

expected

in suites

RAGEverything

result shape

score 0–1 (expected level ÷ top level), label “x.x / n”, confidence, a probability per level, passed at 75% of the scale.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "context_precision"
  ],
  "input": "How is Jev priced?",
  "context": [
    "Price: $42 per billion input tokens. Output free.",
    "Rate limits: 1,200 rpm.",
    "Founded in 2024.",
    "Enterprise plans include ZDR.",
    "Legacy v0 output billing $8/M."
  ],
  "output": "…"
}

expected

Level 1: a minority of chunks (two of five) are relevant.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Mean score

Σ score_i / n

Mean score averages the normalised 0–1 score across items.

Precision, recall and F1

P = TP/(TP+FP) · R = TP/(TP+FN) · F1 = 2PR/(P+R)

Precision is the share of predicted positives that were really positive; recall is the share of real positives the evaluator caught; F1 is their harmonic mean, which is only high when both are.

related evaluators

chunk_relevancejev

Every retrieved chunk gets its own yes/no probability of being relevant to the input, all in the same Jev call. Score is precision (share of relevant chunks); details show the per-chunk verdicts.

context_recalljev

How much of the expected answer is supported by the retrieved context.

context_relevancyjev

Overall relevance of the retrieved context to the input.

noise_sensitivityjev

Did irrelevant context leak into the output as incorrect claims?

questions people ask

Context precision vs context recall?: Precision: how much of what was retrieved is relevant. Recall: how much of what was needed was retrieved. You want both.