JEVAL
jevragasscoremirrors ragas.context_recall / autoevals.ContextRecall

Context recall

Context recall rates how much of the reference answer can be attributed to the retrieved context, as a 0–3 level. It mirrors RAGAS context_recall and autoevals ContextRecall and tells you whether retrieval found what the answer needed.

Run this example in the console 1 jev question · answered inside one request
how it works // frame by frame

What the evaluator does

Jev checks each statement in `expected` against `context` and rates coverage from “not present” to “every statement attributable”.

frame 1/4 · the state
state
expected250k tokens/s and 1,200 requests/min.
contextRate limits: 250,000 tokens per second / 1,200 requests per minute.
question → jev

Rate how much of the reference `expected` answer can be attributed to statements in `context`.

ordered levels
0
not in the context
0%
1
small part supported
0%
2
most supported
4%
3
every statement attributable
96%
3.0 / 3conf 96%pass · threshold 75%
exactly what jev is asked

The question, verbatim

This is the question the API sends for the example below, generated from the same code path the playground and API use. Jev sees the request fields as state and returns a probability for each outcome. Nothing is generated, so there is nothing to parse.

qscore · ordered rubric

Rate how much of the reference `expected` answer can be attributed to statements in `context`.

  1. 0The context does not contain the information needed for the expected answer
  2. 1The context supports a small part of the expected answer
  3. 2The context supports most of the expected answer
  4. 3Every statement in the expected answer can be attributed to the context
when to use it

Reach for it when

  • Measuring retrieval coverage against a golden answer set.
  • Deciding whether a wrong answer was a retrieval miss or a generation error.

Not the right tool when

  • No reference answer (use context_relevancy or chunk_relevance).

Watch out for

  • Requires an `expected` answer written from the corpus; if the reference contains facts the corpus never had, recall is capped.
inputs and example

What to send

required fields
expectedcontext
optional fields
input
in suites
RAGEverything
result shape

score 0–1 (expected level ÷ top level), label “x.x / n”, confidence, a probability per level, passed at 75% of the scale.

POST /api/v1/evaluateopen in console 
{
  "evaluators": [
    "context_recall"
  ],
  "input": "What are Jev's rate limits?",
  "expected": "250,000 tokens per second and 1,200 requests per minute.",
  "context": [
    "Rate limits: 250,000 tokens per second / 1,200 requests per minute."
  ],
  "output": "…"
}
expected

Level 3: both statements are in the context.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Mean score
Σ score_i / n

Mean score averages the normalised 0–1 score across items.

Precision, recall and F1
P = TP/(TP+FP) · R = TP/(TP+FN) · F1 = 2PR/(P+R)

Precision is the share of predicted positives that were really positive; recall is the share of real positives the evaluator caught; F1 is their harmonic mean, which is only high when both are.

related evaluators
context_precisionjev
What share of the retrieved context is actually relevant to the input.
context_entity_recalljev
Share of entities in the expected answer that appear in the context.
faithfulnessjev
Are the claims in the output grounded in the retrieved context?
questions people ask
Why does context recall need an expected answer?
Recall is measured against what the answer should contain. Without a reference there is nothing to recall; use context_relevancy instead.