How is entity recall different from context recall?

Context recall is about statements; entity recall is about the specific names, numbers and dates. A context can support a statement loosely while missing the exact figure.

jevragasscoremirrors ragas.context_entity_recall / autoevals.ContextEntityRecall

Context entity recall

Context entity recall rates what share of the named entities, numbers, dates and terms in the reference answer appear in the retrieved context. It mirrors RAGAS context_entity_recall and is a sharper retrieval check for fact-heavy answers.

Run this example in the console 1 jev question · answered inside one request

how it works // frame by frame

What the evaluator does

Jev lists the entities implied by `expected` and rates how many are present in `context`, from none to all.

frame 1/4 · the state

state

expected$0.042 per million input tokens; output free.

contextPrice: $42 per billion input tokens ($0.042 per million). Output tokens are free.

question → jev

Consider the entities, numbers, dates and terms in `expected`. Rate what share appear in `context`.

ordered levels

none present

a few present

most present

all present

95%

2.9 / 3conf 95%pass · threshold 75%

exactly what jev is asked

The question, verbatim

This is the question the API sends for the example below, generated from the same code path the playground and API use. Jev sees the request fields as state and returns a probability for each outcome. Nothing is generated, so there is nothing to parse.

qscore · ordered rubric

Consider the named entities, numbers, dates and technical terms in `expected`. Rate what share of them appear in `context`.

0None of the entities (names, numbers, dates, terms) in the expected answer appear in the context
1A few of the expected entities appear in the context
2Most expected entities appear in the context
3All entities in the expected answer appear in the context

when to use it

Reach for it when

Fact-dense domains: finance, legal, medical, specs.
Catching retrieval that found the topic but not the numbers.

Not the right tool when

Reference answers that are conceptual rather than entity-rich.

Watch out for

Entities can be present in a different form (Q3 vs third quarter); the rubric allows equivalents, but check with a few gold cases.

inputs and example

What to send

required fields

expectedcontext

optional fields

none

in suites

RAGEverything

result shape

score 0–1 (expected level ÷ top level), label “x.x / n”, confidence, a probability per level, passed at 75% of the scale.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "context_entity_recall"
  ],
  "expected": "$0.042 per million input tokens; output tokens are free.",
  "context": [
    "Price: $42 per billion input tokens ($0.042 per million). Output tokens are free."
  ],
  "output": "…"
}

expected

Level 3: both entities present.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Mean score

Σ score_i / n

Mean score averages the normalised 0–1 score across items.

related evaluators

context_recalljev

How much of the expected answer is supported by the retrieved context.

faithfulnessjev

Are the claims in the output grounded in the retrieved context?

questions people ask

How is entity recall different from context recall?: Context recall is about statements; entity recall is about the specific names, numbers and dates. A context can support a statement loosely while missing the exact figure.