Which ROUGE variant is the score?

ROUGE-1 F1 is the score and drives the pass; ROUGE-2 F1 is shown in the label for reference.

code · freepromptfoodeterministicmirrors promptfoo rouge-n

ROUGE-N

ROUGE-N measures n-gram overlap between the output and a reference: the score is ROUGE-1 F1 with ROUGE-2 reported alongside. It mirrors promptfoo's `rouge-n` with its default 0.75 threshold and runs in code.

Run this example in the console runs in code · $0

how it works // frame by frame

What the evaluator does

Both texts are lower-cased and tokenised; unigram counts are clipped and compared to give precision and recall, combined as F1. Bigrams give ROUGE-2.

frame 1/4 · the two inputs

output

refunds are issued within 5 7 business days after we receive the returned item

expected

refunds are processed within 5 to 7 business days after we receive the item

1.tokenise both

2.clipped unigram overlap

3.precision, recall → F1

0.80score 0.8

runs in code · $0 · ~0 ms · deterministic

when to use it

Reach for it when

Summarisation regression against reference summaries.
Any place a lexical similarity is wanted for free before spending on a judgment.

Not the right tool when

Paraphrase-heavy outputs (use answer_similarity or faithfulness).
Short answers where one word swings the score.

Watch out for

ROUGE rewards copying the reference; it says nothing about truth.

inputs and example

What to send

required fields

outputexpected

optional fields

none

in suites

PromptfooEverything

result shape

score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "rouge_n"
  ],
  "output": "Refunds are issued within 5-7 business days after we receive the returned item.",
  "expected": "Refunds are processed within 5 to 7 business days after we receive the item."
}

expected

High unigram overlap: ROUGE-1 around 0.8.

rolling it up