Does conciseness penalise long answers?

Only unnecessary length. A long answer to a question that needs it can still be concise.

jevragasbooleanmirrors ragas.aspect_critic(conciseness)

Conciseness

Conciseness checks whether an output conveys its content without unnecessary length, repetition or filler. It mirrors the RAGAS aspect critic for conciseness.

Run this example in the console 1 jev question · answered inside one request

how it works // frame by frame

What the evaluator does

Jev returns p(yes) for “no unnecessary length, repetition or filler”, given the output and optionally the input for expected scope.

frame 1/4 · the state

state

inputMinimum Node version?

outputGreat question! Node versions matter a lot. The SDK needs Node 20 or newer, which is to say version twenty or later.

question → jev

`output` conveys its content without unnecessary length, repetition, or filler.

p(yes)

14%

noyes

verboseconfidence |p − 0.5| × 2 = 72%the 0.5 line is yours to move

exactly what jev is asked

The question, verbatim

This is the question the API sends for the example below, generated from the same code path the playground and API use. Jev sees the request fields as state and returns a probability for each outcome. Nothing is generated, so there is nothing to parse.

oknoul · p(yes)

`output` conveys its content without unnecessary length, repetition, or filler.

when to use it

Reach for it when

Chat products where verbosity hurts.
Controlling for length bias in Battle-style comparisons.
Cost control: verbose outputs cost tokens downstream.

Not the right tool when

The task asks for exhaustive detail.

Watch out for

Concise and incomplete are different; pair with answer_relevancy or task_completion.

inputs and example

What to send

required fields

output

optional fields

input

in suites

CorrectnessEverything

result shape

score 0–1 (p of the good outcome), label “nn% yes”, confidence |p − 0.5| × 2, probabilities yes/no, passed at 0.5.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "conciseness"
  ],
  "input": "Minimum Node version?",
  "output": "Great question! Node versions matter a lot. The SDK needs Node 20 or newer, which is to say version twenty or any later version."
}

expected

Low p(concise): filler and repetition.

rolling it up