Does Contains understand negation?

No. It is a substring test. Use closed_qa or llm_judge when meaning matters.

code · freeuipathdeterministicmirrors uipath.eval ContainsEvaluator / autoevals ListContains

Contains

Contains checks whether the expected string appears anywhere in the output, case-insensitively. It mirrors UiPath's ContainsEvaluator and autoevals ListContains and is free.

Run this example in the console runs in code · $0

how it works // frame by frame

What the evaluator does

Normalise both strings and test substring inclusion.

frame 1/4 · the two inputs

output

Your refund has been issued and will arrive in 5–7 days.

expected

refund has been issued

1.normalise case and whitespace

2.substring search

foundscore 1

runs in code · $0 · ~0 ms · deterministic

when to use it

Reach for it when

Required phrases, disclaimers, IDs that must appear.
Cheap smoke tests in CI.

Not the right tool when

Paraphrase is acceptable (use answer_similarity).

Watch out for

A negation around the phrase still passes (“we do not offer refunds” contains “offer refunds”).

inputs and example

What to send

required fields

outputexpected

optional fields

none

in suites

CorrectnessEverything

result shape

score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "contains"
  ],
  "output": "Your refund has been issued and will arrive in 5–7 days.",
  "expected": "refund has been issued"
}

expected

Found: score 1.

rolling it up