JEVAL
code · freepromptfoodeterministicmirrors promptfoo regex

Regex

Regex checks that the output matches a regular expression you supply in `pattern`. It mirrors promptfoo's `regex` assertion, runs in code and costs nothing.

Run this example in the console runs in code · $0
how it works // frame by frame

What the evaluator does

The pattern is compiled as a JavaScript RegExp (a `/…/flags` form is accepted) and tested against the whole output. Score is 1 or 0; an uncompilable pattern fails with a clear label.

frame 1/4 · the two inputs
output
Ticket #48213 has been escalated.
pattern
^Ticket #\d{5}\b
1.compile the pattern
2.test against the output
matchscore 1
runs in code · $0 · ~0 ms · deterministic
when to use it

Reach for it when

  • Format contracts: an ID shape, a date format, a required prefix, one line only.
  • Guarding structured outputs before parsing them.
  • Cheap CI gates that never need a model.

Not the right tool when

  • Meaning rather than shape (use closed_qa or llm_judge).
  • Multi-line JSON or XML (use contains_json, is_xml).

Watch out for

  • Anchor the pattern (^…$) when the whole output must conform; an unanchored pattern matches anywhere.
  • Escape backslashes twice when the pattern travels through JSON.
inputs and example

What to send

required fields
outputpattern
optional fields
none
in suites
PromptfooEverything
result shape

score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.

POST /api/v1/evaluateopen in console 
{
  "evaluators": [
    "regex"
  ],
  "output": "Ticket #48213 has been escalated.",
  "pattern": "^Ticket #\\d{5}\\b"
}
expected

Match: score 1.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Pass rate
passed / total

Pass rate is the share of items whose evaluator result met its threshold.

Accuracy
(TP + TN) / (TP + TN + FP + FN)

Accuracy is the share of items the evaluator got right.

related evaluators
starts_withcode · free
Output starts with the expected string (case-insensitive, whitespace-trimmed).
containscode · free
Output contains the expected string (case-insensitive).
contains_allcode · free
Output contains every string in `values` (case-insensitive). Score is the share present.
valid_jsoncode · free
Output parses as JSON.
questions people ask
Which regex syntax does jeval use?
JavaScript RegExp. Pass either a bare pattern or the /pattern/flags form; the `i` flag makes it case-insensitive.