Why did valid-looking HTML fail is_html?

The check requires every non-void tag to close. Implicitly closed tags like a bare fail on purpose so that output is unambiguous.

code · freepromptfoodeterministicmirrors promptfoo is-html / contains-html

Is HTML

Is HTML checks that the output is HTML with balanced tags, allowing void elements such as br, img and input. Scores 1 for a document or full fragment, 0.5 for HTML inside prose. It mirrors promptfoo's `is-html` and `contains-html`.

Run this example in the console runs in code · $0

how it works // frame by frame

What the evaluator does

Same scanner as is_xml with HTML rules: tag names are case-insensitive and void elements need no closing tag.

frame 1/4 · the two inputs

output

<p>Your order <strong>#77812</strong> has shipped.<br>Track it below.</p>

rules

void: br img input … · case-insensitive

1.walk the tags

2.skip void elements

3.stack empty at the end

htmlscore 1

runs in code · $0 · ~0 ms · deterministic

when to use it

Reach for it when

Email and page generation pipelines.
Detecting whether a model returned markup when plain text was expected.

Not the right tool when

Rendering correctness or accessibility (needs a browser check).

Watch out for

Unclosed <p> or <li> is legal HTML but fails this balance check; it is intentionally strict.

inputs and example

What to send

required fields

output

optional fields

none

in suites

PromptfooEverything

result shape

score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.

POST /api/v1/evaluateopen in console

{
  "evaluators": [
    "is_html"
  ],
  "output": "<p>Your order <strong>#77812</strong> has shipped.<br>Track it below.</p>"
}

expected

Balanced with a void <br>: pass.

rolling it up

Aggregate with

One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.

Pass rate

passed / total

Pass rate is the share of items whose evaluator result met its threshold.

related evaluators

is_xmlcode · free

Output is well-formed XML (tags balance and nest; self-closing allowed). Scored 1 when the whole output is XML, 0.5 when a well-formed fragment is embedded in prose.

regexcode · free

Output matches the regular expression in `pattern` (JavaScript syntax; `/…/flags` form allowed).

questions people ask

Why did valid-looking HTML fail is_html?: The check requires every non-void tag to close. Implicitly closed tags like a bare <p> fail on purpose so that output is unambiguous.