code · freeautoevalsdeterministicmirrors autoevals.Levenshtein
Levenshtein
Levenshtein returns 1 minus the normalised edit distance between the output and the expected string. It mirrors autoevals Levenshtein and is free.
Run this example in the console runs in code · $0
What the evaluator does
Compute the character edit distance, divide by the longer length, subtract from 1.
frame 1/4 · the two inputs
output
calibrated probabilties
expected
calibrated probabilities
1.edit distance = 1 (insert 'i')
2.max length = 24
3.1 − 1/24
0.96score 0.96
runs in code · $0 · ~0 ms · deterministic
Reach for it when
- Near-exact string tasks: OCR, transcription, code identifiers.
- A graded companion to exact_match.
Not the right tool when
- Semantic tasks; a synonym is a large edit distance.
Watch out for
- Length-sensitive: one wrong word in a long sentence scores high.
What to send
required fields
outputexpected
optional fields
none
in suites
CorrectnessEverything
result shape
score 0–1 computed in code, label, passed at an evaluator-specific threshold, no probabilities.
POST /api/v1/evaluateopen in console
{
"evaluators": [
"levenshtein"
],
"output": "calibrated probabilties",
"expected": "calibrated probabilities"
}expected
One insertion over 24 characters: 0.96.
Aggregate with
One result per item is a fact; a dataset of them is a metric. These are the aggregations that fit this evaluator's output shape.
- What is a good Levenshtein threshold?
- jeval passes at 0.8 by default. For OCR-style tasks 0.95 is common; set it from your own data.