playground // one jev request

Playground

The evaluators you already use from autoevals, RAGAS, UiPath and promptfoo, re-expressed as calibrated Jev judgments. Send input, output, expected and context once; every score, probability and confidence comes back in a single request, in a few hundred milliseconds, for a fraction of a cent.

One request: Every evaluator's questions fan out over the same state in a single call. Adding an evaluator barely moves latency.
Per chunk: Grounding evaluators ask one question per context chunk, per step, per tool call. Still one request.
Probabilities: Scores come with per-level distributions and confidence, so thresholds are yours to set in code.
$0.042 per million tokens: Input is priced per token and output is free. A full RAG suite on one item costs well under a hundredth of a cent.

Is the answer grounded, relevant and correct given what was retrieved?

1Start from a RAG example

Edit anything below

Inputtask or questionOutput *the text under evaluationExpectedreference answerContextone chunk per line, 3 chunks, each judged individuallyCriteriarubric for judge-style evaluators

2Choose evaluators

Jev · 6 q0 in code

ragas

autoevals

uipath

jeval

promptfoo

6 evaluators, 1 Jev request

3Request preview

POST /api/v1/evaluateWhat the button sends

{
  "evaluators": [
    "faithfulness",
    "answer_relevancy",
    "answer_correctness",
    "context_precision",
    "context_recall",
    "noise_sensitivity"
  ],
  "output": "Jev 1.13 accepts up to 64k tokens per request. Of that, the state plus the single longest question may use at most 32k tokens.",
  "input": "What is the maximum context length for Jev 1.13 and how is it split?",
  "expected": "64k tokens per request; state plus the longest question is limited to 32k tokens.",
  "context": [
    "Context length: 64k tokens per request; 32k tokens for `state` plus the longest question.",
    "Rate limits: 250,000 tokens per second / 1,200 requests per minute.",
    "Price: $42 per billion input tokens. Output tokens are free."
  ]
}

Response shapeOne result per evaluator, plus usage and a judge comparison

{
  "results": [
    {
      "id": "faithfulness",
      "kind": "score",
      "score": 0.12,
      "label": "0.5 / 4",
      "confidence": 0.91,
      "probabilities": {
        "0": 0.55,
        "1": 0.4,
        "2": 0.05,
        "3": 0,
        "4": 0
      },
      "rationale": "Most claims in the output are not supported by the context or contradict it",
      "passed": false
    },
    {
      "id": "answer_correctness",
      "kind": "score",
      "score": 0.02,
      "label": "0.1 / 4",
      "confidence": 0.96,
      "passed": false
    },
    {
      "id": "noise_sensitivity",
      "kind": "boolean",
      "score": 0.06,
      "label": "94% yes",
      "rationale": "Irrelevant context leaked into the answer",
      "passed": false
    }
  ],
  "usage": {
    "requests": 1,
    "input_tokens": 1087,
    "cost_usd": 0.000046,
    "latency_ms": 220
  },
  "comparison": {
    "calls": 6,
    "est_cost_usd": 0.013,
    "est_latency_ms_sequential": 11400,
    "cost_multiple": 284,
    "speed_multiple_sequential": 52
  }
}