JEVAL
jeval // universal evaluator api // powered by jev

Every evaluator.
One call.

The evaluators you already use from autoevals, RAGAS and UiPath, re-expressed as calibrated Jev judgments. Send input, output, expected and context once; every score, probability and confidence comes back in a single request, in a few hundred milliseconds, for a fraction of a cent.

why jev for evalsstate + questions // structured output
  • one requestEvery evaluator's questions fan out over the same state in a single call. Adding an evaluator barely moves latency.
  • per chunkGrounding evaluators ask one question per context chunk, per step, per tool call. Still one request.
  • probabilitiesScores come with per-level distributions and confidence, so thresholds are yours to set in code.
  • $0.042 / mtokInput is priced per token and output is free. A full RAG suite on one item costs well under a hundredth of a cent.
1 · RAG examplesedit anything below
2 · evaluators
jev · 6 q0 code
ragas
autoevals
uipath
jeval
6 evaluators · 1 jev request
3 · results
POST /api/v1/evaluatethis is the request the button sends
{
  "evaluators": [
    "faithfulness",
    "answer_relevancy",
    "answer_correctness",
    "context_precision",
    "context_recall",
    "noise_sensitivity"
  ],
  "output": "Jev 1.13 accepts up to 64k tokens per request. Of that, the state plus the single longest question may use at most 32k tokens.",
  "input": "What is the maximum context length for Jev 1.13 and how is it split?",
  "expected": "64k tokens per request; state plus the longest question is limited to 32k tokens.",
  "context": [
    "Context length: 64k tokens per request; 32k tokens for `state` plus the longest question.",
    "Rate limits: 250,000 tokens per second / 1,200 requests per minute.",
    "Price: $42 per billion input tokens. Output tokens are free."
  ]
}
response · shapeone result per evaluator · usage · judge comparison
{
  "results": [
    {
      "id": "faithfulness",
      "kind": "score",
      "score": 0.12,
      "label": "0.5 / 4",
      "confidence": 0.91,
      "probabilities": {
        "0": 0.55,
        "1": 0.4,
        "2": 0.05,
        "3": 0,
        "4": 0
      },
      "rationale": "Most claims in the output are not supported by the context or contradict it",
      "passed": false
    },
    {
      "id": "answer_correctness",
      "kind": "score",
      "score": 0.02,
      "label": "0.1 / 4",
      "confidence": 0.96,
      "passed": false
    },
    {
      "id": "noise_sensitivity",
      "kind": "boolean",
      "score": 0.06,
      "label": "94% yes",
      "rationale": "Irrelevant context leaked into the answer",
      "passed": false
    }
  ],
  "usage": {
    "requests": 1,
    "input_tokens": 1087,
    "cost_usd": 0.000046,
    "latency_ms": 220
  },
  "comparison": {
    "calls": 6,
    "est_cost_usd": 0.013,
    "est_latency_ms_sequential": 11400,
    "cost_multiple": 284,
    "speed_multiple_sequential": 52
  }
}