Decision Models
Feature

Ask an LLM which team owns a ticket and you pay for a paragraph, then parse it back into an if statement — slow, billed on output tokens you discard, and breakable. A decision model drops generation entirely: it takes state and typed questions and returns a distribution over exactly the answers you allowed. The schema is the output space, so no invented option can come back.

There are three primitives. choice (unordered selection: routing, intent, tool choice), score (ordered scale: urgency, intensity, risk) and noul (a yes/no probability: phishing, jailbreak, refund request). Options arrive with the request, so unlike a classic classifier there is nothing to train.

The most important third-party evaluation comes from a University of Bonn, Lamarr Institute and Fraunhofer IAIS group, which ran jev-1.13.0 zero-shot across 37 datasets and 346,009 requests for under $10 of API spend, with Qwen3.8-27B and Gemma-4-E4B scored on identical requests. Jev beat Qwen3.8-27B on 27 of 37 datasets, reached 95–99% on IMDB, SST-2, HellaSwag and ARC, and 86.7% on Belebele across 122 languages. Its choice probabilities were well calibrated and supported selective prediction. One caveat: binary noul probabilities rank well but sit poorly against a fixed 0.5 threshold, so a hardcoded cutoff is the wrong way to read them.

The vendor also publishes its weak spots: multi-step maths and planning (System Two work), counting, date comparison and literal reading. TypeSafe's own "jaggedness" page lists maths, counting and date comparison, and the weakest reported category was trading bots.

TypeSafe AI released Jev 1.14.0 on Oct 14, 2026 under the jev-preview alias. It is a set of small, steady improvements across the board, with the largest expected gain in multilingual support. Pricing is unchanged at $0.042 per million input tokens, with output free.

In a small box test on 36 painting crops, each cutting off part of the target, four per-edge Noul questions caught 30 of 36 cut-offs; a single "is all of it inside" question caught 26, and drawing the box as an outline and asking about it caught 20.Vendor-reported

AIfrontier view Box is a first step from judging to locating. It keeps the typed-output philosophy while reaching toward inspection and localisation tasks. But "always returns a box" is a different trap from type safety in decisions — always pair it with an existence question. Related: Local and cloud LLMs — latest

Within a week of Jev, open projects reimplemented the same interface. They split three ways: bidirectional encoders (no KV cache — fastest on CPU), causal decoders with LoRA adapters (long state, more world knowledge), and extraction-only approaches that read logits from existing models.

Watch the naming collision. Laya is called "the open-source Jev", but most of its headline numbers come from a checkpoint fine-tuned on the benchmark's own training split, not from the base weights. And the Jev column in the Laya comparison table is drawn from third-party publications — the Laya author states he had no access to the TypeSafe API. Always read a number together with its conditions: checkpoint, routing, context length and option count.

October brought the first results measured by someone other than the vendors on a shared yardstick. The Artificial Analysis Decision Index v1.0 weights four evaluations equally: knowledge and hallucination (AA-Omniscience-Decision), reasoning (HLE-MultiChoice), tool calling (When2Call) and classification (WildGuardTest). Source: Artificial Analysis

AIMultiple evaluated 50 decision models and put Jev 1.13 first on classification. On browser tasks, though, Jev frequently returned "blocked": Kev-27B completed 40 of 50 against Jev's 17. Source: AIMultiple

AIfrontier view Most "beats Jev" claims — d1-ultra's, Decider v1.1's — rest on the vendor's own metrics. On independent scoring Jev still leads overall, yet Decider wins on cost, Celeris on speed and Microsoft on knowing when not to answer. Pick the axis that matters for your workload rather than a single composite.

The value of a decision model is not speed; it is a confidence you can put in an if statement. If the model says 0.95 about things that happen 70% of the time, every threshold you set is wrong. That agreement between stated probability and observed frequency is calibration — a property of groups of predictions, not of one answer. The standard summary statistic is ECE (expected calibration error), where zero is perfect.

Laya publishes an ECE of 0.081 against Jev's 0.246, but that is the routed measurement — and Laya's own model card warns that shipped probabilities are overconfident and should be recalibrated on held-out data before they drive consequential actions.

In practice, split confidence three ways: act automatically on high confidence, route medium confidence to a stronger model or a human, hand low confidence to a human. Laya's Act/Escalate head is trained with a cost matrix (+1 correct, −3 wrong, −0.5 escalate), so automatic action only beats escalation above 62.5% accuracy. Temperature scaling reshapes a distribution but cannot repair a ranking that is already wrong — the one property where the open models still trail Jev.