Decision Models Compared, October 2026
Jev, Clef, d1, Jeff and Kev on one table
Decision models return only probabilities over a fixed set of options — no generated text. A month after Jev appeared on Sep 15, both hosted APIs and open weights are in place. Here is a side-by-side of price, latency, architecture and licence from primary sources, plus why a benchmark number stops meaning anything once you forget who measured it.Oct 16, 2026
A decision model (System One) takes a state (the material to judge) and typed questions, and returns nothing but a probability for each option. Because it generates no text, there is no output-token cost and no decoding wait. Within a month of Jev defining the shape, Cloudflare's Clef, Liquid AI's d1 and the independent Jeff and Kev shipped the same API. This article puts those five lineages into one comparable table.
- Verified Confirmed on the vendor's own blog, documentation or model card
- Vendor claim The vendor's own evaluation or comparison; not independently reproduced
- AIfrontier view Our interpretation, not an established conclusion
1. What is shared, what differs
What is shared is the interface. There are three question types: noul (a yes-probability between 0 and 1), choice (pick one option, with a probability per option and a confidence value) and score (a 2-to-10-step ordered scale, returned as the probability-weighted value). One request can carry several questions, and the options are passed at request time as criteria — so there is no classifier to retrain.
What differs is four things. (1) Whether the internals are public (Jev and the hosted d1 are closed; Clef, d1-3B, Jeff and Kev publish weights). (2) Which modalities are accepted (only Clef-omni reads audio and video). (3) Context length (Kev-27B is validated at 65,536 tokens; Jev offers 64k with an experimental 262k). (4) Licence (Apache 2.0 is the norm, but some Jeff adapters are non-commercial). With those settled, the comparison becomes price and latency.
2. The full comparison (as of Oct 16, 2026)
Split by whether the weights are published. All prices are per million input tokens; no model charges for output, because none generates any.
Hosted APIs (weights closed)
| Model | Vendor / licence | Input | Context | Input price | Latency (source) |
|---|---|---|---|---|---|
| Jev 1.14.0 | TypeSafe AI / closed | Text (images experimental) | 64k (262k opt-in) | $0.042 | 70–500 ms (vendor) / 524.1 ms median (measured by Cloudflare) |
| d1 | Liquid AI / closed | Text, images | Undisclosed | Free (d1:free) / paid $0.02 | 200–300 ms per text decision (vendor) |
| Solar Decide | Upstage / closed | Text | 512K | $0.10 (third-party listing) | Undisclosed |
| d1-ultra | Liquid AI / closed | Text, images | Undisclosed | $0.042 | Undisclosed (released Oct 15) |
| Microsoft-Decision-1 | Microsoft / Foundry | Text | Undisclosed | ~$31 per million tasks on the AA index | AA Decision Index 54 |
| Decisions API | OpenAI / closed | Text, images (reported) | Undisclosed | $0.10 (third-party listing) | ~150 ms (launch video, reported) |
Open weights (run it yourself)
| Model | Vendor / licence | Architecture | Input | Context | Price / latency |
|---|---|---|---|---|---|
| Clef | Cloudflare / Apache 2.0 | Qwen3.8-27B frozen, rank-256 LoRA plus a routing head | Text, images | 64k | $0.24 (Workers AI) / 209.3 ms median, 238.6 ms p95 |
| Clef-flash | Cloudflare / Apache 2.0 | Qwen3.5-9B, same recipe | Text, images | 24,576 | $0.038 (cut from $0.09 on Oct 9) / 38.8 ms median, 122.4 ms p95 |
| Clef-omni | Cloudflare / Apache 2.0 | Qwen3-Omni-30B-A3B (MoE, 3B active) with the speech-output path removed | Text, images, audio, video | — | $0.15 / ~130 ms text, ~150 ms image, ~1.5 s for a 21-second clip |
| d1-3B | Liquid AI / open | Trained from averaged LFM2.5-VL-3B and LFM2.5-2.6B weights | Text, images | — | Free (self-hosted) / 8 ms on an RTX 4090, 30 ms on M5 Pro, 50 ms on Jetson Orin Nano |
| d1-omni-600M | Liquid AI / open | Bidirectional encoder (LFM2.5-Encoder-350M) plus audio and vision encoders | Text + image, or text + audio | — | Free (self-hosted) / experimental checkpoint |
| Jeff (firelex) | firelex / code MIT, weights Apache 2.0 (some adapters non-commercial) | Qwen3.5-0.8B base plus 15 task-specific LoRA adapters (v1.3 is adapter-first) | Text | 65,536 (accuracy by length in each card) | Free (self-hosted) / 0.25 s per decision on M4 Max, 22–29 ms on RTX PRO 6000, 463 ms–1 s on CPU |
| Decider v1.1 27B | Perplexity / Apache 2.0 | 27B, non-causal attention | Text, images | 8,192 | $0.02 / $17 per million tasks on the AA index (cheapest) |
| Kev 1.0 | Jared Palmer / Apache 2.0 | 0.8B/4B/9B are Qwen3.5-Base with a rank-16 LoRA and pointer head; 27B is a full fine-tune of Qwen3.8-27B | Text | 8,192 (65,536 for 27B) | Free (self-hosted); Kev-4B also on OpenRouter at $0.042 / H100: 18.1 ms (4B), 75.0 ms (27B) |
Verified Mind the name collisions. Jeff 1 (Gestalt Lab, Qwen3-4B plus a LoRA) and Jeff (firelex, 0.8B) are different projects; this article means the latter. Laya (Convai) ships three variants — English, multilingual and task-specialised — and their numbers are not measured under the same conditions.
3. The five lineages
Jev (TypeSafe AI) — the reference point that fixed the formatVerified
Announced on Sep 15, 2026 as "the first System One model"; the current release is 1.14.0 (Oct 14, jev-preview). Input is $0.042 and output is free, with rate limits of 100,000 tokens/s and 40 req/s. Version 1.14.0 added three experimental features — image input, a box question type that returns a bounding box, and 262k context — but TypeSafe states plainly that none of them is production-ready ("danger zone"). Independent evaluation found choice probabilities are well calibrated, while noul probabilities rank well but do not suit a hard-coded 0.5 threshold. Source: TypeSafe AI
Clef / Clef-flash / Clef-omni (Cloudflare) — three generations in one weekVerified
Clef (27B) and Clef-flash (9B) shipped under Apache 2.0 on Oct 1, and Clef-omni followed on Oct 9. They are API-compatible with Jev: change the endpoint and the model name and you have switched. Cloudflare's own measurements put Clef at 209.3 ms median (238.6 ms p95) and Clef-flash at 38.8 ms (122.4 ms p95), well under the Jev figure they measured (524.1 ms / 536.0 ms). On Oct 9 they also cut Clef-flash from $0.09 to $0.038 — about 60% — and sped up Clef itself by 1.7–2.0x. In production, their threat-intelligence team classifies a domain in 2.2 s, against 4.7 s for their fastest general LLM (gpt-oss-120b), which also returned only two classifications. Source: Cloudflare / Clef-omni and the price cut
AIfrontier view Clef-omni is the first product to drop the "text-only" assumption for decision models, folding the transcribe-then-judge pipeline into a single pass. But its decision quality on video trails the main Clef on many published benchmarks (69.3 versus 82.95 on the home-appliance case): "sees every modality" and "is accurate" are separate claims.
d1 (Liquid AI) — open weights as of Oct 7Verified
The hosted d1 launched on Sep 29 as d1:free; the paid tier is $0.02 input with $0 output. Across six real applications it claims to match or beat GPT-6.1 Sol on four, at 19x to 200x lower cost. On Oct 7 Liquid published the open-weight d1-3B and d1-omni-600M. d1-3B scores 48.57 on the Decision Index v0.2.1 public split — ahead of every model under 10B and on par with Decider 35B-A3B, twelve times its size. A larger hosted sibling, d1-ultra ($0.042, text and images), followed on Oct 15. Liquid claims it beats Jev 1.14 across 112 evaluations and leads other decision models on the Artificial Analysis Decision Index by at least 8 points.Vendor-reported Source: Liquid AI (hosted) / d1-3B and d1-omni-600M
AIfrontier view What matters about d1-3B is where it runs, not its accuracy: 8 ms on an RTX 4090 and 50 ms on a Jetson Orin Nano puts decision-making on the device, not in a data centre. Three questions cost only 1.3x the time of one, which suits asking many questions about a single state.
Jeff (firelex) — a 0.8B model in front of your 27BVerified
A 0.8B decision model trained on a home GPU. Version 1.3 switches the base from "compete zero-shot" to adapter-first, shipping 15 task-specific LoRA adapters plus GGUF files. In the author's measurements, letting Qwen3.8-27B make every decision scored 86.6%; Jeff with an adapter scored 94.6%, and Jeff with the 27B as a fallback on low-confidence items scored 95.0% — while time per decision fell from 8.1 s to 0.46 s. Wired into a coding agent as Jeff-Code, it held pass rate at 62.4% versus 62.8% across 1,242 tasks on six benchmarks and cut time per task by 32%. Source: GitHub (firelex/jeff)
AIfrontier view The practical trap is mixed licensing. The code is MIT and the base and most adapters are Apache 2.0, but the sanctions and soc (security-alert triage) adapters are CC BY-NC 4.0 — non-commercial, and aml and trading-desk carry their data sources' terms. "Just take the accurate adapter to production" does not clear. Note too that the v1.3 base deliberately trades away zero-shot accuracy on long, unfamiliar option lists (legal-clauses: 66.0% to 7.4%), so if you want no adapter, use v1.2.
Kev (Jared Palmer) — TypeSafe's SDK works unchangedVerified
Four sizes — 0.8B, 4B, 9B and 27B — released together as Kev 1.0. The three smaller ones put a rank-16 LoRA and a pointer head on a Qwen3.5 base; the 27B is a full fine-tune of Qwen3.8-27B shipping as 51 GB of weights. The practical draw is that TypeSafe's Python SDK connects unchanged: point it at your local Kev server and you have migrated. On the community Decision Index (breadth-v1 test split, held out), Kev-27B scores 52.3 against a reference Jev of 54.0 — and the author states plainly that Jev has only been run on the development sets, so this is not a controlled comparison. Source: GitHub (jaredpalmer/kev)
AIfrontier view Kev's value is that you can train and serve your own. There is a skill that runs the whole loop from finding your questions to serving the result, a one-command HTTPS endpoint, and a deployment that scales to zero when idle. The shift from "buying a decision model" to "operating one" starts with this lineage.
4. Reading accuracy and speed
A benchmark number means whatever the measurer made it mean. Below are Cloudflare's own measurements of their models and Jev under identical conditions (the Jev figures are Cloudflare's too).
| Benchmark | Clef-omni | Clef | Clef-flash | Jev |
|---|---|---|---|---|
| BFCL case exact | 98.2 | 98.47 | 98.76 | 95.75 |
| ToolRet nDCG@10 | 66.6 | 69.19 | 66.43 | 65.28 |
| API-Bank accuracy | 92.7 | 91.93 | 93.11 | 88.19 |
| Home appliances case exact | 69.3 | 82.95 | 97.73 | 52.27 |
| When2Call accuracy | 63.3 | 72.37 | 65.58 | 80.97 |
| BANKING77 macro-F1 | 94.8 | 94.20 | 90.93 | 79.74 |
| CLINC150+OOS macro-F1 | 97.7 | 97.43 | 66.77 | 89.27 |
| BRIGHT nDCG@10 | 42.0 | 45.91 | 39.26 | 47.52 |
| PhishNChips accuracy | 73.2 | 79.60 | 75.05 | 62.55 |
Three things follow. (1) Smaller is not automatically worse — the 9B Clef-flash leads on BFCL, API-Bank and home appliances. (2) Jev still wins some columns (When2Call, BRIGHT). (3) The multimodal Clef-omni trails the main Clef on text. The obvious conclusion shows up in the data: measure your own task.
Latency behaves the same way. TypeSafe quotes Jev at 70–500 ms, yet Cloudflare measured a 524.1 ms median — past the stated ceiling. Laya appears as 5.8 ms (Cloudflare), ~33 ms (the vendor) and 39.5 ms (its model card, on a Tesla T4). Different measurer, place and conditions move the same model.
The value of a decision model is that its confidence can go straight into an if. If it says 0.95 and the event happens 70% of the time, every threshold you set is wrong. Laya's published ECE (expected calibration error, 0 is ideal) is 0.213 on its task-specialised variant and 0.425 on a many-option task, and its own model card warns that shipped probabilities are overconfident and should be calibrated on held-out data before controlling anything important. Jev, independently, is reported to rank noul well but to fit badly with a hard-coded 0.5 threshold. Re-measure thresholds on your own labelled data, for your own task.
On the Artificial Analysis Decision Index v1.0, Jev 1.13 leads at 58, with GLiDE and Microsoft-Decision-1 on 54. Decider v1.1 27B is cheapest at $17 per million tasks, and Celeris-1 Decision fastest at 0.06 s. Every model is overconfident, especially on knowledge and reasoning. d1-ultra was not in the published write-up at the time. Source: Artificial Analysis
5. Choosing by requirement
| Requirement | First pick | Why |
|---|---|---|
| Cheapest at high volume | Clef-flash ($0.038) / d1-3B (self-hosted) | Lowest input price band with a 38.8 ms response — good for piping many deterministic decisions |
| Images as evidence | Clef / Clef-flash / d1 / d1-3B | Built-in vision encoders; Jev's image support is experimental and not production-ready |
| Audio and video | Clef-omni | The only option as of Oct 16, 2026; a 21-second clip is scored in ~1.5 s |
| Long documents | Kev-27B (65,536) / Jev (262k, experimental) | Longest validated context; Solar Decide offers 512K but with little published accuracy detail |
| Maximum accuracy | Jev / Kev-27B / d1-ultra | Top of the Decision Index family of metrics — though rankings move with the measurer |
| Data cannot leave | Clef family / Kev / d1-3B / Jeff | Weights can be self-hosted; Apache 2.0 is the norm |
| Keep existing Jev code | Clef / d1 / Kev / Jeff | Migration is an endpoint and model-name change |
| Put something in front of a 27B | Jeff (0.8B) | 27B alone 86.6% to 95.0% with Jeff, and 8.1 s to 0.46 s per decision, per the author |
Input prices across the major models sit between $0.02 and $0.24, with Jev's $0.042 near the middle. The spread is roughly 10x — an order of magnitude below output-billed general LLMs. So where you place the decision layer matters more than which model tops a leaderboard: a hosted API, your own GPU, or a small model in front of a 27B. As of October 2026 the work is choosing among those three placements, with measured calibration and a licence check attached. Related: the Jev standing hub / the local and cloud LLM hub
Sources
- TypeSafe AI「Introducing System One Models and Jev」 https://typesafe.ai/blog/introducing-system-one-models-and-jev
- Cloudflare「What is a decision model? Introducing Clef and Clef-flash」 https://blog.cloudflare.com/clef-decision-models
- Cloudflare「Clef-omni takes audio, video, image, and text input」 https://blog.cloudflare.com/clef-faster-cheaper-multimodal
- Liquid AI「Introducing d1: The most capable decision model, now with vision」 https://www.liquid.ai/blog/d1-decision-model
- Liquid AI「Open d1: Edge decision models for text, vision, and audio」 https://www.liquid.ai/blog/d1-open
- GitHub「firelex/jeff」 https://github.com/firelex/jeff
- GitHub「jaredpalmer/kev」 https://github.com/jaredpalmer/kev
- Qiita「Comparing System One decision models」 https://qiita.com/nogataka/items/a2f89a94d243b1b715cb
- System One Models (directory) https://systemonemodels.org/
- Liquid AI「d1-ultra」 https://www.liquid.ai/blog/d1-ultra
- Artificial Analysis「Decision Index v1.0」 https://artificialanalysis.ai/articles/artificial-analysis-decision-index
- Hugging Face「perplexity-ai/pplx-decider-v1.1-27b」 https://huggingface.co/perplexity-ai/pplx-decider-v1.1-27b
- Microsoft「Microsoft-Decision-1」 https://commandline.microsoft.com/microsoft-decision-1-model-foundry/