Local LLMs, Cloud LLMs, Jev and Laya
AI moves from "how smart" to "where it sits"

The short version

The question itself has changed

For several years the AI conversation revolved around which model was smartest. Benchmark rankings were the story, and models kept getting larger. In 2026 that frame has visibly broken. The argument is no longer about the ceiling on intelligence; it is about where each piece of work should sit.

The driver is practical. Most of the judgments a running system makes each day are small and repetitive: is this ticket urgent, which queue does it belong in, does this response violate policy, is this document an invoice. Sending each of those to a frontier generative model means paying generation prices and generation latency for something closer to a lookup.

The cloud price war, the arrival of usable local models, and the emergence of decision models look like three separate stories. They point in the same direction. Here is each in turn.

Cloud LLMs: from capability race to price race

On the frontier, capability differences are getting harder to sell, so price and supply terms have become the competitive axis. We have tracked OpenAI cutting GPT-5.6 Luna by 80%, Grok 4.7 being shelved as 4.8 emerged and GPT-6 Astra reached the API, and what "AI IS OVER" signals about price destruction and capex re-examination. The common thread is rapid model turnover and steadily falling prices.

Frontier input pricing generally sits between $0.20 and $10 per million tokens, with output running roughly five times input. On latency, the third-party benchmark TypeSafe AI cites puts frontier chat models between 3 and 329 seconds. Cloud models remain overwhelmingly capable on hard problems — and neither cheap nor fast. Lower prices do not change that structure.

Local LLMs: running on your own hardware became the baseline

The local picture has changed substantially in six months. A summer 2026 survey on DevelopersIO lays out the shift clearly: the mainstream moved to MoE plus 4-bit quantization, quantization stopped being a user-side trick and became something providers ship up front, and speculative decoding (MTP) became practical. The hardware center of gravity widened from 24GB GPUs to 128GB-class unified-memory machines.

The leading choice for general-purpose work including Japanese is Qwen3.6-35B-A3B — 35B total, 3B active, Apache 2.0 — which reaches 108.3 tok/s with speculative decoding in measured runs. On the tooling side, Ollama got substantially faster in the 0.32 line, matching vLLM on single-stream decode.

For commercial use, Qwen3.6 and Gemma 4 are the easy pair — both Apache 2.0. Some lines have stalled: gpt-oss has not been updated since August 2025, and Llama 4 has had no successor since April 2025. The pattern — open model progress driven by Chinese labs and Google — hardened over this period.

The important point is that this is not a cut-down cloud model. Keeping input data in-house, getting consistent latency, and not being exposed to a provider's terms or price changes are the real reasons to choose local. The case for it is data path and operational autonomy, not capability.

A third path: decision models that never generate text

Then, in September 2026, a different layer appeared: models that produce no text at all and return only typed judgments. The two reference points are TypeSafe AI's Jev and Convai Innovations' Laya.

The idea is simple. Code does not want prose; it wants a decision. You can write if (order.total > 100), but the moment the condition is a judgment, ordinary code falls apart. Is this message angry? Is this email about billing? Which of these twelve links continues the checkout? The old options were to train a classifier or to have a general LLM emit JSON and parse it back. Decision models sit between the two: you define the questions at runtime like an LLM, and get a constrained probability distribution like a classifier. What comes back is not a sentence but a value with a probability attached.

Jev: the "smart if statement"

TypeSafe AI was founded by Diogo Almeida, who worked at OpenAI on the instruction-following research that led to ChatGPT. The company left stealth on September 15, 2026, announcing a $40M seed round led by DCVC. Even the naming is pointed: Jev comes from the Jevons paradox — more efficient steam engines raised coal consumption rather than lowering it — and System One from Kahneman's System 1, fast intuitive judgment. The bet is that cutting the unit cost of a decision by orders of magnitude will put decisions where nobody would have called an LLM before.

The interface is three types, which we covered in our earlier report and can now complete with later sources:

The design detail that matters is parallel sampling. Every question in a request is evaluated independently and in parallel against the same state. One answer never becomes context for another, and adding a question barely changes response time. TypeSafe describes this as having no context rot. Because the answer is a distribution over options you defined, a value outside your schema cannot be produced. The company states its 0% type-error rate is derived from that construction rather than measured.

Pricing is $0.042 per million input tokens, with output treated as effectively free — "too cheap to meter." Published latency is 70 to 500 milliseconds. TypeSafe's own workflow evaluations report 193.6x faster and 444.6x cheaper, but the company notes those workflows were built by its own team and that the choice of reference models biases the result. Read those as a ceiling, not a median.

In Japan, FastMetal began offering JEV 1.13 on September 19, 2026 on a flat ¥0.1-per-call, yen-denominated prepaid model. It is also reachable through OpenRouter (typesafe/jev-1.13), and LangChain published a harness-building write-up on September 17.

Laya: an open-weight decision model

The other notable entry is Convai Innovations' Laya. Its role is close to Jev's; the decisive difference is that its weights are open under Apache 2.0 and it runs entirely on your own hardware.

Architecturally it pairs a ModernBERT-large backbone with a 395M encoder, a two-layer decision head, an option-marker scorer and an act/escalate head — 421M parameters total. It is not autoregressive. The question types are the same three (choice, score, noul), and it shares the key property that the answer space is defined at request time, so no retraining is needed for a new taxonomy.

Three checkpoints ship: laya for English, laya-multilingual (mmBERT-base, 100+ languages), and laya-typed-decisions for typed-decision workflows. The Router selects English or multilingual by script and language; typed-decisions normally requires explicit selection or task detection. Weights are under 1GB, and alongside PyTorch/Transformers there are community runtimes for Node.js and TypeScript (ONNX Runtime) and Apple Silicon (MLX).

But the revealing result is not speed — it is how strongly specialization matters. On the typed-decision evaluation (400 cases, 2,000 decisions, four workflows), the fine-tuned laya-typed-decisions reaches 0.766 overall accuracy, while base Laya manages only 0.362 zero-shot, below a majority-class baseline of 0.461. This is not a model you download and expect to make expert judgments out of the box. The intended pattern is to specialize a small decision model on your own domain, calibrate it, then run it cheaply at volume.

Calibration figures are reported with similar honesty. Temperature refitting is said to cut mean ECE (expected calibration error) from 0.466 to 0.081. Which means the raw probability should not be trusted as-is. If your system branches on "risk above 0.90," you need to recalibrate on your own data first.

Limits are equally clear. Choice questions should stay under roughly 20 options, with hierarchy preferred beyond that. Ordinal scoring is a weaker area, and separating adjacent levels is hard. Input is 512 tokens for the English checkpoint and 1,024 for some variants, so it is not built for dropping in a whole long document — it expects selected evidence.

Jev and Laya: what differs, what does not

Framing these two as "which is better" misses the point. The dividing line is not capability but where control sits. Jev outsources operations. TypeSafe describes its probabilities as calibrated, but they still require evaluation on the target distribution. Laya hands you the weights and leaves calibration, specialization and data placement to you. Which you want depends on how sensitive your data is and how much freedom to swap models you need.

The four-layer architecture is where this lands

Put the four stories side by side and they point one way: one general-purpose model answering every call is a prototyping pattern, not a production one. In practice the stack looks like this.

The payoff is not only cost. It is that confidence comes back as a value. With a probabilistic first layer, "handle high-confidence cases automatically, escalate the middle to a bigger model, send low confidence to a human" becomes explicit application code. The automation boundary that used to be a black box becomes a threshold you can read.

In that architecture, being able to swap the decision layer is what makes the system stable. A call path that is not tied to one vendor, with logging and auth in one place, is what keeps price changes and terms changes from becoming rewrites. Given how fast models turned over in 2026, that is not over-engineering.

Where AIfrontier sits: Sign and decision models

Our Sign language is built on category-theoretic types and composition, aiming to guarantee structural correctness statically. Three things overlap with how decision models are designed.

First, deciding the shape of the answer up front. In Sign, types constrain how values can be built; in Jev and Laya, the option space constrains the output. Both give up the freedom to write anything in exchange for only being able to write the right shape.

Second, separating judgment from generation. A decision model judges; turning that judgment into prose stays with an LLM. Sign likewise puts the separation of meaning-determination from execution at the center of its design.

Third, calibration as a shared problem. That a probability is returned is not the same as that probability being trustworthy. Laya's improvement from an ECE of 0.466 to 0.081 is evidence that a returned number cannot simply be used. Drawing the line between what a type can guarantee and what can only be measured empirically is exactly the problem Sign exists to address.

The Playground on our Sign page demonstrates this: building a Jev call from a Sign schema, then branching and escalating on a threshold. Treating a decision model as a component that returns probabilities, and letting the types and thresholds in your code decide whether to adopt the answer — that split is the starting point.

Three things to watch

First, independent validation of Jev. Its headline numbers — 193.6x faster, 444.6x cheaper, frontier-level accuracy — all come from an evaluation format the vendor designed itself. Public benchmark results, and domain-specific accuracy measured by someone else, are what is missing.

Second, whether Laya's specialization is reproducible. The jump from 0.362 base to 0.766 fine-tuned decides a lot. If that holds in other people's domains, "specialize small and run at volume" becomes a general answer.

Third, whether the major labs ship decision models. There is ample commercial reason to. A person opens a chat app a few times a day; software can make thousands of small decisions in the background. Counted in calls, the decision market could be larger than the generation market — which is exactly the Jevons bet.

Primary sources

All figures below are the vendors' own published numbers; we have not independently verified them.