Diogo Almeida, founder, TypeSafe
Models person been superhuman astatine chat for years, truthful wherever is each the automation?
This has been my driving mobility for the past 4 years. At OpenAI, I helped build the methods that made connection models useful astatine pursuing instructions and talking pinch people. That activity ended up arsenic the investigation down ChatGPT. At the time, I thought possibly chat models would lead to AGI, but contempt the hype it became evident to maine that location was thing really large missing.
After 2 years successful stealth, countless method challenges, and investigation breakthroughs… I americium beyond excited to denote that today, TypeSafe AI is releasing our first System One Model: a caller people of frontier models built to make fast, system decisions that package tin usage directly.
We built a caller stack wholly focused connected automation: pinch a caller exemplary architecture, parallel sampler for maximum efficiency, and training method we telephone Reinforcement Learning for Calibrated Decisions (RLCD).
Our first nationalist exemplary is Jev, disposable coming successful early access. Jev achieves akin levels of intelligence connected System One tasks compared to existing LLMs, while being 2 orders of magnitude faster and much efficient. While Jev gives up drawstring generation, it’s optimized for system outputs and can’t hallucinate.
Think of Jev arsenic a frontier-intelligence usability call: unstructured authorities in, typed probabilistic decisions out.
Extraordinary claims require bonzer grounds truthful spot beneath for the receipts. 💅
Frontiers, Old and New
Optimized with | Reinforcement Learning pinch Human Feedback (RLHF) / Reinforcement Learning pinch Verifiable Rewards (RLVR) | Reinforcement Learning for Calibrated Decisions (RLCD) |
Optimizes for | Human preference: writeups and chat responses that quality raters prefer. Verifiable rewards: outputs that tin beryllium programmatically verified. | Calibrated decisions: answers pinch epistemically honorable probabilities connected System One tasks. |
Inputs | Unstructured information (e.g. text) pinch an accent connected sequential messages. | Unstructured information (e.g. text) pinch an accent connected structured programme state. |
Outputs | Strings / generated text. Strings are elastic and tin beryllium anything: chat responses, code, hallucinations, refusals, aliases moreover type-safe system values. To beryllium utilized by software, responses request to beryllium parsed + validated. There is besides ever immoderate consequence that the AI goes disconnected the rails. | Type-safe system values. Possible outputs and building are defined successful advance. The exemplary ne'er makes type errors. All answers are accompanied pinch calibrated probabilities and assurance scores. |
Sampling | Sequential. Generates 1 token astatine a time, each conditioned connected the last. | Parallel. Generates each outputs successful a azygous query. Incredibly businesslike and hardware-aware. |
Cost | Input tokens: from $0.20 to $10 / MTok. Output tokens: ~5x much costly than input tokens. | Input tokens: $0.042 / MTok ($42 per cardinal tokens). Output tokens:(too inexpensive to meter). |
Speed | End-to-end consequence clip is 3 to 329 seconds for frontier models. Fast capable for interfacing pinch humans, but a large bottleneck erstwhile integrated successful code. | End-to-end consequence clip is 70ms-500ms for TypeSafe. This tin scope from 40x-200x faster for the aforesaid levels of frontier intelligence for System One shaped queries. |
Confidence | Even if prompted for a assurance estimate, models thin to beryllium overconfident and inconsistent. If a exemplary tin do a task 95% of the clip but doesn’t opportunity erstwhile it’s successful the 5%, it can’t automate that task. | Always communicates assurance and uncertainty pinch each output. Calibrated: higher assurance intends higher accuracy. More consistent: returns akin answers for akin inputs. |
Use cases | Human-in-the-loop tasks (chatbots, copilots, coding agents). General and powerful, but requires quality oversight because their state besides intends they mightiness spell disconnected the rails.
Demos. The elasticity of strings allows it to beryllium unthinkable for quickly making prototypes that only activity sometimes. | AI-Powered Workflows / smart if-statements. Structured outputs slot into mean package arsenic fuzzy determination rules: classify, route, score, extract, aliases branch wherever hand-written logic is excessively brittle. The surrounding codification constrains their freedom, making them easier to constitute into reliable systems. Map-reducing over large data. Turn petabytes of information into features and insights. Real-time applications. 100ms speeds intends you tin usage AI successful your applications wherever UX is critical.Verify everything. Score, judge, verify, guardrail, and observe jailbreaks of LLM prompts, reasoning traces, and/or outputs. |
Evidence / Technical Results
We emotion skeptics, and are skeptics ourselves.
There are immoderate claims you tin easy verify:
Speed per call: We genuinely are that fast, though our published evals are mostly tally from our laptops connected the West Coast (this is wherever our work is presently based).
Cost per call: We make our pricing transparent. We can’t beryllium it isn’t subsidized; we’ll request the semipermanent to beryllium the sustainability of our pricing (which we expect to spell down, not up).
No type errors: This would beryllium an easy point to falsify pinch conscionable a azygous counter-example, but it is mathematically impossible.
For our bolder claims, we want to supply arsenic overmuch nuance arsenic we can.
Side-by-side demonstration
Our side-by-side demo shows a cardinal quality betwixt our models and LLMs: Jev outputs each probabilities successful parallel alternatively of autoregressively generating by token. Strings are highly powerful and general, but costly. “Giving up” strings really gives america a batch of superpowers!
Nuance:
For group pinch early entree to TypeSafe, present is the actual query.
The query is highly simplified and questions were chosen to person descriptive, human-readable keys truthful that the output connected the surface is understandable.
The authorities is besides a short, dense, and elaborate paragraph, to stress the quality successful sampling methodology. The comparatively shorter input paints our exemplary successful an advantageous light.
For the keen eyed, for the recorded run, the only disagreement pinch GPT-5.6 Terra is connected “Churn likelihood level”. The existent reply seems genuinely ambiguous to us.
We utilized GPT-5.6 Terra pinch default reasoning for this example, because we’ve recovered it to beryllium the astir comparable astatine intelligence to Jev connected average.
Fun fact: a akin demo was what convinced america to spell all-in successful the guidance of System One Models!
Workflow evals
We made a caller type of information to measurement really good AI useful wrong code. We don’t optimize for a crushed truth classification orand let the harness and exemplary to alteration (potentially allowing for overfitting via harness engineering). Instead, we presume location is simply a correct compute chart (a “workflow” represented successful code) and usage the predictions of the largest, smartest, and astir costly outer models arsenic reference probabilities.
Rephrased: each exemplary gets the aforesaid workflow. We trial really they comparison to the mean of the smartest models (in this case, Astra and Fable).
Jev is disconnected the charts – owning the Pareto frontier for almost 2 orders of magnitude. We besides comparison to models pinch a generated punctual doing each the logic successful their chain-of-thought, but this tends to do importantly worse than utilizing the workflow itself.
Note that the calls present are importantly much analyzable than the side-by-side objection above. That’s because they’re much typical of the types of accumulation workloads needed for existent business automation. Below is the simplest of the 4 workflows we’re publishing:
The astir reliable real-world workflows thin to person galore independent, decomposed questions, pinch fine-grained behaviour that’s limited connected probabilities alternatively of discrete decisions. The extremity consequence is discrete branching, but really we get to a last reply involves a batch of domain-specific engineering that needs to beryllium done highly consistently.
See our workflow evals site for each the details: examples, disagreements, afloat queries, and each workflow.
Nuance:
This is wherever the claims of 193.6x faster, 444.6x cheaper connected our location page comes from, and we expect that these are connected the higher extremity of existent world gains.
These contented of these workflows were not deliberately chosen nor constructed to make our exemplary look good, and are not successful our training distribution. However, they were made by individuals connected our exemplary capabilities team, truthful immoderate bias could exist.
We usage the mean of GPT-6 Astra and Fable 5.1 arsenic the reference answer, which biases answers towards OpenAI and Anthropic’s models. We apt underestimate the comparative capacity of our exemplary and DeepSeek’s models.
The LLMs usage our System One LLM wrapper, which constrains LLMs to output system decisions compatible pinch our API. We person recovered this to beryllium the astir meticulous measurement to get decisions from LLMs, but this tends to beryllium slower and much costly than giving decisions without probabilities.
Hallucination and Type-safety
Hallucination and type-safety are intrinsically related, and we deliberation the second is array stakes for automation. Having a hallucinated instrumentality telephone is inconvenient successful an agent, but is an absolute deal-breaker if it’s portion of a strategy pinch latency guarantees aliases it’s buried respective layers heavy successful a dependency chain. Existing models, no matter really smart, still hallucinate and person type errors.
Nuance:
The numbers for LLMs are from OpenRouter i.e., location almost surely is bias here: much analyzable queries mightiness beryllium routed to amended models.
Our number is not empirical. Schema matching is guaranteed, frankincense we tin confidently adhd 0% into the plots.
Fun Demos
Perhaps the astir breathtaking portion of our activity is enabling caller usage cases. We person a batch much to show you, but present are a mates of the team’s favorites:
Doom
We emotion really this doomo doomonstrates real-time intelligence and what tin beryllium doone pinch codification + AI. The technologist down it was worried astir making 10 queries a 2nd (which ends up costing ~$7/hour), but the remainder of america agreed that was little than expected! This is truthful nosy we intend to not only merchandise an in-depth walkthrough, but besides big immoderate events to hack connected this.
Nuance:
The demo is connected system authorities arsenic a information building pinch text, not connected images (yet…)
A non-AI punishment bot could play better, but we wanted a bot that was reactive to different representations of crippled state, and astir importantly… pursuing instructions was cool arsenic heck!
Wikiracing
The nonsubjective of the crippled is to commencement connected 1 Wikipedia page and scope a circumstantial different Wikipedia page utilizing only links you travel crossed while traversing. Each measurement tin mean choosing betwixt hundreds to thousands of links! It’s a awesome playground for demonstrating not conscionable intelligence-per-second, but besides the compounding benefits of not hallucinating pinch high-cardinality choices.
Nuance:
As acold arsenic we know, it was wholly random that some the 2nd and 3rd challenges started pinch “Rubber Duck.” The writer only noticed erstwhile the squad pointed it out.
Our speedups present thin to beryllium a batch little than successful erstwhile demos. That’s because this is against the non-reasoning modes of the models (except Astra which was group to the lowest reasoning setting). This is besides why Jev tended to decorativeness successful less steps (a motion of greater intelligence). This was to make the demo much bearable to watch. The LLMs look overmuch worse astatine this task than pinch reasoning enabled.
Jev supports a cardinality up to 255. For the higher cardinality choices, we do a 2 stage-system of scoring independently past making an definitive choice, hence the occassional slowdown.
What’s next
We’re still successful Jev’s early days. We person a batch much successful the pipeline and are truthful excited to support connected shipping 🔥.
Today, we are opening early access and bringing developers disconnected the waitlist arsenic quickly arsenic we can. We want to perceive which decisions you request to automate, wherever Jev works, and wherever it falls short. Tell america what sci-fi you want to build!!
We started TypeSafe because we judge that AI needs an interface package could dangle on. We can't hold to spot caller usage cases continuously diffuse done the organization and economy.
We Give A FAQ
Where do the names “System One Models” and “Jev” travel from?
We were inspired by Daniel Kahneman, Thinking, Fast and Slow. The exemplary people sanction draws connected the favoritism betwixt fast, intuitive System 1 reasoning and slow, deliberate System 2 reasoning.
“System 1 thinking” has besides implied error-prone. For reasons we will get into successful the future, we judge System One Models tin beryllium made much reliable than its alternatives.
We named Jev aft William Stanley Jevons. We expect instrumentality intelligence to travel a akin way to coal, aft steam-engine ratio led to an summation successful demand. Every bid of magnitude driblet successful the costs of intelligence unlocks orders of magnitude much usage cases.
Why was a caller training algorithm needed?
What usage cases is Jev bully for?
Is Jev conscionable a smaller LLM?
How does Jev execute against nationalist benchmarks?
Where does our training information travel from?
These are results are kinda crazy - really is it possible?
English (US) · 
Indonesian (ID) · 
