Clef: Open-weight decision models, and new RL fine-tuning platform

“we’ve been testing our new Clef model on our Threat Intelligence team to help us classify website domains” Not sure if any of this testing is in production but their website classifications need lots of work. I use the cloudflare ZTNA agent on my work laptop and often run into uncategorized websites. Just last night I could not get into my insurance company’s website because it was uncategorized so I got a block page. If Clef can decide this in 2 seconds I shouldn’t have to enter a support request and wait hours for it to get categorized. Especially the main page for a Fortune 500 company.

Am I hearing this right, that they made a decision model based on Typesafe's new paradigm, and actually made a model better than Jev based on Typesafe's own ranking?And it's only been a few weeks. reply: A lot of people claim to have made better than jev, there's a jev benchmark, I have tried many of those models and they eventually end up failing, a non trivial task which doesn't seem like much but reminds me of the svg pelican bench is games, have one of these decision/classifier models play a game, hook it up to the input, most of the ones that are supposedly on jev level end up playing a terrible game, showing that they are very narrow. Cloudflare doesn't compare to the top open bench alternatives, I just finished downloading it and will compare it to jev for non trivial tasks tonight.

I'm a big fan of Cloudflare products. I recently stuck Jev in front of a Cloudflare-hosted Ollama model for chat/username moderation, so I was excited to test out Clef. The setup is user send a message -> Jev does first pass to see if it's toxic/hate speech/profane, if Jev is unsure then Ollama on Workers AI takes a deeper look.Clef was 2-3x slower and worse (it caught less hate speech) than Jev. Overall disappointing. reply: In "How we trained Clef" section Cloudflare mentioned that they initially based their Jev-like system on DiffussionGemma (DJev) model but then changed to Qwen with post training for Clef, but never mentioned any reason and justification for the change.Although they included the benchmark against DJev and Clef is better, perhaps if you can test it to see the real-world performance.

Open weights, not open source.The weights have permissive licensing, but the data and training pipeline are not published to reproduce them from their proprietary Qwen starting points. Weights are not "source." reply: Ok, we've put weights instead of source in the title, at least until someone comes along and says that's not right either!

Jev = $0.042/m input, output free Clef = $0.24/m input, no output price listedAt 300 tokens per call, you'd get:One million decisions on Jev cost about $12.60. One million decisions on Clef cost about $72.Would probably make sense to self-host Clef, if you have the capability/resources. If not... reply: Because Clef is a larger model, larger context, can process images. Plus its open weights, so you can run it yourself.Since it benches better than Jev, Jev is probably smaller and easier to host. They could also be losing a lot of money.

Pricing is $0.24/million input tokens which is ~6x compared to Jev. Clef-flash is at $0.09 which is way more competitive. reply: Yeah I also thought it was strange their pareto frontier didn't include cost.

Clef is based on Qwen3.8-27B and Clef-flash is based on Qwen3.8-9B (edit: actually Qwen3.5-9B). So, similar in spirit to Kev by my understanding, but based on a newer model. reply: > and Clef-flash is based on Qwen3.8-9BThere is no official qwen 3.8 9bFrom the model card:> Clef-Flash is post-trained from Qwen/Qwen3.5-9B. See Clef for the larger variant.

I love what Cloudflare is doing generally so I was excited to try Clef in my evals on a real task vs Jev: should an agent's knowledge-base write go to human review?Quality: close (recall 0.98 vs 1.00) Hosted p50: Clef ~850ms, Jev ~110ms Clef-flash: over-escalatesData + script: https://github.com/nicia-ai/admission-decision-eval

For smaller scale decision model that runs on CPU, check https://news.ycombinator.com/item?id=49923223

It's amusing that this blog post explained Jev's underlying design far more clearly than their onslaught of bull posts and marketingThis entire time they could have just said "decision model" but they kept using vague, flowery wording. I have no idea why

A question someone not trainined in AI/ML field, Is a decision model that easy to crete that there are floods of these JEV alternatives already?Or are companies/people already building this based on say an arXiv docs? n---The pricing is ... hm more expensive but not at the point I won't give it a try due to the embeded vision encoding

This is awesome.I bet the competition will result in research into how to make these decision models several more orders of magnitude faster and cheaper.Here's a challenge problem - look at a 1M context window and produce N decisions (different queries) from it in 50-100ms.

I’ll go out on a limb and suggest that I don’t think a Jev-like model is particularly useful unless you can fine tune it. The Jev API has zero ability to pass in a prior [0], and, if you can neither pass in a prior nor fine tune for your system, you will get an output that may be almost meaningless.I’d love to see someone build a model of this sort that can actually accept priors and do something intelligent with them.[0] You can feed Jev a prior as text. I’ve tried it. It works poorly.

With all these new Jev-like models popping up, has anyone actually started building anything with them yet? It's odd how quickly they've multiplied despite being relatively niche in their use cases, as far as I can tell. I suppose they're simple and cheap enough to make that it's a sort of 'why not' thing for a lot of these companies.

> This means that a human does not necessarily need to be in the loop for agentic decisions anymore — agents can programmatically gather context, make decisions, and take actions on tasks, or defer to a human when needed.1. Humans are already not in the loop for lots of LLM agent actions. Isn't that just a function of how much you trust it and not some completely new paradigm? Am I missing something?2. How can it gather context if it just outputs a single decision?One guess: Maybe it's decision can be "gather more context and re-run me"? But an LLM can be much more expressive about what context it needs.

I found the paragraph about how much networking data they have weird. I mean, if you already have all that data why didn't you train your models already using that? Why did you need clef to begin with?

> This contrasts with the world of Large Language Models (LLMs), which are largely non-deterministic, but are open-ended enough to reason...This seems misleading. Decision models do not produce deterministic output. Repeated calls can product different decisions just like an LLM with structured outputs.

Can these decision models be used to enhance the intelligence of video game adversary and friendly NPC "AI"?

Is there a Jev-like model I can run on my Mac? Something like Ollama? Or what's the best way to play with it? Is there a cheap/free API service eg on OpenRouter?

Why give cloudflare your data ?At0M: A 60M local Jev at 16 ms latency and 79% accuracy on Typed Decisionhttps://at0m.pienomial.com/ https://news.ycombinator.com/item?id=49920350

Related: an intelligence cache for decision model data: https://cachev.devI built this for my own needs, and thought others might find it useful too.

I wonder if a good usecase for this would be cloudflare's WAF rules. Give broader request context to the decider and let it pick type of challenge/block traffic directly.Perhaps that may be too costly atm

it's great to see Cloudflare releasing consumer edge level models.

Can someone explain how so many folks managed to build decision models within days or weeks after Typesafe came out with Jev? Is this concept of decision models been in the works for a while? Is it easy to copy?

Oldy enough I tired this 2h ago as I was testing jev on cf and was like oh this should be a better replacement but it takes 3s which is useless to me

Commodities get commoditized. Nothing to see here.

Competition in this area is great and kudos for releasing something that we can try out today.

in a quick mini-bench here n=250 of clef vs jev, clef came out 5.2x more expensive, a lot slower (p50 of 350ms vs 1.9s) with only marginally better results (78.6% vs 79.8%)