Every major AI shop has a ton of in-house classifiers already, big, small, generalist, specialized. Some are used in inference pipelines (e.g. safeguards), some are used in data preparation, training, analysis and investigation, research, various one-off and intermediate tasks etc. Offering them on a public API doesn't always make business sense. I don't see much substance to this buzz, looks like people that are new to all this are discovering that classifiers exist, they are more efficient at classification, and many tasks commonly done with generative models are classification in disguise. Which is not bad at all, a fresh look at their use is great to have.
Correct me if I'm wrong, but a zero-shot classifier like Jev is fundamentally different to a classifier with a fixed task (e.g. for safeguards), unless they trained a general purpose system to complete the safeguard task, which seems unlikely.
But zero-shot classifiers with this level of intelligence, world knowledge, ergonomics, cost profile, and ease of use are new.
I feel like good engineering doesn't just ignore those things, or at least it didn't before recently. Now I guess social media has added a pressure to reduce everything to a hot take.
Fixed guard today is not very fixed. For ex, the safeguard qwen released is a full 4b llm model. It has no different to normal llm model arch except tuned for this specific purpose,
So it is tuned specifically to classify content for safeguarding? I'm not familiar with this particular model, but it most likely has a specific classifier head that is tuned for the safeguard task. This is completely different to zero-shot classification.
I fed into the hype at first. Testing Jev and Laya, they both suffer from the same issues as LLMs that stop them being useful beyond limited classifications.
I can't see any benefits that a typical ML classifier would not be better at.
Agreed. I tested Jev on OpenRouter this past weekend and it’s “okay” but a specific classifier is significantly better. It used to require skill to import sklearn (ok, not really), but now it’s literally one prompt and upload your Excel file or whatever and you can get your classifier out. It’ll run free, instant, more accurate.
This is predicated on you having training data already. I approach Jev more like Langchain -- you can prototype something new extremely fast and cheap, and if the use case works well enough, rip it out and build something bespoke. If it doesn't, you didn't spend a bunch of time curating a training dataset anyway.
I think the main argument would just be that because the model is general, you don't need to retrain it from scratch for a new problem - just tweak the input prompt. For a typical classifier there's a lot more hassle - collecting the data, training it yourself, retraining under distribution shift... In that sense Jev seems great for prototyping or small-scale use cases.
Counterargument: this works for quick prototyping, but for any serious business, you will eventually develop a benchmark/eval to track how well the general model is working, and once you have that dataset, you might as well train a specific model
Jev's bet is that if it works well enough for random use cases that nobody complains, then management won't feel a need to develop a benchmark/eval, and they won't need to employ all those data science guys.
I'd also add that they're hoping Jevon's Paradox also leads to a whole new segment of users who would have never reached for a classifier in the first place, given the barrier to entry.
And if you do get complaints or feedback on the classification, have a dev log into the user's account, tweak the Jev prompt a little until the issue goes away, and push it to production
Or not. And replace the generalist with the next generalist that gets you +15% on that benchmark for the same price, or gives you the same benchmark performance for half the price.
One advantage of using generalist models is that the generalists are improving - regardless of whether you're doing anything about it.
Using Jev as a plain classifier is the least interesting case. See robotic control, navigation, computer use examples, none of it possible with a classifier.
That's the point, they're classification in disguise. Agentic game engines/mods started doing this long ago due to the latency requirements (although they're typically using small BERT-like models that need to be finetuned, or low TTFT generative models and structured outputs). New or newly discovered use cases are great, sure.
It starts to break down once you go over 20 classifications. Which is very basic routing that can easily be done with typical ML models for cheaper and faster.
Entertainingly, OpenAI had a general purpose zero-shot classifier API built on GPT-3! Just no one ever cared that much about it, so I guess it got dropped somewhere along the way since 2020/2021.
For me, I think the big deal is that it promises to be general and broadly applicable and high quality. That's new and special. But we'll wait to see if the claims actually hold.
Agreed. This isn't new. I led a research team at a Fortune 500 that used a transformer based classifier approach in a commercial product as far back as 2022 and we didn't come up with it. It was already common enough that we found the inspiration for our implementation on some web forum. Models like RouteLLM have been around for a long time. The news here isn't that a new model type came about, its that a large percentage of people messing around with this stuff that are new to AI just learned that not all transformer based implementations need to be autoregressive.
It took us a few hours to implement that one we used in 2022. This isn't about usability its about a huge population working on this stuff not really knowing what is available until it becomes a meme.
> Every major AI shop has a ton of in-house classifiers already, big, small, generalist
I think building generalist classifier is some open ended research task, where frontier labs can contribute: different internal reasoning, instruction tuning, building datasets and benchmarks, building and distilling super large models.
This article is extraordinarily hard to read. It’s tummelvisioned on OpenAI and things like tool calling which are only relevant to the extent that llms have been tuned to make relative choices, but this applies to all LLMs. Also, some really outdated references. LLM written, perhaps?
Also, moat discussion is the lowest form of discussion. I don’t care if jev has a moat. Did it get the interface right? What other past ideas have we overlooked that if given some love, could kick the door down like jev did?
Really silly stuff.. people wanting to talk about moats when there’s no castle. Moat talk merely projects the illusion of being engaged but, much more often than not, it’s hollow engagement.
Partially. I'm a terribly slow writer and get stuck on phrase choice, but I'm good at content ideas, outlines, and editing text already on the page. So I have AI do the bit that I'm not as good at.
Process: First, actually have ideas :D Then, I write an outline for what I want to talk about at basically a sentence-by-sentence level. (This is me yelling things at my computer.) And then I have the AI convert a chunk at a time into prose. I reread it and rework it to be my voice.
Then I have the AI help with things like subject titles and social posts.
I am not personally like philosophically or ethically opposed to having LLMs help or even write text... the issue is that I see so much LLM-written text that is just _bad_, and very hard for me to read or extract meaning out of, especially relative to it's often long length.
People think they are bad writers, but usually LLMs are actually worse (although they are great writers of catchy slogans and phrases, and then put together an article out of them, which I find just exhausting to try to get more than a "vibe" out of).
What you describe sounds like a fairly reasonable approach, but I suspect the parts the person I was replying to were reacting to was areas were you had not been as succesful at reworking it to be your voice. Which are probably also the parts pangram flagged as likely LLM written.
Pangram gives you a handful of free tokens, it would be interesting if you wanted to see what parts are the 20% pangram is flagging as LLM, and reflect on if they went through your process differently. Perhaps they were the parts you didn't spend quite as much time reworking it to be your voice. (I 100% believe you, because I've been running things through pangram a lot lately, and it's actually pretty rare for it to flag mixed content, instead of 100% likely AI or 100% likely human).
There was recently a post on HN that said if you want to avoid this, you really can't use any words at all that are written by the LLM, you can use it for suggesgting structure or points, or reviewing your work in various ways, but if you accept even a single phrase it provides... it's not going to be "reworked into your voice", it's going to be picked up by people (at least those of us who have become sensitive to it) as AI, because it's like, headline-speak.
(I can't find the article now, because I'm trying to quit facebook so can't log in to find my own post of it there, have to stop using that as bookmarks substtitue!)
Of course, that's not welcome advice if what you want AI for is "phrase choice".
I'm just here to say, LLMs are not good at phrase choice either. Although they may be quick at it. I feel like it's asking the reader to do the work of trying to extract meaning from slop that the author didn't have the energy to use to encode it well in the first place. I don't have time to try to read sentences that the writer didn't have time to write, i find myself bailing out quicker and quicker at signs of AI slop buzzword headline-speak.
Panagram seems fun! I tried it out with a large chunk of my post (whole thing wouldn't fit). Panagram says it is 98% human. Feels right, to me b/c I aggressively edit whatever comes out. The thing that I'm avoiding is the ominous blank page - I just freeze. If there's text there I can always reshape it. And I do, heavily.
I find it unlikely. OpenAI is all in training models with reasoning with RL, and Jev-like models are the total opposite. They are made to not reason at all to be fast. If you want to add reasoning on top, you might as well use a conventional LLM because you lose the price and speed benefits when you output auto-regressive tokens. I don't think OpenAI will even bother with this.
> My main assumption is that Jev is using something quite close to a conventional large language model. As evidence of this, Latent Space reports that many of the early clones are indeed LLM-based.
Not proof that this is the case with Jev though. It might use non causal text encoder for the state, which could make sense given that it's very good for its price.
I don't see it fundamentally any different than knowing when to use a tool. Is this tool like RAG an important enough corner case to train for it? I dunno.
LLMs already shell out and write code to solve certain problems. This is just a special case of that.
It's a special case for an LLM, and you can use an LLM with structure output to get similar results, but you can engineer specifically for that case to get better results per dollar for it. That's why there is little reason to adapt GPT 5.6 Sol or wathever for this task; it can already do it (at a high cost). For OpenAI to compete with Jev they have to maintain another line of models, something like "GPT-5.6-instant-decision", that is small, fast and cheap, in the scale of GPT-5 nano.
Note that I don't think OpenAI is incapable of doing it, but I just don't think they will bother with it.
Keeping people looped into your product is pretty important, but yeah, there's not clean way currently to separate "structured" outputs from the token stream and to start using a different billing structure there. And I also appreciate that they aren't going to be keen on gving free or near free output either, so gotta figure that.
In the olden days we call this classifier, usually assignment 2 of Machine Learning 101. BERT (well, GLiNER specifically) and diffusion are calling and want their Large Classifier Models back.
Or OpenAI just buys them outright. Buying your upstart competitor seems to be in the Silicon Valley Ten Commandments. The Fed whussed-out on breaking up FB and Insta last year, so there's never going to be any kind of remediation to worry about.
And for Jev, everyone has a price, and OpenAI's raised an historical amount of funding.
Why would OAI need to follow Jev? I really think this is paid by Jev. Jev itself won't have lunch money in a shortwhile because there are literally 10s of free alternatives available which can be run locally on basic consumer hardware. Terrible utility aside, there is no sensible business proposition in Jev.
I'm having a really hard time wrapping my head around why Jev is getting so much hype. It feels manufactured to me. I don't think they've proven a significant market for their product, and there's no independent benchmarks that prove anything. To me that's doesn't pass the smell test.
But if they had to show how well their product worked they might give away the whole game... because they'd have to compare their "noul" class against an NLI benchmark for instance, and possibly show they're losing to cross encoders and give away the fact that they are just rebranding NLI. Or rerankers (choice) or zero-shot classifiers.
The AI hype cycle is always looking for the next big thing. It really doesn't take much for enthusiasts to get very excited and push something into the stratosphere. Just not having a vibe coded website, and someone that made ChatGPT is enough to set them apart. Hitting pain points like pricing and speed and also implicitly mentioning llms (even if it's to mention it can't generate text in contrast to llms) make it seem like a major step-up.
If OpenAI releases something similar to what Jev does, then that would be like admitting defeat. Their whole spin is AGI and world ending danger. Why would somebody with an AGI at home make something like Jev which is intended to be a part of some SW the AGI is going to replace anyway.
A lot of business automation doesn't need AGI, doesn't want to pay for AGI if they don't have to, and would be better off using a classifier than something increasingly unreliable with a mind of it's own.
There are many automation pipelines that use LLMs because there was no choice, but the multi-way classification that Jev provides is exactly what they need, and is going to be way faster and cheaper, as well as having the benefit of calibrated probabilities and structured output that can be relied on.
I would assume that they would fold this type of classification into their responses API next to existing ancillary tooling that they already ship. I think OpenAI positions themselves as being wholly focused on AGI - but I think their business model leans towards lock-in via superior tooling (Codex, ChatGPT, etc.). This feels like an easy win for them without muddying the larger vision.
The short answer is: they need to make money now to fund continued improvements towards AGI. This is extremely obvious and these "well if they had AGI..." arguments are clearly in bad faith.
You have to run a legitimate business and make money. After the "takeoff," anything is fair game. It could result in value accruing to capital (lab shareholders) and everyone else is screwed, it could result in the status quo being maintained but people do way more and GDP goes up by a lot, it could result in a post scarcity society etc. Theres a reason why its called the singularity - you can't see past the event horizon.
In any case it is optimal for OpenAI to create good products that generate revenue NOW vs going for some straight shot pie in the sky model that is "AGI." SSI is attempting to do this and I suspect they are about as close as anyone else is, and I'd honestly assume they are likely further away than OpenAI or Anthropic at the moment. But one of these classes of companies has actual revenue that is arguably good for the economy, the other is just a glorified research lab that has produced nothing of value.
AGI is like water, and these surfaces are the plumbing. Agents and the internet's existing APIs are how AGI will actually be delivered in the short term.
But you're completely right: our current APIs were optimized for humans, and some like Excel date from the 80s. So in the long term, agents will highly likely create their own interfaces optimized for when no human is in the loop.
Which is exactly why companies stick with current APIs for now. Fable 5.1 already emits alien-like reasoning traces, and the recent OpenAI agent swarm accident just proved this.
The people that buy into "AGI is here" and the people that understand what Jev is and how useful it would be to hook it up to an LLM are two separate circles, so there's no "defeat" being admitted.
There's a big difference between what AGI can accomplish and what it will accomplish. We must also ask why OpenAI wants to summon their God, and how they would want the average person's life to change if it comes. I think they will try to clone Jev just to add another revenue stream.
It's just another tool. Luna exists for a reason: it's the right tool for the job. If they release AGI and it costs $1 and 5 seconds to decide "is the customer asking for a refund", then that's a terrible use case for AGI if another tool can do it with 95% accuracy for $0.002 and 50ms.
> If they release AGI and it costs $1 and 5 seconds to decide "is the customer asking for a refund", then that's a terrible use case for AGI
Is it? If AGI is here then by the time I test and deploy that the AGI will be most likely cheaper and smarter because it improved itself (for example by implementing it's own Jev for stupid prompts like this), so why invest into a more complex solutions?
I don’t think that was a great example since there’s only so many refunds a customer is going to ask for. And it’s saving time that otherwise maybe would have to go through a human. The rate is low enough that a more expensive model makes more sense.
Though for tasks where you are trying to search through billions of documents, social media posts, etc. and extract certain information, where each individual post is of low value and only the data in aggregate is valuable, then that’s where you’d want something cheaper and faster.
Such as if you want to look at all posts on X in the last few months and find how many have a negative or positive sentiment about the economy (or are unrelated).
Of course you could use a special-purpose model for this, but the whole point of something like Jev is to ask whatever questions you want without having to train something new.
Well sure, in the future you may also be able to ask AGI to "please just run my life", while you stay in bed.
In the meantime, today, in the real world, there are businesses wanting to automate well-defined business flows, who don't want some stroppy AGI with a mind of it's own to instead decide to hack into something, or reward hack and make the customer happy by just wire transferring $1M of company money into their account.
Why would it need to? There is a deterministic flow here for the actions that are allowed. AGI isn't needed for this at all if you can map out the flow and use a classifier to decide which route to follow.
I'm not exactly following through with the claim, can someone explain how the built-in classification would not necessitate more tokens used, or be much different from turning on reasoning? Not that I don't see the difference, I just doing see how OpenAI would do it well.
AFAIK Jev is nothing special technically so it's easy to embed it as an another tool for the LLM? For many batch tasks it can still be quite a token saver I think.
Or they can even offer it as a standalone API if deemed worth it.
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
> It's output probabilities are calibrated to actually mean something
Don't fall for marketing BS so easily.
Jev can output drastically different probabilities if you simply reorder the list of choices. And Jev's "confidence" output is fake/redundant - it's just a formula applied to probabilities, it conveys no additional information.
I bet they will eventually "fix" (read hide under the rug) the ordering problem by ordering the list on the backend before feeding to the model.
It seems that anyway most of the value is in the speed and cost.
If it really matters to you whether whether some business-specific classification confidence is above/below some specific threshold (vs just relative order), then you'd be better off training or fine tuning a custom model for that. Maybe that is something that TypeSafe are planning to also provide?
Grammars do risk pushing models off distribution in a way that impacts their output quality in a way Jev allegedly does not suffer from. Additionally, Jev's ability to answer questions independently is also exciting. Using an LLM to answer multiple questions in one generation has the property of earlier answers influencing later ones. TBD how many of TypeSafe's claims stand up, but my testing so far is promising. I hope they author some papers on their methods as well, but that might destroy their moat.
If you really know what grammers did, grammer is a filter to mask out option llm provided but you don't like.
It does not change potential distribution in any means. It DROPS part of answer model returned directly.
The text generation model go wild because model relies on previous section it answered to continue later section. And because now it contain item model have no idea, it is completely screwed.
In the case you only require model to answer one of a,b,c,d and don't care about later segment at all. It don't really matter.
What I mean is that, in general, constrained decoding can push model output off into less probable regimes. This is well studied; see for example https://arxiv.org/pdf/2606.21619. The mask may only retain very improbable logits. In pathological cases, the constrained output may be little better than noise filtered through the constraint. When using existing structured output APIs, it may not be possible to even know.
You don't even bother text after the [a] at first place in this case
Your question is something like
anwser only a,b,c,d for following question
a. b. c. d....
the model output possibility of next character
a: 0.8 b: 0.7 c: 0.3 f: 0.2 d: 0.1
If the list contains option you did not provide.
The model is confused anyway, it don't matter if you use grammer to filter out the bad option or not, the answer is screwed already.
Yes, agreed. I was speaking in general, of course. This particular topic is of interest to me, so thinking of the edge cases and confounds vs Jev.
In your example, I would expect an LLM to do fine and if you have access to the raw logits you can measure whether or not it was confused and assign a confidence to the answer it gave.
I do think that Jev handles more than this though and, in my early testing, does things that are not easily accomplished with guided decoding techniques.
The way jev actually internally work could be interesting though. I believe most llm are only tuned to return the first or second logits(or a few more) correctly as that is what the sampler would choose anyway. Do they alter existing model for better behavior across all options? Or they distilled one to have the proper behavior? We can only guess without the actual implementation.
Yes! I really hope they release some papers on their techniques. I am very curious.
I ran it through MMLU a few days ago and it scored ~90% so seems to have a lot of general world knowledge trained in. Makes me think your speculation is right. I have some credits left, might try and think of an experiment. I saw a gist where someone was asking it which model it was and it was picking qwen a lot, but who knows...
Although the underlying model is unknown. If it expose input token count, the tokenizer may be probable though. Most tokenizer segemnts wildly different in CJK inputs. It can probably be used to fingerprint the tokenizer based on token count if it is using existing tokenizer.
Not an expert at all here, but I saw a comment on the jev post saying that it you constrain an LLM suck that it outputs a valid structure, if the token with the highest probability is not the one that you expected because of the structure (and so you pick the valid lower one) this means the LLM was already confused and your answer is less likely to be correct anyways.
the algorithms technically, sure, however the outcomes definitely depend on data quality and coverage like any other training method, this is well known
Read the paper. They train RLCR on existing big math problems. They subtract a brier score penalty from the correctness reward. No new confidence labels are needed.
It's hard to say without knowing their architecture, but I'd guess something like block attention. You can process the prompt separately from the classifications into a latent space and then do some kind of late interaction with the encodings from the classifications.
There are plenty of other ways to do zero shot classification that would result in more "token usage" (really just having to reprocess everything for each class), but the pricing and the way they describe it narrows it down somewhat.
Normal LLM will do the classification on the text that is generated. Jev just returns the classification and confidence.
It has the advantage of speed and the confidence not being hallucinated.
But LLMs start to generalise on the pattern, rather than the classification that you want the more examples you have to train on.
LLMs start to break down as well the more classifications you have. Laya (Open source paper Jev is based on) even mentions that over 20 classifications and it starts to fail rapidly.
20 is around the level of sentiment analysis or minor intent routing. There are cheaper, smaller and easier ML models for that level of classification.
That is, if you force any llm to return json and a confidence it can also do that too and mostly likely it will he better at any one shot classification task than Jev.
LLMs have the great quality of knowing more due to the depth and richness of the training data. If Jev is trying to classify anything outside of its training data, it’s going to do a terrible job.
I think the idea is that the latent thinking space in the LLM will be roughly the same for similar quality results - so the majority of executing well could be stripping back and fine tuning an existing LLM.
Rather than focusing on OpenAI in particular, let's just say that there are many smart people at other AI labs and if it seems like it will be popular, this technique will probably be copied. What would prevent them from adding another API?
Hopefully there will be some decent benchmarks and gateways for switching between providers easily.
I'd be surprised if they weren't aqui-hired by one of the big labs as a unicorn.
The ability to use classifiers under the hood for the larger models has the potential to dramatically improve cost and throughput, allowing them to increase margin on a very similar service.
Mostly one guy sending a fleet of agents to add features at unsustainable speeds who wants validation from a large org? I don't see many similar vibes at all, the founder already had massive impact at OpenAI and before that, Google Brain.
Extremely fast hype and a very short-lived moat, those are the parallels that I see. The OpenClaw guy was smart to cash out at peak hype, and chances are Jev will be overrun by big players and open-weights pretty soon too. I really love the idea, but just don't see any sustainable moat here. People are already coming knocking [0], it should only be a matter of time until we get something like Qwen3.9-Classify (or a classify mode just baked into a multimodal LLM).
comments are pretty weird here, there's no real moat to what jev is doing, it is certain that frontier labs are going to release their own jev and there are even open source alternatives (although nowhere near as accurate as jev).
so maybe typesafe's real plan is to front run and releasing their own new models for some time until they can get acquired which seems to be the only rational objective
I don't think it's unreasonable to think this, but I do think the burden of proof is on your side. Between the SaaS-pocalypse narrative that never materialized, and inexplicably losing their first-mover advantage to Anthropic, OpenAI's track record is not great when it comes to jumping on these micro paradigm shifts.
If the headline said "Frontier labs are about to eat Jev's lunch" it might be an easier sell. But if we're gonna include Anthropic, I think part of their success is actually making products for which there is demand. It will take time for something like that to come out of this new "decision model" paradigm.
the real win with jev isnt beating a trained classifier on accuracy, its that you can change what you're classifying by editing a prompt instead of retraining and redeploying a model.
People are desperately trying to cope themselves into thinking that there are alternatives to scaling up transformers to AGI/actual competition to OpenAI or Anthropic. Jev, continual learning, linear attention, local models, non-transformer architectures etc. Imo these are just random technologies that nerdsnipe your average twitter or hackernews user and give them some hope that some underdog can take a slice of the pie.
In reality, none of these really matter. The frontier labs can easily do something like this but likely havent because the size of this market is too small and it is not on the critical path to AGI.
When you have as many resources as OpenAI and Anthropic, theres basically no point in putting compute towards random bets that don't have a predictable return. And at this point, scaling up transformers is almost a surefire way ot putting money in via training and getting money out via increased capabilities AND it speeds up your own business by factors of X. Sidequesting a Jev like product is falling for twitter hype and is likely not going to happen, definitely not by Anthropic, and I'd bet probably not by OpenAI either.
Isn't this more and more likely on all shapes of model evolution? The providers will all copy each other.
And in this case I hope its true. I've been classifying a lot of email and while OpenAI `text-embedding-3-small` has been very helpful for fast and cheap embeddings, initial tests with Jev are very promising and much more ergonomic.
The right part: autoregressive LLMs are indeed generating “probabilities” (scare quotes very much intentional). During pre-training and any SFT steps, those probabilities are nudged toward the probabilities, over the training distribution, of the next token conditioned on the previous tokens. (This is an explicit property of most training recipes: KL divergence is a “proper scoring function”.)
So if you prompt with “Paris is a city in ”, the next token probabilities estimate the probabilities over the input distribution that the next token in the sentence is the first token of France or of something else.
But there are huge caveats:
1. That is not at all the same thing as the probability that Paris is France under any distribution that you care about (the population of the various Parises, for example).
2. None of this necessarily usefully applies to RL or, as the article discusses, tool calling. The output probability of a tool call is not some Platonic idea of a probability that the input is worthy of a tool call. It’s a the result of a training process that tried to teach the model to be useful and to achieve its goals.
3. I suspect that reasoning makes this all much worse. Suppose that you prompt with “a help desk user with IP=a.b.c.d says they’re ‘in Paris’. What country are they in?” The model has been trained to generate a reasoning trace, which may well start with “let me think of where Paris could be. It could be in France or in Texas etc. The user was speaking English…” See the problem? The model is reasoning well, but it reasoned “France” before “Texas”, so the logprob for France was probably higher than “Texas”. At the end of the reasoning trade there will be an answer, but the logprobs for that answer are, at best, some representation of the probabilities of the answer conditioned on the sampled reasoning trace. And that is not the probability distribution that a Jev user wants.
I did so a month ago, I developed a deterministic agent framework that works with a set of predefined intents, it is instantaneous and fully deterministic.
Would it be intesting/useful to use jev to generate a block of text like LLMs do ?
Like asking it to pick the n + 1 word given the starting text (using it's choice primitive), but also asking n + 2,n+3 and so on at the same time.
Would it give coherent or useful results ? Or does the fact that it computes it "all at once" means it cannot make one of it's answer influence the other ones ?
Everyone was obsessed with classification prior to transformers, then we had 5+ years of everyone (rightfully) obsessed with next-token prediction. What's this sudden resurgence of interest in classifiers? I thought we all agreed that ML tasks generally require something far more advanced than pretrained classifiers. My timeline was also absolutely filled with mentions of Jev, which makes me think it's a successful viral marketing campaign, like langchain. It's now so popular that the dialog is whether or not [frontier company] is poised to catch up to it or not? We already have openjev...anyone can use it. I don't get it, and usually that means it's marketing.
Jev doesn't code. It can be used with LLMs to simplify coding and token consumption, but still and LLM is needed. Will they complement each other? How can Jev replace LLMs? Are they even competing?
You can easily use Jev without an LLM (consider Jev used to make truthy decision branches in a script). It's not meant to replace LLMs. The tech is not a competition. But OpenAI is certainly in competition with TypeSafe, they'll want to keep people in their own ecosystem!
Reading logits is the cornerstone of ML. It’s almost like many of the people reporting on and fawning over AI have no technical background and never knew about ML classifiers or calibration.
Even the article itself has the title as a question: "Will OpenAI eat Jev's lunch?". A more useful title would be "OpenAI is Positioned to Compete with Jev".
Personally, I don't really care what OpenAI does here. What's going to be far more exciting is when DeepSeek, Qwen, or GLM start integrating classifiers into their open models.
Exactly, the future is open models. Which is also the reason why those overvalued companies such as OpenAI will lead to a market crash as soon as investors realize that.
I'm not sure it makes as much sense to use the same bigger models for the things Jev does. Part of what makes Jev appealing is the cost/speed. I can definitely see them putting out S1 spins of their smaller models.
That's not what I meant. I'm thinking more of AI systems that combine multiple modules the same way the brain has different regions. LLMs are just part of the bigger picture here. They're good at tackling a certain types of problems, but other approaches are better for different kinds of problems. Having a system that combines a generative model and a classifier for example would make it a lot more efficient and accurate because it has a bigger toolbox instead of using one algorithm for every problem whether it fits or not.
There are plenty of other use cases aside from agentic coding. However, even here a classifier has a lot of obvious benefits. There are plenty of scenarios where a model has to make a choice, and programming/system tool use happen to be deterministic sets of choices that can be evaluated.
> programming/system tool use happen to be deterministic sets of choices
I'm not following. There are way more tools in the world, and likely available on every machine, than can fit in any model context. One can create a small list, but that will be very org dependent. Programming is very open ended, I struggle to see the 'deterministic set of choices' here.
I'm looking at Jev/Kev like models for some control over tool calling and similar, but I also think that fine-tuning out behaviors (like qwen 3.8 wanting to use rkt all the time) may be better in the long run. Instead of having to check/change every tool call, we can refine the main LLM to have high enough likelihood of using the tools we actually have, that we won't need to check every call.
Jev is definitely useful, but also being held like a hammer in search of a nail right now, approaching OpenClaw like fanfare and hype levels. No one talks much about OpenClaw these days, I would not be surprised if Jev has a similar trajectory.
I'm not talking about checking/changing every tool call, I don't even know what you mean by that to be honest. The question is about selecting the tool to use in a particular scenario.
The whole point of Jev/Kev is efficiency. If you can make decisions faster, cheaper, and more reliably that's the benefit. Even within the model architecture itself, you could use a classifier as a router within MoE to pick the expert that's likely to give the best outcome.
Nerd sniping the hackernews/twitter crowd with "large scale transformer-based language model alternatives."
People don't want to believe something as unsatisfying as "Scaling up LLMs" can yield something as profound as AGI/be useful, and just hope that literally anything else can take their mindshare away, and this just happens to be the new rage. Along with clearly-not-frontier-level open source models, non-transformer based architectures, etc.
Beats me also - this feels unreliable, extremely niche, and over-hyped. I don't trust LLMs even when they explain their reasoning; the idea of trusting a black-box classifier like this seems insane.
In my company, and I think in most companies that are using AI at all, one of the first ways it got integrated is as a classifier, to tag orders based on feeding all their data into a prompt and asking for a structured output.
I think demand for tools that are more tailored for this type of integration is high. I don't really understand why Jev is supposed to get my company's decisions right more than an LLM, but regardless of the tech I think people are just excited about the possibility of iterating faster, more explainability, higher-level tools that are specifically created to help hone classifiers etc.
I think the point of Jev is to thread the needle of the gap between non-LLM classifiers and LLMs.
Classifiers like classical NNs require:
- annotated data, potentially a lot of it
- training
- inference
#2 and #3 aren’t a big deal if you have an ML engineer, but #1 will always be a potential headache no matter who you are. The tradeoff is that they could be quite fast, cheap, and you can get probabilities, not just classes.
With LLMs you get:
- zero shot classification (no dataset or training required)
- potentially can use third party model providers like OpenAI off the shelf. Don’t even need to host your own model.
The downside to LLMs is that they are comparatively slow and expensive to traditional classifiers. Historically they also were prone to hallucination or malformed responses, though not as much these days. You also can technically get log-probs back, but these aren’t equivalent to the classifier probabilities.
Jev gets you the zero-shot, zero-infra benefits of LLMs, while being closer to the speed and cost of traditional ML classifiers, as well as both classification and probability responses.
Yeah but they weren't that great, you couldn't ask for arbitrary classifications after the model was trained. You are underestimating what they've done here, even if it does seem a little overhyped.
Classification models lend themselves to sparsity and explainability. The good ones are very simple and economic to run on a laptop. If someone told you before 2022 that a json classifier was a product you would have laughed it off.
To add to this the more difficult problems in classification done on scale have always been about collecting "good" -ve examples, enough data to calibrate on every confidence interval and debugging outliers. And those are solved on a case by case basis by the company pursuing its own peculiar version of the problem.
Am I the only one who thinks this is just all hype?
I just had Claude and Jev combine forces last night. I've built a few personal browser extensions in the past and thought it would be fun to copy an experiment I saw on twitter where Jev classifies comments/posts etc as slop or not.
Fed Claude an api key from typesafe and a link to documentation, and within about 10 minutes I had a view of HN that was populated with a little ranking as to sloppiness of each comment.
When your mind has been wired a bit to LLM latency, it feels extremely fast, and for such a subjective rating I think it did a good job.
Feels like it sits in a space between traditional ML classification and the frontier models. I can't think of a 'real' production use case for it in my sphere of influence, but certainly some will. And of course there will be five Jev competitors by the end of the year.
Literally never heard of Jev before now. Trying to figure out if it's really a big deal, or if OP is just Jev marketing, and where I would learn more about it that isn't just LLM-produced slop. What a world.
> Back when I was at GitHub working on Copilot I had the opportunity to work with a very new and very raw internal API for GPT-4. Out of the gate, we knew something was way off because, after an initially very coherent response, the model would have trouble wrapping up. It would end every response with something like "Let me know if you have any other questions. Have a nice day. Have a great week. Have a good time. Have a wonderful life. Have a special day. ..." and it would keep on like this until it hit the response token limit.
Asking again (didn't get an answer in prior discussion):
As there have been a lot of Jev related submissions, can someone point me to a simple guide on how I can use it? For example, say I have a script/workflow where I use OpenRouter for LLM calls, and at some point I want to do a simple classification. Can I still use OpenRouter with some Jev model...?
With all the excitement around Jev, I suspect we'll see hundreds of options, it doesn't sound like Jev is that hard to replicate, given all the 3rd parties who are getting pretty damn close, or even better, results within a week.
The more likely scenario is either OpenAI or Anthropic just go pay some highly inflated price to buy Jev. Mainly just for its people and the PR, not tech.
Which is likely what all the VC, hype machine, and overinflated claims are really about anyways.
The tech etc is easily replicated. The hype / name, not.
I seem to remember reading that the Jev-founder-guy is ex-OpenAI anyways. So that's how these things often roll.
That article is a bit myopic. People and companies don't want to feed all of their ideas, projects, and intellectual property to a morally unscrupulous oligopoly. We learned that lesson the hard way with the last batch of tech monopolies, and the shift towards majority open weight models proves the trend.
Open weight classifiers and open weight LLMs will be burned onto silicon cards in a few years after the models begin to stabilize. They'll be in PCs and laptops. That's going to capture a HUGE chunk of the market.
If you need more horsepower, you'll rent the same silicon safely from AI services cloud providers without handing your data over to Anthropic and OpenAI.
I tried looking into this, but frankly, I have a very hard time understanding Jev. If you’re going to offload half the work to a generic classifier, then you’re not getting the full value of the intelligence from the LLM why is this better? I get there’s going to be a speed up but I care about quality more than speed I guess.
Every major AI shop has a ton of in-house classifiers already, big, small, generalist, specialized. Some are used in inference pipelines (e.g. safeguards), some are used in data preparation, training, analysis and investigation, research, various one-off and intermediate tasks etc. Offering them on a public API doesn't always make business sense. I don't see much substance to this buzz, looks like people that are new to all this are discovering that classifiers exist, they are more efficient at classification, and many tasks commonly done with generative models are classification in disguise. Which is not bad at all, a fresh look at their use is great to have.
Correct me if I'm wrong, but a zero-shot classifier like Jev is fundamentally different to a classifier with a fixed task (e.g. for safeguards), unless they trained a general purpose system to complete the safeguard task, which seems unlikely.
Correct, but zero-shot classifiers are also not new.
But zero-shot classifiers with this level of intelligence, world knowledge, ergonomics, cost profile, and ease of use are new.
I feel like good engineering doesn't just ignore those things, or at least it didn't before recently. Now I guess social media has added a pressure to reduce everything to a hot take.
Fixed guard today is not very fixed. For ex, the safeguard qwen released is a full 4b llm model. It has no different to normal llm model arch except tuned for this specific purpose,
So it is tuned specifically to classify content for safeguarding? I'm not familiar with this particular model, but it most likely has a specific classifier head that is tuned for the safeguard task. This is completely different to zero-shot classification.
I fed into the hype at first. Testing Jev and Laya, they both suffer from the same issues as LLMs that stop them being useful beyond limited classifications.
I can't see any benefits that a typical ML classifier would not be better at.
Agreed. I tested Jev on OpenRouter this past weekend and it’s “okay” but a specific classifier is significantly better. It used to require skill to import sklearn (ok, not really), but now it’s literally one prompt and upload your Excel file or whatever and you can get your classifier out. It’ll run free, instant, more accurate.
This is predicated on you having training data already. I approach Jev more like Langchain -- you can prototype something new extremely fast and cheap, and if the use case works well enough, rip it out and build something bespoke. If it doesn't, you didn't spend a bunch of time curating a training dataset anyway.
I think the main argument would just be that because the model is general, you don't need to retrain it from scratch for a new problem - just tweak the input prompt. For a typical classifier there's a lot more hassle - collecting the data, training it yourself, retraining under distribution shift... In that sense Jev seems great for prototyping or small-scale use cases.
Counterargument: this works for quick prototyping, but for any serious business, you will eventually develop a benchmark/eval to track how well the general model is working, and once you have that dataset, you might as well train a specific model
Jev's bet is that if it works well enough for random use cases that nobody complains, then management won't feel a need to develop a benchmark/eval, and they won't need to employ all those data science guys.
I'd also add that they're hoping Jevon's Paradox also leads to a whole new segment of users who would have never reached for a classifier in the first place, given the barrier to entry.
And if you do get complaints or feedback on the classification, have a dev log into the user's account, tweak the Jev prompt a little until the issue goes away, and push it to production
Or not. And replace the generalist with the next generalist that gets you +15% on that benchmark for the same price, or gives you the same benchmark performance for half the price.
One advantage of using generalist models is that the generalists are improving - regardless of whether you're doing anything about it.
Training a classification model is trivial these days, even for a number far bigger than what Jev can do.
Using Jev as a plain classifier is the least interesting case. See robotic control, navigation, computer use examples, none of it possible with a classifier.
That's the point, they're classification in disguise. Agentic game engines/mods started doing this long ago due to the latency requirements (although they're typically using small BERT-like models that need to be finetuned, or low TTFT generative models and structured outputs). New or newly discovered use cases are great, sure.
Prompt ingestion is going to be the biggest differentiator.
Being able to route prompt to features that then route to special models would be a really solid implementation.
It starts to break down once you go over 20 classifications. Which is very basic routing that can easily be done with typical ML models for cheaper and faster.
Thanks for the breadcrumb!
Entertainingly, OpenAI had a general purpose zero-shot classifier API built on GPT-3! Just no one ever cared that much about it, so I guess it got dropped somewhere along the way since 2020/2021.
For me, I think the big deal is that it promises to be general and broadly applicable and high quality. That's new and special. But we'll wait to see if the claims actually hold.
"I don't see much substance to this buzz..."
Agreed. This isn't new. I led a research team at a Fortune 500 that used a transformer based classifier approach in a commercial product as far back as 2022 and we didn't come up with it. It was already common enough that we found the inspiration for our implementation on some web forum. Models like RouteLLM have been around for a long time. The news here isn't that a new model type came about, its that a large percentage of people messing around with this stuff that are new to AI just learned that not all transformer based implementations need to be autoregressive.
> Agreed. This isn't new. ...
It doesn't have to be new, it just has to be consumable by devs.
You could send text before Twilio. You could process credit cards before Stripe.
Jev, at the end of the day is an easy to use API.
Everyone seems to forget that usability is a thing.
It took us a few hours to implement that one we used in 2022. This isn't about usability its about a huge population working on this stuff not really knowing what is available until it becomes a meme.
> Every major AI shop has a ton of in-house classifiers already, big, small, generalist
I think building generalist classifier is some open ended research task, where frontier labs can contribute: different internal reasoning, instruction tuning, building datasets and benchmarks, building and distilling super large models.
If your "master AI" is good enough, it should be able to find and learn about and use specialized tech AI like Jev if it suits your goals
and then whatever tech it is will be absorbed/assimilated/Sherlocked into the leading products anyway
This article is extraordinarily hard to read. It’s tummelvisioned on OpenAI and things like tool calling which are only relevant to the extent that llms have been tuned to make relative choices, but this applies to all LLMs. Also, some really outdated references. LLM written, perhaps?
Also, moat discussion is the lowest form of discussion. I don’t care if jev has a moat. Did it get the interface right? What other past ideas have we overlooked that if given some love, could kick the door down like jev did?
Really silly stuff.. people wanting to talk about moats when there’s no castle. Moat talk merely projects the illusion of being engaged but, much more often than not, it’s hollow engagement.
> LLM written, perhaps?
pangram says... 20% of content likely AI written, 80% of content likely human written.
Eventually humans are going to start writing like AI if we read enough of it.
Partially. I'm a terribly slow writer and get stuck on phrase choice, but I'm good at content ideas, outlines, and editing text already on the page. So I have AI do the bit that I'm not as good at.
Process: First, actually have ideas :D Then, I write an outline for what I want to talk about at basically a sentence-by-sentence level. (This is me yelling things at my computer.) And then I have the AI convert a chunk at a time into prose. I reread it and rework it to be my voice.
Then I have the AI help with things like subject titles and social posts.
¯\_(ツ)_/¯
Hi, thanks for engaging!
I am not personally like philosophically or ethically opposed to having LLMs help or even write text... the issue is that I see so much LLM-written text that is just _bad_, and very hard for me to read or extract meaning out of, especially relative to it's often long length.
People think they are bad writers, but usually LLMs are actually worse (although they are great writers of catchy slogans and phrases, and then put together an article out of them, which I find just exhausting to try to get more than a "vibe" out of).
What you describe sounds like a fairly reasonable approach, but I suspect the parts the person I was replying to were reacting to was areas were you had not been as succesful at reworking it to be your voice. Which are probably also the parts pangram flagged as likely LLM written.
Pangram gives you a handful of free tokens, it would be interesting if you wanted to see what parts are the 20% pangram is flagging as LLM, and reflect on if they went through your process differently. Perhaps they were the parts you didn't spend quite as much time reworking it to be your voice. (I 100% believe you, because I've been running things through pangram a lot lately, and it's actually pretty rare for it to flag mixed content, instead of 100% likely AI or 100% likely human).
There was recently a post on HN that said if you want to avoid this, you really can't use any words at all that are written by the LLM, you can use it for suggesgting structure or points, or reviewing your work in various ways, but if you accept even a single phrase it provides... it's not going to be "reworked into your voice", it's going to be picked up by people (at least those of us who have become sensitive to it) as AI, because it's like, headline-speak.
(I can't find the article now, because I'm trying to quit facebook so can't log in to find my own post of it there, have to stop using that as bookmarks substtitue!)
Of course, that's not welcome advice if what you want AI for is "phrase choice".
I'm just here to say, LLMs are not good at phrase choice either. Although they may be quick at it. I feel like it's asking the reader to do the work of trying to extract meaning from slop that the author didn't have the energy to use to encode it well in the first place. I don't have time to try to read sentences that the writer didn't have time to write, i find myself bailing out quicker and quicker at signs of AI slop buzzword headline-speak.
Panagram seems fun! I tried it out with a large chunk of my post (whole thing wouldn't fit). Panagram says it is 98% human. Feels right, to me b/c I aggressively edit whatever comes out. The thing that I'm avoiding is the ominous blank page - I just freeze. If there's text there I can always reshape it. And I do, heavily.
prodigycorp is entitled to their opinion
I find it unlikely. OpenAI is all in training models with reasoning with RL, and Jev-like models are the total opposite. They are made to not reason at all to be fast. If you want to add reasoning on top, you might as well use a conventional LLM because you lose the price and speed benefits when you output auto-regressive tokens. I don't think OpenAI will even bother with this.
> My main assumption is that Jev is using something quite close to a conventional large language model. As evidence of this, Latent Space reports that many of the early clones are indeed LLM-based.
Not proof that this is the case with Jev though. It might use non causal text encoder for the state, which could make sense given that it's very good for its price.
I don't see it fundamentally any different than knowing when to use a tool. Is this tool like RAG an important enough corner case to train for it? I dunno.
LLMs already shell out and write code to solve certain problems. This is just a special case of that.
It's a special case for an LLM, and you can use an LLM with structure output to get similar results, but you can engineer specifically for that case to get better results per dollar for it. That's why there is little reason to adapt GPT 5.6 Sol or wathever for this task; it can already do it (at a high cost). For OpenAI to compete with Jev they have to maintain another line of models, something like "GPT-5.6-instant-decision", that is small, fast and cheap, in the scale of GPT-5 nano.
Note that I don't think OpenAI is incapable of doing it, but I just don't think they will bother with it.
Keeping people looped into your product is pretty important, but yeah, there's not clean way currently to separate "structured" outputs from the token stream and to start using a different billing structure there. And I also appreciate that they aren't going to be keen on gving free or near free output either, so gotta figure that.
If there's money to be made, I'm sure sama will find a righteous reason to offer it.
system 2 is just an llm with a forced toolcall IMO
In the olden days we call this classifier, usually assignment 2 of Machine Learning 101. BERT (well, GLiNER specifically) and diffusion are calling and want their Large Classifier Models back.
https://github.com/vllm-project/vllm/pull/57250
There is one benefit that Jev has, that it is not OpenAI and thus it's probably less likely to steal your own work.
> that it is not OpenAI
For now. Any company that grows to OpenAI/Anthropic's size and gets VC money is ought to become greedy.
Or OpenAI just buys them outright. Buying your upstart competitor seems to be in the Silicon Valley Ten Commandments. The Fed whussed-out on breaking up FB and Insta last year, so there's never going to be any kind of remediation to worry about.
And for Jev, everyone has a price, and OpenAI's raised an historical amount of funding.
Not steal, but keep it indefinitely, per the JEV TOS
They absolutely need ZDR
I have been told that ZDR is available upon manual request.
I can confirm, I sent their sales team an email asking for ZDR, and they provided it within an hour.
Enterprise I thought.
Or use it through Vercel [0] with per call zdr.
https://vercel.com/changelog/typesafe-ai-jev-now-available-o...
At this point, we shouldn't trust company even with ZDR.
This is a tired argument that needs actual evidence to go beyond the level of a conspiracy theory.
TIL the terms of service are a conspiracy theory.
Which clause? For instance openrouter lists openai as "Logs: this provider may retain prompts, but does not use them for training."
https://openrouter.ai/openai/gpt-6-astra#providers
Why would OAI need to follow Jev? I really think this is paid by Jev. Jev itself won't have lunch money in a shortwhile because there are literally 10s of free alternatives available which can be run locally on basic consumer hardware. Terrible utility aside, there is no sensible business proposition in Jev.
I'm having a really hard time wrapping my head around why Jev is getting so much hype. It feels manufactured to me. I don't think they've proven a significant market for their product, and there's no independent benchmarks that prove anything. To me that's doesn't pass the smell test.
But if they had to show how well their product worked they might give away the whole game... because they'd have to compare their "noul" class against an NLI benchmark for instance, and possibly show they're losing to cross encoders and give away the fact that they are just rebranding NLI. Or rerankers (choice) or zero-shot classifiers.
The AI hype cycle is always looking for the next big thing. It really doesn't take much for enthusiasts to get very excited and push something into the stratosphere. Just not having a vibe coded website, and someone that made ChatGPT is enough to set them apart. Hitting pain points like pricing and speed and also implicitly mentioning llms (even if it's to mention it can't generate text in contrast to llms) make it seem like a major step-up.
Why wouldn't i use something like these?
- https://github.com/NandhaKishorM/laya
- https://github.com/deepanwadhwa/OpenDecision
For context on what "Jev" is: https://news.ycombinator.com/item?id=49717558
If OpenAI releases something similar to what Jev does, then that would be like admitting defeat. Their whole spin is AGI and world ending danger. Why would somebody with an AGI at home make something like Jev which is intended to be a part of some SW the AGI is going to replace anyway.
A lot of business automation doesn't need AGI, doesn't want to pay for AGI if they don't have to, and would be better off using a classifier than something increasingly unreliable with a mind of it's own.
There are many automation pipelines that use LLMs because there was no choice, but the multi-way classification that Jev provides is exactly what they need, and is going to be way faster and cheaper, as well as having the benefit of calibrated probabilities and structured output that can be relied on.
I would assume that they would fold this type of classification into their responses API next to existing ancillary tooling that they already ship. I think OpenAI positions themselves as being wholly focused on AGI - but I think their business model leans towards lock-in via superior tooling (Codex, ChatGPT, etc.). This feels like an easy win for them without muddying the larger vision.
Why release an image model, or a video model, or custom agents if AGI will just make them all obsolete?
Why build codex if AGI will replace SWEs?
Why build excel integrations if AGI will replace spreadsheets?
The short answer is: they need to make money now to fund continued improvements towards AGI. This is extremely obvious and these "well if they had AGI..." arguments are clearly in bad faith.
You have to run a legitimate business and make money. After the "takeoff," anything is fair game. It could result in value accruing to capital (lab shareholders) and everyone else is screwed, it could result in the status quo being maintained but people do way more and GDP goes up by a lot, it could result in a post scarcity society etc. Theres a reason why its called the singularity - you can't see past the event horizon.
In any case it is optimal for OpenAI to create good products that generate revenue NOW vs going for some straight shot pie in the sky model that is "AGI." SSI is attempting to do this and I suspect they are about as close as anyone else is, and I'd honestly assume they are likely further away than OpenAI or Anthropic at the moment. But one of these classes of companies has actual revenue that is arguably good for the economy, the other is just a glorified research lab that has produced nothing of value.
AGI is like water, and these surfaces are the plumbing. Agents and the internet's existing APIs are how AGI will actually be delivered in the short term.
But you're completely right: our current APIs were optimized for humans, and some like Excel date from the 80s. So in the long term, agents will highly likely create their own interfaces optimized for when no human is in the loop.
Which is exactly why companies stick with current APIs for now. Fable 5.1 already emits alien-like reasoning traces, and the recent OpenAI agent swarm accident just proved this.
the point of this article perplexes me. the implication is we should all just take our Quietus. you decide.
I didn’t realise they were competing. If anything Jev seems complimentary to LLMs.
ASI isn't here yet. It could benefit people in the interim and help make the ASI.
The people that buy into "AGI is here" and the people that understand what Jev is and how useful it would be to hook it up to an LLM are two separate circles, so there's no "defeat" being admitted.
There's a big difference between what AGI can accomplish and what it will accomplish. We must also ask why OpenAI wants to summon their God, and how they would want the average person's life to change if it comes. I think they will try to clone Jev just to add another revenue stream.
without getting too far into it i would just like to note that i am in both these circles
It's just another tool. Luna exists for a reason: it's the right tool for the job. If they release AGI and it costs $1 and 5 seconds to decide "is the customer asking for a refund", then that's a terrible use case for AGI if another tool can do it with 95% accuracy for $0.002 and 50ms.
> If they release AGI and it costs $1 and 5 seconds to decide "is the customer asking for a refund", then that's a terrible use case for AGI
Is it? If AGI is here then by the time I test and deploy that the AGI will be most likely cheaper and smarter because it improved itself (for example by implementing it's own Jev for stupid prompts like this), so why invest into a more complex solutions?
I don’t think that was a great example since there’s only so many refunds a customer is going to ask for. And it’s saving time that otherwise maybe would have to go through a human. The rate is low enough that a more expensive model makes more sense.
Though for tasks where you are trying to search through billions of documents, social media posts, etc. and extract certain information, where each individual post is of low value and only the data in aggregate is valuable, then that’s where you’d want something cheaper and faster.
Such as if you want to look at all posts on X in the last few months and find how many have a negative or positive sentiment about the economy (or are unrelated).
Of course you could use a special-purpose model for this, but the whole point of something like Jev is to ask whatever questions you want without having to train something new.
Because AGI will also handle whatever is happening after your "is the customer asking for a refund?" question. Replacing whoever is doing that refund.
Well sure, in the future you may also be able to ask AGI to "please just run my life", while you stay in bed.
In the meantime, today, in the real world, there are businesses wanting to automate well-defined business flows, who don't want some stroppy AGI with a mind of it's own to instead decide to hack into something, or reward hack and make the customer happy by just wire transferring $1M of company money into their account.
Why would it need to? There is a deterministic flow here for the actions that are allowed. AGI isn't needed for this at all if you can map out the flow and use a classifier to decide which route to follow.
I'm not exactly following through with the claim, can someone explain how the built-in classification would not necessitate more tokens used, or be much different from turning on reasoning? Not that I don't see the difference, I just doing see how OpenAI would do it well.
AFAIK Jev is nothing special technically so it's easy to embed it as an another tool for the LLM? For many batch tasks it can still be quite a token saver I think.
Or they can even offer it as a standalone API if deemed worth it.
Jev seems to have three benefits:
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
> It's output probabilities are calibrated to actually mean something
Don't fall for marketing BS so easily.
Jev can output drastically different probabilities if you simply reorder the list of choices. And Jev's "confidence" output is fake/redundant - it's just a formula applied to probabilities, it conveys no additional information.
I bet they will eventually "fix" (read hide under the rug) the ordering problem by ordering the list on the backend before feeding to the model.
It seems that anyway most of the value is in the speed and cost.
If it really matters to you whether whether some business-specific classification confidence is above/below some specific threshold (vs just relative order), then you'd be better off training or fine tuning a custom model for that. Maybe that is something that TypeSafe are planning to also provide?
Just to clarify:
> 2) It generates structured output natively - guaranteed to be correct
It's not guaranteed to be correct: it's guaranteed to be _formatted in a particular way_. You can get the same thing with grammars on any LLM.
Jev and Jev-like models have other advantages, but I feel like people forget grammars exist for LLMs.
Grammars do risk pushing models off distribution in a way that impacts their output quality in a way Jev allegedly does not suffer from. Additionally, Jev's ability to answer questions independently is also exciting. Using an LLM to answer multiple questions in one generation has the property of earlier answers influencing later ones. TBD how many of TypeSafe's claims stand up, but my testing so far is promising. I hope they author some papers on their methods as well, but that might destroy their moat.
If you really know what grammers did, grammer is a filter to mask out option llm provided but you don't like.
It does not change potential distribution in any means. It DROPS part of answer model returned directly.
The text generation model go wild because model relies on previous section it answered to continue later section. And because now it contain item model have no idea, it is completely screwed.
In the case you only require model to answer one of a,b,c,d and don't care about later segment at all. It don't really matter.
What I mean is that, in general, constrained decoding can push model output off into less probable regimes. This is well studied; see for example https://arxiv.org/pdf/2606.21619. The mask may only retain very improbable logits. In pathological cases, the constrained output may be little better than noise filtered through the constraint. When using existing structured output APIs, it may not be possible to even know.
You don't even bother text after the [a] at first place in this case
Your question is something like
anwser only a,b,c,d for following question a. b. c. d....
the model output possibility of next character a: 0.8 b: 0.7 c: 0.3 f: 0.2 d: 0.1
If the list contains option you did not provide. The model is confused anyway, it don't matter if you use grammer to filter out the bad option or not, the answer is screwed already.
Yes, agreed. I was speaking in general, of course. This particular topic is of interest to me, so thinking of the edge cases and confounds vs Jev.
In your example, I would expect an LLM to do fine and if you have access to the raw logits you can measure whether or not it was confused and assign a confidence to the answer it gave.
I do think that Jev handles more than this though and, in my early testing, does things that are not easily accomplished with guided decoding techniques.
The way jev actually internally work could be interesting though. I believe most llm are only tuned to return the first or second logits(or a few more) correctly as that is what the sampler would choose anyway. Do they alter existing model for better behavior across all options? Or they distilled one to have the proper behavior? We can only guess without the actual implementation.
Yes! I really hope they release some papers on their techniques. I am very curious.
I ran it through MMLU a few days ago and it scored ~90% so seems to have a lot of general world knowledge trained in. Makes me think your speculation is right. I have some credits left, might try and think of an experiment. I saw a gist where someone was asking it which model it was and it was picking qwen a lot, but who knows...
Anyway, thank you for the interesting discussion!
Although the underlying model is unknown. If it expose input token count, the tokenizer may be probable though. Most tokenizer segemnts wildly different in CJK inputs. It can probably be used to fingerprint the tokenizer based on token count if it is using existing tokenizer.
Not an expert at all here, but I saw a comment on the jev post saying that it you constrain an LLM suck that it outputs a valid structure, if the token with the highest probability is not the one that you expected because of the structure (and so you pick the valid lower one) this means the LLM was already confused and your answer is less likely to be correct anyways.
Is this actually true ?
(3) seems to be the hard one, you have to have training data with accurate probabilities, maybe, but perhaps not since people are primed to trust
No, you don't. You do RLCR, similar to that proposed here:
https://arxiv.org/pdf/2507.16806
yes, and... pretty much everything in the Ai field comes back to "data makes more difference"
Sure, and most days it doesn't rain.
depends on where you live, an important feature for data points about weather pattern probabilities
the underlying data set needs to be representative
RLVR and RLCR really don't need a whole bunch of special data.
the algorithms technically, sure, however the outcomes definitely depend on data quality and coverage like any other training method, this is well known
I don't think you've ever done either of these training steps. You are just handwaving.
you know what they say about making assumptions, yea?
and then you are going to ignore all the research and results that clearly show otherwise? why?
what might we infer about the importance of data from a learning algorithm like decision trees?
Read the paper. They train RLCR on existing big math problems. They subtract a brier score penalty from the correctness reward. No new confidence labels are needed.
Existing datasets, different reward function.
It's hard to say without knowing their architecture, but I'd guess something like block attention. You can process the prompt separately from the classifications into a latent space and then do some kind of late interaction with the encodings from the classifications.
There are plenty of other ways to do zero shot classification that would result in more "token usage" (really just having to reprocess everything for each class), but the pricing and the way they describe it narrows it down somewhat.
Normal LLM will do the classification on the text that is generated. Jev just returns the classification and confidence.
It has the advantage of speed and the confidence not being hallucinated.
But LLMs start to generalise on the pattern, rather than the classification that you want the more examples you have to train on.
LLMs start to break down as well the more classifications you have. Laya (Open source paper Jev is based on) even mentions that over 20 classifications and it starts to fail rapidly.
20 is around the level of sentiment analysis or minor intent routing. There are cheaper, smaller and easier ML models for that level of classification.
Jev is just as non deterministic as any llm.
That is, if you force any llm to return json and a confidence it can also do that too and mostly likely it will he better at any one shot classification task than Jev.
LLMs have the great quality of knowing more due to the depth and richness of the training data. If Jev is trying to classify anything outside of its training data, it’s going to do a terrible job.
I think the idea is that the latent thinking space in the LLM will be roughly the same for similar quality results - so the majority of executing well could be stripping back and fine tuning an existing LLM.
The claim of how they are doing it is likely wrong.... if you had to bet, it's likely an encoder model of some sort.
Rather than focusing on OpenAI in particular, let's just say that there are many smart people at other AI labs and if it seems like it will be popular, this technique will probably be copied. What would prevent them from adding another API?
Hopefully there will be some decent benchmarks and gateways for switching between providers easily.
I'd be surprised if they weren't aqui-hired by one of the big labs as a unicorn.
The ability to use classifiers under the hood for the larger models has the potential to dramatically improve cost and throughput, allowing them to increase margin on a very similar service.
The OpenClaw vibes are hard to miss here.
Mostly one guy sending a fleet of agents to add features at unsustainable speeds who wants validation from a large org? I don't see many similar vibes at all, the founder already had massive impact at OpenAI and before that, Google Brain.
Extremely fast hype and a very short-lived moat, those are the parallels that I see. The OpenClaw guy was smart to cash out at peak hype, and chances are Jev will be overrun by big players and open-weights pretty soon too. I really love the idea, but just don't see any sustainable moat here. People are already coming knocking [0], it should only be a matter of time until we get something like Qwen3.9-Classify (or a classify mode just baked into a multimodal LLM).
[0] https://archerhume.com/posts/jevs-architecture-unmasked
except the output is indistinguishable from hallucination.
comments are pretty weird here, there's no real moat to what jev is doing, it is certain that frontier labs are going to release their own jev and there are even open source alternatives (although nowhere near as accurate as jev).
so maybe typesafe's real plan is to front run and releasing their own new models for some time until they can get acquired which seems to be the only rational objective
I don't think it's unreasonable to think this, but I do think the burden of proof is on your side. Between the SaaS-pocalypse narrative that never materialized, and inexplicably losing their first-mover advantage to Anthropic, OpenAI's track record is not great when it comes to jumping on these micro paradigm shifts.
If the headline said "Frontier labs are about to eat Jev's lunch" it might be an easier sell. But if we're gonna include Anthropic, I think part of their success is actually making products for which there is demand. It will take time for something like that to come out of this new "decision model" paradigm.
the real win with jev isnt beating a trained classifier on accuracy, its that you can change what you're classifying by editing a prompt instead of retraining and redeploying a model.
People are desperately trying to cope themselves into thinking that there are alternatives to scaling up transformers to AGI/actual competition to OpenAI or Anthropic. Jev, continual learning, linear attention, local models, non-transformer architectures etc. Imo these are just random technologies that nerdsnipe your average twitter or hackernews user and give them some hope that some underdog can take a slice of the pie.
In reality, none of these really matter. The frontier labs can easily do something like this but likely havent because the size of this market is too small and it is not on the critical path to AGI.
When you have as many resources as OpenAI and Anthropic, theres basically no point in putting compute towards random bets that don't have a predictable return. And at this point, scaling up transformers is almost a surefire way ot putting money in via training and getting money out via increased capabilities AND it speeds up your own business by factors of X. Sidequesting a Jev like product is falling for twitter hype and is likely not going to happen, definitely not by Anthropic, and I'd bet probably not by OpenAI either.
Isn't this more and more likely on all shapes of model evolution? The providers will all copy each other.
And in this case I hope its true. I've been classifying a lot of email and while OpenAI `text-embedding-3-small` has been very helpful for fast and cheap embeddings, initial tests with Jev are very promising and much more ergonomic.
I put more thoughts here: https://housecat.com/blog/classifying-email
You might find my old post interesting https://arcturus-labs.com/blog/2025/03/31/supercharging-llm-...
I think the article is part right and part wrong.
The right part: autoregressive LLMs are indeed generating “probabilities” (scare quotes very much intentional). During pre-training and any SFT steps, those probabilities are nudged toward the probabilities, over the training distribution, of the next token conditioned on the previous tokens. (This is an explicit property of most training recipes: KL divergence is a “proper scoring function”.)
So if you prompt with “Paris is a city in ”, the next token probabilities estimate the probabilities over the input distribution that the next token in the sentence is the first token of France or of something else.
But there are huge caveats:
1. That is not at all the same thing as the probability that Paris is France under any distribution that you care about (the population of the various Parises, for example).
2. None of this necessarily usefully applies to RL or, as the article discusses, tool calling. The output probability of a tool call is not some Platonic idea of a probability that the input is worthy of a tool call. It’s a the result of a training process that tried to teach the model to be useful and to achieve its goals.
3. I suspect that reasoning makes this all much worse. Suppose that you prompt with “a help desk user with IP=a.b.c.d says they’re ‘in Paris’. What country are they in?” The model has been trained to generate a reasoning trace, which may well start with “let me think of where Paris could be. It could be in France or in Texas etc. The user was speaking English…” See the problem? The model is reasoning well, but it reasoned “France” before “Texas”, so the logprob for France was probably higher than “Texas”. At the end of the reasoning trade there will be an answer, but the logprobs for that answer are, at best, some representation of the probabilities of the answer conditioned on the sampled reasoning trace. And that is not the probability distribution that a Jev user wants.
> autoregressive LLMs are indeed generating “probabilities”
I find it slightly more helpful to say they generate plausibility
The next step is to rediscover Eliza :)
I did so a month ago, I developed a deterministic agent framework that works with a set of predefined intents, it is instantaneous and fully deterministic.
It works thanks to FlintParser (https://github.com/gioblu/NPC-Forge/blob/main/src/FlintParse...) which can transpile plain English to any Programming Language. IMHO this is the future of AI.
Check it out: https://github.com/gioblu/NPC-Forge
Can't they eat everybody's lunch simply by typing "Astra, please copy this product?"
They certainly have the token budget for it.
Jack of all trades, master of none.
Jev is well positioned to fast-follow BART zero shot classification
https://huggingface.co/facebook/bart-large-mnli
Not directly related, but still jev related:
Would it be intesting/useful to use jev to generate a block of text like LLMs do ?
Like asking it to pick the n + 1 word given the starting text (using it's choice primitive), but also asking n + 2,n+3 and so on at the same time.
Would it give coherent or useful results ? Or does the fact that it computes it "all at once" means it cannot make one of it's answer influence the other ones ?
Plenty of examples online, e.g.
https://github.com/kyle-pena-nlp/jevchat
https://www.reddit.com/r/LLM/comments/1winnju/jev_the_new_ai...
Despite Typesafe claims that Jev is not an LLM, it obviously is.
Everyone was obsessed with classification prior to transformers, then we had 5+ years of everyone (rightfully) obsessed with next-token prediction. What's this sudden resurgence of interest in classifiers? I thought we all agreed that ML tasks generally require something far more advanced than pretrained classifiers. My timeline was also absolutely filled with mentions of Jev, which makes me think it's a successful viral marketing campaign, like langchain. It's now so popular that the dialog is whether or not [frontier company] is poised to catch up to it or not? We already have openjev...anyone can use it. I don't get it, and usually that means it's marketing.
Jev doesn't code. It can be used with LLMs to simplify coding and token consumption, but still and LLM is needed. Will they complement each other? How can Jev replace LLMs? Are they even competing?
You can easily use Jev without an LLM (consider Jev used to make truthy decision branches in a script). It's not meant to replace LLMs. The tech is not a competition. But OpenAI is certainly in competition with TypeSafe, they'll want to keep people in their own ecosystem!
I’m shaking my head in disbelief.
Reading logits is the cornerstone of ML. It’s almost like many of the people reporting on and fawning over AI have no technical background and never knew about ML classifiers or calibration.
I heard that Jev is amazing at classifying people who understand ML, and people who don't.
Even the article itself has the title as a question: "Will OpenAI eat Jev's lunch?". A more useful title would be "OpenAI is Positioned to Compete with Jev".
Jev certainly feels vulnerable but whether it’s oai or someone else that goes after them seems unclear.
Wouldn’t be surprised if every single AI house spins up a copy
But like they usually also have an embeddings endpoint
I think the original idea originates from this author : https://laya.convaiinnovations.com/
Nope. Here's the closest I got https://arcturus-labs.com/blog/2025/03/31/supercharging-llm-... - 1.5 years ago! But I never really did anything with it. And I wasn't thinking about reinforcement learning anything.
Personally, I don't really care what OpenAI does here. What's going to be far more exciting is when DeepSeek, Qwen, or GLM start integrating classifiers into their open models.
Exactly, the future is open models. Which is also the reason why those overvalued companies such as OpenAI will lead to a market crash as soon as investors realize that.
I'm not sure it makes as much sense to use the same bigger models for the things Jev does. Part of what makes Jev appealing is the cost/speed. I can definitely see them putting out S1 spins of their smaller models.
That's not what I meant. I'm thinking more of AI systems that combine multiple modules the same way the brain has different regions. LLMs are just part of the bigger picture here. They're good at tackling a certain types of problems, but other approaches are better for different kinds of problems. Having a system that combines a generative model and a classifier for example would make it a lot more efficient and accurate because it has a bigger toolbox instead of using one algorithm for every problem whether it fits or not.
they write code and use tools, most of that does not need to be part of the "system" that is trained up
There are plenty of other use cases aside from agentic coding. However, even here a classifier has a lot of obvious benefits. There are plenty of scenarios where a model has to make a choice, and programming/system tool use happen to be deterministic sets of choices that can be evaluated.
> programming/system tool use happen to be deterministic sets of choices
I'm not following. There are way more tools in the world, and likely available on every machine, than can fit in any model context. One can create a small list, but that will be very org dependent. Programming is very open ended, I struggle to see the 'deterministic set of choices' here.
I'm looking at Jev/Kev like models for some control over tool calling and similar, but I also think that fine-tuning out behaviors (like qwen 3.8 wanting to use rkt all the time) may be better in the long run. Instead of having to check/change every tool call, we can refine the main LLM to have high enough likelihood of using the tools we actually have, that we won't need to check every call.
Jev is definitely useful, but also being held like a hammer in search of a nail right now, approaching OpenClaw like fanfare and hype levels. No one talks much about OpenClaw these days, I would not be surprised if Jev has a similar trajectory.
I'm not talking about checking/changing every tool call, I don't even know what you mean by that to be honest. The question is about selecting the tool to use in a particular scenario.
The whole point of Jev/Kev is efficiency. If you can make decisions faster, cheaper, and more reliably that's the benefit. Even within the model architecture itself, you could use a classifier as a router within MoE to pick the expert that's likely to give the best outcome.
maybe open source even eats Jevs lunch first
see here: https://news.ycombinator.com/item?id=49800574
I'm looking forward to next week when we never have to hear about Jev again.
Why is everybody so obsessed with it? There are 2 Jev posts on the front page even now, I feel like I'm taking crazy pills.
Nerd sniping the hackernews/twitter crowd with "large scale transformer-based language model alternatives."
People don't want to believe something as unsatisfying as "Scaling up LLMs" can yield something as profound as AGI/be useful, and just hope that literally anything else can take their mindshare away, and this just happens to be the new rage. Along with clearly-not-frontier-level open source models, non-transformer based architectures, etc.
Beats me also - this feels unreliable, extremely niche, and over-hyped. I don't trust LLMs even when they explain their reasoning; the idea of trusting a black-box classifier like this seems insane.
For certain tasks, it seems much, much more efficient. That's not nothing. People have been using LLMs for various classification tasks.
In my company, and I think in most companies that are using AI at all, one of the first ways it got integrated is as a classifier, to tag orders based on feeding all their data into a prompt and asking for a structured output.
I think demand for tools that are more tailored for this type of integration is high. I don't really understand why Jev is supposed to get my company's decisions right more than an LLM, but regardless of the tech I think people are just excited about the possibility of iterating faster, more explainability, higher-level tools that are specifically created to help hone classifiers etc.
Great, we don't need 15 thousands posts per hour across social media channels. We had classification NN before LLMs as well.
I think the point of Jev is to thread the needle of the gap between non-LLM classifiers and LLMs.
Classifiers like classical NNs require:
- annotated data, potentially a lot of it - training - inference
#2 and #3 aren’t a big deal if you have an ML engineer, but #1 will always be a potential headache no matter who you are. The tradeoff is that they could be quite fast, cheap, and you can get probabilities, not just classes.
With LLMs you get:
- zero shot classification (no dataset or training required) - potentially can use third party model providers like OpenAI off the shelf. Don’t even need to host your own model.
The downside to LLMs is that they are comparatively slow and expensive to traditional classifiers. Historically they also were prone to hallucination or malformed responses, though not as much these days. You also can technically get log-probs back, but these aren’t equivalent to the classifier probabilities.
Jev gets you the zero-shot, zero-infra benefits of LLMs, while being closer to the speed and cost of traditional ML classifiers, as well as both classification and probability responses.
Yeah but they weren't that great, you couldn't ask for arbitrary classifications after the model was trained. You are underestimating what they've done here, even if it does seem a little overhyped.
this one feels closer to the claw cycle
Classification models lend themselves to sparsity and explainability. The good ones are very simple and economic to run on a laptop. If someone told you before 2022 that a json classifier was a product you would have laughed it off.
To add to this the more difficult problems in classification done on scale have always been about collecting "good" -ve examples, enough data to calibrate on every confidence interval and debugging outliers. And those are solved on a case by case basis by the company pursuing its own peculiar version of the problem.
Am I the only one who thinks this is just all hype?
I think it’s about sample efficiency. You could finetune your own jev using Lora with very little data
I'm confused. Why OpenAI and not Anthropic? I don't see anything here that is specific to OpenAI.
It's not just OpenAI. It can be any frontier-level lab that has more funding than Jev.
I just had Claude and Jev combine forces last night. I've built a few personal browser extensions in the past and thought it would be fun to copy an experiment I saw on twitter where Jev classifies comments/posts etc as slop or not.
Fed Claude an api key from typesafe and a link to documentation, and within about 10 minutes I had a view of HN that was populated with a little ranking as to sloppiness of each comment.
When your mind has been wired a bit to LLM latency, it feels extremely fast, and for such a subjective rating I think it did a good job.
Feels like it sits in a space between traditional ML classification and the frontier models. I can't think of a 'real' production use case for it in my sphere of influence, but certainly some will. And of course there will be five Jev competitors by the end of the year.
gosh, for wanting TypeSafe to survive, this fellow just handed OpenAI detailed instructions and ideas for defeating them...
chaotic neutral
Literally never heard of Jev before now. Trying to figure out if it's really a big deal, or if OP is just Jev marketing, and where I would learn more about it that isn't just LLM-produced slop. What a world.
> Back when I was at GitHub working on Copilot I had the opportunity to work with a very new and very raw internal API for GPT-4. Out of the gate, we knew something was way off because, after an initially very coherent response, the model would have trouble wrapping up. It would end every response with something like "Let me know if you have any other questions. Have a nice day. Have a great week. Have a good time. Have a wonderful life. Have a special day. ..." and it would keep on like this until it hit the response token limit.
I love this!!
I'm glad someone noticed :P - It was hilarious once we figured out what was going on. We had literally removed it's ability to shut up.
Asking again (didn't get an answer in prior discussion):
As there have been a lot of Jev related submissions, can someone point me to a simple guide on how I can use it? For example, say I have a script/workflow where I use OpenRouter for LLM calls, and at some point I want to do a simple classification. Can I still use OpenRouter with some Jev model...?
With all the excitement around Jev, I suspect we'll see hundreds of options, it doesn't sound like Jev is that hard to replicate, given all the 3rd parties who are getting pretty damn close, or even better, results within a week.
vLLM has a PR very close to merging: https://github.com/vllm-project/vllm/pull/57250
Kev is an open Jev: https://github.com/jaredpalmer/kev
If Typesafe/Jev has 2-3 years of financial runway, this problem might solve itself.
{ "answer": "might", "probabilities": { "might": 0.99, "won't": 0.01 }, "confidence": 0.99 }
The more likely scenario is either OpenAI or Anthropic just go pay some highly inflated price to buy Jev. Mainly just for its people and the PR, not tech.
Which is likely what all the VC, hype machine, and overinflated claims are really about anyways.
The tech etc is easily replicated. The hype / name, not.
I seem to remember reading that the Jev-founder-guy is ex-OpenAI anyways. So that's how these things often roll.
love JEV, but you are right - how much time before a Frontier Lab release the same?
That article is a bit myopic. People and companies don't want to feed all of their ideas, projects, and intellectual property to a morally unscrupulous oligopoly. We learned that lesson the hard way with the last batch of tech monopolies, and the shift towards majority open weight models proves the trend.
Open weight classifiers and open weight LLMs will be burned onto silicon cards in a few years after the models begin to stabilize. They'll be in PCs and laptops. That's going to capture a HUGE chunk of the market.
If you need more horsepower, you'll rent the same silicon safely from AI services cloud providers without handing your data over to Anthropic and OpenAI.
their pipeline would be to just prompt it's internal next gen models to create a jev copy given all the data they have as a first pass.
I tried looking into this, but frankly, I have a very hard time understanding Jev. If you’re going to offload half the work to a generic classifier, then you’re not getting the full value of the intelligence from the LLM why is this better? I get there’s going to be a speed up but I care about quality more than speed I guess.