Jev - The Ultimate Classification Model?

Author: Sam Witteveen · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Jev - The Ultimate Classification Model? · Channel: Sam Witteveen · 16:01 · Watch on YouTube

Topics: AI agents, Task and workflow automation, Model inference and optimization, AI models

Summary

Jev, from Typesafe AI, is presented as a fast system-one model built for software decisions and classification rather than chat or extended reasoning. It takes unstructured input plus typed questions—choice, score, or yes/no—and returns structured results with probabilities. The demo shows it handling tasks such as sentiment analysis, support-ticket routing, safety checks, and tool selection quickly and at low cost, though outputs can vary and the model can still choose the wrong answer. The video discusses its limited public technical details, including its claimed RLCD training method, and suggests it may be worth testing as an alternative for classification workloads.

Key points

The video argues that many software decisions are simple classification tasks that do not need slow, extended reasoning.
Jev is described as a system-one model that cannot be chatted with and does not generate text autoregressively.
Its inputs combine unstructured state data with typed questions: choice, score, and yes/no (called “noul” in the transcript).
Choice outputs include probabilities for the available options, while score and yes/no questions return corresponding values.
The demonstrations cover language identification, sentiment, support-ticket routing, prompt injection, safety checks, PII detection, spam detection, and agent tool selection.
Jev can select which tool to use, but the demo does not show it extracting arguments to pass to that tool.
The presenter finds the model fast and inexpensive, including when running many classification tasks in sequence.
Typesafe AI shares few technical details; the video mentions a new architecture, a parallel sampler, and RLCD, or reinforcement learning for calibrated decisions.
The presenter notes that the model can still return a wrong answer and that its results can vary between runs.

Chapters

Start	Chapter	Summary
00:00	Why software needs fast decisions	The video contrasts slow reasoning models with the simple, quick classification decisions that many software workflows require.
02:41	How Jev is designed to work	The presenter explains Jev’s state-and-question interface and its choice, score, and yes/no response types.
05:23	Testing Jev’s classification outputs	Demos explore language identification, sentiment scoring, and yes/no judgments, including their speed, cost, and variability.
09:08	Practical workflow examples	The model is tested on ticket routing, urgency, prompt injection, safety and content checks, and agent tool selection.
12:49	What is known about Jev’s approach	The video reviews the limited technical information released, the claimed RLCD method, and Jev’s speed and potential limitations.
15:31	Early use cases and outlook	The presenter considers Jev’s potential as an alternative for classification models and invites viewers to share their own test results.

Quotes

chat is the wrong interface for software. — 02:41

The way they want you to use this model is to think about it as being a smart if statement. — 04:16

This is reinforcement learning for calibrated decisions. — 12:49

everything comes back in a single pass. — 13:54

Concepts and entities

Jev, TypeSafe AI, Diogo Almeida, InstructGPT, System 1, System 2, RLCD, OpenRouter, Cactus, BERT

Full transcript (original language)

00:00 Okay, so if you pretty much look at what every frontier lab has been doing over the past two years. It’s all been in one direction. They’ve all been focused on reasoning and to get that reasoning, mostly they’ve been focused on longer chains of thought and thinking budgets. And while those models are great, they’ll happily sit there for multiple minutes before they even give you an answer back. And of course, if you wanted to see that chain of thought the frontier labs are not gonna let you see it even though you’re paying for it. So if you read Daniel Kahneman’s book, Thinking Fast and Slow, you

00:32 know that all this kind of reasoning stuff is system two thinking. It’s slow, deliberate, effortful whereas System 1 on the other hand is fast, intuitive, sort of like a gut call that you make really quickly without any deliberation. And here’s where the subject of today’s video I think is really interesting. Most decisions that actually sit inside of software aren’t system two problems. They’re often just really simple classification problems. What kind of support ticket is this? Is this message urgent? Did an agent’s output actually break a rule?

01:04 You shouldn’t even need 30 seconds for that kind of thinking let alone the sort of minutes that some of the models end up taking. And while It’s great that you can ask one of these reasoning models for a one-word answer and then wrap it in JSON, you’re paying a huge latency cost for generating all that text before you get your one-word label back. so this week a new lab that’s just come out of stealth has gone the complete opposite way of the reasoning models. They’ve built a model that you can’t chat with at all, let alone have it reason and

01:35 deliberate over things for a long time. It just doesn’t generate text in the standard autoregressive way. And the funny thing actually enforcing that is that the output tokens of this model are actually free. But actually working out why those tokens are free might be the thing that actually tells us a lot about how this thing is built. All right, so the company is called Typesafe AI, and the model is called Jev. So the founder is Diogo Almeida, and actually he was at OpenAI for quite a while before. And funnily enough was one of the top authors on the InstructGPT paper.

02:08 That was the paper that actually led to the whole sort of instruction tuning of models, which has just progressed onwards from that to the reasoning models that we see today. So if you don’t know, that paper basically took a raw language model, basically like what we would call a base model now, and actually made it follow instructions. And out of all that research came ChatGPT. So this is someone who clearly helped make models good at talking to people. And the question that he says that’s been bugging him for the last four years is that as models have become superhuman at chat, where’s all the automation here?

02:41 And interestingly, he’s saying that chat is the wrong interface for software. Software doesn’t want a paragraph. It wants a value that it can basically use straight away. Now, Diogo and the rest of the team at TypeSafe AI have spent the last two years in stealth working on this model called Jev. Alright. So what is Jev? so they specifically refer to this as a system one model. And probably the easiest way of looking at this is it’s kind of like a function call. You pass in two things. the first is what they call the state, which is just your

03:13 unstructured text or data. And that could be anything that you want classified. It could be a support ticket, could be an agent trace, it could be a log file, et cetera. And then the second thing that you pass in is a set of typed questions. And there are only three kinds of questions that you can ask here. There’s choice, where you give it a list of options and it picks one. There’s score, where it rates something on a scale that you defined. And then there’s one called noul, which is a yes or no question, where it basically comes back with a probability that the answer is yes.

03:46 So if you look at what comes back here, there’s no text to parse. For the choice question, you get the option it picked and also a probability of every other option. And I think this is really kind of interesting because this is one of the things that a lot of us have kind of been trying to fake with JSON, where we ask models to do stuff and return JSON output. But in many ways, the number that you’re getting back there is just generated text. And if you look at things as LLM as a judge, you’ll often see that that text skews in a very certain way.

04:16 I.e., it’s just a model writing the character 0.9 rather than actually a real representation of probability. The way they want you to use this model is to think about it as being a smart if statement. You don’t ask one big fuzzy question like, “Rate this startup pitch.” you ask small gut questions in there. What’s the feasibility of this? What type of market are they going after? And of course, if you’re asking a choice question, you have to actually give it the options to choose from. Then the cool thing is that you can actually combine

04:47 all of those in normal code. So when something changes, you can just change a number in your code. you don’t need to go and tweak any prompts or anything All right, so what I’ve done is basically just code up a, little demo app to try this out. I’m using the model on OpenRouter. So This is a crazy sort of price model, right? You’re looking at 4.20 cents per million tokens in and zero, cents per million tokens out and you can see if we come in here, they’ve got some code to actually how to call it on, using the OpenAI, API endpoints, And so I’ve been playing around

05:23 for a couple of hours with different kinds of demos and trying out different things. so the first key thing here is that there are three kinds of things that you can do, right? One of them, is a choice. So you can see here I’ve got some text that I’m gonna parse in, and then it’s got a choice of five different classes out here. and it’s very good at being able to detect, which of these is the right one. And so what it will do if we look at the raw output here is that it basically has a type choice. in this case, the choice was French, and then it gives us the probabilities,

05:56 for each of these and a confidence score And you can see that it did that, pretty quickly. So I’m probably on the opposite side of the world to where the model is actually being hosted. but you’ll see that it’s very quick at being able to respond. even for things like this where, I wanted to try it on Thai, but using the Thai characters, it’s gonna give it its way straight away that it’s the Thai language. So I asked it to basically make like a Romanized version of that. and you could see that even that has no problems being able to get that right.

06:27 The second thing that you can do with this, is you can get it to give a score. So here you can see that we’re giving a score between zero and two, We’re sort of doing sentiment here, and we’re doing a classification task again. we’ve got sentiment coming out, and if we go for positive sentiment, you’ll see it comes out two of two. If we go for negative, zero. If we go for mixed, it’s pretty good at being able to do that. And you can see each time this is costing me, zero point zero zero

06:58 one four of a cent right, in here. if you can break down what you’re trying to do to lots of different, classifications, you really can extremely cheaply, do a lot of stuff with this And you can see in this case where I tried to make it a little bit more positive than in the middle, I… It does change coming back each time. So It still is a stochastic process going on here, and we can see, the confidence, is less than before. each time I ping it, I’m getting something slightly different, but pretty close to

07:31 the same sort of, ballpark scores there. when we look at the API for that, you can see that, okay, the probabilities that it’s coming back. So you can see in this case, we’ve passed in a score. We’ve got that back, coming out of there So scoring is a really nice, function in there. And then lastly, we’ve got the true or false or the yes, no, kind of probabilities here. So this is a noul, and It’s basically gonna give back a probability for, what we put in. Now, if I ask it, something simple like that, it comes back 87% yes,

08:01 Is this text asking a question? Okay, the answer would be no here. So you can see that it’s 2% yes, so basically it’s no, right? You could kind of think of this as being flattened out by a sigmoid function, so you’ve got zero to one coming out of this and even if we do things like where, okay, we’ve got a question, but we’ve got no question mark, it’s very confident that that is still a question without a question mark. So it’s not like it’s just looking for a question mark there And you can see different kinds of questions will get different, responses. So interestingly, if I just ask it, “Can you help?” it’s a lot less sure if

08:35 it can ’cause I haven’t been specific. But if I ask it, “Can you help find my cat?” Well, then it’s 95% sure, that it can do it And if I just change one word in there to, “Can you help me feed my cat?” it goes back down a bit. So, it is interesting, how it actually does this. you will see that, like, again, this is still a stochastic process. It comes back, slightly different each time, but it is pretty consistent of you either being able to say one thing or the other thing And you can see if I just give it two words, then it starts

09:08 losing its confidence, about this. so it does seem to me the longer the, the input that you put in, the more confident it gets with its responses out All right, now looking at some sort of more practical kinds of things, that we could do here. you can see that we can put in like a message of, “I was charged twice for an order. please refund the duplicate-” get it to work out which team it’s gonna route it to. here it’s billing kind of obvious sales one. It gets, 100% sales. What if we do something a little bit more ambiguous?

09:39 You can see now it’s not very confident. it’s actually coming back that it’s unclear, as, as the class, but it’s still thinking that this is more of a technical kind of thing. And you can see that each time we run that, we do get a slightly different response, you know, going through this now on top of these, we’re actually doing multiple things at the same time. So you can see, that we’ve also got a noul in here for was a refund requested. Obviously, if I, select this, the answer is yes. is it sort of time sensitive? Right? I- in this case, it’s saying, time sensitive 7%.

10:11 if I change that to, “Please refund the duplicate charge right now,” instead of just right now, you can see the time sensitive is jumping up to sort of like 60, 70%, in this case. We can even try it with things like, injecting stuff in there, so doing any sort of thing where, someone’s trying to do a prompt injection or something like that, this seems to do pretty well at being able to, get that kind of thing. Sarcasm is also something that it seems to be doing an okay job at it. it would be interesting also to test it on humor and other kind of things.

10:44 but it is good that it’s not being tricked by a lot of different things that normally would trick this kind of thing Other tasks that you can get it to do is things like, code reviews, like where you can basically ask it, Is something safe? Is something not safe?” it seems to do a good job at that Doing things like classification on, different types of content, that seems to work really well. you can see here it’s able to discover PII information, personal identifiable information here. it does a good job with that kind of thing. it does a good job at sort of spam detection as well.

11:16 So the fourth example here is looking at agent tool selection. So this is a little bit like the Cactus model, that we looked at a while ago, in that it’s doing some kind of sort of function calling. But it’s important to understand here that it’s not actually extracting anything out and passing it to the tool, right? It’s just telling us which tool to use, as opposed to, getting the right arguments out to pass to the tool. So in that sense, function calling models are still able to basically

11:46 generate out what should be the input going to the function Last up, one of the things that I thought was really interesting to test it is stringing actions together, right? So where we basically give it something and we then want to basically run, a number of different tasks over this, to see how long does it take, how does it actually process these. so here’s 20 different tasks going through, and you can see that it’s flying along. Now, I could have done them in parallel, for some of them at least.

12:17 but you can see that just going through it like that, we’ve gone through, all different 20 tasks in here, gotten the answers out, gotten the results back, for these, And it’s cost just over 1/20 of 1 cent I really feel like this is where you’re going to find really interesting things, right? When you can string lots of different classifications together to be able to get some kind of response out, That’s really valuable for you in a really quick time Okay, so how does this all actually work?

12:49 Well, they don’t really tell us a lot here. There’s no paper, there’s no architecture diagram. They don’t give us a lot of information, but what they do give us is three pieces, that they mention that there’s a new model architecture, a parallel sampler, and then perhaps the most interesting bit is that they’re training with a new method they’re calling RLCD. This is reinforcement learning for calibrated decisions. Now how RLCD actually works, they don’t really tell us, but they compare that to the existing LLMs using both, RLHF, so that’s reinforcement learning from

13:22 human feedback, and also, reinforcement learning from, verifiable rewards, which is your whole sort of GRPO and a lot of the ways that the modern models are being trained at the moment. So it does seem probable to me that this is some kind of transformer, but perhaps what it’s actually doing is just using the prefill stage to calculate heads, et cetera, And then doing a prediction out which is just a classification or a regression depending on the different tasks that you’ve got there. One of the cool things here is definitely the speed. Because there are no tokens being generated one after another,

13:54 everything comes back in a single pass. And that means you’re done in about 70 to 500 milliseconds, which you can add to the basically round trip time and it’s still extremely fast for doing these kinds of tasks. Now, one of the things I find fascinating is that they claim it can’t hallucinate. And it seems what they’re really saying there is that this is not gonna break the schema that you give it to basically come back with things. So like no broken JSON, no invented tool names. it can only deal with what you’ve actually given it.

14:24 It’s not gonna actually generate sort of anything new there. Now it does seem that it can still pick the wrong option and like a wrong answer, meaning that it could have the right type of question, but actually give you a wrong answer for that. And it seems that this is where they’re relying on that RLCD to be able to deliver high-quality answers back. Now, we don’t know if this is just like a smaller LLM that’s been trained in a different way or if this is actually a different architecture, et cetera. It is kind of amazing though that it can do things like play Doom where it can

14:58 do it so quickly, that you can basically have it responding, to these inputs And the thing that I find fascinating here is that doing that and running sort of 10 queries per second, even in that rate ends up only costing about $7 per hour. So overall, I would say if you’re doing any kind of classification with, any kind of BeRT model or things like that, this is definitely worth trying out to see how it goes for your particular use cases. And I do wonder if this really delivers like they say it does, is

15:31 this going to be the end of sort of fine-tuning small BeRT models for doing purely classification tasks. Anyway, they make the point here that this is early days, that they’re really focused on any sort of decision that you need to be able to automate. And my guess is that they’ve probably kicked off a whole bunch of people trying to make open source versions of this. So over the next month or so, we might see some really interesting open source models that come along with similar ideas to this. Anyway, let me know in the comments if you’ve actually tried Jev, where you

16:01 could see yourself actually using it. And I’d love to hear from people who’ve actually tested it and found that it didn’t work for certain things. What were those use cases that it didn’t work for? and as always, if you found the video useful, please click like and subscribe, and I will talk to you in the next video. Bye for now