CLM 8B - Contrastive Language Models: Fastest AI Decision-Maker

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: CLM 8B - Contrastive Language Models: Fastest AI Decision-Maker · Channel: Fahd Mirza · 11:33 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Model inference and optimization, AI models

Summary

The video introduces CLM 8B, a contrastive learning model from Stanford and Nvidia research that scores candidate actions or answers rather than generating text. The presenter explains how it uses a frozen Qwen 3 8B encoder and small trainable projection heads, then installs and runs it locally. Demonstrations include customer-support decisions and a relationship scenario, alongside benchmark claims about speed and accuracy. The final ranking test produces a poor result, which the presenter notes as a limitation.

Key points

CLM applies contrastive learning to agent decision-making; the presenter notes that contrastive learning itself is not a new idea.
Rather than generating text, the model scores states and candidate actions or answers.
The system uses a frozen Qwen 3 8B encoder with small trainable projection heads.
The presenter says the projection heads are 75 MB and describes the trainable component as 20 million parameters.
The model is presented as Apache 2 licensed and free to use and fine-tune.
In a Chrome Dino comparison described in the video, CLM scores 91 points at 16 ms latency, while the comparison model scores 17 points at 150 ms.
In the customer-support playground, CLM scores urgency, department, and customer frustration in 379 milliseconds.
The presenter cites a Deep Sea benchmark result of 81.6% accuracy in 79 ms for CLM, compared with 71.1% in 449 ms for the competition.
The final ranking demonstration gives a poor recommendation, and the presenter says ranking is not there yet for the model.

Chapters

Start	Chapter	Summary
00:01	Introducing CLM 8B	The presenter distinguishes CLM’s use of contrastive learning for agent decisions from the older idea of contrastive learning itself.
03:17	How the model works and local setup	The video explains the frozen Qwen encoder and trainable projection head, then shows the model being served locally.
06:24	Customer-support demonstration	A playground example shows CLM scoring a customer complaint for urgency, responsible department, and frustration.
09:30	Further tests and limitations	The presenter tries a relationship scenario and a ranking task, noting that the final ranking result is poor.

Quotes

Clip connects images and text using contrastive learning. — 00:31

Instead of generating text, CLM learns to connect states and actions. — 01:37

No paragraphs written, no thinking out loud, just instant decisions. — 07:25

So, a ranking is not there yet for this model. — 11:33

Concepts and entities

CLM 8B, Contrastive learning, Qwen3 8B, Jev, CLIP, vLLM, DeepSeek, Fahd Mirza, NVIDIA, Stanford

Full transcript (original language)

00:01 You look at AI social media and every new model finishes the game and changes everything forever. Same thing happened with this model from Nvidia and a team from Stanford CLM 8 billion. People have started saying that it is something very new, a new class of model which we never saw before. Look, that is not really accurate. If you look at my channel, more than 2 years ago, we covered this contrastive

00:31 learning before. I covered clip right here on this channel. Clip connects images and text using contrastive learning. Not a new idea at all. But what is different is something else. As I said, a team from Stanford and Nvidia research has taken that same powerful idea and applied it to agent decision-making. Meet contrastive learning models of CLM in 8 billion parameter posted just few hours ago and the idea is quite

01:03 exciting. There is no doubt about it. That is why we are going to install it locally and I am going to test it out. More importantly, I am going to unpack in as simple words as possible what exactly is happening under the hood. This is Fahad Mirza and I welcome you to the channel. If you want to help out the channel, please become a member from the home page or in the first pin comment. Recently, I also have started this free weekly AI newsletter which you can subscribe at fahadmirza.substack.com.

01:37 So, what exactly is this model? Look, instead of generating text, CLM learns to connect states and actions. The result is CLM 8 billion. This is built on a frozen country 8 billion encoder with tiny trainable projection heads that are only 75 meg. Yes, that’s right. So, if I go to the model card, go to files, you see the model size is just 75 meg. Nothing more.

02:12 And it matches much larger models on agentic benchmarks while running up to nine times faster. I will also show you that very shortly. Another great thing, Apache 2 licensed, free to use and free to fine-tune. Let’s get the installation underway. I’m going to use this Ubuntu system. You can even run it on your nail. Although I have one GPU for the transparency. If you’re looking to rent a GPU or CPU on very good price, you can find the link to Vast Compute in video’s description.

02:44 First up, let me get clone the repo and I will drop the link in video’s description. It is going to install a few things. While that happens, let’s talk a bit more around a few of the things which they have shared. So, this is CLM versus Jev playing the Chrome Dino game zero shot. And yes, Jev is another thing where everyone was thinking it’s something very groundbreaking that is just a classification model, but a lot of exciting new features. Anyway. So, CLM is scoring 91 points at just 16

03:17 ms latency. Jev only reaches 17 points at 150 ms. Same GPU, but CLM is reacting nine times faster because it scores candidates instead of generating tokens. Everything is installed. Let me now serve it with vllm. First time when you run this, it also downloads that Qwen 3 8-billion which is frozen, but we run this model on top of it. I will also explain that.

03:49 So, while that Qwen gets downloaded, let me explain what is happening here. So, this diagram is what sums up everything. Let me explain in simple words. Because this is showing exactly why CLM is so lightweight. Input text goes into a frozen Qwen 3 8 billion. No gradient updates. No training. Completely locked. It takes the final token’s hidden state, normalizes it into unit length via L2 norm, then passes it

04:20 through a tiny MLP projection head, and that head is only thing being trained. Just 20 million parameters. The output is a unit length embedding vector that represents either a state or an action. So, this is a machine learning explanation. Let me explain it in human terms. So, again, your text goes in and goes into Qwen 3 8 billion, which is completely frozen, meaning we are not touching it, not retraining it, not trying to update it.

04:51 You can consider it like, you know, it’s a very smart reader that just reads your text and produces a list of numbers representing its meaning or semantic. Then we shrink and clean up those numbers, so they are all on the same scale, like converting different currencies to one standard currency. Then those clean numbers pass through a tiny neural network, just a few simple math layers. And that tiny network is the only thing that actually gets trained.

05:22 Everything is locked. That is why the whole trained component is just 75 meg instead of a gigabyte. So, hopefully that makes sense. If it still doesn’t, let me know in the comments with your question, then I will be happy to answer. And we have got this Qwen 3 8 billion running on the GPU. This is encoder in this terminal, acting as the the that converts text into numbers. Let me show you the VRAM consumption. Just 1.38 billion, nothing fancy, just

05:53 under 30 gig of VRAM. And now, in the next terminal, we are starting the CLM server itself, which loads those tiny 75 meg projection heads and sits on top of the encoder. Once both are running, we can hit the playground in the browser or even I think should be okay in CLI too, ask CLM to make decisions in real time. Let me run this. So, you see it has downloaded that model and now it is being served at localhost

06:24 at port 8700. Let me copy it and open it here. And there you go, this is our uh playground running. Let me turn on the dark background. There you go, much better. I will also make the font a little bigger if I can. Okay, so now look at this thing. On the left, we have a customer complaint as the state, as you can see here at the top. And customer is saying my invoice was

06:55 charged twice and nobody answers the phone. So, this is a state, an angry customer charged twice who cannot reach anyone. Below that, we have three questions we are asking CLM about this situation, as you can see here. There is an is this urgent, which team should handle this, and how frustrated is the customer. And then, um you can also see that not only that, CLM is not going to generate any text. It just reads the situation and then

07:25 instantly scores this answer. Let me run this to show. You see how fast that was? 379 milliseconds for three answers at once. It says urgent, 84.5% yes, department billing at 99% confidence, customer frustration very angry 100% confidence. As you can already see, no paragraphs written, no thinking out loud, just instant decisions. This is exactly what you want running inside a

07:55 customer support agent making routing decisions in real time and that is the use case of this model. And this is the right time to show you one very interesting thing. Look at this image which they have shared. Let me unpack it in simple words. You can consider this diagram like a dating app. I know some people get a bit upset that I use dating a lot, but you know, that is what it is. During training it looks at thousands of situations and actions

08:27 and learns which ones belong together. So, Mario about to hit a cactus belong with jump, not run left. Once that matching is learned, at game time you just show it the situation. It swipes through the options and picks the best match in milliseconds. No writing an essay about what to do, just you know, instant matching. Look at this agentic benchmark. So, just you know, imagine you ask five people to fix a bug in your code. Instead of

08:58 reading all five answers yourself, you hand them to GLM, oh sorry, CLM and tells you which one is actually correct. That’s what these benchmarks test. CLM picked the right fix 81.6% of the time on Deep Sea beating the competition at 71.1% and it did it in 79 milliseconds versus 449 milliseconds. Faster and more accurate, that’s the whole story and we just saw it first hand in this playground.

09:30 Okay, let’s do another test. Let’s see if CLM can save a marriage faster than it routes a customer support ticket. So, state is that wife, you forgot our anniversary again and you are still at work at 10:00 p.m. So, now this is just a yes and no question that gives you a probability. Like is this true or how confident are we? So, my knowledge is that survival, am I in serious trouble? And then next question is action. These are few of the choices, what should I do right now?

10:01 Apologize or ignore or maybe I would need to do all of these except last one. What is the danger level? Fine, couch tonight, full disaster. And then of course you can add more, just click on plus level. And then add whatever you like. I’m just going to go with CLM latest. Let me run this and look at the speed. And you can see that model is really trying to help out and it knows what is happening. So, it says yes, I am in serious trouble, which is true.

10:32 And I should apologize and then buy a flower. Ignore. So, you know, ignore and keep working. It has put keep working at the end. So, no further video maybe today. Danger level is yes, it is couch tonight, not full disaster. It’s not as bad. So, look, pretty good model. And then of course yes on curl and this you can also do the ranking if you like. So, primarily what it means is that instead of asking questions, you know,

11:03 multiple candidates, uh you just throw the multiple candidates at CLM and it tells you which one is the best answer. So, the question is what should I do when my wife is angry? And then let me paste some of the answers here. There you go. So, these are the answers. Let’s run this. And there you go. So, it has ranked them in order at what should I uh do out of all of these. And I mistakenly just duplicated this,

11:33 but still if you look I think the response is terrible in this one. There you go. So, it says keep working and apologize tomorrow, which is not good. It is totally different than what we did earlier. So, a ranking is not there yet for this model. But other than that, this one was quite good as you already saw the asking stuff. So, that’s it. Let me know your thoughts in the comments. Hopefully, this was useful. If it helped out, please become a member. Thank you for all the support.