Clef 27B Locally: Multimodal Decision-Maker From Text, Images and Video

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Clef 27B Locally: Multimodal Decision-Maker From Text, Images and Video · Channel: Fahd Mirza · 09:30 · Watch on YouTube

Topics: AI evaluation and benchmarks, AI models, Local and open-source models, Multimodal models, AI safety

Summary

The video presents Cloudflare’s 27-billion-parameter CLF multimodal decision model, which returns probabilities for predefined options rather than generating text. The host tests it on a social-engineering scenario and describes its parallel decision-making, then evaluates image and video interpretation. Further tests cover benchmark comparisons, an Uzbek-language news screenshot, and a chemistry diagram. The host concludes that the model handles a broad range of text, image, video, and classification tasks.

Key points

The host describes CLF as a 27-billion-parameter multimodal decision model that accepts text, images, video, or JSON.
Rather than generating text, the model returns calibrated probabilities for options in a single forward pass.
In a social-engineering test, the host says the model identified the threat and assigned a 76% probability to social engineering.
The host illustrates how multiple questions can be answered in parallel from one input.
In an image test, the model inferred emotional dynamics and possible motives from a single picture.
In an eight-second video test, the model interpreted a man with a candle in a frozen forest as likely participating in a ritual, with a low probability of danger.
The host says CLF led several listed benchmarks, while another model performed better on deciding when to escalate to a human.
In a screenshot of an Uzbek-language news site, the model identified the language, topic, tone, and a mention of the Ukraine conflict.
The final test involved reading a chemistry titration curve, where the host reports high-confidence answers across the requested categories.

Chapters

Start	Chapter	Summary
00:01	Introducing CLF	The host introduces CLF as a large multimodal decision model and explains how it differs from a text-generating chatbot.
03:08	Parallel decisions and image interpretation	The host demonstrates parallel answers to multiple questions and tests the model on an image-based relationship scenario.
05:48	Video test and benchmark comparison	The model interprets a short video, after which the host discusses its performance on a set of benchmarks.
07:24	Language and chemistry tests	The host tests a news screenshot in Uzbek and a chemistry titration diagram, then summarizes the model’s range of tasks.

Quotes

This is not a chatbot. It does not write a single token. — 00:32

It returns calibrated probabilities for every option in one single forward pass. — 01:05

No generation, no waiting, just instant parallel decision. — 03:39

I think for an 8-second clip with no dialogue and no context, that is a remarkably poetic and accurate read. — 06:20

handled all of it is what a truly multimodal decision model looks like at this stage in AI — 09:30

Concepts and entities

Clef, Cloudflare, decision model, multimodalidade, engenharia social, Mast Compute, Fahd Mirza

Full transcript (original language)

00:01 Decision models are everywhere. You lift a stone and you will find a decision model. Even Cloudflare has come up with their own decision model which is CLF. Before I tell you all about this decision model which is bit different than the rest of the lot look at these two videos where not only we have covered um these models separately also I have done a comprehensive comparison of Jev Kev open and heaps of others. So

00:32 if you’re interested in what exactly I’m talking about go search and watch these videos. So what has happened with this one? So look, Cloudflare has come up with this 27 billion parameter multimodel decision model. Correct? This is not a small one as we have been looking at others. This is not a chatbot. It does not write a single token. You give it a situation as text or image or video or JSON. Ask it typed questions and it returns calibrated

01:05 probabilities for every option in one single forward pass. So you can just consider everything we have tested with CLM, LIA, Jeff and Julia, but now with eyes. It sees images. It processes video frames and it still answers the same way. Fast structured decisions with no generation overhead. Apache 2 fully open source. Let’s run it. This is Fhat Miraza and I welcome you to the channel. I really want to give a huge shout out

01:35 to Mast Compute who are giving me this GPU to run this. So really thank you to them. If you’re looking to rent a GPU on very good price with a discount coupon, just check the video’s description. And you know what I have started this free weekly AI newsletter which you have to subscribe please. You can access it at faja.substack.com. Now let me quickly show this is the output of a test which I have done with this cleft model and this is the code.

02:06 So sorry where is the code? This is a code. Now we are running the same social engineering trap we used in our last two or three videos. Same employee as you can see that employee is saying that employer needs um emergency fake access unverifiable manager approval and then there are few options which we are giving to the model to see if model is able to detect it or not.

02:38 If you remember in our other videos, CLM and Jev caught it, but others like Kev, Lia, OpenJ fell for it. If you look at this result here, you see that KF caught it. 5.9% chance of granting access, 76% social engineering probability escalate to security. That puts it firmly in the same camp as CLM Jev. The bigger model, better security instinct. But you know

03:08 this is not the differentiator in my opinion. Differentiator is something else which is multimodality. Before I do the multimodal test, look at this diagram. On the left is your input a real support ticket saying the API is down and orders are blocked. Below that you define your question with their possible answers. Which team and is it urgent? Both go into the decision model together. On the right, the answer fire back in parallel. Technical wins at

03:39 100%, urgency is yes at 100%, two completely different questions, one single forward pass answered at the same time. That is what makes this category different from a chatbot. No generation, no waiting, just instant parallel decision. Now let me show you a multimodal test. Okay, so for the multimodality, let’s go with this situational test. We have a woman at Leonyi partners caught between two men. As you can see in these

04:12 pictures, her older boss who is infatuated with her, just wants to have fun casually and a younger colleague she has feelings for who is purely after her career gains. And in this script and prompt, we are asking CLF to read the image. understand the emotional dynamics and tell us who loves who, who is faking it, and how this whole thing ends for her. So, let me take you to the terminal. I’m going to run it. It is

04:43 going to load the model. While it loads, let me also show you the VRAMm consumption. So, you can see it is consuming this much VRAM with KV cache and stuff. So it is going to jump around this 27 billion parameter model fully loaded. So you see under 54 gig of VM and this is the answer returned by KF and you can already tell that KF has decided that she is 94.5%

05:15 stressed about work not the men and that neither man has her genuine attention at 71.9%. The boss is 92.7% not in love just using her. Uh, Matio, the subordinate, is only 47.7% money motivated, which is surprisingly generous, I would say. And the outcome is messy, but she survives. Honestly, not a bad read for a model looking at just one image in one shot.

05:48 Okay, next up, I’m going to do this video test. So, this is an AI generated video. By the way, all the images and videos I’m using, I generated it myself with AI. Now, if you look at it, this is an 8-second video of a bearded man alone in a frozen forest holding a lit candle at dusk. I’m going to ask Clev to watch it and tell us his emotion, whether he’s in danger, what the candle means, and how this story ends. And this is the response of KF. You can

06:20 see that KFF read it as a ritual at 72.2% confidence, memorial candle at 60.2% 2% and only 16% chance of danger. And the outcome is that he survives alone. I think for an 8-second clip with no dialogue and no context, that is a remarkably poetic and accurate read. If you have a quick look at this benchmark, you can see that cleft dominates almost every benchmark here and the numbers are not even close in most cases.

06:51 total uh or sorry tool retrieval, API banking, clinical classification, cleft leads them all, including Jeff, which has been the gold standard until now. The one place Jeff still win is when to call when where knowing when to escalate to a human matters most, which is interesting because that is arguably the most human judgment dependent task in the list. Lia is at the bottom across the board, which confirms what we saw in our showdown. Great for its size but a different weight class entirely. Okay,

07:24 let’s do another test. Next test is coming from Usbakistan. This is Kund. One of Usbakistan’s biggest news sites entirely in Usbuk ser. We asked KF in this uh prompt to go and pick up this you know Ukraine conflict mentioned at 97.8 8 something like that. It will do that. But first identify the language, understand the headline and tells us what topic are

07:55 being covered without any translation. This is image. I just screenshotted that news site. Full credit to them um for letting me use this. So this is the result by the way. So look at this. Cle identified USB at 95.9% confidence. So multilinguality is awesome. Picked up the Ukraine conflict as I said earlier 97.8%. 8% which is right there in the ticker correctly called the tone neutral at 93.1% and got the main topic as politics. The UN mentioned scored low at

08:28 39.6% which is a one miss since BMT in USB means United Nations but if you are from US Pakistan please also correct me. Overall a strong read of a language most models would completely fail on. And for the final test, I’m going to go with this chemistry titration curve showing a diaprotic acid being neutralized with NaOH. So I’m asking CLF to identify the diagram type the acid the count the

08:59 equivalence points and read the final pH value directly of the graph. And this is the answer. Perfect score across the board. Uh titration curve at 99.1% droic acid at 97.8 rate two equivalence point at 96.7 really really good so KF read a university level chemistry diagram with near perfect accuracy so text image video foreign language newspaper relationship drama security threats and chemistry diagram cleft

09:30 handled all of it is what a truly multimodal decision model looks like at this stage in AI let me know your thoughts if you want to help out the channel please become a member Just go to the homepage of channel and then choose this link or in the first pin comment and don’t forget to subscribe to this newsletter.