I Ran Liquid's New Decision Model on an 8GB Laptop. Here's What Happened.

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I Ran Liquid’s New Decision Model on an 8GB Laptop. Here’s What Happened. · Channel: Prompt Engineer 48 · 09:24 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Local and open-source models, Multimodal models, AI safety

Summary

The video tests Liquid AI’s D13B, a 3.12-billion-parameter decision model, on an 8GB RTX 4060 laptop. D13B answers named yes/no, choice, and score questions in one pass without generating text, with three-question latency of 70–83 milliseconds depending on quantization. It performs well in several tests, including image questions, command-risk checks, judging a factually incorrect answer, and customer messages in multiple languages. However, it misses sarcastic refund intent, and the presenter cautions that the model is not a chat model or a standalone safety layer.

Key points

D13B is a decision model, not a chat model: it takes a state and named questions and returns probabilities or scores without generating text.
Its question types are null (yes/no), choice (select among options), and score (estimate among ordered levels).
The presenter ran it locally using llama.cpp release B115552; older llama.cpp versions may not support the required endpoint.
On the Q8 model, three questions took a median of 83 milliseconds across 60 warm calls; one question took a median of 23 milliseconds.
The vision test correctly identified two cats on a sofa, taking 533 milliseconds.
In a small shell-command test, it identified destructive or risky commands, but the presenter warns against relying on it as the only safety layer.
It correctly judged an answer of Lyon to the question about France’s capital as wrong.
It handled tested Spanish, Hindi, Japanese, and German messages without translating them, but interpreted a sarcastic refund request literally.
Q4_K_M produced the same team choices in the tested text messages, reduced three-question latency to 70 milliseconds, and used 3.3 GB of VRAM versus 4.5 GB for Q8.
The presenter notes that the benchmark figures are Liquid AI’s internal results and that the model’s knowledge score is its weakest column on the cited table.

Chapters

Start	Chapter	Summary
00:01	What D13B does	The presenter introduces the local test and explains D13B’s decision-question formats and intended uses.
02:06	Running the model and testing customer triage	The video covers the llama.cpp setup, then tests refund intent, team routing, urgency, and repeated-call latency.
04:11	Vision, guardrails, judging, and languages	D13B is tested on an image, shell commands, a factuality judgment, and customer messages in several languages.
06:48	Failure case, concurrency, and Q4 results	The presenter demonstrates a sarcasm miss, measures concurrent and long-input performance, and compares Q4 with Q8.
08:23	Benchmark context and final takeaways	The presenter puts the benchmark figures in context, notes the model’s limitations and license, and summarizes the laptop tests.

Quotes

It is not a chat model and it does not write text. — 00:32

The median is 83 milliseconds on Q8 — 03:41

I’m not claiming it catches everything — 05:13

So, it took the sentence literally. — 06:48

If you wanted to write or explain, this is the wrong tool. — 08:53

Concepts and entities

D13B, Liquid AI, LFM 2.5 VL3B, llama.cpp, GGUF, Q8, Q4_K_M, LFM 1.0, Decision Index 0.2.1

Full transcript (original language)

00:01 All right, in this video we are going to test D13B from Liquid AI and you can see right here that I sent it one customer message and three questions and it answered all three in one pass. Look at this line, output tokens zero. [music] It did not write anything, refund yes, 0.99, team billing 0.98, urgency [music] score all in about 80 milliseconds. Now mind you, everything you’re seeing is running on my local system, [music] this AGB’s RTX 4060 laptop. So, let’s go ahead and see what this thing really is and where it breaks. So, here’s the

00:32 model card. D13B is a decision model, 3.12 [music] billion parameters post trained from LFM 2.5 VL3B, which means it also takes images. Now, this is important and the card says [music] it itself. It is not a chat model and it does not write text. What you do is you give it a state that can be a string, any JSON, [music] or an image and you give it named questions. There are three types, null, which is a yes or no, and it gives you the probability of yes, [music] choice, where you describe the options and it gives you the pick plus the probability for every option, and

01:04 score, where you give two to 10 ordered levels [music] and it gives you an expected level with a confidence. And all of that comes out of one forward pass. So, there’s no JSON to parse out of a paragraph, no retry because it rambled, and no waiting for tokens to stream. Now, why would you want this? If you look at the speed table on the card, one question on an RTX 4090 is 8 milliseconds. That one is with compile [music] turned on. On a Jetson Orin Nano, it is 50 milliseconds. On an Apple M5 Pro, it is 30. So, this is the kind of thing you put in front [music] of

01:34 every request, a router, a moderation check, a guardrail, a judge, because it costs you almost nothing. And the card lists [music] use cases like routing and triage, intent classification, reranking, LLM as a judge, scoring, [music] and agent guardrails. We are going to test a few of those ourselves on my laptop, not on [music] their table. So, for running it locally, there is a GGUF repo and here are the files. [music] Q8 is 2.87 GB, Q4 KM is 1.67 GB, and there is a separate MM proj file for

02:06 vision. 583 MB at Q8. There are also BF16 [music] and F16 files at 5.4 GB if you have the room. Now, one thing [music] to note, this does not work with just any llama.cpp. The system one endpoint was merged into llama.cpp in pull request [music] 29818 and I used release B115552, the [music] Windows CUDA 12.4 build. If your llama.cpp is older, update it first. The command is simple. llama server -m with the Q8 file, –mmproj

02:37 [music] with the mm proj file, ngl 99 and it loads. You can see in the log it says decision [music] model type LFM2-D1 and it is listening on port 8080. Let’s go ahead and send the first one. The message is, “I was [music] charged twice this month. Please refund one of them.” Three questions, is the customer asking for a refund? That is a null. Which team should handle this? Billing, technical, or fraud? That is a choice. [music] And how urgent is this? Three levels, can wait today, blocking the customer now. And you can see the answer. Refund

03:11 0.99 team billing with 0.984 and technical is 0.01. Fraud is half a percent. [music] Now urgency, the score is 0.76 on a scale of 0 to 2. And if you look at the probabilities, it is spread [music] out. 34% can wait, 56% today, 10% blocking. And the confidence for that one is only 0.34. >> [music] >> So it tells me it is unsure and I can write a rule if confidence is under >> [music] >> 0.5, send it to a human usage. Says 154

03:41 input [music] tokens, zero output tokens. Now that one call can be a lucky one. [music] So I wrote a small Python script and did 60 warm calls rotating over five different customer messages with three questions in one pass. The median is 83 milliseconds [music] on Q8 and the slowest of the 60, the 95th percentile is 87. [music] With one question, the median is 23 milliseconds. So, compare that to the card, a 4090 [music] with three questions is 21 milliseconds. My laptop is about four times slower [music] than that and it is still not

04:11 even a tenth of a second. For a 3 billion parameter model on an 8GB laptop GPU, I’m happy with that. Next, the image. This is the same photo from the readme, two cats on a sofa. State is [music] just a line of text saying it is a photo for a pet sitting request and the image goes in as base 64. [music] Questions, which animals, cats, dogs, or birds? Are they on a sofa? And how many, one, two, or three, or more? Cats, 0.989, [music] sofa, 0.963,

04:41 two, 0.979. The count [music] is right. It took 533 milliseconds with 831 input tokens and again, zero output tokens. Slower than text [music] as expected because the image has to go through the vision encoder first. Now, this one I really wanted to try [music] because it is the agent guardrail use case. I give it a shell command as the state and ask [music] two questions. Would this destroy data and how risky is it? Harmless, needs review, [music] or dangerous? Last layer on my projects

05:13 folder, harmless, 0.968, >> [music] >> rm -rf / with no preserve root, dangerous, 0.978, and git push force origin main, dangerous, 0.965, and destroy data [music] at 0.86. Each of these came back in 60 to 90 milliseconds. So, you can put this in front of an agent’s tool calls and block anything where the danger probability crosses >> [music] >> a line you pick. Mind you, this is four commands. I’m not claiming it catches everything, so please do not use this as [music] your only safety layer. One

05:45 more, LLM as a judge. The state is a JSON object question, what is the capital of France? Answer, Lyon, because it is the biggest [music] city. And I ask, is the answer factually correct? It says 0.058, and on the quality score it puts 95% on the level called wrong. That is the correct call, and it took 82 [music] milliseconds. The card lists 16 languages, so let me test that. Same three questions, same script, but now the customer writes in Spanish, [music] they charged me twice this month, refund 0.989, team billing 0.983, then the same

06:17 sentence in Hindi, and you can see the Devanagari text right here. Refund 0.987, billing 0.982, [music] Japanese refund 0.987, billing 0.982, Japanese refund 0.987, billing 0.984, and a German message that says the app crashes every time [music] I open settings. Refund is 0.026, team >> [music] >> So, I did not translate anything, and I did not change a single question. The question stayed in English, and the customer writes in whatever [music] language they want. Now, let me try to

06:48 break it. This is a message with sarcasm. Oh, great, another free trial that charges my card. Thanks for nothing, totally not asking for a [music] refund, I would never. A person reads that and knows this customer is angry about a charge, and the model [music] says refund 0.02. So, it took the sentence literally. It does get the team [music] right, billing. But, if your pipeline depends on refund intent, sarcasm like this will slip [music] through. So, that is something you want to test with your own messages before you trust it. Next, load Lama server [music] started with four slots. So, I

07:20 sent 64 requests with four workers at the same time. That finished in 5.57 seconds, [music] which is 11.5 requests per second with three questions each. The median per request goes up to 178 milliseconds because they’re sharing the GPU. And then a long input, I built a fake [music] chat log of 3,040 tokens with a refund request at the very end. It found [music] it, refund 0.98 in 511 milliseconds. The card says 102 milliseconds for a 3.4 thousand [music] token state on a 4090. So, my laptop is about five times slower there and it is

07:51 still half a second. Now, the Q4 model, I ran [music] the exact same script on Q4KM. All five text messages gave the same team choice as Q8 and the refund probabilities are 0.99 and 0.991 on the first one. The cats came back as [music] cats and two three question latency drops from 83 to 70 milliseconds and the VRAM used goes [music] from 4.5 gigabytes to 3.3. That includes the vision projector and four slots >> [music] >> at 32K context. So, if you’re short on memory, Q4KM looks like the sensible

08:23 pick. Now, let me be straight about the numbers on [music] the decision index 0.2.1 table, D13B scores 48.57. Win now 12B, which is four times the size, scores [music] 50.02. So, it is not number one on that table. It is ahead of the 36 billion Decider, [music] which is 47.11 and the 9.7 billion JPT at 46.89. So, what you can say is it is the best under 10 billion on this index [music] as the card says. And these are Liquid’s own internal benchmarks. They scored it with

08:53 the official scorer, but it is [music] not a leaderboard submission and my five messages, four commands and one sarcastic message are [music] not a benchmark either. The knowledge score is 23.8, which is the weakest column on the table. And again, it is [music] not a chat model. If you wanted to write or explain, this is the wrong tool. License is LFM 1.0. [music] So, read it before you ship anything. So, in this video, we have seen how D13B answers [music] typed questions in a single pass with zero output tokens, 70 to 83 [music] milliseconds for three questions on an

09:24 8GB laptop. Correct on the cats, correct on the guardrail, correct in Hindi and Japanese and wrong on the sarcasm test, which I showed you, too. If you’re interested, you can go ahead and try this on [music] your own system. The links are in the description and let me know what you wrote with it. Credit [music] to Liquid AI for the model. I’ll see you in another one. Check out the next video.