I Ran Nace.AI's Drex v1.5 Locally (llama.cpp, Q4_K_M, Real Demos)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I Ran Nace.AI’s Drex v1.5 Locally (llama.cpp, Q4_K_M, Real Demos) · Channel: Prompt Engineer 48 · 07:32 · Watch on YouTube

Topics: AI evaluation and benchmarks, Cybersecurity and software security, Model inference and optimization, Local and open-source models

Summary

The video tests Drex V1.5, a decision model that returns probabilities rather than generated text, running locally on an 8 GB laptop through a llama.cpp server. The presenter explains the model’s question types and reported benchmarks, then describes quantizing a Q8_0 file to Q4_K_M so it fits on the GPU. Demos cover support tickets, a phishing email, a product review, and questions about a longer document. The presenter notes that the long-document test was slowed by limited video memory and that the Q4_K_M results were not compared against the official versions.

Key points

Drex V1.5 is presented as a decision model that scores options with probabilities instead of generating text.
It supports choice, yes-or-no (null), and ordered-scale (score) questions.
The model card describes a Qwen 3.5 9 billion architecture and reports long-context accuracy results.
The original model files total about 17.9 GB, which would not fit on the presenter’s 8 GB GPU.
A Q8_0 GGUF file ran out of host memory with 26 GPU layers; using 10 layers loaded but led to paging and a 334-second first request.
Quantizing the file to Q4_K_M reduced it from 9.5 GB to 5.6 GB, allowing it to run entirely on the GPU.
The demos show probability scores for ticket routing and urgency, phishing detection, review sentiment, and document questions.
The long-document request took 113 seconds on the presenter’s machine because another process was using about 5 GB of video memory.
The presenter did not verify whether their Q4_K_M version matches the BF16 and Q8_0 versions’ answers.

Chapters

Start	Chapter	Summary
00:00	Introducing Drex V1.5	The presenter introduces Drex as a probability-based decision model and previews running it locally on an 8 GB laptop.
02:06	Building and fitting the local server	The presenter describes building the llama.cpp server and quantizing the model to Q4_K_M after the larger file proved impractical.
03:17	Decision-model demos	The video tests ticket classification and urgency, phishing detection, and product-review scoring.
04:55	Long-context test and benchmark context	The presenter tests questions about the model card, discusses the slow local run, and compares claims and reported benchmarks.
05:58	Caveats and wrap-up	The video covers benchmark and game-arena results, cautions that Q4_K_M was not validated against other versions, and summarizes the setup.

Quotes

It does not write a single word. — 00:00

Each question takes one forward pass. — 01:02

So, that is not usable. — 02:45

the whole point of a model that gives you numbers and not a sentence. — 04:55

I did not run that comparison, so I cannot tell you my Q4KM matches it everywhere. — 06:59

Concepts and entities

Drex V1.5, Nace.AI, Qwen3.5, llama.cpp, Q4_K_M, Q8_0, Hugging Face

Full transcript (original language)

00:00 Hi, I’m prompt engineer. In this video, we look at Drex V1.5, a decision model that answers with probabilities, and I run it on my own 8GB’s laptop. All right. In this video, we will see Drex V1.5, a 9 billion parameter model from Nays AI. And you can see that when I send it a support ticket, it comes back with billing at 0.986, and it did not write a single word. Now, mind you that everything here is running on my local system, this 8GB GPU GPU laptop that I have here through a llama server I built myself. Let’s go ahead

00:31 and test this out. So, this is the model page on Hugging Face, Nays AI/Drex V1.5. If you look here, it says it’s a decision model. You give it a state, which can be text or JSON, and typed questions, and it returns a probability for each option. It does not generate text. Each question takes one forward pass. There are three question types: choice, where you name the options, null, which is a yes or no and gives you a

01:02 probability of yes, and score, which is an ordered scale, lowest first. It’s built on the Qwen 3.5 9 billion architecture, 32 layers, BF16 weights, about 18 GB’s on disk. It has a small head.pt file, only 8 MB’s, and that head is what scores each option. The headline claim on the card is number one under 10 billion parameters on decision index 0.311 with a score of 58.08. There is a second table on the card for

01:32 long context. From 8,000 to 32,000 tokens, it reports 89.5% accuracy. And from 32,000 to 128,000 tokens, 93.4%. And if you truncate the same inputs to 8,000 tokens, those drop to 76.5 and 78%. So, the card is saying the long input is the thing that matters here. Now, if you open the files tab, you can see four safetensors shards, about 4.9, 5, 4.9, and 3 GB, and the total is about 17.9 GB. That will not fit on my 8 GB card.

02:06 There is also a GitHub fork nice AI/llama branch drex v1.5. The description says it adds a post/v1/system on endpoint, and it forwards that to a llama server from the drex v1.5 branch of their llama.cpp. And when you do a llama create, it detects the pointer tensors in the GGUF and marks the model as decision instead of completion. So, what I did is clone the drex v1.5 branch of their llama.cpp and build llama server with CUDA for my RTX 4060. That took several minutes of compiling.

02:45 Then I downloaded their Q8_0 GGUF, which is 9.5 GB, and here’s the honest part. I started it with 26 layers on the GPU, and it ran out of host memory. With 10 layers, it loaded, but my first request took 334 seconds because the laptop was paging. So, that is not usable. What you can do is quantize it yourself. I ran llama.cpp quantize with Q4KM. It took about 135 seconds, and the file came down from 9.5 to 5.6 GB. Now, the

03:17 whole thing sits on the GPU, and I start the server with embedding on, pooling none, and 16384 context. Okay, first demo. The ticket says, “I was charged twice for the same order. Please refund the extra payment.” Three questions: Which team does the customer ask for a refund, and how urgent? You can see that team is billing at 0.986, refund is 0.992, and urgency lands on urgent at 0.418 with soon at 0.305. So,

03:51 that one is not confident, and that is fine. A double charge is not an outage. Now, I change the ticket to checkout page returns a 500 error for every customer since 9:40. Team goes to technical at 0.958. Refund drops to 0.006, and urgency is urgent at 0.949. The same three questions, different state, and the numbers move the way you would expect. Next, I give it an email that says your account will be suspended in 24 hours. Click this link and enter

04:24 your password and OTP. Is it fishing? 0.9994. What should the mail filter do? Block at 0.987. Quarantine at 0.013. And this one answered in under a second. Then a product review. Battery last 2 days. Screen cracked after 1-ft drop. Support took 3 weeks. Sentiment on a 5-point scale comes out as 1.65 with mixed at 0.617. Mention support 0.996, and the main

04:55 complaint durability at 0.55 and support at 0.45. So, here it is split, and the probabilities show you that, which is the whole point of a model that gives you numbers and not a sentence. For a longer input, I passed the model card text itself, 4,389 tokens. Does the document say the HTTP server requires an authorization header by default? 0.045, which is correct. The card says no header is required. Does it say the

05:26 model generates free-form text? 0.002. Which base architecture? Qwen at 0.9999. Mind you, this one took 113 seconds on my machine because another process was holding about 5 GB of my video memory. On their side, they report 0.65 seconds median at 8K to 32K tokens on an A10G. So, do not read my number as the model’s number. Now, the claims the 58.08

05:58 comes from NACE’s own run of the official kit. Jeff 130 town 0 is at 57.296 and bespoke nimble is at 57.29 and the card itself says those three are inside the board’s 0.9 point tie band. On Jeff bench, Drex gets 86.2% and Jeff gets 87.0. The server binds to 127.0001 and has no authentication. So, add that before you expose it and the license is NACE AI open rail M for the model and

06:29 Apache 2.0 for the Kev code. There’s also a game arena chart on the card. Eight open spiel games, 32 games each against Jeff. It reports 122 wins, 47 draws, and 87 losses, which is 56.8% and the card says Drex beats Jeff on 20 of the 37 benchmarks. So, on 17 it does not. Also, the numbers I showed you came from a four-bit quantization that I made from their eight-bit file. The card says

06:59 their BF16 and Q80 versions gave the same answers as the Python server. I did not run that comparison, so I cannot tell you my Q4KM matches it everywhere. So, in this video we have seen how we can build the Drex V15 llama server, quantize the G GUF to 5.6 GB, and send it tickets, an email, and a review on an 8GB laptop. If you’re interested, you can go ahead and try this out on your own system as well and let me know how it goes. The model is NACE AI/Drex V155.

07:32 I’ll see you in another one. Check out the next video. The model page and the GitHub fork are linked in the description. Thanks for watching and I’ll see you in the next one.