GLiNER2.5-Decide: 340M Local Text Classifier on CPU (Tested vs Keywords)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: GLiNER2.5-Decide: 340M Local Text Classifier on CPU (Tested vs Keywords) · Channel: Prompt Engineer 48 · 05:44 · Watch on YouTube

Topics: AI evaluation and benchmarks, AI models, Local and open-source models, Privacy and data protection

Summary

The video tests a 340-million-parameter GLiNER2.5 Decide model for classifying support tickets locally on a CPU. On 30 hand-labeled tickets, it matches a keyword router for intent at 83%, while its priority and sentiment scores are lower; adding descriptions to labels improves the results. Multi-label mode catches some useful tags, but the model always selects from the provided labels and cannot explain its choices. The presenter sees it as useful for private routing and tagging, while recommending human or larger-model oversight.

Key points

The model classifies text against supplied labels in a single forward pass, without generating explanations or chat responses.
The presenter ran it locally on a laptop CPU, without an API.
On 30 self-written support tickets, intent accuracy was 83%, tying a keyword-based router.
Priority accuracy was 63% and sentiment accuracy was 73%; all three classifications were correct for 40% of tickets.
Adding short descriptions to labels raised intent accuracy to 93% when asked alone, and improved priority and sentiment scores.
With descriptions, all three classifications together were correct for 57% of tickets.
Multi-label mode returned multiple tags, including delivery and churn risk for one example.
The model must always choose one of the supplied labels, so the presenter emphasizes defining an appropriate ‘other’ label.
On the presenter’s laptop, three questions took about 620 ms, one took about 528 ms, and a batch of 30 took about 8.9 seconds.
The presenter recommends it for fixed-label routing and tagging, with a human or larger model in the loop.

Chapters

Start	Chapter	Summary
00:03	Model overview and benchmark	The presenter introduces a 340-million-parameter local classifier and questions whether its reported benchmark reflects real-world performance.
01:33	Testing support-ticket classification	The presenter compares the model with a keyword router on 30 tickets and reports the initial accuracy results and failure patterns.
02:36	Improving results with label descriptions	Adding descriptions to labels improves classification accuracy, especially for intent and for getting all three classifications right.
03:36	Multi-label mode in practice	The presenter tests multi-label tagging and shows examples where the model identifies billing, data loss, delivery, and churn risk.
04:40	Trade-offs, speed, and recommendations	The video explains the model’s fixed-label and privacy trade-offs, shares CPU timings, and recommends human oversight.

Quotes

You give it a text and a list of labels, and it picks the label — 00:33

the little model only tied with a bunch of if statements. — 02:05

the labels you write matter a lot — 03:36

It always has to pick one. — 04:40

I would keep a human or a bigger model in the loop. — 05:44

Concepts and entities

GLiNER2.5-Decide, Fastino, DeBERTa V3, classificação de texto, classificação multilabel, roteador baseado em palavras-chave, fine-tune

Full transcript (original language)

00:03 All right, in this video, we will see a tiny 340 million parameter model that reads a customer message and decides what to do with it, and you can see that it gives me the answer in one pass on a plain CPU. Now, mind you that everything here is running on my local laptop, no API, no tokens. Let’s go ahead and test this out and see if it is really as good as it sounds. So, this is Gleena 2.5 decide from Fast Tina. It’s a 340 million parameter model I bet 2.0 license built on a Debater V3

00:33 large encoder, and the idea is very simple. It does not chat, it does not reason, it does not explain anything. You give it a text and a list of labels, and it picks the label intent, routing, priority, sentiment, all in a single forward pass. Now, if you look here at the benchmark on their own suite of 17 domains with 200 examples each, it scores 60.2%. The bigger 1 billion version gets 59.6, so the small one is actually slightly ahead. Now, 60% is not a number that makes you jump out of your chair, so I

01:03 did not want to just trust this table. I wrote my own test. Setup is really simple. You do pip install Gleena 2, and then you load it with auto extractor.from pre-trained. That’s it. Now, one thing I did hit a few missing packages on my side, transformers and paft, and on Windows, you want to set UTF-8 mode, otherwise the banner crashes the console. Doesn’t matter, it’s a minute of fixing. And then you call classify text with the text and a dictionary of labels, and you can see that it says refund request first try.

01:33 So, here’s what I did. I wrote 30 support tickets myself with tricky ones in them. A person who got charged twice and rent is due today. A person who says the support is amazing, but the word refund is in the sentence. Some small talk that is not support at all. And I labeled every ticket myself, and I asked the model three questions at once, intent from eight labels, priority, and sentiment. For comparison, I also built the laziest router you can think of, plain keywords. And mind you, I wrote those keywords after seeing my own tickets, so that baseline is a bit

02:05 friendly to itself. All right. So, the results intent 25 out of 30, that is 83%, and the keyword router also got 83%. So, honestly, on my test, the little model only tied with a bunch of if statements. That is not what I was hoping for. Priority was 63%, and sentiment was 73. And if you need all three correct on the same ticket, it is only 12 out of 30, so 40%. So, for urgency, I would not trust it blindly. Now, if you look at the misses, there is a pattern. Do you have an office in

02:36 Berlin? Got sent to speak to a human. What’s your favorite color? Also went to speak to a human. The model has to pick something, and when the message does not fit, it forces a label. So, the other bucket is the weak spot. But here’s the tip. The model card says labels can carry a description. So, instead of a plain list, I pass a dictionary, the label, and a short sentence that says what it means. And for other, I wrote anything unrelated to support, small talk, compliments, general questions. Same model, same tickets, and now intent goes from 83 to

03:06 93%. 28 out of 30 when I ask it alone. Then I did the same for priority. Urgent means money, data loss, a deadline, or a lockout today. Low means a question or a minor annoyance. And priority goes from 63 to 73%. Sentiment moves a little from 73 to 77. Now, when I ask for all three together, intent drops slightly to 90%, but the full ticket, all three correct, goes from 40% to 57%. So, the labels you

03:36 write matter a lot, and that is really great because it costs nothing, no training, just a better sentence. Now, there is one more thing I wanted to try. Real tickets are not about one thing, so there is a multi-label mode. You pass the labels, you set multi-label to true, and a threshold. I use .4 and it returns every label above that. So, it made seven tags: billing, account access, delivery, software bug, legal threat, churn risk, and data loss. And let’s go ahead and see charged after canceling, refund me or I dispute with

04:09 my bank, it says billing. Good. The app deleted all my project files, it says data loss. Good, although it did not add software bug, which I would have. And this one is nice, package marked delivered but nothing at my door and I’m about to take my business elsewhere. It gives me delivery and churn risk. Two tags and a keyword router would never catch the churn risk. So, that is really amazing for a model this small. So, how is this different from just asking a big LLM to classify? Well, an LLM writes tokens, you write a prompt,

04:40 you parse the answer, and sometimes the answer comes back in a slightly different shape. Here, there is no prompt template, no generated text, you give the labels, and you get one of your labels back. Always one of your labels, and that is also the catch. It always has to pick one. That is why the other bucket and the descriptions matter so much. And since it is only 340 million parameters, you can run it on your own machine, so the ticket text never leaves your system. Mind you, it cannot explain why it chose something. So, if you need a reason, that is a job for a bigger model.

05:13 Now, for speed, on my laptop CPU, three questions at once take about 620 ms and a single question about 528. And if I send all 30 in a batch, it takes about 8.9 seconds. So, roughly 300 ms each. Fast INO says 167 ms on a 48-core server. And I believe it, my laptop just has fewer cores. It’s not instant on a laptop, but it is free, private, and the same answer every time. So, would I use it for routing and tagging where you have a fixed list of

05:44 labels and thousands of messages? Yes, especially with descriptions, and you can fine-tune it, too, for priority or anything with open-ended messages. I would keep a human or a bigger model in the loop. So, in this video, we have seen how a 340 million parameter model decides what to do with text, what it gets right, and where it fails. If you’re interested, you can go ahead and check this out and try this out on your own system as well, and let me know how it goes. The model is Gleaner 2.5 beside from Fast T know. I’ll see you in another one. Check out the next video.