Ollama's New Decision Models: tev1 + nimble (Tested on 8 GB)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Ollama’s New Decision Models: tev1 + nimble (Tested on 8 GB) · Channel: Prompt Engineer 48 · 08:03 · Watch on YouTube

Topics: AI agents, Task and workflow automation, AI evaluation and benchmarks, Local and open-source models

Summary

The video introduces Ollama’s System 1 decision models, Tev 1 and Nimble, which classify or score input text and return probabilities rather than conversational responses. The presenter demonstrates installing and calling the models through Ollama’s /v1/system1 endpoint, then uses them in a Gradio app for ticket triage, moderation, routing, and grading. In a 30-ticket test, Tev 1 4B achieved perfect team classification, while Nimble performed best on detecting whether a customer was angry but was slower and exceeded the presenter’s 8 GB GPU capacity. A conventional Qwen model with a JSON schema matched Nimble’s test accuracy and was faster, but the decision models provide probabilities and more consistent confidence signals; the presenter cautions against relying on them alone for high-stakes decisions.

Key points

System 1 models take text as a state and return answers with probabilities; question types include choice, null (true/false), and score.
A single call can ask up to 64 questions about the same text.
The models require Ollama 0.35 or later and use the /v1/system1 endpoint on port 11434.
The presenter demonstrates ticket triage, moderation, model routing, and answer grading in a Gradio app.
On the presenter’s 30-ticket test, Tev 1 0.8B scored 86.7% on team classification and 76.7% on detecting anger; Tev 1 4B scored 100% and 83.3%, respectively.
Nimble scored 96.7% on both test tasks but took about 1.5 seconds per call and partly ran on the CPU because it did not fit the 8 GB GPU.
A Qwen 3.5 4B model forced to use a JSON schema scored 96.7% on both tasks in 268 milliseconds, matching Nimble’s accuracy and beating it on speed.
The presenter highlights probabilities as a way to flag uncertain predictions for human review, and says the decision models can offer more consistent confidence information than chat models.
The request format is strict, Nimble may not fit smaller GPUs, and Tev 1 is experimental; the models should not be the sole check for high-stakes decisions.

Chapters

Start	Chapter	Summary
00:00	Introducing Tev 1 and Nimble	The presenter introduces Ollama’s decision models, explains their System 1 approach, and describes the available model sizes and setup requirements.
02:14	Calling the System 1 endpoint	The presenter tests the API, fixes the required question fields, and demonstrates choice and score results with probabilities.
03:19	Examples in a Gradio app	The models are used for ticket triage, moderation, routing requests to different model sizes, and grading answers.
04:57	Benchmark results and GPU limits	The presenter compares the models on 30 support tickets, reports speed and accuracy, and notes Nimble’s memory demands on an 8 GB GPU.
06:33	Comparison, strengths, and cautions	A JSON-constrained Qwen model is compared with the decision models, followed by a discussion of probability signals, consistency, and limitations.

Quotes

These are not chat models. They don’t talk. They decide. — 00:00

You give it some text, they call that the state, and you give it questions and you get back an answer with a probability. — 00:34

That is why this is fast. — 02:46

So, on accuracy and speed, it matched Nimble and beat it on speed. — 06:33

So, do not let it be the only check on a high-stakes decision. — 07:33

Concepts and entities

Ollama, tev1, Nimble, TypeSafe, Together AI, Bespoke Labs, System 1 models, Qwen3.5, Gradio

Full transcript (original language)

00:00 All right. In this video, we will see two brand new models on Ollama, Tev 1 and Nimble. And you can see here that I type a support ticket and it tells me the team and how sure it is in about half a second. Now, mind you, these are not chat models. They don’t talk. They decide. And everything here is running on my local system on this 8 GB GPU laptop that I have here. If you watched my videos where I trained my own Jeff style model, this is the same idea and now Ollama supports it. Let’s go ahead and test this out. So, what is this? Ollama put out a blog post that says it now supports Jeff style decision models.

00:34 These come from a company called Type-Safe and they call them System 1 models. The idea is really simple. You give it some text, they call that the state, and you give it questions and you get back an answer with a probability. No paragraph, no JSON that you have to hope is valid. There are three kinds of question choice, where you pick from a list, null, which is basically true or false, and score, where you place it on a rubric. And you can ask up to 64 questions about the same text in one call. Now, there are two models. Nimble is 9 billion parameters from Bespoke Labs and it is Apache 2.0. Tev 1 is from

01:08 Together AI and it comes in a 4 billion and a 0.8 billion size. Both are fine-tuned from Qwen 3.5. They say Nimble gets a 75.7% on their public benchmark and Tev 1 4B gets 73.3. Those are their numbers, not mine. In a minute, we will run our own. First, if you do not have Ollama, go to ollama.com, click download and install it for your system. Really simple, on Windows, it is one installer. Now, this is one extra step that you need to do. These models need Ollama 0.35 or later, so open a terminal and say ollama {dash}

01:42 {dash} version. Mine says 0.35.0, so we are right at the minimum. Now, go ahead and pull the models. I say ollama pull tev1, then ollama pull nimble, and ollama pull tev1:0.8b. I already have them, so this finishes in a second on your side. Nimble is a 9.5 GB file. So, it will take some time on your connection. And if I say all on my list, you can see all three here. Tev one is 4.5 GB. The small one is 811 MB. And Nimble is 9.5 GB. Remember that

02:14 number because my card has only 8 GB. Now, let’s make our first call. It is not the chat endpoint. It is /v1/system1 on port 11434. I send a model, a state, and questions. So, what I have done is I have written a question called team, type choice, and I ask which team should handle this, and I hit enter, and we get an error. It says instructions must be a non-empty string. So, every question needs an instructions field. Fine, let’s add it. Run again. Now, it says choice criteria must map option keys to descriptions. So, for a

02:46 choice, I need to give a criteria map, billing, payments, and refunds. Tech, bugs, and outages. Sales, pricing, and upgrades. Go ahead and run it again. And now a score question. I added and it says score criteria must be an array of descriptions. So, for a score criteria is a list, not a map. Third time is the charm. Now, if you look here, we get billing at 0.9975, the customer is angry at 0.87, and the urgency at 0.65. And the model wrote only four tokens. That is why this is fast. In Python, it is the same. I use

03:19 requests, I post the model, the state, and the questions, and I read the answers. Six lines. I will put this file in the description. Now, I have put this into one small Gradio app with four tabs, so you can try it. First tab is ticket triage. I type, “My card was charged twice, and I want a refund now.” Click the side, and you can see it says billing at 0.99. The bars here are the probabilities. Tech and sales are basically zero. Now, let’s try the API one. The API returns a 500 error, production is down, and And goes to

03:50 tech. Good. And now a hard one. Do you have a discount for nonprofits? Tev1 says sales really good. It is not matching the word billing. It is reading what I mean. Second tab is moderation. Three yes or no questions on the same message. Spam, personal data, abusive, a scam message with a card number. Spam is 0.97, personal data 0.95, and I block anything above 0.8. Now, here’s one where I want to be honest. Hi, I’m John. Call me on a phone number. Personal data is 0.98, which is right. But, Tev1 gave spam 0.66 for a friendly message. Nimble

04:27 gave 0.49. That is why I set the threshold at 0.8 and not at 0.5. Third tab is a router. A cheap model or an expensive one. What is the capital of Australia? Small model. A fault-tolerant event sourcing architecture. Big model. 0.85. This is where I would use it in my own apps because you save money before the expensive call even happens. Last tab is a grader. A clean answer about photosynthesis gets 2.03 out of three on clarity. And the messy one with the IDK

04:57 in it gets 0.81. And correctness drops, too, from 0.84 to 0.44. Now, let me switch to Nimble. Same ticket, same tab. It is slower. And it also says sales at 0.89. We will see why in the numbers. Now, the part I really wanted to see, I wrote 30 support tickets myself, each with a correct team and a correct angry or not. This is my small test, not a public benchmark. So, treat it as a good start. Tev1 0.8b, the tiny one, 86.7%

05:30 on the team, 76.7 on angry, 235 milliseconds, and it takes about 900 megabytes of my GPU. Tev1 4b, 100% on the team, 83.3 on angry, 470 milliseconds, and 4.7 gigabytes all on the GPU. Nimble 96.7 on the team and 96.7 on angry. So, it is best at reading the mood, but it is 1.5 seconds per call. And if you look here at Ollama PS, it says 40% CPU, 60% GPU. That 10 GB

06:02 does not fit my 8 GB card. So, part of it runs on the CPU. So, my take on an 8 GB laptop TV 14B is the one to pull. If you have 12 GB or more, Nimble is worth it. And the 0.8B is for when memory is really tight. Also, one gotcha. If Tev 1 is already loaded and you call Nimble, I got a 500 error out of memory on the host side. Once Tev 1 unloaded, the same call worked. So, on a small card, Ollama stopped the first one. Now, you might say, “Why not just ask a normal small

06:33 model for JSON?” So, I tried it. I pulled Qwen 3.54B, gave it the same 30 tickets, and forced a JSON schema. And I have to be honest with you, it got 96.7 on the team and 96.7 on angry in 268 milliseconds. So, on accuracy and speed, it matched Nimble and beat it on speed. So, what does the decision model give you? Two things. First, the probability Tev 14B made no mistakes on the team, but the 0.8B did make four, and every one of them had a probability below

07:03 0.77. Nimble’s one miss was at 0.83. So, you can say, “If it is under 0.9, send it to a human.” Second, consistency. I sent one ticket to the chat model eight times. Seven times it said billing, once it said tech, and the chat model tells you nothing about how sure it was. That is a difference. So, the good things. Local, free, no API key, answers in a few hundred milliseconds, you get probabilities you can act on, and you can batch 64 questions in one call. The

07:33 not-so-good things. The request shape is picky. If you guess it like I did, you will hit those three errors, though the messages do tell you what is missing. Nimble does not fit an 8 GB card. The TEV 1 models are marked experimental and the official page says it can be wrong. So, do not let it be the only check on a high-stakes decision. So, in this video we have seen how to install Ollama, pull TEV 1 and Nimble, make a call to the system 1 endpoint and use them in an app. If you’re interested, you can go ahead and try this out on your own

08:03 system as well and let me know how it goes. In the next video, I will run my own trained TEV model on the same 30 tickets against Nimble and TEV 1. I will see you in another one. Check out the next video. Thank you.