Julia: 144M Parameters, Picks Answers in 44 ms on a CPU (vs Jev)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Julia: 144M Parameters, Picks Answers in 44 ms on a CPU (vs Jev) · Channel: Prompt Engineer 48 · 05:12 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, AI models, Local and open-source models

Summary

Julia 1 is a 144.3-million-parameter decision model that selects among options rather than generating sentences, and it can run locally on a CPU. The video compares its benchmark results with reference figures for Jev, then demonstrates Julia’s choice, score, and yes-or-no question types, multilingual inputs, and fast responses. Tests also reveal that its answers can change with question wording and that it can make confident mistakes or force a choice when the input is unsuitable. The presenter suggests using it for structured decisions such as prioritizing support tickets, while recommending that users test it on their own questions before relying on it.

Key points

Julia 1 comes from Supersonic Labs, is built on mm bird small, and has 144.3 million parameters.
The model is available as a roughly 550 MB download under the Apache 2.0 license.
It takes a state, a question, and 2 to 20 possible answers, then returns an answer with probabilities.
The video compares Julia’s model-card results with supplied Jev references, while noting the limitations of those comparisons.
Julia supports choice, ordered-scale scoring, and yes-or-no questions, and the demo includes English, Portuguese, and Hindi.
On the presenter’s laptop CPU, 100 short calls took 4.4 seconds in total, or about 44 milliseconds per call.
Question wording can substantially change results; the model also produced a confidently incorrect assessment of a customer complaint.
The presenter suggests decision tasks such as flagging urgency or checking answers against a rubric, and advises testing before trusting the model.

Chapters

Start	Chapter	Summary
00:00	What Julia is and how it compares with Jev	The video introduces Julia as a compact decision model and outlines its inputs, access model, and benchmark comparisons with Jev.
01:34	Question types and local CPU demo	The presenter explains Julia’s choice, score, and yes-or-no tasks, then demonstrates its setup, speed, and multilingual behavior.
03:07	Where Julia fails	Examples show that wording can change a result and that Julia can make confident mistakes or be unsuitable for open-ended inputs.
04:10	GGUF availability and use cases	The presenter mentions an untested GGUF route and suggests using Julia for structured decisions that might otherwise require keyword rules.
04:41	Practical takeaway	Julia and Jev are contrasted as downloadable and hosted options, and viewers are advised to test the model on their own tasks.

Quotes

it picks the right answer in about 44 milliseconds. — 00:00

It’s not a chat model. It only compares the answers that you hand it. — 00:31

about 44 milliseconds each on a CPU no GPU. — 02:05

Same model, same text, different wording. — 03:07

Either way, test it on your own questions and options before you trust it. — 04:41

Concepts and entities

Julia 1, Supersonic Labs, Jev, TypeSafe, mm bird small, Hugging Face, GGML, llama.cpp, Apache License 2.0

Full transcript (original language)

00:00 All right. In this video, we will see Julia, a decision model with just 144 million parameters. You can see that I’m running it here on my laptop CPU and it picks the right answer in about 44 milliseconds. Now, mind you, it cannot write a single sentence. Let’s go ahead and see why that’s actually the point. So, Julia comes from Supersonic Labs and Julia 1 is the first model in the Julia family. It’s built on mm bird small, which is a multilingual encoder and the whole thing is 144.3 million parameters. The checkpoint is 550 megabytes and the

00:31 license is Apache 2.0. Here’s the idea. You give it three things, a state, a question, and two to 20 possible answers. And it gives you back one answer with a probability for every option. It’s not a chat model. It only compares the answers that you hand it. Now, this idea is not new. And this is where Jev comes in. Jev is the first system one model from a company called Typesafe. Their line is unstructured state in typed probabilistic decisions out. They say a response takes 70 to 500 milliseconds and input costs 4.2 cents

01:03 per million tokens. Output is free. The difference is access Jev is an early access behind an API and Julia as a download. Different companies same shape of problem and Julia’s own model card puts Jev numbers right next to its own so we can compare them. So let’s look at the card on their type decisions test. Julia gets 73.15% and the Jev reference is 72.7. on AG news with four labels 94 against 91. On the emotion set 86 against 48 but on banking 77 Julia gets

01:34 64 and Jeff is at 87. Now mind you the Jeff numbers are supplied references not a fresh Jeff run. The classification tests are just 100 examples each and banking 77 uses a short list not a native 72 option call. So this is a good start not a guarantee. Julia answers three kinds of question. Choice picks one option from a list score gives you a number on an ordered scale and null is the yes or no one with a probability that it’s yes. Getting it is simple. We download the repo from hugging face

02:05 which is about 550 megbytes. PIP install it and call load model on my machine. It takes about 20 seconds to load and after that it stays in memory. Let’s go ahead and ask it something. The state is I was charged twice for the same order. The question is which team should handle this billing, shipping or access? And you can see it says billing with 85%. Now the speed I ran 100 calls in a row. That’s 4.4 seconds in total about 44 milliseconds each on a CPU no GPU. Mind you these are short inputs so a long

02:36 state will take longer. It’s also multilingual. I wrote the same complaint in Portuguese and it says billing over 99%. Then in Hindi I wrote that my package has not arrived and it says shipping again over 99%. You can also ask several questions about one state in a single call here. The customer says they were charged twice, nobody’s answering, and they want a refund today. One call, three answers. The team is billing. Urgent is true at 99.8% and the anger score comes out around 1.1, which is annoyed. Now, let’s see where it

03:07 breaks because it does. The state, our production database is down and customers can’t check out. I asked, is this an emergency needing immediate human attention? And it says no. 84% false. That’s wrong. So what I did is I rewrote the question and gave true and false real descriptions. Should this page the on call engineer right now, not urgent or urgent outage? And now it says true with basically 100%. Same model, same text, different wording. The second one I could not fix. A customer says the food was cold and the waiter ignored

03:38 them for 40 minutes. On a fivepoint scale, it said neutral 99.97% with better wording. It still said neutral 76%. and as a choice between negative neutral and positive neutral at 100. So that’s a real miss. A few more limits. If you ask it something unrelated like just hello, it still has to pick one and it said billing at 70%, give it one option or 21 and it throws an error because it only takes 2 to 20 and math is not what it’s for. I asked 17 * 23 and it got it right at 59%. But

04:10 the model card says that ability is not established. Now there’s also a GVUF version from GGML ORG. It comes as two files Q8 at 168 MGB and BF16 at 303 mgabytes. The card says to run it with llama serve and the model name. And it needs the new /v1/system on endpoint in llama.cpp. I haven’t run this path yet. So go ahead and try it and let me know how it goes. So where would you use something like this? Smart if statements wrote a support ticket. flag something

04:41 as urgent, score a reply, check if an answer fits a rubric, things where you’d normally write a pile of keyword rules and keep fixing them. Jev is the same bet at API scale. Julia is the one you can download, read the code off, and run on your own CPU. Jev is the hosted one in early access. Either way, test it on your own questions and options before you trust it. So, in this video, we have seen Julia A1 144 million parameter decision model and how it lines up against Jev. If you’re interested, you can go ahead and try this on your own

05:12 system as well and let me know how it goes. The model is Julia 1 from Supersonic Labs. I’ll see you in another one. Check out the next video.