d1-omni-600M Locally: A Decision Model for Audio, Vision, and Text

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: d1-omni-600M Locally: A Decision Model for Audio, Vision, and Text · Channel: Fahd Mirza · 08:05 · Watch on YouTube

Topics: AI evaluation and benchmarks, Local and open-source models, Multimodal models

Summary

The video reviews Liquid AI’s D1 Omni, a roughly 587-million-parameter decision model that handles text, images, and audio and returns answer probabilities rather than generated text. In tests, it performed well on a customer-support text example and correctly answered two visual questions, though its judgment of the main source of stress was uncertain. The audio tests produced mixed evidence: the model classified clips in 13 languages, but the presenter questioned whether it understood their meaning; a simple English spoken question worked better. The model is described as small enough for edge hardware, using under 2 GB of VRAM in the presenter’s test, and its benchmark results are discussed with a call for comparisons against other models.

Key points

D1 Omni is a 587-million-parameter model built on Liquid AI’s LFM 2.5 encoder.
It combines a SigLip 2 vision tower, a 17-layer fast conformer for audio, and a shared trunk for modalities.
Unlike a generative model, it scores answer options with probabilities instead of writing responses.
In a customer-support text test, it correctly identified a refund request, the billing team, and high urgency.
In the image test, it correctly detected two phones and distress, but identified work as the main stressor with only 51% confidence.
The model used less than 2 GB of VRAM when loaded on the presenter’s GPU.
Across 13 short speech clips in different languages, it returned classifications, but the presenter questioned whether it captured their semantics.
A simple English spoken question was classified as a request for information.
The presenter says the benchmark results look strong but would like to see comparisons with other models.

Chapters

Start	Chapter	Summary
00:01	Introducing D1 Omni	The presenter introduces Liquid AI’s multimodal decision model, outlines its architecture, and describes the setup for testing it locally.
02:12	Text and vision tests	The model handles a customer-support text example well and answers visual questions accurately, although one image judgment is uncertain.
04:22	Architecture and multilingual audio tests	The presenter explains how the model combines inputs and scores options, then tests audio clips in 13 languages and discusses the limits of the results.
07:03	English audio check and benchmarks	A simple English speech example is classified as a request for information before the presenter comments on benchmarks and model size.

Quotes

One pass, no generated text, and all three answers are correct for this message. — 02:43

So I don’t think so you should be reading especially with in that answer too much. — 03:49

So the model never really writes a sentence it just tells you how likely each answer is all at once. — 05:27

So, I think, um, the model tells you how the speech is delivered, not what it means. — 06:31

But, anyway, with just 600 meg, not a bad size at all. — 08:05

Concepts and entities

D1 Omni 600M, decision model, Liquid AI, LFM 2.5, SigLip 2, fast conformer, Mask Compute, Fahad Mirza

Full transcript (original language)

00:01 I am really sick and tired of all these decision models raining like cats and dogs every day and they are more similar than different. And I guess they are easy to produce because everyone is sort of producing these models every hour of the day. But this model D1 Omni 600 million, which is yet another decision model, comes up with audio as a modality too and that is the reason why I am covering this model in this video. This is Fahad

00:31 Mirza and I welcome you to the channel. And if you want to see the proof why I am fed up of these decision models, just go and search for decision on my channel and you would see that we have covered heaps of them in the last two two two and a half weeks. We have compared them and we have done hands-on videos on every model out there. So, what exactly is this new model from Liquid AI? Well, this is a 587 million parameter model built on Liquid

01:06 AI’s own LFM 2.5 encoder. It has a SigLip 2 vision tower for images and a 17-layer fast conformer for audio with every modality running through the same shared trunk. I will also talk a bit more around this architecture later, but this model is small enough to run on edge hardware and it answers with probabilities, not generated text like any other model. So, same decision model idea we have covered in the previous days. So, now it

01:37 has eyes and they say it also has got ears. So, audio is the most promising thing which I’m looking forward to to see if it actually works or not. This is Fahad Mirza and I welcome you to the channel. Please also subscribe to this free weekly AI newsletter at fahadmirza.substack.com. I’m going to use this Ubuntu system. I have one GPU card, but you can easily run it on your CPU, too. If you’re looking to rent a CPU or VM or GPU on very good price, you can find the link to Mask Compute in video’s description.

02:12 So, what exactly I’m going to do first? I’m going to run a test with text. So, while it runs and loads, let me show you the script. So, look at this one. What we are testing here is one customer message that I was charged twice and nobody answers the phone. With three questions asked together, as you can see here, is a customer asking for a refund, which team should handle it, and how urgent is it?

02:43 If I go back here, you can see that a refund requested is at the top, billing at second, and urgency at around 1.90 out of two, which is blocking the customer now. One pass, no generated text, and all three answers are correct for this message. So, text modality is pretty good here. But, we will also stretch it later down the road. But, for the next one, let me quickly test out the vision. So, for the vision test, I’m going to use this AI-generated image where we are

03:17 showing this D1 Omni, the Leoni partner image, a woman stressed at her desk holding two phones and asking three questions. What’s stressing her most, as you can see in this script? Is she holding two phones? And does she look overwhelmed? And these are all the questions. Now, look at the result here. It confirmed she’s holding two phones at 99.7% and looks distressed at 99.9% both correct from the image. The primary

03:49 stress answer came back as work at only 51% which is close to a coin flip I would say. So I don’t think so you should be reading especially with in that answer too much. Now let me run it again and while it runs let me show you the VRAM consumption if it there is any. I will just let it run. And you see so just around still under 2 gig. So under 2 gig VRAM it consumes the when the model is fully loaded on to the GPU.

04:22 Okay, so that’s done. Let’s now do the audio test. Right before that audio test I just want to quickly show you one architecture diagram this one. Let me explain it in very simple words very quickly. So this diagram which shows the whole idea in one picture. On the left you feed in three things the state which means your text and image or an audio clip clip just what we are doing. The question type and the options you want the model to pick from. A tokenizer turns the text into small

04:54 pieces the model can read and the image or audio goes through its own encoder which converts the picture or sound into the same kind of pieces. Everything that lands in one long row and passes through a single shared encoder with 16 layers where every piece can look at every other piece at once. That is why the model can compare your questions against the evidence in the state in one pass. On the right hand side the decision had reached the result and gives you a probability for each answer. For a yes no question or now you get

05:27 something like 0.93. For a choice each option gets its own score and then so on and so forth. So the model never really writes a sentence it just tells you how likely each answer is all at once. Now let’s do one more test here. So for now of audio test what I have done, I have fed the model 13 short speech clips in 13 languages, from English, Spanish to Yoruba, Persian, Russian, and then, you know, all the languages from across the world. This is

06:00 a script, as you can see. Now, I’m not going to play the clips, all of these, but what I’m going to do, I’m going to show you the output where there is transcription, too. So, each clip is read aloud news or encyclopedia text, and the model had to pick from information, action, complaint, or statement. Now, look at the result, and please also check out your language. All 13 languages have returned an answer with no errors. 12 were labeled information

06:31 with confidence, highest on Portuguese and Spanish at over 99%, and lowest on Persian at 65%. The only odd one out was Czech, labeled action at just 51%, which is I would again say a coin flip, really. So, I think, um, the model tells you how the speech is delivered, not what it means. So, it really misses the semantics there. So, I don’t think so this model really understands the content of any clip. All it does is just covers the language. So,

07:03 I’m not really still sure about, um, the language side of things of this model. Especially when it comes to the audio. So, just to align with model’s training a bit more, I ran one more check on a short spoken question from, um, English, because the data set which I used earlier, maybe that was too hard for this. And you can see that the question is simple, that what does speaker want, and then is it information, action,

07:34 complaint, or statement. When I ran that, and it is only English, you can see the model at least can read a spoken question as a request for information because the script just says what is happiness. That’s it. Now, before I wrap up, I just wanted to touch upon this benchmark. I mean, I really don’t see any serious contender here unless I’m missing something. Like Decider, I haven’t even covered this model. So, no J F nothing else. So, on

08:05 all of these benchmarks, this model has performed really well. But, I think it would be good to put it against CLM or J F to see how exactly this thing performed. But, anyway, with just 600 meg, not a bad size at all. Let me know what do you think and they also have another one in 3 billion. But, text and vision, no audio there if you’re interested. Let me know what do you think about this model. Again, if you want to help out the channel, please go to the home page, become a member or in the first pinned comment. Thank you for all the support.