Saluki 27B: a 54 GB model squeezed into 7.89 GB (88 vs 84 on tool calling)
Read the Portuguese translation →
Author: Prompt Engineer 48 · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: Saluki 27B: a 54 GB model squeezed into 7.89 GB (88 vs 84 on tool calling) · Channel: Prompt Engineer 48 · 09:23 · Watch on YouTube
Topics: AI agents, AI evaluation and benchmarks, Local and open-source models, Privacy and data protection
Summary
Section titled “Summary”The video reviews Underdog’s Saluki, a 27-billion-parameter model compressed from roughly 54 GB to 7.89 GB, and surveys the company’s app, privacy claims, benchmarks, Hugging Face model card, and limitations. The presenter highlights Saluki’s results on a 120-task tool-calling test, where it scored 88 compared with 84 for the full-size model, while stressing that the test is modest and the result does not prove Saluki is smarter. The video also covers areas where Saluki falls short, including competition math, letter-level puzzles, and formatting errors in some parallel tool-call replies. The presenter notes a discrepancy between the website and model card about image support and advises checking which file and settings are being used.
Key points
Section titled “Key points”- Saluki is a 27B model compressed to 7.89 GB, which the video says is small enough for a 16 GB laptop.
- Underdog presents its app as local and private, with features for email, calendars, meeting notes, dictation, and errands; the presenter says they did not install or test it.
- The website lists privacy and security checks, distinguishing requirements marked as passed from assessments still underway.
- Saluki scored 88 of 120 on Underdog’s tool-calling test, compared with 84 for the full-size model; the company cautions that the test is modest and run-to-run variation may explain part of the difference.
- On coding tasks, Saluki fixed 30 of 50 GitHub issues, compared with 33 for the full model; on Web Walker Q&A, it answered 60 of 150 questions.
- The model card lists an Apache 2.0 license, GGUF files, Llama.cpp support, and quick-start instructions that the presenter reads but does not run.
- For fast tool calls, the model card recommends turning thinking off and setting temperature to zero; general use settings keep thinking on.
- Saluki scores lower than the full model on AIME math benchmarks and MUSR, and is described as weakest on letter-level puzzles.
- The video notes a discrepancy about image support: the website says the release does not take images, while the model card describes an optional vision add-on.
- The presenter warns that some auto-generated setup snippets specify f16, which does not match the 7.89 GB file, so users should check which file they download.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Saluki and the Underdog app | The presenter introduces Saluki’s compressed size and tool-calling result, then reviews Underdog’s app, its advertised privacy features, and the status of its security checks. |
| 03:04 | Underdog’s models and Saluki benchmark results | The video compares Underdog’s three models and walks through Saluki’s tool-use, coding, browsing, and instruction-following benchmark results, including the company’s caveats. |
| 06:44 | Hugging Face model card and setup | The presenter reviews the model card’s files, license, download and server commands, recommended settings, and supported local apps. |
| 08:20 | Limitations and closing notes | The video covers Saluki’s weaker math and puzzle performance, formatting issues, the image-support discrepancy, and a final recap. |
Quotes
Section titled “Quotes”a model 1/7 the size scored higher on tool calling. — 00:00
Good to know is honest, built for agents, not math Olympiads. — 04:39
they don’t claim it’s smarter than the original. — 05:42
For fast tool calls, turn thinking off and set temperature to zero. — 07:48
So, check the card for what you actually download. — 08:51
Concepts and entities
Section titled “Concepts and entities”Saluki 27B, Underdog, Wolf 4B, Underdog 27B, llama.cpp, Hugging Face, Berkeley function calling leaderboard, SWE-bench Verified, Apache License 2.0, IQ2_Mix.GGUF
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 All right. In this video, we will see Saluki, a 27 billion parameter model that underdog squeezed from about 54 GB down to 7.89 GB, and you can see that right here on their page. Now, mind you, that file is small enough for a 16 GB laptop, and it still solved 88 out of 122 calling tasks, while the full 54 GB model it came from solved 84. So, a model 1/7 the size scored higher on tool calling. I want to go through everything on their website, the underdog homepage,
00:31 the Saluki blog post, and the Hugging Face model card, including the parts where they say it falls short. Let’s go ahead and look at it. First, who is underdog? This is underdog.ai, and the headline is the most loyal AI. It’s an app that gives you your own AI for email, calendar, meeting notes, dictation, and errands. And the line under it says, “100% local AI, 100% private.” If you look here, there are six items. Number one is sensitive tasks, doctor’s appointments, prescription refills, tax paperwork. The idea is that it finds what you need and gets the next step ready for your
01:02 approval. Then we have errands, email, calendar, meeting notes, and dictation. And on the right, you can see a chat demo. Someone asks it to schedule a checkup with Dr. Park, and it searches the mail for the clinic, checks the calendar, and drafts a request. Down here, the download section says Mac and Windows, both marked beta, Linux, iPhone, and Android are on the way. And at the top, it says invite-only beta, so you need an invite link to get in right now. Mind you, I have not installed the app for this video. Everything I’m telling you about it is what their page
01:32 says. So, I’ll point at the page each time. This part of the page is the reason they call it loyal. There’s a block titled “Loyal Means Loyal,” and it lists specific things. The model runs on your computer, so your content isn’t sent to a cloud AI service. Mail syncs directly between your computer and your email provider. Saved memories stay on your machine, and you can see, edit, or delete them in settings. It strips remote images, scripts, and tracking pixels on site. Every account gets its own vault, and the key never leaves your computer. And there’s a switch called
02:03 naval mode one switch, and underdog never calls home. Now, a privacy page is easy to write, so look at the next block who has checked Leviathan Security Group, the app defense alliance assessment, all 20 requirements passed, Google approved underdog to read your Gmail and calendar. Then two marked underway, a team that hacked OpenAI trying to break it every day, and Jane Manchun Wong taking the Mac app apart. Passed means passed. Underway means they haven’t finished, and I’m going to say it exactly like that. Scrolling up, there’s a backer strip, and recent
02:34 Horowitz, Hustle Fund, Hummingbird, Anthology Fund in partnership with Anthropic, SV Angel, and Breakneck, plus angels including Patrick Collison, Naval Ravikant, Guillermo Rauch, and Noam Brown. They say this first round was raised in April 2026. There’s a wall of love with posts from people. Ben Horowitz says he always loves an underdog. Guillermo Rauch says every human and every company must own their intelligence stack. And Thomas Wolf from Hugging Face says most personal AI should run on your device. Then the
03:04 manifesto card own your AI. Don’t let it own you with a note that the creator Sigil is building it for himself. And the team section says a tiny team that comes from Apple, Meta, Amazon, Baidu, Google Brain, Ramp, Mr. Beast, Teal Fellowship, and Meta Superintelligence Labs. They’re hiring. Now, the models. The page introduces three. Wolf 4B, the fast one, 115 tokens a second, which they say is 21 times faster than you can read. Soluki 27B, the light one, 96% of
03:36 the full 27B at a seventh the size, 7.89 GB, fits a 16 GB laptop. That’s the one this video is about. And Underdog 27B, the smart one. They show a score of 33.7 against 31.9 for Claude Opus, 4.6 max at 28 tokens a second. There’s a fine print line under it that I want you to see. Speeds are estimates, the scores come from the artificial analysis intelligence index. Underdog 27B is built on trend 3.8 27B and the scores
04:08 are trend scores. And Saluki’s 96% is its average across nine benchmarks against the full-size trend 3.8 27B. Now, the blog post dated 7 October 2026 they start with a claim from artificial analysis that trend 3.8 27B beats Claude Opus 4.6 max, the frontier model from 6 months ago. They squeezed it into 7.89 GB that runs on standard Llama CPP and they call the result Saluki. It’s open
04:39 source and they say you’re welcome to use it even without citing them. They also call it a preview checkpoint with full training to be published soon. The best for list agents that use tools, local assistance that follow instructions closely and private offline work. And good to know is honest, built for agents, not math Olympiads. It likes to think things through, so for snappy tool calls you turn thinking off and it’s text in text out. Here’s the headline, Underdog Bench 120 tasks from the Berkeley function calling leaderboard, 24 in each of five kinds of
05:10 tool use, picked and frozen on 27 September before any of these models were tested. And you can see the bars, their other build called main 91 Saluki 88, Mia 2-bit 85, the full-size trend 84 ISTA 2-bit 76, ternary bonsai 2 70. The table has one more column, how many of the full models 84 solved tasks each one still solves. Saluki 76, that’s 90%, main 77, bonsai 68, Mia and ISTA 63. And look at the breakdown by task type on
05:42 calling the one right function, Saluki got 24 out of 24 on picking the right function from several 23. That’s 47 of 48 against 31 for Mia. Now, they themselves say they don’t claim it’s smarter than the original. 120 tasks is a modest test, and part of the lead is likely run-to-run variation. Next, several tool calls at once like checking the weather in Paris, London, and Tokyo on 100 tasks. Strict scoring, Saluki 42, the full model 35, Bonsai 11. With forgiving scoring for small formatting
06:13 slips, Saluki goes up to 55. And they said plainly Mia is the clear leader here. 69 strict and 70 forgiving with a bigger 9.1 GB file. I like that they put that in the table. Then the coding test, 50 real GitHub issues from SWE-Bench verified. Saluki fixed 30, the full model 33, and Saluki still fixed 27 of the 33 the full model fixed. Web browsing, Web Walker Q&A, 150 questions. Saluki answered 60, that’s 40%. Bonsai
06:44 55, and their own main build 48, which they say is one reason they’re launching Saluki. And on IFEVAL, 541 prompts, Saluki scores 93.5 loose and 90.9 strict against 91.5 public for the full model, and they count six unfinished prompts as failures. Now, the Hugging Face page, Conway Research {slash} Underdog Saluki 27 B1.0, 250 likes, about 59,000 downloads last month. License Apache 2.0 and tags for GGUF. Llama CPP, 2-bit,
07:18 tool calling, and agents. Under files and versions, you can see the main file IQ2 mix.GGUF, 7.9 GB, and two vision files, 928 MB and 629 MB. The quick start is two commands, and I’m reading them off the card. I have not run them for this video. First, Hugging Face CLI download with the file name, then llama.server {dash} m the file {dash} {dash} tinja {dash} ngl –faon, –c32768.
07:48 That ginger flag turns on the chat template that handles tool calls and thinking, and the server speaks the OpenAI chat API on port 8080. For settings, general use is thinking on, temperature 0.6, top p 0.95, top k 20. For fast tool calls, turn thinking off and set temperature to zero. The card also lists local apps, llama.cpp, LM Studio, Jan vLLM, oobabooga on slot desktop, and PI, and the view all local apps link has more, like Lemonade,
08:20 Hermes Agent, and Open Claw. One thing to watch, those auto-generated snippets say colon f16, which doesn’t match the 7.89 GB file. So, check which file they pull. Now, the limitations, because the card lists them on competition math, AIME 2025, it scores 79.2 against 96.7 public for the full model, and 80.0 against 94.6 on AIME 2026. That’s 82 to 85% of the full score. MUSR is 67.5 against 79.6.
08:51 It’s weakest on letter-level puzzles, like palindromes, about a fifth of parallel call replies have small formatting slips, and with thinking on, it can reason at length before it answers. One more thing, and this is a small mismatch between their own pages. The block says this release doesn’t take images, the Hugging Face card says text by default and images with an optional vision add-on, those MM proje files. So, check the card for what you actually download. So, that’s Saluki, 7.89 GB, 88 out of 120 on tool calling, Apache 2.0,
09:23 and runs on stock llama.cpp, thanks to and the trend team, which Underdog credits on both pages. If you’re interested, go ahead and try it on your own system, and let me know how it goes. I’ll see you in another one. Check out the next video.