LEANN: 201 GB vs 6 GB?! I Installed This INSANE Vector DB
Read the Portuguese translation →
Author: Prompt Engineer 48 · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: LEANN: 201 GB vs 6 GB?! I Installed This INSANE Vector DB · Channel: Prompt Engineer 48 · 03:36 · Watch on YouTube
Topics: Claude Code, Data, databases and embeddings, Local and open-source models, Privacy and data protection, RAG, search and information retrieval
Summary
Section titled “Summary”The video introduces LEANN, a vector-search system that aims to reduce storage by keeping a pruned graph and recomputing embeddings for only a few nodes during search. The creator installs it on a Windows laptop and tests it with Pride and Prejudice, finding the small book’s graph file is 82% smaller than storing all embeddings, while searches on CPU take around 48 seconds. Offline search works after the internet drops, but tests with a 4-billion-parameter model produce incorrect answers. The creator emphasizes that this is a small, three-question test rather than a benchmark, and says LEANN may suit people prioritizing disk space and privacy.
Key points
Section titled “Key points”- The README claims LEANN needs 6 GB for 60 million text chunks, compared with 201 GB for a normal vector database; the creator did not test that scale.
- LEANN stores a pruned graph of similar chunks and recomputes embeddings for a small number of nodes during search.
- The storage savings come with added computation at search time.
- Installation took 4 minutes 48 seconds, mostly because of PyTorch; the first build crashed because the Windows console could not display emoji.
- For a 1,776-chunk Pride and Prejudice index, the graph file was 482,000 bytes, 82% smaller than storing all embeddings for that book.
- The whole folder was 5.3 MB, including the text and a keyword index.
- A CPU search took 48 seconds, and a warm-up attempt did not speed things up on Windows.
- Search and answering continued offline after the internet dropped; the creator says the data stayed on the laptop.
- In three questions, retrieval often found useful passages, but the 4-billion-parameter model gave incorrect answers.
- The creator considers LEANN worth exploring for disk savings and privacy, while cautioning that the test was not a benchmark.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | The storage problem and LEANN’s approach | The video contrasts the claimed storage needs of a normal vector database and LEANN, then explains LEANN’s graph-based approach and its search-time computation trade-off. |
| 00:31 | Installation and the first index | The creator describes installing LEANN, fixing a Windows console error, and building an index of Pride and Prejudice. |
| 01:31 | Storage and search results | The small book’s graph file is 82% smaller than storing its embeddings, while a CPU search takes 48 seconds. |
| 02:33 | Testing answers with a small model | The creator runs three questions with a 4-billion-parameter model and finds that it makes mistakes despite retrieval finding useful passages. |
| 03:05 | Verdict and caveats | The creator recommends considering LEANN for disk savings and privacy but stresses the test was limited and notes the README’s untested recall claim. |
Quotes
Section titled “Quotes”And I have not tested that scale, but I did install it on my laptop today. — 00:00
You save disk, but you pay compute at search time. — 01:01
A single novel can’t show that so I won’t pretend it does. — 02:02
That’s three questions, one book, not a benchmark. — 03:05
Concepts and entities
Section titled “Concepts and entities”LEANN, RAG, embedding, BM25, UV, PyTorch, Pride and Prejudice, Qwen, Ollama, Claude Code
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 60 million text chunks. A normal vector database needs 201 GB for that. Lean says six. That’s the readme’s table, not mine. And I have not tested that scale, but I did install it on my laptop today. And some of it surprised me. [music] Here’s the story. You have years of notes, PDFs, chats, code. You want to ask questions over them. So, you try Rag. Step one, turn every chunk into an embedding, a list of hundreds of numbers. Step two, store all of them.
00:31 That second step is the problem. The vectors are often bigger than the text itself. The readme’s own table shows a normal database at 3.8 GB for a data set where lean needs 324 megabytes. So you have two choices. Fill your drive or send your private data to a cloud service and pay per query. Neither is great if the data is your email. So the real question is, do we need to store every embedding at all? Lean’s answer from the paper is no. It keeps a graph that connects similar chunks, prunes that graph hard, and when you search, it only computes embeddings for the few
01:01 nodes on the search path. Think of a map without photos of every street. You walk the map and take a photo only where you’re going. That’s selective recomputation. The trade is obvious. You save disk, but you pay compute at search time. Keep that in mind because it shows up in my test. Install it. Two commands with UV. It took 4 minutes 48 seconds on my machine, mostly PyTorch. Then my first build crashed with a unic code error. Lean prints emoji and my Windows console can’t show them. The fix is one setting t is t
01:31 you equals one. After that it built pride and prejudice 770 kilobyt 1776 chunks 3 minutes on the CPU. Now the storage claim the graph file is 482,000 bytes storing every embedding the normal way. 1776 chunks times 384 numbers would be about 2.7 megabytes. So on this small book, the graph is 82% smaller, not 97 and the whole folder is 5.3 megabytes because it also keeps the text and a keyboard index. The 97% is for huge
02:02 collections. A single novel can’t show that so I won’t pretend it does. Search who runs away with Wickham. It returns the passage about Lydia and Wickcham being found in London. That’s the right place. But look at the clock. 48 seconds and the others took up to 76. This is on CPU each command code. I tried the warm-up demon and it did not speed things up on Windows. That’s the price of recomputing. Then something nice. My internet dropped midsession and hugging face couldn’t resolve. I set offline mode and it all kept working. Embedding search even the answer. Nothing left my laptop. Now ask with a 4 billion
02:33 parameter Quen running in. Two more snacks. My Olama host has no HTTP in front. So I passed the host explicitly and the warm-up left a stale Python process holding the port. So I killed it. Then the answers. First question, why does Elizabeth refuse Darcy? Retrievo missed the proposal and a model set no proposal exists. Wrong. Second, retrieval did find Darcy’s real line, but the small model decided she accepts him. Third, it found the sentence about Collins switching from Jane to Elizabeth, then still misread it. So in my test, search found useful passages most of the time, and the 4 billion
03:05 model was the weak link. That’s three questions, one book, not a benchmark. So is it worth it if your problem is disk and privacy? Yes. a small index fully offline MIT license. The readme also shows it plugged into claude code reporting 2.1 times better early recall than BM25 on 30 tasks. I haven’t tested that. If you need fast answers today, expect about a minute per search on a CPU laptop and use a bigger model than mine. The repo link is in the description. If you want me to test a cloud code setup next or the email index, tell me in the comments. And if
03:36 you like videos where I install things, show you the crashes and the real numbers, subscribe to my channel, Prompt Engineer. See you in the next one.