Baidu's Unlimited OCR on an 8GB Laptop: 25 Tests

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Baidu’s Unlimited OCR on an 8GB Laptop: 25 Tests · Channel: Prompt Engineer 48 · 15:04 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, Local and open-source models, Multimodal models

Summary

The video evaluates Baidu’s Unlimited OCR using a 4-bit quantized model on an 8GB laptop, testing 25 custom document images and scoring the output against the text written into each image. It performs well on clean printed text, code, layouts, and several formatting challenges, but produces subtle errors in names, numbers, email addresses, receipts, equations, and some non-Latin scripts. Switching between Gundam and base resolution modes substantially changes results, sometimes improving them and sometimes making them worse. The presenter concludes that the model is useful for local document conversion, but consequential details need human verification and the results should be treated as a first look from one setup.

Key points

The tested AWQ 4-bit model file is 2.83GB, but runtime GPU memory peaked at 6.8GB because the library decompresses weights to 16-bit during inference.
The original 16-bit model crashed with a segmentation fault on the laptop; the 4-bit model also had CUDA out-of-memory errors on some first-run tests.
Scoring used Python’s SequenceMatcher to compare generated text with the exact text in each custom image; it was the creator’s own metric, not an official benchmark.
In the first round using Gundam mode, eight tests were exact, 15 scored 0.9 or higher, and the average score was 0.85.
Using the better result from Gundam or base mode for each image raised the average to 0.90, with nine exact tests and 16 scores of 0.9 or higher.
Strong results included clean printed text, code, rotated text, two-column reading order, release-note markdown, mixed fonts, and large signs.
Notable weaknesses included a receipt with spaced-out prices, equations, chart values, and several tested non-Latin scripts; Japanese and Russian performed better in the examples.
Base mode corrected the low-contrast test from 0.37 to 1.0 and improved some language tests, but worsened the presentation-slide score; the receipt was unchanged.
Errors can look convincing, including an invented surname, a missing order-number digit, and an incorrect email address, so names, numbers, and emails need checking.
The presenter emphasizes that this was one run on one laptop with one quantization and is not a definitive verdict on the model.

Chapters

Start	Chapter	Summary
00:00	Model, setup, and limitations	The presenter introduces the 25-image OCR test, explains the model’s claimed long-document approach, and describes quantization, memory use, and initial setup issues.
02:35	Scoring method and first test results	The video explains its text-comparison metric and inference setup, then tests printed text, invoices, receipts, code, tables, and multiple scripts.
07:15	Remaining image tests and initial scoreboard	Tests cover small and low-contrast text, rotation, blur, layouts, forms, charts, and symbols before the presenter compares resolution modes.
10:57	Comparing Gundam and base modes	The presenter reruns seven challenging images in base mode and shows that changing resolution can improve some results while worsening others.
13:01	Scoreboard, verdict, and caveats	The video summarizes the scores, identifies strengths and weaknesses, warns about plausible but incorrect details, and frames the test as a first look.

Quotes

The failures are the interesting part. — 00:00

It invented a surname. — 04:40

The whole failure in test 12 was the tiling, not the model, and not the four-bit weights. — 10:57

A wrong answer that looks right is worse than an error. — 14:03

treat it as a first look, not a verdict on Baidu’s model. — 15:04

Concepts and entities

Unlimited OCR, Baidu, Sahil Chakra, AWQ, Gundam, base mode, Hugging Face, GitHub, RunPod

Full transcript (original language)

00:00 25 documents, 18GB laptop. Eight of them came back letter perfect and one of them made up a person’s name. This is unlimited OCR from Baidu in a 4-bit version that fits in about 2.8GB. I did not read the benchmarks. I made 25 test images, ran every one of them on my own GPU and scored what came out. The good ones are really good. The failures are the interesting part. So, stay for those. First, where this comes from. This is the Hugging Face page for Baidu {slash} unlimited OCR. It is a vision language

00:32 model for document parsing, MIT license, and the page shows about 4,500 likes and over 1 million downloads. The code lives on GitHub and on the day I recorded this, it shows 27.2 thousand stars, 2.9 thousand forks, two contributors, and no tagged releases. So, this is a fresh research drop, not a mature product. Keep that in mind. And here’s a paper on arXiv called unlimited OCR works. The abstract says the problem with language model OCR is that the key value cache grows as the output gets

01:04 longer, so generation slows down. Their fix is called reference sliding window attention, which keeps the cache constant, and they say it can transcribe dozens of pages in a single pass under a 32,000 token limit. So, the claim is long documents without the slowdown. I’m not going to test dozens of pages today. I’m going to test the thing most of you actually care about. Does it read a normal image correctly on a normal laptop? To fit it on my card, I’m using a community quantized version by Sahil Chakra. This is the page. It is AWQ

01:35 4-bit weights with 16-bit activations, and the model file is 2.83GB. The original on disk is 6.3. Now, here’s a detail that matters if you have an 8GB card. When I loaded it, it took 12.9 seconds and used 2.6GB of video memory. But, after the first image, memory peaked at 6.8, the library decompresses the weights back to 16-bit as it runs. So, the file is small, the run time is not that showed up right away. My first full run on all 25 images

02:05 had seven of them die with CUDA out of memory on an 8-GB card. I cleared the cache between images and re-ran only those seven. Those are the numbers you will see. I also tried the original 16-bit model on this laptop and it crashed with a segmentation fault at load. I did not dig into why, so I’m not going to guess. One more honest thing, my very first attempt returned empty strings for six images and got stuck on a long generation for another. I changed four things at once, the prompt to the native document parsing one, the resolution

02:35 mode to Gundam, the no repeat n-gram setting from the read me, and a cap of 8,000 tokens on the output. After that, it worked. I cannot tell you which of the four did it because I did not isolate them. How I scored it, each image has the exact text I drew on it. After the model answers, I stripped the bounding box tags and compared the two strings character by character with Python sequence matcher. One means identical. This is my own simple metric, not an official benchmark. So, use it to compare tests against each other, not against the paper. Here is the whole thing in code because

03:07 it is shorter than you would think. Load the tokenizer and the model with trust remote code bfloat16 on CUDA, then one call model.infer with the tokenizer, a prompt, the image file, a base size of 1,024, and the image size and crop mode that choose between Gundam and base. With the eval flag on, it hands you the text back as a string. A word on speed because you’ll want a number. My per image times run from 3.4 seconds for the parking sign up to 62 seconds for the

03:37 dense page, and most are between 5 and 30. But, I was doing this with a browser and an editor open on a shared 8-GB card with the memory pressure I described. So, this is not a clean benchmark. It tells you a laptop can do it, not how fast a tuned server would. Quick note on prompts. The model card lists a small vocabulary, free OCR for plain text, document parsing for the native mode, convert the document to markdown with the grounding tag for layout, parse the figure for charts, and a locate prompt to find one phrase and get its box. In this video, I only used

04:10 document parsing and the markdown prompt on a table, the two columns, and the release notes. My very first one used free OCR on the first image, and it dropped a middle line, but I did not repeat that with the final settings, so I will not compare the prompts. That is a test for another video. Test one, a clean three-line paragraph, Arial 34 pixels, score 1.0, exact match. It took 18 seconds, but this was the first image, so that includes the one-time wait decompression. You can see the model returns the whole paragraph

04:40 with a bounding box around it. Test two, an invoice in a monospace font, score 0.97. Look at the second line. I wrote “Bill to Sahil Chakra”, and the model read “Sahil Achanta”. It invented a surname. It is a four-bit model, and I cannot say if the full one would do the same, but for an invoice, that is exactly the kind of mistake that costs money. Test three, a cafe receipt, 0.79, and this is the first real failure. Latte 450 is right, then the croissant line becomes c.450,

05:10 then c.5, and the totals are gone. Receipts with wide gaps between the item and the price are where it fell apart for me. Test four, a handwriting style font, 0.99. This is a handwriting font, not a real pen, so do not read this as handwriting recognition. It did read “Dear Sam”, the time, and “Priya”. It added a dash before “bring”, which I did not write. Test five, code in a dark editor, exact match 1.0. The indentation for the loop body and the hash comment are all there. For anyone pulling code out of screenshots, this one is

05:42 encouraging. One more thing to look at before the next block. This is the raw answer for the release notes page before I strip anything out. Every piece of text comes with a tag that with a label like title or text and four numbers, which are the box left, top, right, bottom on a grid. So, you do not only get the words, you get where they sit on the page. That is what lets you rebuild a layout or highlight a field and it is why I strip the tags only for scoring. Test six, a table with the markdown prompt style. 0.82, the structure is

06:13 right, pipes and four rows, but the header says param instead of params and the speed units come back with a capital T and the last row ends in t/u. Structure good, small characters wrong. Test seven, European languages one per line 0.56 and this is where I got worried. Hello world is fine. Bonjour le monde and hallo welt both come back as the single word environ. Hola mundo becomes hola m. The Russian line is perfect, the Greek is mostly wrong. Remember this one because I come back to it. Test eight, Chinese, Japanese,

06:45 Korean 0.73. The Japanese sentence is exact. In the Chinese line it dropped the first two characters and the Korean line is empty, so one out of three fully. Test nine, Hindi 0.64. It started right, namaste, then drifted. The second line came out twice with different spellings. If your documents are Indic scripts, test your own samples before you trust this build. Test 10, math 0.59. It gives me LaTeX, which is the correct format and the first equation e = mc squared is almost there. It has a straight two. The

07:15 quadratic formula keeps the plus minus, but writes the square root as separate letters as sqrt and a summation line is wrong. Do not use this for equations yet. Test 11, tiny text 16 pixels, five lines of the same sentence 0.91. But look closer, I drew five lines and it returned six. It added one. Small text, extra line. Test 12, low contrast gray on gray 0.37. The lowest score in the hard group, low contrast text came back as low contract text in capitals. And the last line ends in should still cowl.

07:48 It reads the shape and guesses the rest. Hold on to this one, too. Test 13, a sentence rotated 12°. 1.0. I expected this to fail. It did not. Exact match. Full sentence. Test 14, noise and blur. .99. Two lines, noisy, blurred with a Gaussian. One wrong letter. It wrote PCR where I wrote OCR. Funny, since it is the one word it should know. Test 15, two columns. 1.0. Left column first, then the right. The reading order is correct, and it gave me two separate

08:19 boxes. Layout is one of the strong points here. Test 16, a release notes page with a heading and a bullet list. Markdown prompt. 1.0. Heading, three bullets, and the email address, all correct. If your goal is markdown out of a screenshot, this is the happy path. Test 17, order numbers, a tracking code, a phone number, and an IBAN. .95. The IBAN, the phone, and the tracking number are exact, but the order number lost a digit. 88-1204-771. And there’s a line of random Chinese

08:50 characters at the top that I never drew. Verify numbers. Test 18, a blue parking sign. 1.0. Large white bold letters. No parking. Exact. Test 19, a presentation slide. .94. Three bullets. Exact. The title, Why local models matter, came out as YIN models MA. The big yellow title on a dark background was the part that broke, and the small body text was fine. Test 20, a form. .86. Name, date of birth, and phone are correct. The city, Pune,

09:22 came back as two Chinese characters, and the signature line became the words no text in brackets. I put an empty signature line, so that is arguably honest. Test 21, a dense page. 14 repeats of the same sentence at 26-pixels, 0.79 and 62 seconds, the slowest of the run. Because the same sentence repeats, the repeat guard may also be fighting the text itself. I would not use my score here as proof of anything except that long outputs are slower. Test 22, mixed fonts in one line, bold,

09:54 italic, mono, serif, comic, 1.0, every word in order. Test 23, a bar chart with the parse the figure idea, 0.4. It found a title and it drew a chart box around the bars, but it did not read the values 122 10 170 300. So, it detects the chart. It does not extract the data with the document parsing prompt. Anyway, I did not test the other chart prompt in this run. Test 24, white text over a color gradient, 1.0. Test 25, symbols, euro sign, percent,

10:26 degrees, and email, 0.9. The euro price, the percent, and the temperature range with the degree sign are all exact. Email came back as I I k a n at prompt at e.com. Right shape, wrong address. Same lesson as the invoice. Names and emails, check them. Before I give you the scoreboard, I want to go back to the worst ones because something bothered me. Every one of these 25 images is small, around 1,000 pixels wide. The mode I used, Gundam, is designed for large pages and it crops the image into tiles. So, I took the

10:57 seven worst or most suspicious images and ran them again in base mode. Same model, same prompt, only the resolution setting changed. No tiling, one pass at 1,024. Low contrast text, this was the 0.37 in round one. In base mode, the answer is low contrast text is hard to read, but the model should still get it. Exact match, 1.0. The whole failure in test 12 was the tiling, not the model, and not the four-bit weights. European languages, 0.56 goes to 0.85.

11:28 Bonjour le monde is back and hallo welt is back. Hola mundo is now hola mono, so it is still wrong by one letter and the Greek line is still wrong. It comes back in Cyrillic letters something like Geza Ukozi. So, it did not read it as Greek at all. Big jump, still not clean. Hindi, 0.64 goes to 0.82. Fewer duplicate lines and the second line is now one line, not two, but the words are still not right. Namaste Dunathi instead of Namaste Duniya, better not fixed. Now, the opposite. The presentation

11:58 slide was 0.94 in Gundam. In base mode, it drops to 0.8. The title is now right, but look at the second bullet, cost no per token bill becomes cost colon and then a LaTeX underline and the word not twice. It invented math formatting inside a bullet. So, base mode is not strictly better. It fixed some images and broke others. The receipt, no change at all, 0.79 in both. The same C.450 and C.5 lines. So, the resolution was not the problem there. I think it is the wide gap

12:29 between name and price, but that is a guess. I did not test it. And math, 0.59 against 0.58, nothing changed. And the chart, in base mode, the score goes up to 0.62. But look at what it returns, the title and then a few empty lines. It still does not read the values. The score went up only because it stopped writing the wrong markup. It still cannot read the bars with this prompt. So, here is the practical lesson. Resolution mode changed the answer on six of seven images and not always in the same direction. For small images, try base mode first. For big

13:01 pages, Gundam. And if the answer matters, run both and compare them. I did not test that on any big pages, so that last part is a rule of thumb from the readme, not from my data. So, here is the scoreboard, all 25. In round one, with Gundam mode, eight were exact, 15 scored 0.9 or better, and the average was 0.85. After round two, if I take the better of the two modes for each image, nine are exact, 16 are 0.9 or better, and the average goes up to 0.90. The bottom of the list is math, the chart, the Chinese line, the receipt, the dense page, the table, and

13:33 Hindi. Let me put the four findings in one place before the verdict. One, clean printed text, code, and layout came back exact or close to it. Two, the resolution mode is a real setting that can move a score from 0.37 to 1.0, so it is not a detail. Three, the errors are quiet, a wrong surname, a missing digit, a wrong email. Four, an 8 GB card works, but only just because the four-bit weights unpack to 16-bit when it runs. What it is good at from my data, clean

14:03 printed text, code, rotated text, two-column layout, big signs and slides, markdown pages, mixed fonts, and low contrast. Once I use base mode, what it is bad at, receipts with wide gaps, equations, chart values, and the non-Latin scripts I tried other than Japanese and Russian. The failures also have a pattern. It almost never returns garbage. It returns confident, well-formed text that is slightly wrong, a fake surname, a missing digit, an email at e.com. A wrong answer that looks right is worse than an error. So,

14:33 anything with names, numbers, or emails needs a human check or a second pass. Should you use it? If you want to turn clean screenshots, code, slides, and normal documents into markdown on your own machine, yes, this is worth an afternoon. If you need exact receipts, Indic scripts, or math, run your own samples first in both modes, and maybe try the 16-bit model on a bigger card. I could not on this laptop. Everything in this video is real output from my own run from files on my disk. The test

15:04 images, the model answers, and the scores are all generated by the scripts, and I did not edit the model output. This was one run, one laptop, one model quant, so treat it as a first look, not a verdict on Baidu’s model. If you’re looking for cheap and reliable GPUs, I use RunPod, and the link is in the description. That is it. Subscribe because next I want to run the multi-page mode on a real PDF and see if the constant memory claim holds. And tell me in the comments what document I should break it with. See you in the next one.