This Tiny 0.8B AI Reads Documents Better Than OCR?

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: This Tiny 0.8B AI Reads Documents Better Than OCR? · Channel: Prompt Engineer 48 · 05:53 · Watch on YouTube

Topics: AI evaluation and benchmarks, Local and open-source models, Multimodal models, RAG, search and information retrieval

Summary

The video tests LightOn OCR 3, a small vision-language model that converts document images into Markdown and HTML tables, against Tesseract on five document types. On the creator’s 8 GB RTX 4060 laptop, the 0.8B model correctly preserves information and structure in examples where Tesseract loses layout or misreads content. However, processing is extremely slow on this setup, so the creator says it is not practical for laptop use and could not verify the much faster speed reported for an H100 with vLLM. The video recommends it for structured document extraction when suitable GPU infrastructure is available, while Tesseract remains a better choice for fast, simple text extraction from clean scans on a CPU.

Key points

LightOn OCR 3 is a vision-language model that outputs page content as Markdown, including tables as HTML; a grounding prompt can return labels and bounding boxes.
The model card supports only two prompts: an empty prompt or grounding; the video stresses that this is a document reader, not a chat model.
The video focuses on the 0.8B model and compares it with Tesseract 5.4 using five generated document images.
On the handwritten-style note, LightOn OCR read details such as “6 eggs” and “6 p.m.” correctly, while Tesseract misread the number six as an at sign.
For the table, invoice, and chart, LightOn OCR preserved or reconstructed useful structure that Tesseract returned as flat text or incomplete content.
On the research paper, Tesseract read the text in order, while LightOn OCR represented the equation as LaTeX and headings as Markdown.
The model took 1.71 GB of VRAM to load on the creator’s 8 GB RTX 4060 laptop, but inference times ranged from 19 seconds to about 1,960 seconds per page.
The creator could not reproduce the reported 4.78 pages per second, which was measured on an H100 with vLLM, and could not test the 4B model on the laptop.
The video concludes that LightOn OCR 3 may suit structured extraction on capable GPU infrastructure, while Tesseract is preferable for fast, simple CPU-based extraction from clean scans.

Chapters

Start	Chapter	Summary
00:00	LightOn OCR 3 and the comparison	The creator introduces the 0.8B model, compares its table output with Tesseract, and outlines the document tests.
00:33	Model features and benchmark claims	The video explains the model’s supported outputs and prompts, reviews reported benchmark scores, and notes their limitations.
02:06	Local setup and document tests	The creator describes the laptop setup and compares LightOn OCR with Tesseract on a note, table, invoice, paper page, and chart.
04:51	Speed results and recommendations	The video reports slow laptop inference, contrasts it with the unverified H100 speed claim, and explains when to use LightOn OCR or Tesseract.

Quotes

Tesseract gave me a flat line of text. — 00:00

So, this is not a chat model. It is a document reader. — 01:04

Light on OCR returned an actual HTML table with the header row — 03:14

So, the accuracy is great and speed on a laptop like mine is not usable until you run it on a proper server. — 05:21

If you need fast, simple text from clean scans on a CPU, Tesseract is still the one. — 05:53

Concepts and entities

LightOnOCR 3, Tesseract 5.4, OLMo CR bench, Infinity Parser Pro, Surya, Transformers, vLLM, RunPod

Full transcript (original language)

00:00 This is a table, and this is what a 0.8 billion parameter model gave me back from it. Every number, every column as a real HTML table. Same image, same laptop. Tesseract gave me a flat line of text. It’s an 8 GB RTX 4060 here, and the model takes up 1.7 GB. Now, I tested five types of documents, and on one of them, the old OCR turned six eggs into at eggs. Stick around because the speed number at the end is not what I expected. So, this is Light on OCR 3, and it came

00:33 out on October 8th. There are three sizes: 0.8B, 1B, and 4B. And you can see here that the license is Apache 2.0, so you can use it for work. Now, what’s different from normal OCR is that this is a vision language model. You give it a page image, and it writes the page back as markdown with tables as HTML. And if you send a single word grounding as the prompt, it also gives you a label and a box for every block with coordinates from 0 to 1,000. So, text, title, table, chart, formula, each

01:04 with its position. Mind you, the model card says only these two prompts, empty or grounding, are supported. Anything else is out of distribution. So, this is not a chat model. It is a document reader. Now, the claim on OLMo CR bench, the blog says the 0.8B model scores 85.5, and the 4B scores 86.3. And you can see here that the 4B is only 0.8 points better at five times the size. But hold on, look at the top row.

01:35 Infinity Parser Pro scores 87.6, and it’s 35.1 billion parameters. So, this is not the best model. It’s the best small one in this list, ahead of Surya at 83.3 and Mr. OLMo CR at 81.9. And one more thing, these are the developers’ own numbers. They also say simple text normalization changes can move scores noticeably. So, treat 85.5 as a guide, and that is why I made my own test. Now, for running it, mind you that everything here is on my local system.

02:06 This is the RTX 4060 laptop GPU with 8 GB. The model card says you need Transformers 5.5.4 or later. So, I made a fresh environment and installed that. One thing I had to do first, my GPU had 7.5 GB already used by two llama processes. So, I stopped those, and then it had room. I loaded the model in BFloat16, and it took 5.5 seconds to load and 1.71 GB of VRAM, and the baseline is Tesseract 5.4, the standard open-source

02:38 OCR. Same five images go to both. The images are ones I generated myself, so I know the right answer for every word. I’ll tell you now the handwriting one is a handwriting font, not a real pen, so keep that in mind. Test one, the handwritten style note. Tesseract gave “trees with the Gigan 2K Abasmati rice and that eggs in a liter of milk.” It read the number six as the at sign twice. So, even the phone call time became at p.m. Light on OCR gave me every line right, 2 kg, 6 eggs, 6 p.m., room 204.

03:14 Test two, the table. Tesseract got the numbers right, but you can see that it’s just rows of text, the columns are gone. Light on OCR returned an actual HTML table with the header row, and all 15 numbers match, and the total row is 4,985, which is correct. Now, in grounding mode, it also tells me the title is at this box, and the table is at this box, and you can see the table box lines up with the real table on the image. Test three, the invoice. This is where I thought Tesseract would be fine, and the

03:46 text is all there, but look at the order. The item column, then QRD, then bill two, then the amounts. The line items are split apart so you can’t tell which price belongs to which service. Light on OCR kept the bill to block together and the item table with each row intact. GPU compute hours $41.2048 and the total is $202.96, GST $30.96 and the bank account and IFSC code are exact. For anything that goes into accounting, that is the difference.

04:18 Test four, a research paper page with two columns. Tesseract actually did well here. The text reads in the right order, but the equation came out as QK {question mark} T. Light on OCR wrote it as LaTeX, softmax of QK transpose over square root of D and the headings became real markdown headings. Test five is the one I was waiting for the chart. Tesseract found the title, the number 85% and model D. That’s all. Light on OCR gave me a table with the chart’s data. Model A 62, B 71, C 78, D

04:51 85. It read the bar heights. And remember, this is not the 4B model, this is the 0.8B. Now, the speed number. Tesseract took under 1 and 1/2 seconds for every page on my laptop. Light on OCR took 19 seconds for the note, 131 for the table, 454 seconds for the invoice and for the paper page about 1960 seconds. That’s over half an hour for one page. And that’s the honest answer on Windows.

05:21 Some of the models layers fall back to plain PyTorch because the fast kernels aren’t installed. The blog reports 4.78 pages per second for the 0.8B, but that’s an H100 with vLLM. I could not reproduce that here, so I won’t claim it. Also, I could not test the 4B. It doesn’t fit in 8 GB at BF16. So, the accuracy is great and speed on a laptop like mine is not usable until you run it on a proper server. So, who should use this? If you need tables, invoices, or charts turned into structured text, and you can run it on a

05:53 GPU with VLLM. This replaces a lot of pipeline. If you need fast, simple text from clean scans on a CPU, Tesseract is still the one. If you’re looking for cheap and reliable GPUs, I use RunPod, and the link is in the description. So, that’s light on OCR three on five documents. Tell me in the comments which document type I should break it with next, and I’ll see you in another one.