Unlimited OCR 2 is here!

Author: 1littlecoder · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Unlimited OCR 2 is here! · Channel: 1littlecoder · 11:52 · Watch on YouTube

Topics: Model inference and optimization, Multimodal models

Summary

The video introduces Baidu OCR 2, describing how its sparse reference sliding window attention (SRSWA) is designed to process long documents while limiting the model’s attention and KV-cache memory. The presenter explains the technique’s page gate and token sieve, and cites benchmark results for long-document OCR. They then demonstrate running the model on a RunPod 4090 instance using Transformers, converting a PDF into pages, and performing multi-page inference. The demo checks extracted tables and an equation, and the presenter ends by showing how to stop and terminate the pod.

Key points

Baidu OCR 2 uses sparse reference sliding window attention (SRSWA) to handle long-document parsing.
Traditional page-by-page OCR can struggle when context, such as a paragraph, spans multiple pages.
SRSWA selects relevant document references using page gate and token sieve, rather than looking back across every page.
The presenter describes a fixed set of 512 reference tokens, arranged in 16 blocks of 32, along with a 128-token window over recent output.
The video reports a score of 77 for Baidu OCR 2 on Marathon Doc Bench and compares it with scores cited for other models.
The demo uses a RunPod instance with a 4090 GPU and Transformers; the presenter also mentions SGLang for serving a production endpoint.
PDFs are converted into pages with PyMuPDF, and multi-page inference sends all pages in one request to preserve cross-page context.
The demo inspects extracted markdown and checks table values and an equation against the PDF.
The presenter recommends stopping and terminating the RunPod pod after the demo.

Chapters

Start	Chapter	Summary
00:00	Introducing Baidu OCR 2	The presenter introduces Baidu OCR 2 and outlines the video, including a demo on a multi-page PDF.
03:03	How SRSWA works	The video explains how page gate and token sieve select references to reduce the model’s attention and KV-cache demands, then cites benchmark results.
05:41	Setting up the model and running inference	The presenter walks through using a RunPod GPU instance, Transformers, and PyMuPDF to run multi-page PDF inference.
08:47	Checking the extracted results	The presenter reviews the output markdown and verifies examples of extracted table data and an equation.
10:51	Ending the demo and shutting down	The presenter recommends trying the model, describes terminating the RunPod instance, and wraps up the video.

Quotes

I’m using it on a RunPod instance, very cheap, less than $1 per hour. — 00:30

And that became what as RSWA, reference sliding window attention. — 02:33

Baidu does this with two steps. One is called page gate and the second one is called token sieve. — 03:34

There are two ways to do inference according to Baidu. — 07:46

we successfully managed to run unlimited OCR 2 from Baidu with a sparse reference sliding window attention, S R S W A. — 11:22

Concepts and entities

Baidu, Baidu OCR 2, SRSWA, RSWA, KV cache, Marathon Doc Bench, RunPod, Transformers, SGLang, PyMuPDF

Full transcript (original language)

00:00 One of the most popular OCR model, the Baidu’s unlimited OCR has got a newer version. The newer version is called Baidu OCR 2 and this model comes with something that is called SRSWA which stands for sparse reference sliding window attention. And as you can see on my screen right now, we are actually using that OCR model with this PDF. This is approximately 13 pages PDF. So, we are using this OCR model on this PDF to

00:30 get every text that is possible. So, I’m going to show you in this video first of all what is unlimited OCR 2 and how you can use it. I’m using it on a RunPod instance, very cheap, less than $1 per hour. This is Baidu unlimited OCR 2. So, you can see here they are saying that they are pioneering the era of OCR 3.0 and uh from long to unlimited horizon parsing. And this is being enabled by a new technique that they have figured out which is called SRSWA.

01:01 Before we try to understand what unlimited OCR 2 is doing, let us try to understand how traditionally OCR works. So, the classical approach is very simple. You chop the document into multiple pages and you read each of the pages on its own and then you finally try to glue the results back together. It works pretty well if there is only one page in isolation. So, if each page is an individual read, it doesn’t have to have context across the pages, then it works extremely fine. But, imagine like there are like one paragraph there

01:32 is one paragraph which is like split across two pages and then this becomes a really hard problem to solve. And that is exactly why attention comes to help. So, now the model like these models, they write their output one token at a time. And before writing each token, the model looks back over everything that it has in context for every image. So, we have got like three images, then it is going to look at three images and every word that it has written. So, the input and output. And

02:02 this look back is technically called as attention. To avoid redoing the work, the model keeps a memory of all the past context. And this is called KV cache. And now, as you know that the number of pages as it grows, it is going to look back more pages. It is going to write more tokens. And that means the KV cache is going to grow. And that is the first fix that Baidu did with the first model, which is unlimited OCR 1. So, when you copy a book, what do you do? You don’t reread everything that you have already

02:33 written. So, you look at the original and glance at the last few words that you wrote and then write the next word. And everything older just fades. So, this way you are actually keeping the KV cache minimum. And that became what as RSWA, reference sliding window attention. And this was powering the first unlimited OCR model. Now, the unlimited OCR 2 model comes with a newer fix, which is SRSWA. That is sparse reference sliding window

03:03 attention. And the new word here is sparse. That is very important for us to understand. Now, if we go back to the same analogy where a person was copying the book, now you know that they don’t look at all the 300 pages once. If you have got 300 pages document, they look at the page that they are trying to copy right now or maybe one page before. So, the writing is equivalent of tokens and the number of pages that they are looking is the look back, the attention that is happening. So, sparse reference sliding window attention does the same

03:34 thing. For each new token that the LLM, the large language model in this case, the OCR model is writing, instead of looking back every single page that was before, the model very meticulously picks out only the most relevant pieces. And Baidu does this with two steps. One is called page gate and the second one is called token sieve. And you can go read the paper. I’ll link it in the YouTube description for you to understand the details. So, the because of this, the result is a small fixed set of working references, which has got only 512 tokens that it

04:06 has to keep at a given point, chosen in 16 blocks of 32. So, 16 by 32 512 tokens. So, 128 token window over its current own recent output stays just like before. So, it’s going to have like 128 token window over its recent output. And the memory for the parts of the document that is no longer needed are released. That means it constantly flashes out the KV cache, and it looks at very few items that are selected by page gate and token sieve.

04:37 This way, now what Baidu has done is it has tried to fix the number of documents, number of pages it has to look back, and it has also made sure that you’re releasing KV cache in such a way that when you produce new tokens, you’re not filling in the memory. So, this way the computer is not increasing. Because of this, because of this changes that they have done, this model is according to a benchmark called Marathon Doc Bench, where they’ve tested this model. This model can handle hundreds of

05:08 pages long, and Baidu’s unlimited OCR 2 has scored 77. And the next best model, which has a vision model language model large language model but with a vision, is Kimi K3 has scored 33.7, and G Gemini’s flagship Gemini 3.8 flash the flagship flash model has scored 28.6. So, not only that this is going to help you look through all the pages in one single pass, but it can also help you do the same thing with highest accuracy. And that is the whole gist of why this

05:41 is very important. And this innovation brings state-of-the-art OCR model. And the machine in which currently I’m running is I’m currently running it on runpod no sponsor or anything. So this is currently running on a 4090 which is kind of a consumer based GPU and this is going to charge me less than $1 less than $1 per hour. So even if I run this continuously for $1 the compute is going to cost me less than $1. First spin up an environment deploy a new pod. When you deploy a new pod make sure that you are deploying it with CUDA and

06:11 everything and then get a 4090 machine or any machine that you have got like minimum 24 GB RAM the graphics memory. Once you have that you can open your Jupiter notebook. Install the required libraries. We are trying to make sure that we have all the libraries exactly like Baidu has given. Baidu has very clearly said that use transformers like I’m showing. If you are trying to just test bed like you want to do like once or twice. But if you want to do like a surf like if you want to like create a product on top of it or keep this endpoint always running on serverless then use SG lang.

06:43 So currently we are using transformers. So as you can see here we are using transformers but if you wanted to for like real production use case where you want to serve this then you can use SG lang as a serving platform. And here I’m using hugging face transformers. Once we install the required library I’m going to import torch and I’m going to import auto model and auto tokenizer. I’m going to call the model in this case like define the model Baidu unlimited OCR. It is literally coming from here and after we have that in place then define the tokenizer define the model and as you

07:15 can see here as we are importing the model we are moving it to CUDA the GPU. And takes like very few seconds. Once we do that if you want PDF to be enabled then you need this very short thing. I I just thought like we are going to use PDF so I’ve added it. We are using a library called PyMuPDF and after that the small utility function and once you do that we send the PDF we upload the PDF into our current environment wherever if you have got the PDF already in your working folder, good and fine. Then we convert all those PDFs into

07:46 pages and we send those pages. There are two ways to do inference according to Baidu. So, one is what we call as infer multi, which is what we are doing. The second one is what they call as infer model.infer, which is a single page inference. In a single page inference, in a multi-page inference, all the pages of a document are passed in one request. So, the context stays like the problem that we described at the start, like where a paragraph is cut into multiple pages, that will be handled in this case. But if you have only one page,

08:17 you’ve got only one JPEG, then you don’t have to do it, you can save further memory. So, in that case, you just have to A single page is encoded with dynamic tiling, which they call as like Gundam mode. So, that is something that you need to keep in mind. Right now, I’ve written the code for multiple page inferences because we are dealing with PDFs. So, we are going to do send the tokenizer, the pages, the output path where the result has to be saved, and then the result, and we are printing the token as well. So, you can see here, sorry, the text that we captured. All

08:47 the text that we captured is here. I’m going to go to the output. And we have multi. Okay, this is the input PDF, and you can see the result here. So, we have the raw markdown file. So, everything that we have gotten from this PDF is available here. That was the raw file. Here it is like a parsed file. And we can go pick one of the elements. Like for example, we can go see how the table is looking here. This is in page number 10. And what is this table called? This is called

09:18 COSCHL scene. So, I’m going to go back here to my result and then search for it. And we have the table here. So, you can see what we are seeing here is Sorry, what we are seeing here is this benchmark, COSCHL scene, and the accuracy is 77.8717, and we can cross verify it. So, the it is converted into a table first of all in HTML or markdown table and the exact value is there Nsynth which is 72.2900.

09:49 And oh my goodness, I keep on going for the wrong one. 72. This is So, let’s pick one more and then cross check. So, in this case, let’s go further down. I’m going to go to page number 11 and in this case I’m going to look for voice moss. Come back here, look for voice moss. We have voice moss in multiple places. Okay, so here we have voice moss which is in the table 0.81347. So, point my goodness. 0.81347. So, as you can see here, it

10:20 managed to capture the table pretty well and it has all the text and let’s see how it has done the equation. So, I’m I’m also figuring out the time like it’s a live demo. So, let’s see um so, this is in page number six choice categorical cross entropy. So, let’s see where do we have Okay, we have the equation here. We have equation that explains This is the equation. Okay, so this is the equation. So, let’s see how it looks.

10:51 If it looks exactly like how it has been p i k p i k Perfect. Um k belongs to c of i uh lower case k and uh Okay, it looks pretty good. Um I would strongly encourage you to try it out. I think that takes us to the end of the video. All you have to do is now close the notepad. In this case, notebook. In this case, I’m going to download it and uh keep it ready so that I can share it with you on GitHub and um shut down the run pod. Uh let us go and

11:22 do the same thing here. So, stop the pod. Stop the pod. Make sure all the data is gone like no other pod is running. Terminate the pod. Terminate the pod. After you terminate the bot, everything is gone. And uh less than $1, we successfully managed to run unlimited OCR 2 from Baidu with a sparse reference sliding window attention, S R S W A. And um I hope this video was helpful to you in understanding a very new OCR fresh off the oven.

11:52 See you in the video. Happy prompting.