I Let an AI Reverse-Engineer a Game Binary, Then I Graded It

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I Let an AI Reverse-Engineer a Game Binary, Then I Graded It · Channel: Prompt Engineer 48 · 06:42 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Cybersecurity and software security, Open source and software ecosystems

Summary

The video tests REA, an AI-assisted reverse-engineering tool, on a stripped OpenTTD binary in a sandbox, with the game’s source code kept separate for checking its claims. Across multiple runs, the agent correctly identified pathfinding structures, route-ending conditions, settings offsets, and the route-cost formula, but it also made a confident false claim and could not recover runtime default values. The creator built a small pathfinder from the agent-derived specification and demonstrated how signals and other costs affect routes, while noting important differences from the real game. The four runs cost $4.39, and the video emphasizes checking both an AI’s positive claims and its conclusions about what is absent.

Key points

REA is an MCP server that lets an agent use Ghidra, Hopper, or IDA to inspect compiled programs.
The creator chose OpenTTD 15.3, built and stripped the binary, and kept the source code separate to verify the agent’s claims.
Stripping alone left many exported symbols, so the creator relinked without the RDynamic flag before the main test.
The first run correctly identified six path-segment fields and 13 segment-ending reasons, but incorrectly claimed penalty strings were absent.
After being given offsets from the first run, the agent mapped all 24 settings to offsets and correctly recovered the pathfinding cost formula.
The agent could not determine the settings’ default values because they are filled in at runtime, and it appropriately refused to guess.
A pathfinder built from the agent-derived specification passed 10 tests based on hand-calculated costs and demonstrated route changes in response to signals.
The clone differs from the real game in some respects, and it was not tested against the real game on the same track layout.

Chapters

Start	Chapter	Summary
00:00	Introducing REA and choosing a test binary	The creator introduces REA, explains its limitations around proprietary software, and chooses an open-source OpenTTD binary for a verifiable test.
01:34	Sandbox setup and preparing the binary	The creator describes the isolated WSL setup, removes exported symbols, and extends Ghidra’s startup timeout after the initial analysis times out.
03:05	First analysis: accurate findings and a false claim	The agent identifies pathfinding structures and segment-ending reasons, but misses the cost formula and wrongly says penalty strings are absent.
04:38	Recovering the cost formula and building a clone	The agent recovers the route-cost formula, and the creator uses the findings to build and demonstrate a tested pathfinder.
05:41	Limits, caveats, and results	The agent cannot determine runtime default values, and the creator explains the clone’s differences, testing limits, and total run cost.

Quotes

The formula behind that number was pulled out of a game file with no source code and no symbols by an AI agent. — 00:00

13 out of 13 in the right order. — 03:05

It was confident, and it was wrong. — 03:36

And it refused to guess. — 05:41

So those 10 tests check my reading of the spec, not the game. — 06:11

Concepts and entities

REA, OpenTTD, Ghidra, Claude Code, A*, WSL 2

Full transcript (original language)

00:00 This train just took a detour because of a red signal and it worked out the cost of that detour as 1984. The formula behind that number was pulled out of a game file with no source code and no symbols by an AI agent. All right, in this video I’ll show you how and where it got stuck. Mind you, all of this ran in a sandbox on my laptop. So, the tool is called REA. It’s on GitHub under Morluto and it stands for reverse engineer anything. It’s an MCP server, which means an agent like Claude code can call it and under the hood it

00:31 drives Ghidra, Hopper, or IDA to read compiled programs. The license is MIT. Now, before I run anything, one thing, the star numbers I found don’t agree. My notes said 25,000 in a day, the repo page showed about 124,000, and a skills directory showed 14,000. I can’t tell you which is right, so I won’t quote one. What I can tell you is the NPM package was created on the 12th of July, so it’s about 3 months old, not a day old. And its own disclaimer says it won’t

01:02 help with copying proprietary software or getting around DRM. That matters for the next part. I first wanted to do this on RimWorld, but I don’t own it and reading that disclaimer, taking apart a paid game, and cloning it is exactly what the tool says it won’t help with. So, I switched to OpenTTD version 15.3. It’s open source, so I can build the binary myself, strip it, and give the agent only the stripped file, and then I keep the real source on the side to check every claim it makes. That’s the whole test. Everything runs inside WSL 2 Ubuntu on

01:34 this machine. I turned off the Windows drive mounts and the interrupt. So, if you look here, the C drive mount is empty and the tool can’t touch my files. I tried to run pod CPU pod first, but there was no capacity in that data center, so I went with WSL. Ghidra 12.1.4 is healthy. Node is 24.21. And one thing I did differently from the readme, I didn’t run the full setup command. I added REA to Claude code as a single MCP entry myself, so I know exactly what changed. Then I run the

02:05 agent with only the REA tools turned on. No shell, no file rights, no web. Now, I stripped the binary, so no debug info and no symbols, and then I searched it for the word Yapf, which is the name of OpenTTD’s pathfinder. And you can see that it still finds full function names like Yap train choose track. So, strip is not enough here. The game links with a flag called RDynamic, which leaves 19,080 exported symbols in the file. I relinked without that flag, and the exports dropped to 55. That’s the

02:35 file I used for the real test. There’s one more thing worth noting. 11 compiler strings still mention a function called add new node with the track follower types. Real programs leak like this, so I left them in. Also, the first analysis took about 17 minutes in Ghidra, and REA’s default deadline is 330 seconds. It timed out. The readme mentions a setting for that, REA_GhidraStartupTimeoutMS, and I raised it to 45 minutes. The question was simple. How does a

03:05 train choose its route at a junction, and show evidence for every claim? 52 turns, 19 minutes, 97 cents. What it found, six fields of a path segment and 13 reasons a segment can end, decoded one by one from a string table. I checked all of them against the real source, 13 out of 13 in the right order. It also found the search node, parent, cost estimate, and signal counters, and it said it looks like a best first search. That’s right, it’s A*.

03:36 But, it did not find the cost formula, and it said so. And there’s one claim it got wrong. It told me the binary has no penalty strings, nothing like real slope. But, they’re right here, real slope penalty curve 90 first red. It was confident, and it was wrong. So, that’s a good reminder, check the negatives too. For run two, I corrected it. I told it those strings exist and gave it the offsets from run one. So, mind you, this one is not blind and I locked the real source, so the agent’s user can’t read it. The permission is denied. I checked.

04:06 28 minutes, 84 turns, $2.26. And here’s what it came back with. It found that each setting has a tiny function that returns an offset into one settings object and it mapped all 24 settings to offsets eight by following the name, 14 from the order of the stubs. I checked everyone against the real struct, 24 out of 24. Then the cost, a straight piece of track costs 100, a diagonal piece 71, a 45° turn adds one penalty, and 90° turn adds another. Slope, crossing station, red

04:38 signals, and a bonus for green signals, and the distance estimate, the smaller of the two distances times 71 plus the difference minus one times 50. That’s exactly the real one. What it did not find are the default values. I’ll come back to that. Now, I wrote a spec only from what the agent found. No game source where the agent didn’t establish something, the spec says gap. Then I built a pathfinder. 10 tests with the costs worked out by hand from the spec. A straight line is 400 for four tiles. The turn south turn east test is

05:10 171 + 71 + 200 + 100. They all pass. And this is the page. The train starts here, the station is there, and there’s a red signal on the short route. So, it takes the detour and the total is 1984. If I turn that signal green, the route goes straight and the cost drops to 500 because a green signal gives a bonus. And if I make it a red exit signal, the penalty is 10,000. So, it goes round again. You can change the penalties with these sliders and watch the route change. Let’s go ahead and run the train.

05:41 I ran two more attempts to get the default values. How big is the slope penalty by default? The first one stopped halfway because my account hit a usage limit. The second one finished 21 minutes 57 cents and it found nothing. It said the registration code has no default in it and the settings object is empty in the file, so they get filled in at runtime. And it refused to guess. That’s the right answer, even if it’s not the one I wanted. So the clone uses the game’s published defaults and the page says that. And three more things I should tell you.

06:11 The real slow penalty only applies going uphill and mine applies both ways. I didn’t implement signal reservation or platform lengths and I haven’t run my clone against the real game on the same track layout. So those 10 tests check my reading of the spec, not the game. So in this video we have seen how an agent can pull the structure, the cost formula, and 24 settings out of a stripped game file and where it couldn’t. The four agent runs together cost $4.39. If you’re interested, you can go ahead and try REA on a program you build yourself and let me know how

06:42 it goes. I’ll see you in another one. Check out the next video.