500B Tokens Later: Letting AI Agents Decompile a First-Person Shooter

Unless you explicitly need byte-matching decompilation, there are significantly faster ways to produce a decompilation/C which is functionally equivalent. I need to post about this. What's been working for me is that for every function, Agent A is tasked with writing some code which is semantically equivalent to the original assembly, but not necessarily exactly the same. Agent A also writes tests. Agent A submits the implementation of the function and tests to the harness for it to judge. The harness runs both the original function and the submitted function in a virtual machine/simulator/emulator (the tests define function inputs and starting state). The harness will only accept the implementation if 1) the read/write sequence to RAM is identical to the original function's, and 2) there must be complete line and branch coverage of the original function being decompiled.I've found this to be robust for decompiling games, while giving the agents enough freedom to write code that is readable and not waste a ton of time making sure e.g. instruction ordering, register assignments, etc. are all exactly the same.... reply:...

The reason this cost so much is because the AI has the ridiculous goal of getting identical assembly output.The agents would have had to mess around with compiler versions, optimisation options, and the phase of the moon as well.If you just went for functional equivalence, it would probably cost 10x or 100x less tokens.Another false economy was using Sonnet instead of a more intelligent model like Sol 6.1 (1), which would have cost more per token, but is 100x or so better at reverse engineering and coding and therefore can chew through the source code much quicker and make fewer mistakes, meaning less work needing to be scrapped.In my testing doing a similar task, I ran multiple sonnet for weeks and burnt through ~$1000 in tokens to get 20% completion and output that was pretty bad. After switching to Sol 6.1, it finished the whole task in around 2 days, cost around $50, and it did it with zero supervision and a single /goal.(1): struggle to use Opus for reverse engineering, too many safeguards. OAI has virtually none, and uses way less tokens so is more economical. reply: Identical output isn’t ridiculous....

I struggle to understand what legal leverage they used to threaten him to remove every detail about the game. Can someone post the game name and company name?So, if you reverse-engineer game X and post reverse-engineered code, what exactly do you infringe, how and in which jurisdiction? What changes if it is done via LLM?(I understand that LLM decompilation is absolutely out of hand right now and something surely will come to trample the fun. But what and when? I suppose american LLMs will have their system prompt updated to forbid any reversing help and report suspicious activity straight to legal hotline) reply: Call of Duty: Modern Warfare 2 (2009) (from a previous version of the page)

Interesting, if all software can be decompiled and copied what is the future of software. Will all software be SaaS? A time where the majority of PCs will be terminals? Game consoles are almost there. It's only a small jump for all software to go that way. reply: Most software organizations pay for is effectively a SaaS or something where software is a minor part of the artifact and support is where the real money is.

A previous version of the page said the game was Call of Duty: Modern Warfare 2 (2009).

Can anyone think of any lessons to draw from this for normal development, where we don't have oracles to serve as guardrails? I write specs but the agents still find ways to insert bugs between the lines. Oh well, job security.

- i want to very very badly see a post of this using LM studio and one of the open source models- please someone do it

In case anyone is curious about the obvious question of which game they are talking about... based on months-old reddit posts (which seem like links to prior progress reports of the same project) the game in question appears to be Call of Duty: Modern Warfare 2 (the original 2009 version).https://www.reddit.com/r/ReverseEngineering/comments/1vxig19...

To save folks looking this up:>At current 2026 API rates, 500 billion AI tokens would cost roughly $100,000–$750,000 depending on the model, with most flagship models in the $150–$400 per million input tokens range