This Nature Paper Turns Any Research Paper Into an AI Agent - INSANE

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: This Nature Paper Turns Any Research Paper Into an AI Agent - INSANE · Channel: Prompt Engineer 48 · 07:12 · Watch on YouTube

Topics: AI agents, Health and science

Summary

The video explains Paper to Agent, a system that turns research papers and their code into MCP servers whose validated tools can be run by chat agents. The authors report strong benchmark results across computational biology and other fields, while showing that some papers cannot be converted and open-ended scientific reasoning still needs human guidance. Examples include applying AlphaGenome and Scanpy, and combining multiple paper agents to analyze a psoriasis locus. The video cautions that most evaluations are the authors’ own, and the presenter says they did not independently run the servers.

Key points

Paper to Agent packages a paper’s executable methods, resources, and workflow instructions as an MCP server.
Its tools run the paper’s code and link back to source code, rather than relying only on retrieval from the paper text.
A multi-agent pipeline finds code, sets up the environment, extracts methods into tools, and tests outputs against references.
For AlphaGenome, the authors report 22 validated tools built in about 45 minutes for $14.
On the reported benchmarks, Paper to Agent outperformed direct repository access on tutorial questions, novel questions, and a collection of 100 biology papers.
The system converted 74 of 100 computational biology papers; the other 26 had issues such as missing code or data, broken environments, or scripts that did not generalize.
The authors also report successful applications to Scanpy workflows, non-biology papers, and a multi-agent analysis combining three papers.
The presenter notes that the benchmarks are mostly the authors’ own, that independent replication matters, and that humans remain responsible for hypotheses and mechanisms.

Chapters

Start	Chapter	Summary
00:00	The idea: turning a paper into an agent	The video introduces the limitations of using research code and explains how Paper to Agent packages methods as an MCP server.
01:32	Building and validating paper tools	The authors’ multi-agent pipeline extracts tools from a paper’s code and tests them against expected outputs before making them available.
02:33	Benchmarks and research examples	The video reviews reported AlphaGenome results, a Scanpy workflow, and examples of where the method’s performance and limits become apparent.
04:06	Scaling up and testing beyond biology	The authors report conversion and validation results across 100 biology papers, along with tests on out-of-scope queries, adversarial repositories, and non-biology papers.
05:39	Human-guided science and takeaways	A multi-paper analysis illustrates the role of human-selected hypotheses, and the video closes with caveats about the evaluations and the limits of automation.

Quotes

The core idea is to turn the paper into an MCP server. — 01:01

The agent isn’t writing fresh analysis code every time. — 02:33

open-ended reasoning stays human in the loop. — 05:39

hypotheses and mechanisms stay with the human. — 06:41

I have not run them myself for this video — 07:12

Concepts and entities

Paper to Agent, MCP, AlphaGenome, Claude Code, Claude Sonnet 4, Scanpy, Hugging Face Spaces, RunPod

Full transcript (original language)

00:00 Here’s the normal way to use a new method from a paper, find a repo, install dependencies, fight the environment, work out which inputs go where. Now, here’s what this Nature paper proposes instead. You ask, “Interpret this variance effect on chromatin accessibility in muscle cells.” And the paper itself answers. It’s called Paper to Agent from Stanford. In this video, I’ll show you how it works, what the numbers actually say, and where it breaks. >> [music]

00:30 >> The authors’ point is simple. A paper is a passive object. If it describes a method, you still have to find the code, install it, configure it, and understand the inputs. Their example is Alpha Genome, a model that predicts how a single DNA mutation affects regulation. Using it in code means environments, API keys, building variant objects, picking output types. A biologist who could really benefit from it often just stops there. Earlier fixes like papers with Code Binder and Code Ocean did help reproducibility, but says the barrier to

01:01 customizing and applying the code to your own project was still there. The core idea is to turn the paper into an MCP server. MCP, the Model Context Protocol, is the standard way to expose tools to a language model. Each paper server has three parts. Tools, executable functions for the paper’s methods. For Alpha Genome, a tool takes a variant and returns predictions and plots. Resources, the static stuff, the manuscript, code, supplements, data, and prompts. Short instructions that walk the agent through a multi-step workflow in the right order. You host that

01:32 server, for example, on Hugging Face Spaces and connect any chat agent. The authors use Claude Code with Claude Sonnet 4. Importantly, this is more than retrieval over the paper text. The tools actually run the paper’s code, and every tool carries a link back to the original source code, so you can trace where an answer came from. How do they build it? A multi-agent pipeline on Claude Code. One agent finds the code, an environment agent sets up the software, an extraction agent turns the core methods into tools. Then a testing agent runs them and keeps refining until the

02:03 results match the reference outputs. Under the hood, the method section describes a central orchestrator agent that hands work to specialized sub agents through a six-step pipeline. Each sub agent has a structured prompt for its role, the tools that it is allowed to use, and the output it must produce. Steps run one after another and in parallel inside a step when they can. The paper defines a path expected files generated numbers inside tolerance, figures match the reference. Tools that keep failing are excluded from the server. That is the answer to code

02:33 hallucination. The agent isn’t writing fresh analysis code every time. It calls a tool that was validated and locked. For Alpha genome, the authors say this produced 22 tools, all passing in about 45 minutes, costing $14 on a laptop. Now the numbers on 15 tutorial based questions, the paper two agent agents scored 98.7% Claude with direct repo access got 82.7, BioMed 37.3. On 15 novel questions, 100% versus 78.7 and 56. Five runs, two human

03:04 graders, and it was faster with median run time about two to four times lower. But look at the open-ended questions. 30 research-style queries, paper two agent got 82.7%, Claude plus repo 56.7, and BioMed 72.2. That gap is much smaller. Then the part I like, they asked why a variant on chromosome 1 is linked to LDL cholesterol. The agent planned, scored the variant, made plots, and picked sort 1. The original Alpha genome paper emphasized SCLSR2 and PSR C1. The

03:35 authors don’t hide it. All three score high, and all three are significant in GTX. So that’s not the agent beating the paper, it’s a reminder of how hard causal genes are at these loci. Second case, Scanpy for single-cell data. Seven tools, about 45 minutes, $13. The interesting part is the MCP prompt. It encodes the whole pipeline in order. Quality control, normalization, clustering, cell type annotation. You give a file path and it runs on four public data sets. The authors report it

04:06 matched human results. Same cell and gene counts after filtering, same top marker genes. They also identified a third paper called tissue for spatial transcriptomics and across seven different single cell data sets, the agent adjusted its own parameters to fit each instead of using one fixed recipe. That matters because a locked tool is only useful if it is also flexible about its inputs. Now scale 100 computational biology papers, no manual cleanup, 74 were successfully turned into agents. That means 26 were not. The reasons

04:38 missing code, missing data or model files, broken environments, scripts that don’t generalize. For the 74, 599 tools were proposed and 593 passed validation on 300 questions, 91.2% against 80.3 for Claude code on the repo with the same model. About 20 cents and 1.6 minutes per query against 38 cents and 4.3 minutes. The paper then stress test the idea in a permuted test where the question does not belong to the paper. It correctly rejected 100% of out

05:08 of scope queries. In adversarial tests, the authors injected broken dependencies, wrong file paths, typos and deprecated API calls into repositories and paper two agents still recovered working servers. It even built a working server from a repository with no executable tutorials. And it is not only biology. On 42 execution tasks from 10 non-biology papers covering things like statistics, econometrics, game theory and astrophysics, it scored 98.1%. When code could not be turned into tools at all, the resource layer alone, just

05:39 the structured paper content, reached 89% on 100 synthesis questions. A Claude browser used baseline got 82 and it was 34 times cheaper and 15 times faster. The authors also say it plainly, open-ended reasoning stays human in the loop. And my read how easily a paper becomes an agent could become a kind of reproducibility score. Last case, three paper agents work together, AlphaGenome, a CRISPR screen paper, and a T-cell perturb-seq paper. AlphaGenome predicts GPR137 at a psoriasis locus. The co-scientist then correlates

06:09 perturbation signatures, GPR137 knockdown correlates at about 0.6 but only after stimulation, not at rest. Notice the human picked the strategy. That’s the paper’s model. Scientists set the hypothesis, agents run the analysis. What I find most interesting is the method the agent came up with, correlating a regulatory element perturbation signature against gene knockdown signatures. Across two independent screens is not proposed in either source paper. The authors call it a flexible technique for causal gene prioritization. The paper also runs a

06:41 second case, linking AlphaGenome to an ADHD genome-wide study in the supplement. So, what should you take from this? First, the claim is narrow and well-tested, tools validated against the paper’s own results, then locked. Second, the evaluations are mostly the authors’ own benchmarks and papers created by their own experts, so independent replication matters. Third, a quarter of the biology papers could not be converted at all. And fourth, the paper itself says that hypotheses and mechanisms stay with the human. If you publish code, this is a good reason to

07:12 make it clean enough that an agent can run it. The code is public on GitHub and the three MCP servers are on Hugging Face Spaces. I have not run them myself for this video, so everything you saw is the authors’ reported results. If you’re looking for cheap and reliable GPUs, I use RunPod and the link is in the description. So, here’s my question, which paper would you turn into an agent first? Tell me in the comments.