Eagle 4 Faster than DSpark, MTP, DFlash for LLM inferencing

Author: Data Science in your pocket · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Eagle 4 Faster than DSpark, MTP, DFlash for LLM inferencing · Channel: Data Science in your pocket · 05:13 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, Open source and software ecosystems

Summary

Eagle 4 is a speculative decoding system that pairs a draft model with a harness for existing inference engines, rather than being a standalone chat model. The video explains how it proposes token trees for a target model to verify and describes its sliding-window draft layer, low-rank candidate head, and append-only cache. In the cited Qwen 3.8 27B benchmark, Eagle 4 reaches 3.59× the vanilla decoding speed, outperforming MTP, D Spark, and D Flash in that test. Results vary by engine and load, and the video notes limitations around supported configurations, correctness reporting, and licensing.

Key points

Eagle 4 is the fourth generation of the Eagle speculative decoding family and is delivered as Eagle Harness for supported inference engines.
Speculative decoding uses a draft model to propose candidate tokens, which the target model verifies in one forward pass.
The video says speed-up depends on how many draft tokens are accepted and the costs of drafting and verification.
The described drafter includes a sliding-window draft layer, a low-rank candidate head, and an append-only draft cache.
In the cited Qwen 3.8 27B, single-request benchmark, Eagle 4 achieves 3.59× vanilla decoding, compared with 2.22× for MTP, 2.57× for D Spark, and 2.80× for D Flash.
Other reported results include 3.33× on SGLang at concurrency one, 2.99× on vLLM at a single request, and 1.35× on vLLM at 64 concurrent requests.
The video says vLLM requires offline tree calibration, while SGLang adjusts tree width during decoding.
Stated limitations include text-only use, no pipeline parallelism, unquantized KV cache for vLLM trees, and unqualified multi-node support.
Eagle 4 is described as source available rather than open source; commercial use is permitted under a $20 million revenue threshold, while competing offerings require a separate commercial license.

Chapters

Start	Chapter	Summary
00:00	Eagle 4 and the decoding bottleneck	The video introduces Eagle 4 and its reported speed-up over vanilla decoding and other drafters.
01:02	Speculative decoding and benchmark comparison	It explains candidate-tree verification and compares MTP, D Spark, D Flash, and Eagle 4 on the cited benchmark.
02:04	Inside the drafter and decoding process	The video describes Eagle 4’s draft components, token-tree expansion, and engine-specific verification approach.
03:39	Reported results and setup	It gives additional results across engines and concurrency levels, then outlines installation and configuration.
04:41	Correctness, limitations, and licensing	The video discusses correctness reporting, supported-use limitations, licensing terms, and the project’s overall pitch.

Quotes

Every serving stack has the same bottleneck, one token at a time. — 00:00

Eagle 4 reaches 3.59. — 01:34

And at 64 concurrent requests, the gain falls to 1.35 times. — 03:39

Eagle four is source available, not open source. — 04:41

Concepts and entities

Eagle 4, Eagle Harness, speculative decoding, MTP, D-SparK, DFlash, SGLang, vLLM, Qwen3.8, eaglehq/eagle4

Full transcript (original language)

00:00 Every serving stack has the same bottleneck, one token at a time. Eagle 4 attacks that bottleneck and its paper puts it ahead of three well-known drafters, MTP, D Spark, and D Flash. On Qwen 3.8, 27 billion, a single request across 4B 300s, it measures 3.59 times vanilla decoding and it is not a model you chat with. It is a drafter plus a harness that plugs into the engine you already run.

00:30 Eagle 4 is the fourth generation of the Eagle family for speculative decoding. The idea is old and simple. Let a small draft model guess the next few tokens, then let the big model check all of those guesses in one forward pass. Eagle 4 is not a checkpoint that answers questions. It ships as Eagle Harness, a pip installable package that adds one speculative method to S Jiao Lang, to vLLM, or to Token Speed. The measured pair is Qwen 3.8, 27 billion, with its

01:02 Eagle 4 draft. Speculative decoding trades compute for latency. The draft proposes a tree of candidate tokens. The target verifies every node of that tree in a single forward pass. The accepted path is committed plus one bonus token. Speed up is the product of two things. How many draft tokens survive verification and how little the draft and the verify cost. MTP, D Spark, and D Flash all optimize that same ratio. The

01:34 differences are the draft architecture and how the tree is sized. This is figure one of the Eagle 4 paper. Qwen 3.8, 27 billion. The speed bench qualitative split. 4B 300s, a single request, temperature zero. MTP reaches 2.22 times vanilla. D Spark, 2.57. D Flash 2, 2.80. Eagle 4 reaches 3.59. That is about 28% faster than the best

02:04 of the other three and 62% faster than MTP. Note what is compared here. Speed-ups over the same vanilla baseline, not raw tokens per second. So, what is inside the drafter? Three pieces. A sliding window draft layer. So, each guess only looks at a recent slice of context. A low-rank candidate head. A small rank our estimate short lists a few tokens per vocabulary shard and only those candidates are res-scored with the full language model head. And

02:35 an append-only draft cache. So, every round extends what came before instead of rebuilding it. The draft reads three hidden states from the target and both dense and latent mixture of experts drafts are implemented. One decode round. The draft grows a token tree one level at a time, expanding the best K frontier nodes. A node is ranked by its parent’s score times its own probability. Then the cost model decides how wide the round should be for measured draft and verify costs at this

03:07 batch size and context length. On SG Lang, it re-decides after every second level. On VLLM, the tree is static with a calibrated depth stop instead. One target forward verifies the whole tree. The paper is one kind of evidence. The repo ships another. On SG Lang, 1B 300 at concurrency one, goes from 103 to 343 tokens per second, 3.33 times. For B300s at concurrency eight, 1.91 times. On

03:39 AMD, 1MI 355X at concurrency eight reaches 2.07 times. VLLM on 4B 300s, single request measures 2.99 times. And at 64 concurrent requests, the gain falls to 1.35 times. Serving it is meant to be boring. Install one patched engine from a pinwheel index, then install Eagle Harness from PyPI. Run Eagle Harness, check to confirm the engine exposes

04:10 every hook the plugin needs, then start the server. For SGLang, Aquinn 3.827B target, tensor parallel four, draft depth 12, top K four, verify with 16. On vLLM, you calibrate the tree once, offline, while the GPUs are idle. Speculative decoding is only useful if it is correct, and Eagle four keeps the target’s output distribution. The project publishes its greedy gate, exact

04:41 matches, near ties, and FP32 arbitration, reported separately for every configuration. A near tie is not bitwise equality, and the docs say so. The limits are equally specific. Text only, no pipeline parallelism, and unquantized KV cache for vLLM trees. Multi-node is not qualified, and token speed is still experimental. And then the license, which is the part nobody expects. Eagle four is source available, not open source. Commercial

05:13 use is permitted under a $20 million revenue threshold, and building a competing offering needs a separate commercial license. So the pitch is narrow and real. One drafter that adapts to batch size and hardware. A published 3.59 times, a measured 1.35 at high load. The code is on GitHub at eaglehq/eagle4.