Back to Featured
LMSYS BlogSGLang Team2026-07-06 00:00July 6

SGLang Integrates DSpark Speculative Decoding: Confidence-Driven Variable-Length Verification

Blog DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification Speculative decoding trades extra compute for fewer decode steps, and the trade sours as load grows: at batch size B with K speculative tokens the target verifies B K tokens every step, and past a po... SGLang Team

AI Briefing

The SGLang team integrated the DSpark speculative decoding algorithm into its open-source inference engine. The algorithm uses a semi-autoregressive block drafter to generate a group of tokens at once, then uses a confidence head and sequential temperature scaling (STS) to dynamically assign a variable verification length to each request, trimming the cost of ineffective verification under high load. SGLang supports both dense models (such as Qwen3) and sparse models (such as DeepSeek-V4), using full CUDA graphs to handle irregular per-request verification lengths. It offers three verification modes: `static` (full length), `compact` (production path), and `cap-accept` (acceptance-cap measurement). The release also introduces optimizations such as zero-overhead scheduling, an online scheduler based on an offline cost table, and fused Triton kernels. In tests using DeepSeek-V4-Flash on H200, DSpark achieved a better throughput-latency tradeoff than MTP and the non-speculative baseline across the entire concurrency sweep.

Source: LMSYS Blog · lmsys.org

Finished reading the news? Let’s talk about your business.

Leave your contact details, and I’ll get in touch with you personally.

WECHAT

Add me on WeChat

Press and hold or select the WeChat ID below to copy it.