SGLang Integrates DSpark Speculative Decoding: Confidence-Driven Variable-Length Verification
Blog DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification Speculative decoding trades extra compute for fewer decode steps, and the trade sours as load grows: at batch size B with K speculative tokens the target verifies B K tokens every step, and past a po... SGLang Team
AI BriefingThe SGLang team integrated the DSpark speculative decoding algorithm into its open-source inference engine. The algorithm uses a semi-autoregressive block drafter to generate a group of tokens at once, then uses a confidence head and sequential temperature scaling (STS) to dynamically assign a variable verification length to each request, trimming the cost of ineffective verification under high load. SGLang supports both dense models (such as Qwen3) and sparse models (such as DeepSeek-V4), using full CUDA graphs to handle irregular per-request verification lengths. It offers three verification modes: `static` (full length), `compact` (production path), and `cap-accept` (acceptance-cap measurement). The release also introduces optimizations such as zero-overhead scheduling, an online scheduler based on an offline cost table, and fused Triton kernels. In tests using DeepSeek-V4-Flash on H200, DSpark achieved a better throughput-latency tradeoff than MTP and the non-speculative baseline across the entire concurrency sweep.
Source: LMSYS Blog · lmsys.org