Breaking through the limits of DeepSeek-V4-Pro services: a multi-scenario optimization method on H20
Blog Pushing the Limits of Serving DeepSeek-V4-Pro DeepSeek-V4-Pro is a 1.6-trillion-parameter Mixture-of-Experts (MoE) model released with both FP8 and FP4 weights. Models at this scale naturally benefit from accelerators such as NVIDIA Blackwell GPU... Tianyu Zhang, Yusong Gao, Yun Zhang
AI BriefingFor the 1.6-trillion-parameter MoE model DeepSeek-V4-Pro, the LMSYS team approached B300 performance on H20 GPUs through scenario-based serving configurations. A single-node H20-141GB reference implementation achieved 271 output tokens/s, narrowing the performance gap with B300 at 383.7 tokens/s to 1.42×.
Source: LMSYS Blog · lmsys.org