RadixArk, Google, and Meta jointly open-source SGLang-TorchTPU, enabling SGLang to natively provide production-grade PyTorch inference on TPUs
SGLang-TorchTPU: PyTorch-Native Production Serving on TPUs
AI BriefingRadixArk’s SGLang team has partnered with Google and Meta to open-source SGLang-TorchTPU. Built on Google’s TorchTPU, it enables the same SGLang engine to serve production workloads on both GPUs and TPUs.
Also reported by 2 other sources
- vLLM releases a TorchTPU backend, enabling vLLM to run on Google Cloud TPUsvLLM Official Blog · 27 days ago
- Google released native TorchTPU inference engines for vLLM and SGLangGoogle Developers Blog · 27 days ago
Source: LMSYS Blog · lmsys.org
Event follow-up1
View all event coverage- Google released native TorchTPU inference engines for vLLM and SGLangGoogle Developers Blog · 27 days ago