Google released native TorchTPU inference engines for vLLM and SGLang
Announcing New vLLM and SGLang Inference Engines on TPU
AI BriefingIn a PyTorch-related announcement, Google introduces native PyTorch inference engines for vLLM and SGLang based on TorchTPU, developed with Inferact and RadixArk respectively, allowing developers to deploy models on TPUs using familiar PyTorch tools.
Also reported by 2 other sources
- vLLM releases a TorchTPU backend, enabling vLLM to run on Google Cloud TPUsvLLM Official Blog · 27 days ago
- RadixArk, Google, and Meta jointly open-source SGLang-TorchTPU, enabling SGLang to natively provide production-grade PyTorch inference on TPUsLMSYS Blog · 27 days ago
Source: Google Developers Blog · developers.googleblog.com
Event follow-up1
View all event coverage- Google released native TorchTPU inference engines for vLLM and SGLangGoogle Developers Blog · 27 days ago