Trending
Hot topicsCalmed down

Google releases a TorchTitan TPU fork

2 Articles2 Reporting sourcesUpdated 28 days ago

Learn about this first

Google has released a TPU fork of TorchTitan, hosted at github.com/google-pytorch/torchtitan. Adapted from Meta’s TorchTitan implementation, the fork enables PyTorch models to train on Google Cloud TPUs. Alongside the native TorchTPU backend, it provides SimpleFSDP, Graph Trainer, Pallas kernels, and an optimization playbook.

Google says that, for most models, achieving good MFU on TPUs mainly involves a series of configuration changes to an existing recipe, rather than writing new code.

AI-generated from reports · updated 27 days ago

Latest developments

Google Releases TorchTitan TPU Fork and Provides a Model Optimization Playbook

Event progress1

  1. Introducing TorchTitan on TPU integration and optimization methods

    Google Developers Blog: Google releases a TPU fork of TorchTitan and a model optimization playbook

Finished reading the news? Let’s talk about your business.

Leave your contact details, and I’ll get in touch with you personally.

WECHAT

Add me on WeChat

Press and hold or select the WeChat ID below to copy it.