Google Open-Sources TorchTPU, Putting Native PyTorch on Its TPUs

PUBLISHER ARTICLE PREVIEW

From the original article

Google announced on October 20, 2026, at PyTorch Conference 2026 that TorchTPU, its native PyTorch stack for Tensor Processing Units (TPUs), is now available as open source on GitHub, the company said on the Google Cloud blog. The software was co-developed with the PyTorch team at Meta and is designed to run existing PyTorch workloads on TPUs with minimal code changes.

With the release, TPUs join GPUs and CPUs as a native option for PyTorch, Google said, spanning training, serving, and deployment through tools including vLLM, SGLang, llm-d, TorchTitan, Ray, TorchRec, Hugging Face, and LiteRT. Developers can debug models in eager mode and then move to production-level performance through torch.compile to optimize TPU utilization and reduce memory overhead.

Jana van Greunen, Director of PyTorch Engineering at Meta, said in the announcement that TorchTPU allows Meta to co-develop hardware flexibly with its core machine learning systems team while supporting major LLM, Ads, and Recommendation systems workloads. “It enables us to deploy our PyTorch workloads to Google TPUs, provide direct feedback to the engineering teams, and build the ecosystem flexibility everyone needs,” she said.

Google partnered with Hugging Face so that open-weight models on the Hugging Face Hub run natively on TorchTPU. In Transformers v5, deploying a model to TPUs is done by passing device_map="tpu", and the Diffusers library runs natively on TPUs with support for eager mode, torch.compile, and tensor parallelism across multiple accelerators. Google said it also built dedicated CI/CD pipelines for continuous nightly testing of the Hugging Face Transformers suite on TPU infrastructure.

Jeff Boudier, VP of Product at Hugging Face, said the collaboration puts the platform’s hardware-choice commitment into practice.

For serving, the native PyTorch vLLM backend on TorchTPU was developed in partnership with Inferact. Google said it supports continuous batching, quantization, and multi-chip parallelism on TPU hardware using custom Pallas kernels, while preserving upstream worker models, schedulers, and API servers so existing inference stacks require no changes. Simon Mo, CEO and co-founder of Inferact, said the integration brings first-class vLLM support to TPUs.

A native PyTorch backend for SGLang, developed with RadixArk, targets structured and agentic workloads.