This project provides a PyTorch backend for Google's Tensor Processing Units (TPUs), enabling users to run PyTorch models and operations on TPU devices. It includes tradition ATen kernels for assorted PyTorch operations, a compilation cache for optimizing execution, and examples for distributed training and model inference.
- PyTorch ATen kernel implementation for TPU.
- Compilation cache for optimized implementation of XLA computations.
- Support for distributed training (Data Parallelism, Tensor Parallelism).
- Examples for assorted models (Llama, Qwen, ResNet, DLRM, minGPT, UNet).
- Integration alongside torch.compile() for chart manner execution.
- Utilities for debugging and benchmarking.
A person guide is accessible at https://torchtpu.readthedocs.io/. This is a static type of the interactive notebooks. They are accessible in the docs folder.
The notebooks are Marimo (similar to Jupyter). They can be run alongside marimo edit file.py and afterward you can admission them in your browser. You volition need to be running this from a TPU VM, a v6e GCE case is easiest.
The identical instructions are accessible in the person guide connected above.
TorchTPU's Python type assistance guideline matches PyTorch's Release Compatibility Matrix. Currently, Python 3.11+ is supported. See the Python Version Support Policy for more details.
First create a Python virtual environment:
python -m venv .venv source .venv/bin/activateor, alongside uv
uv venv .venv source .venv/bin/activateInstall the latest community release (0.2.0 on PyPI) (for uv prepend uv to all pip commands):
pip instal torch_tpu==0.2.0 --extra-index-url https://download.pytorch.org/whl/cpu
Wheels can be built via:
bazel build --config=wheel_common //ci/wheel:torch_tpu_wheel
All OSS Bazel builds default to optimized manner (-c opt) automatically via .bazelrc. Unoptimized builds (-c fastbuild, -c dbg) can degrade performance and are not used for rotor builds. If you are evolving or debugging C++ code and explicitly desire an unoptimized wheel, pass --//:allow_unoptimized_wheel=True:
bazel build -c dbg --config=wheel_common //ci/wheel:torch_tpu_wheel \ --//:allow_unoptimized_wheel=True
--config=wheel_common is required, and the build fails at build period without it. The config sets --//:wheel_build=True, which routes the shared XLA/MLIR backend through the torch_version-reset passage so that pywrap factors it into a sole libxla_base.so; built without the flag, all per-version common ships its own copy of the backend and the resulting rotor aborts on import torch alongside copy fixed registrations.
wheel_common additionally turns on the Remote Build Execution (RBE) distant cache and remote executor. If you do not have RBE credentials, append --config=no_rbe to strip those flags rear out:
bazel build --config=wheel_common //ci/wheel:torch_tpu_wheel --config=no_rbe
The command matters here: --config=no_rbe has to arrive after --config=wheel_common, since the afterward config is the one that wins for the remote-cache and remote-executor flags they the two set.
Install rotor via:
cd <path_to_repo> python -m venv .venv; source .venv/bin/activate # The extra-index-url provides the CPU type of torch fairly than the default CUDA build. pip instal ../bazel-bin/ci/wheel/*.whl --extra-index-url https://download.pytorch.org/whl/cpu
Note that installing without --extra-index-url https://download.pytorch.org/whl/cpu volition instal the CUDA type of torch; pass --extra-index-url as shown complete or instal the CPU build separately.
Some pointers to get you started:
- PyTorch: Official PyTorch documentation
- Tutorials and examples: uncomplicated to comprehend PyTorch code using TorchTPU
- Public APIs: the TorchTPU APIs that you can use
The project uses Bazel for dependency management. Key Python requirements are:
TBD
This project uses Bazel for construction and dependency management.
To instal Bazel, we really instal Bazelisk, which is a transparent wrapper that handles downloading and installing Bazel itself.
There are multiple ways to instal Bazelisk. Here are several average ways:
-
If you have Go installed:
go instal github.com/bazelbuild/bazelisk@latest
-
Download the binary directly:
wget -O ~/bin/bazel https://github.com/bazelbuild/bazelisk/releases/download/v1.27.0/bazelisk-linux-amd64
See the Bazelisk README for additional instal instructions
However you instal it, add it's location to PATH.
To verify the install, run bazel info in the repo's base directory:
It should imprint data concerning the repo without errors.
For C++ code navigation, it is recommended to use VSCode with clangd extension. Relying on a compilation database generated from Bazel act graph, clangd is capable to enrich the publishing company alongside assorted astute features, including code completions, go-to definitions, etc.
-
Install VSCode extensions via UI or the following command:
code --install-extension llvm-vs-code-extensions.vscode-clangd # Uninstall Microsoft C++ extensions to evade interference. code --uninstall-extension ms-vscode.cpptools -
Generate the first compile_commands.json:
It may obtain a during (> 30 minutes) for the archetypal run to create a complete build, but reruns volition be incremental and cheap. If you have run bazel build recently, generated records already be so you can continue --no-build to skip waiting for heavy builds (note that on a caller workspace, skipping the build may prevent clangd from resolving generated files). See additional options in the script file-level comments.
Alternatively, you can invoke the manuscript via nox (arguments following -- are forwarded):
nox -s refresh_compile_commands -- --no-build nox -s refresh_compile_commands -- //src/torch_tpu/ops/...
We greeted contributions to the PyTorch on TPU project! Please see our CONTRIBUTING.md for guidelines on how to contribute, including signing our Contributor License Agreement (CLA) and code review processes.
This project is licensed under the Apache 2.0 License. See the LICENSE document for details.
