Google's eighth-generation Cloud TPU for post-training and low-latency inference of large MoE models.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
TPU 8i is one of the eighth-generation Tensor Processing Units from Google, introduced at Google Cloud Next 2026 alongside TPU 8t, which targets large-scale training. TPU 8i is optimized for post-training and real-time reasoning inference. Google states an 80% performance-per-dollar improvement over previous generations for low-latency inference of large mixture-of-experts models. A YouTube summary of the announcement cites 288 GB of HBM, nearly 400 MB of on-chip SRAM, a new "Boardfly" network architecture and up to 2x performance per watt; these figures come from a creator's recap, not from the official page text.\n\nGoogle has since expanded the eighth generation with TPU 8i Max, which has double the compute, SRAM and HBM bandwidth of TPU 8i and fits twice the silicon in the existing rack footprint, and TPU 8tx, aimed at secure and confidential inference. The TPU software stack includes JAX, PyTorch via TorchTPU, OpenXLA, MaxText, Tunix and vLLM, with deployment managed through GKE and AI Hypercomputer.\n\nTPU 8i is accessed as a rented accelerator through Google Cloud, not as hardware you can buy. Pricing for the eighth generation was not listed on the pricing page provided; that page lists Ironwood, Trillium and earlier generations per chip-hour. Regional availability of 8i is not specified in the provided material.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Eighth-generation TPU built for inference, with 288 GB of high-bandwidth memory and almost 400 MB of on-chip SRAM. It uses the new Boardfly network architecture and delivers 80% better performance than the previous generation. The 8th-gen TPUs offer up to 2x better performance per watt.
Closest neighbours by meaning, not just by category.
Hardware · Google
Google's eighth-generation TPU for large-scale AI pre-training, scaling to 9,600 chips per superpod.
Hardware · Google
Google's custom AI accelerators, offered through Google Cloud for training and inference of machine learning models.
Library
Open-source streaming MoE inference framework for running large sparse models on laptops, phones and single-GPU PCs.
Hardware · OpenAI
Custom AI inference chip announced by OpenAI and Broadcom, built to serve ChatGPT-style workloads faster and cheaper.
API · OpenAI
OpenAI high-speed inference tier that serves model output far faster than standard, reportedly around 14x.
Hardware · Groq
Inference provider built on its custom LPU chips, offering fast LLM inference via cloud API (GroqCloud) and dedicated capacity.
