How to Choose Between GPU Clusters and Custom Silicon for AI Training

How to Choose Between GPU Clusters and Custom Silicon for AI Training

Google's eighth-generation TPU announcement in April 2026 split the product line in two for the first time: TPU 8t for training, TPU 8i for inference. The same week, Cerebras filed a public S-1 for a twenty-three-billion-dollar IPO backed by a twenty-billion-dollar OpenAI deal. Both events point to the same inflection: the era of "one chip for everything" is ending. Training and inference workloads have diverged enough that a unified architecture is now a compromise, not a convenience.

This guide is for builders making compute decisions in that new landscape. It compares GPU clusters against custom silicon across the dimensions that actually matter: cost, flexibility, lock-in, and operational reality.

The Split: Why Unified Silicon Is Dying

Google's move is the clearest signal. TPU 8t scales to 9,600 chips per superpod with 121 ExaFlops of training throughput, a 3D torus topology, and a claimed 2.7 times training price-performance improvement over the previous Ironwood generation. It is built for scale-out training of massive models.

TPU 8i is a different chip entirely. It uses a Boardfly topology that caps at seven hops versus sixteen in a torus, carries three times the on-chip SRAM, and includes a Collectives Acceleration Engine that reduces latency fivefold. It is optimized for low-latency serving of mixture-of-experts and reasoning models, with the KV cache kept on-chip. Google claims eighty percent inference price-performance improvement and double the performance-per-watt.

The strategic read is not the raw specs. It is that Google, with full vertical control over its stack, concluded that training and inference are different enough to warrant different silicon. If Google thinks so, builders renting compute should pay attention.

Background: Google co-designed TPU 8t/8i with DeepMind, and Gemini runs on TPUs. The split-architecture decision was not made in a vacuum. It reflects real workload divergence observed at hyperscale. Nvidia has not yet split its product line this way, though the Vera Rubin architecture is expected to address the same tension.

When a company that owns the entire stack from chip to model decides one chip is no longer enough, the rest of the market should question whether their one-size-fits-all approach still fits.

GPU Clusters: What You Actually Get

Nvidia GPUs remain the default for a reason. The ecosystem is unmatched: PyTorch, JAX, CUDA, and thousands of optimization libraries assume you are running on Nvidia silicon. If your team knows CUDA, debugging is faster. If you need a specific kernel or a community-optimized training recipe, it probably exists for GPUs first and custom silicon second.

The business model is flexible. You can rent A100s or H100s by the hour from any major cloud provider. You can buy them and depreciate them over three years. You can move workloads between providers with minimal friction because the software layer is portable. For teams that value flexibility over peak efficiency, this is the right default.

• The hidden cost is talent and tooling.* Every ML engineer knows CUDA. Fewer know TPU's XLA compiler or Cerebras's software stack. When you hit an obscure numerical instability at batch size 32K, the Stack Overflow thread probably exists for GPUs. For custom silicon, you might be filing a support ticket and waiting. The time saved on compute costs can be lost on debugging time, especially for teams without dedicated infrastructure engineers.

The tradeoff is cost at scale. GPU cloud pricing has crashed roughly eighty percent from peak levels, but for sustained training runs on thousands of chips, the per-FLOP economics of custom silicon still win. The question is whether your workload is large enough and consistent enough for that win to matter. A team training a 7B parameter model on a few dozen GPUs will not see the TPU advantage. A team training a frontier-scale model on thousands of chips will.

Custom Silicon: The Case for Specialization

Google's TPUs offer the clearest alternative. The 2.7 times training price-performance claim and 80 percent inference improvement are significant if they hold up in independent benchmarks. Both chips use Google's Axion ARM-based CPU hosts, ditching x86 entirely. Native PyTorch support is in preview, which reduces the migration friction for teams already using standard frameworks.

Cerebras represents a different bet: wafer-scale integration. The Cerebras WSE-3 chip is a single wafer-scale engine designed for inference decode-phase latency, where Cerebras claims it outperforms Nvidia. The S-1 filing reveals $510 million in revenue for 2025, up seventy-six percent year-over-year, with a twenty-billion-dollar-plus Master Relationship Agreement with OpenAI for 750 megawatts of inference compute.

The Cerebras story has caveats. Eighty-six percent of 2025 revenue came from two UAE entities. GAAP profitability of $237.8 million is entirely a one-time $363.3 million non-cash accounting gain from restructuring a liability. The non-GAAP net loss actually widened to $75.7 million. The company is also transforming from a hardware shipper to a hyperscale cloud operator, a transition it has never executed at scale.

The AWS partnership is binding but narrow: disaggregated inference where AWS Trainium handles prefill and Cerebras handles decode, within Amazon Bedrock. It validates the architecture but does not prove Cerebras can run a cloud business.

The custom silicon bet is real, but the commercial validation is thinner than the press releases suggest. Read the S-1 footnotes before you read the headlines.

The Tradeoffs That Matter

• Lock-in.* GPUs run everywhere. TPUs run on Google Cloud. Cerebras runs on Cerebras Cloud or AWS Bedrock disaggregated inference. If you build on custom silicon, your portability is constrained. For startups that might need to migrate providers or sell their training infrastructure, this matters more than raw FLOPS.

• Ecosystem maturity.* The debugging and profiling tooling around TPUs and Cerebras is years behind Nvidia's stack. When a training run fails at 3 AM, the speed of diagnosis often matters more than the theoretical throughput of the chip.

• Pricing models.* GPU rentals are commodity-priced and transparent. TPU pricing is bundled into Google Cloud contracts with volume discounts that are hard to benchmark. Cerebras pricing is opaque and evolving as the company shifts from selling chips to selling capacity.