AWS has launched Trainium3 UltraServers, its latest custom AI chip platform that packs up to 144 chips into a single integrated system and delivers 4.4 times more compute performance than Trainium2. It’s AWS’s most aggressive push yet to convince organisations they don’t need NVIDIA GPUs for AI training and inference, and the company is backing it with claims of 40% better energy efficiency and up to 50% lower costs.
Trainium3 is built on 3nm process technology and designed specifically for large-scale AI workloads. The chip delivers up to 362 FP8 petaflops per UltraServer and supports 4 times lower latency than Trainium2, which AWS says enables faster training for large models and more responsive inference for production applications. In testing with OpenAI’s GPT-OSS model, AWS claims Trainium3 delivers 3 times higher throughput per chip and 4 times faster response times than Trainium2.
The performance improvements come from architectural changes including optimised interconnects, enhanced memory systems, and better data movement between chips. AWS has also upgraded networking infrastructure with NeuronSwitch-v1, which doubles bandwidth within each UltraServer, and enhanced Neuron Fabric networking that reduces communication delays between chips to under 10 microseconds. These improvements matter because distributed AI workloads are limited by how quickly data can move between processors, not just raw compute power.
For organisations that need to scale beyond a single UltraServer, AWS offers EC2 UltraClusters 3.0, which can connect thousands of UltraServers containing up to 1 million Trainium chips. That’s 10 times the scale of the previous generation and enough capacity to train frontier models on trillion-token datasets. It’s also significantly larger than most organisations will ever need, but AWS is clearly positioning Trainium as viable infrastructure for the largest AI projects.
Customers including Anthropic, Karakuri, Metagenomics, Neto.ai, Ricoh, and Splashmusic are already using Trainium and reporting cost savings of up to 50% compared to alternatives. Amazon Bedrock is serving production workloads on Trainium3, which is a strong signal that AWS trusts its own chips for critical services. Decart, an AI lab focused on generative video, is achieving 4 times faster frame generation at half the cost of GPUs using Trainium3.
AWS’s collaboration with Anthropic is particularly telling. Project Rainier connected more than 500,000 Trainium2 chips into the world’s largest AI compute cluster, five times larger than the infrastructure used to train Anthropic’s previous models. Trainium3 builds on that foundation, and AWS is clearly betting that custom silicon can match or exceed NVIDIA’s performance at lower cost.
AWS’s roadmap reveals its longer-term ambitions. Trainium4 is already in development with at least 6 times the FP4 processing performance, 3 times the FP8 performance, and 4 times more memory bandwidth than Trainium3. It’s also being designed to support NVIDIA NVLink Fusion interconnect technology, which means Trainium4, Graviton, and EFA will work together seamlessly within common MGX racks. That integration suggests AWS is hedging its bets, offering customers flexibility to mix Trainium and NVIDIA GPUs in the same infrastructure.
For South African organisations evaluating AI infrastructure, Trainium3 is worth considering if you’re already on AWS and running large-scale training or inference workloads. The cost savings are compelling, and AWS’s software stack has matured to the point where Trainium is a realistic alternative to GPUs for many use cases. The downside is that Trainium only works on AWS, which means deeper platform lock-in.
Trainium’s real test is whether it can match NVIDIA’s ecosystem. NVIDIA’s advantage isn’t just hardware; it’s software, tooling, and a decade of developer familiarity. AWS is making progress, but it’s not clear whether Trainium will ever be as flexible or widely supported as NVIDIA’s platforms.