Research Finder
Find by Keyword
Is Disaggregated Inference AWS's Contrarian Infrastructure Bet?
Trainium handles prefill, Cerebras CS-3 handles decode, EFA handles the seam, and NVIDIA gets to handle the competitive anxiety
03/16/2026
Key Highlights
- AWS and Cerebras announced a collaboration deploying Cerebras CS-3 wafer-scale systems inside AWS data centers, with inference delivery exclusively through Amazon Bedrock, launching in the coming months.
- The architecture breaks inference into two stages: AWS Trainium handling compute-intensive prefill, and the Cerebras CS-3 handling memory-bandwidth-intensive decode; the stages connect via Amazon's Elastic Fabric Adapter networking.
- Cerebras and AWS aim at agentic coding as a primary use case, the companies arguing that because agentic workloads generate ~15X tokens per query than conversational AI, inference speed becomes a first-order constraint.
- AWS becomes the first cloud provider to deploy Cerebras's disaggregated inference solution, with planned support for leading open-source LLMs and Amazon Nova models on the combined hardware stack.
- A caveat, this disaggregated architecture is workload-specific by design, with Cerebras acknowledging that many customers will need access to both aggregated and disaggregated configurations depending on prefill-to-decode ratios in their workloads.
The News
AWS and Cerebras Systems announced a collaboration to deliver what the two companies describe as the fastest AI inference available on a public cloud, with deployment through Amazon Bedrock expected within the next several months. The solution is architected around inference disaggregation, pairing AWS Trainium-powered servers for the compute-intensive prefill stage with Cerebras CS-3 systems for the memory-bandwidth-intensive decode stage, connected by Amazon's Elastic Fabric Adapter high-speed networking and operating within the AWS Nitro System security boundary. AWS is positioned as the first and exclusive cloud provider for Cerebras's disaggregated inference configuration, with plans to support leading open-source LLMs and Amazon Nova models on the combined stack. The announcement arrives as AWS simultaneously carries significant Trainium commitments from both Anthropic, named as AWS's primary training partner, and OpenAI, which has committed to consuming two gigawatts of Trainium capacity through AWS infrastructure.
Analyst Take
This announcement seems a straightforward acceleration of AWS's beyond-NVIDIA hedge. For years, AWS investments in their custom Trainium and Inferentia silicon allowed the company to offer cost-efficiency alternatives to GPU procurement. This Cerebras partnership moves beyond this to something more deliberate: AWS is placing wafer-scale silicon, Cerebras's most unorthodox architecture, directly inside its own data centers. Inside the wire, not on a partner's managed infrastructure, not in a co-location arrangement, but in AWS facilities under the Nitro security model. For a hyperscaler that controls margin by controlling hardware, that is a significant give on the margin side. Our observation is that this move suggests that AWS infrastructure leadership may be concluding that the GPU memory wall for decode-phase inference is a structural problem, and throwing more money at NVIDIA Blackwell hardware can't solve it alone. The contrarian read is acknowledged right in the announcement: AWS and Cerebras hedge the deployment scope in a way that the headline does not: disaggregation is ideal for large and stable workloads. Many enterprise inference workloads are neither.
What Was Announced
The technical core of the announcement is a disaggregation architecture that breaks LLM inference into two computationally distinct stages: prefill and decode. Prefill, the stage in which the model processes the input prompt to build a key-value (KV) cache, is both computationally intensive and natively parallel. Decode, the sequential token-by-token generation stage, is computationally light but memory-bandwidth intensive, and it typically accounts for the majority of end-to-end inference latency in long-output scenarios. The announced offering is Trainium-plus-CS-3, architected with Trainium exclusively handling prefill computation, then tossing the KV cache to the Cerebras CS-3 via EFA networking, and the WSE-3 wafer-scale engine exclusively handles decode. According to Cerebras performance claims, the CS-3 aims to deliver thousands of output tokens per second, a step change from hundreds on GPU-based systems. Accomplishing this by stashing all model weights on-chip in SRAM, thus achieving memory bandwidth that Cerebras claims is thousands of times greater than the fastest GPU's HBM. The combined disaggregated configuration targets what the companies describe as five times more high-speed token capacity in the same hardware footprint compared to aggregated configurations.
The Cerebras CS-3 is built around TSMC’s 5nm Wafer Scale Engine 3 (WSE-3) processor. The architecture is designed to eliminate off-chip communication for intra-wafer operations, targeting the primary source of inference latency in GPU cluster configurations that require model-parallel distribution across multiple chips. The entire solution runs within the AWS Nitro System, meaning CS-3 instances and Trainium-powered instances are designed to operate with the same security isolation and operational consistency that AWS customers expect from native EC2 workloads. Later in 2026, the two companies plan to support Amazon Nova model families on this stack, which would make Nova models available at inference speeds not currently achievable on GPU-based Bedrock endpoints.
Market Analysis
The broader context here is that inference disaggregation has, by late 2025, become mainstream in software. NVIDIA's Dynamo inference framework, announced at GTC 2025, treats prefill and decode workers as first-class citizens and supports disaggregated serving across TensorRT-LLM, vLLM, and SGLang. Academic implementations such as DistServe from UCSD's Hao AI Lab, which introduced the concept in mid-2024, saw rapid adoption across the industry through 2025, becoming what researchers describe as the default playbook for production-grade LLM serving. What AWS and Cerebras are doing is not software disaggregation. It is hardware disaggregation at the silicon level, routing prefill and decode to fundamentally different processor architectures, connected by a purpose-built hyperscale network fabric. This is architecturally more aggressive than anything NVIDIA's Dynamo achieves, because Dynamo disaggregates across homogeneous GPU pools. The AWS-Cerebras solution disaggregates across two entirely different compute paradigms.
The modality here is agentic AI inference, rapidly accelerating in 2026, particularly long-output, high-token-count workloads generated by coding agents and autonomous reasoning models. Cerebras observes that agentic coding applications generate approximately 15 times more output tokens per query than conversational AI, and the bottleneck in those workflows is decode throughput. As enterprise agentic deployments scale, the cost-per-token implications of disaggregated versus GPU-only inference has to emerge from something buried in DevOps to a line item that CFOs and procurement teams must scrutinize directly. The WSE-3's SRAM-based memory architecture is arguably better suited to this workload profile than HBM-equipped GPUs, where memory bandwidth per unit cost remains constrained by packaging economics as well as availability of scarce HBM allocations. NVIDIA's response, anticipated at GTC 2026 this week, is expected to center on the Vera Rubin platform and NVLink Fusion, which aims to support more flexible hardware disaggregation configurations. That NVIDIA is reportedly fast-tracking its own form of heterogeneous interconnect suggests the competitive thesis of the AWS-Cerebras announcement is being taken seriously at Santa Clara. It is also a case for why NVIDIA is accelerating its own product map to an annual revision pace.
From a customer-impact standpoint, enterprises that standardized on Amazon Bedrock for model access now have a zero-friction path to wafer-scale inference performance. Frictionless in that those enterprises don’t need a new cloud vendor, new API contract, and new identity or access management configuration. This is a meaningful moat against competitors from a customer procurement argument for AWS accounts that are already running Bedrock production AI workloads. Outside AWS, Oracle's enterprise customer base, which skews toward large-scale transactional and data-intensive workloads, is particularly well-positioned to benefit from inference economics that favor high-throughput, long-output agentic scenarios. OCI's track record of deploying specialized, high-performance compute configurations demonstrates credibility as a future participant in this architectural shift. The friction reduction alone, independent of the performance argument, justifies consideration for organizations managing large agentic coding deployments or real-time AI applications where inter-token latency is a user experience constraint.
Looking Ahead
HyperFRAME will be monitoring whether the AWS-Cerebras disaggregated architecture moves beyond workload-specific optimization towards a general-purpose inference tier within Amazon Bedrock. The performance case for decode-phase specialization is well-understood. The operational case is going to be the more challenging action: how AWS routes workloads between aggregated and disaggregated configurations at runtime, how it prices the two modes differently, and how it handles the KV-cache transfer latency across EFA under variable load. A disaggregated architecture that is faster at peak but introduces routing overhead under mixed workloads might just erode the user experience gains the announcement promises. We will also be watching how Anthropic, as AWS's named primary training partner and a major Bedrock model provider, responds to the prospect of Claude model families running on CS-3 decode hardware. That specific combination has not been announced, but it is a logical extension of the roadmap. The outcome of the Cerebras IPO, anticipated in Q2 2026, will determine whether this architecture scales beyond AWS to additional hyperscalers or remains an exclusive.
Stephen Sopko | Analyst-in-Residence – Semiconductors & Deep Tech
Stephen Sopko is an Analyst-in-Residence specializing in semiconductors and the deep technologies powering today’s innovation ecosystem. With decades of executive experience spanning Fortune 100, government, and startups, he provides actionable insights by connecting market trends and cutting-edge technologies to business outcomes.
Stephen’s expertise in analyzing the entire buyer’s journey, from technology acquisition to implementation, was refined during his tenure as co-founder and COO of Palisade Compliance, where he helped Fortune 500 clients optimize technology investments. His ability to identify opportunities at the intersection of semiconductors, emerging technologies, and enterprise needs makes him a sought-after advisor to stakeholders navigating complex decisions.