VAST Data and AMD Extend Enterprise AI Inference Beyond GPU Memory

This announcement advances VAST's evolution from a high-performance data platform to an AI Operating System. Existing capabilities built on the DASE architecture, including Context Memory, unified data services, metadata management, streaming, databases, and AI orchestration, now extend directly to persistent inference context.

LLM inference depends on KV cache, which preserves computed attention state between requests. VAST reports that a 128,000-token context window generates 20 GB to 50 GB of KV cache per user depending on model and data type. Aggregate requirements across concurrent sessions exceed available GPU memory, forcing recomputation of previously processed context.

The benchmark tested GPT-OSS 120B at 120,000-token input length on a single MI355X with tensor parallelism of one, running vLLM 0.22.1, LMCache, and ROCm, connected over 800 Gbps NFSv3 with RDMA and multipathing. VAST reported up to 9x TTFT improvement and 9.7x throughput against a baseline requiring recomputation at each decode turn.

VAST tested three configurations: GPU with LMCache offloading to VAST storage, GPU with LMCache offloading to local host RAM, and GPU without cache. Local host RAM delivered the lowest P99 TTFT at every concurrency level above 20 requests, reaching 204 seconds at 300 concurrent requests against 1,918 seconds for VAST and 1,977 seconds for the no-cache baseline. The reported 9x advantage occurs at lower concurrency, where the performance curves diverge. At higher concurrency, VAST closely tracks the no-cache baseline.

Local RAM offload is bounded by host memory; the tested LMCache configuration capped host cache at 300 GB and served a single node. DASE provides a shared, persistent context tier accessible across a global namespace, with capacity independent of host memory. Any compute node draws from a common context pool without duplicating context on individual hosts. That distinction defines the deployment case for external KV cache.

Native lifecycle management reinforces VAST's architectural differentiation. Cached context contains user data subject to retention and deletion requirements. VAST applies existing lifecycle policies to expire and delete KV cache, extending metadata services, security controls, multi-tenancy, and namespace management to inference state. Enterprises deploying persistent cache inherit compliance mechanisms already governing their data.

In our view, the collaboration strengthens AMD's enterprise inference position. NVIDIA supplies comparable cache tiering natively through Dynamo and NIXL within a CUDA ecosystem. AMD closes that capability through partnership. The supporting cloud roster, including Core42, Crusoe, Vultr, TensorWave, and 5C, indicates deployment traction. TensorWave describes itself as an AMD-centric AI cloud. Vultr is notable because it positions the VAST-AMD architecture as part of a validated enterprise AI stack spanning infrastructure, accelerated compute, enterprise data, cloud operations, and AI software. Vultr runs a deliberate two-lane GPU strategy, with AMD Instinct and NVIDIA deployments within the same cloud platform. That dual footing positions Vultr to provide practical operational comparisons between AMD-native and NVIDIA-based inference architectures, rather than relying on experience with a single GPU ecosystem.