Can Scality’s Object Storage Improve AI Inference Performance and Economics?

Key Highlights

• Scality introduced AI Inference Factory, integrating open-weight models, disaggregated inference serving, a control plane, and Scality AI Data Infrastructure (ADI).
• Scality ADI provides shared KV-cache storage, allowing GPUs to retrieve previously computed context and reduce repeated computation.
• An optimized connector and RDMA data path support low-latency retrieval, with Scality reporting up to 72× faster access than recomputation in laboratory testing.
• Enterprise adoption will depend on the performance and economics of the integrated storage, networking, and inference stack, alongside security, governance, and resilience.

The News

Scality introduced AI Inference Factory, an open-code software stack for deploying and operating AI inference on enterprise-owned infrastructure. The offering combines validated open-weight models, a disaggregated inference-serving layer built on vLLM and NVIDIA Dynamo, management services, and Scality ADI for model data and inference state. Scality provides the integrated stack as a software license or fully managed service, targeting enterprises, government agencies, and neocloud providers. For more information, read Scality’s official press release.

Analyst Take

AI inference is placing new demands on enterprise storage. Large language models generate key-value (KV) caches that preserve computational state for reuse, reducing repeated processing. Long-context reasoning and agentic workloads can generate substantial cache volumes, while expensive GPU high-bandwidth memory (HBM) limits how much state can remain available to individual computing resources.

HyperFRAME Research has previously examined KV cache as an extension of established memory and storage hierarchies. As reusable inference state moves from GPU memory into host DRAM, NVMe, and shared storage, data placement and retrieval become part of inference scheduling and economics. The decision depends on whether retrieving previously computed context consumes fewer resources than recreating it on the GPU.

Scality addresses this architectural problem through object storage. ADI stores reusable KV-cache blocks as objects, while its optimized AI Connector and RDMA networking move data into GPU memory. Scality's engineering work extends beyond object storage. Changes to the serving connector and data transfer path help reduce retrieval latency, demonstrating why storage and inference software must be evaluated together.

Testing the Architecture Against Competing Approaches

Scality's August 2026 laboratory tests used one eight-GPU server, five storage servers, and four 100 Gbit/s RDMA links. The company reported a warm time-to-first-token (TTFT) of 166 milliseconds for a 14,000-token context, compared with 102 milliseconds from host DRAM, with retrieval 14 times faster than recomputation. A separate GLM-5.2 test involving approximately 439,000 tokens achieved a 72-fold improvement over recomputation. Using eight vLLM instances, Scality maintained TTFT between 355 and 373 milliseconds as shared cache capacity expanded from 69 GB to 8 TB. These vendor-reported NVMe results demonstrate faster retrieval than recomputation under the tested conditions. Comparable benchmarks against competing storage implementations remain unavailable.

The economic benefit depends on workload characteristics. Applications that repeatedly reuse large contexts can avoid significant GPU recomputation. Short prompts, unique contexts, and low cache-hit rates offer fewer opportunities for savings.

HyperFRAME Research Lens: State of the Enterprise AI Stack (Q3 2026) found that just 27% of respondents had reached AI production and met their original objectives or ROI expectations. Improving accelerator productivity can contribute to better AI economics when the complete infrastructure justifies its cost. The same research found that 72% of infrastructure and operations respondents cited security and compliance as a challenge to scaling AI. These concerns extend to shared inference state, where sensitive information, access controls, tenant isolation, and infrastructure dependencies require attention.

MinIO's MemKV, layered alongside AIStor, provides a dedicated NVMe-based KV-cache tier using RDMA and NVIDIA NIXL. Its architecture bypasses conventional object and file protocols in the active transfer path, separating context memory from persistent S3 object storage. MinIO and Cloudian participate in NVIDIA's CMX ecosystem, while Scality integrates shared KV-cache storage into ADI through an optimized retrieval path. VAST, WEKA, and DDN offer additional architectural approaches to managing inference state.

Scality is extending its responsibilities beyond enterprise object storage into the integration and support of an AI inference software stack. The company states that it maintains no Scality forks of any component and has submitted its vLLM connector improvements upstream. Maintaining compatibility among serving engines, models, networking, and storage will be an important test of the offering's long-term value.

What Was Announced

Management services handle authentication, routing, scheduling, and metering across the vLLM and NVIDIA Dynamo serving layer, which coordinates separate prefill and decode operations. Scality ADI maintains reusable KV-cache blocks that GPU servers retrieve through its optimized AI Connector and RDMA networking. ADI also stores model weights and enterprise datasets. In laboratory testing, Scality loaded approximately 55 GB of Gemma-3 27B model weights in 1.9 seconds, nearly ten times faster than its tested local NVMe configuration.

The stack supports models including Mistral, Gemma, Qwen, and DeepSeek on standard servers from Dell, HPE, Lenovo, and Supermicro. Scality provides access to the code it ships. The company states that ADI supports flash, HDD, and optional tape within a common storage namespace, addressing different performance, capacity, and retention requirements. Scality's published KV-cache measurements used NVMe storage, and its documentation leaves protection policies for the fastest KV-cache tier unspecified.

Looking Ahead

Long-context workloads with frequent reuse offer the greatest opportunity to avoid repeated GPU computation. Context length, concurrency, cache-hit rates, and network performance all influence the return. Enterprises should measure latency, GPU utilization, and cost per token using the full inference configuration. Results from independent testing and Scality's HDD tier would help establish the economics beyond its NVMe-based laboratory tests.

Shared KV cache also brings sensitive inference data into the storage environment. Prompts and their derived computational state may contain proprietary information, making tenant isolation, access controls, and cache expiration important parts of the architecture. Security policies need to follow that information as it moves between GPUs and storage.

Losing KV cache typically means recomputing context, which consumes GPU capacity and can delay responses. Losing a checkpoint, conversation history, or persistent agent state can have more serious consequences. Protection and recovery policies should account for the cost of rebuilding each type of state and its effect on service continuity. A failure in shared storage or networking can also affect multiple GPU servers. Scality is taking responsibility for an integrated inference stack whose components evolve independently. Customers should understand how updates are validated and who resolves problems involving the serving software, network, and storage. Vendor accountability contributes to the value of the offering.

Storage is becoming part of the inference memory hierarchy. Its business value comes from making expensive GPU capacity more productive, improving response times, and reducing the cost of delivering AI services. Scality's opportunity is to translate its laboratory performance into those results while providing the security and resilience enterprises expect.