Research Finder
Find by Keyword
VAST Brings Context Memory to Agentic AI: Why Inference Is Becoming a Data Platform Problem
A storage-centric look at how memory-centric inference architectures are reshaping AI infrastructure economics, governance, and scale.
01/07/25
Key Highlights
- AI inference is evolving into a continuous, stateful workload that depends on shared access to active working data.
- Memory-centric inference architectures focus on efficiency, predictability, and cost by reducing unnecessary data movement and duplication.
- VAST’s collaboration with NVIDIA reflects broader industry alignment around tighter integration between data platforms and accelerated computing.
- By 2026, storage and data platform teams are likely to play a more central role in AI inference infrastructure decisions.
The News
VAST Data announced new capabilities that extend its AI Operating System to support memory-centric inference architectures, developed in close collaboration with NVIDIA and designed to run natively on NVIDIA BlueField-4 DPUs.
The announcement positions inference context, the active working data used by AI models during execution, as a shared and persistent infrastructure resource. VAST describes this approach as foundational for agentic AI, long-running AI services, and large-scale production deployments where context continuity over time and across systems, efficiency, and predictability are required. For additional detail, read the VAST Data press release and the accompanying VAST Data blog.
Analyst Take
This announcement reflects a broader industry realization: AI inference is evolving from a stateless compute activity into a continuous, data-driven service. As inference workloads mature, performance, cost, and operational behavior are increasingly shaped by how systems manage active working data during execution.
For storage administrators and infrastructure leaders, the implications are practical and familiar. As organizations deploy agentic AI, meaning AI systems that execute multi-step tasks, retain state, and operate continuously, the data used during inference grows in size, persistence, and importance. Treating that data as transient application memory introduces inefficiency, unpredictability, and cost pressure at scale.
Across production environments, repeated reconstruction of inference context has emerged as a hidden source of latency and resource waste. These inefficiencies compound under concurrency, reducing GPU utilization and driving infrastructure expansion that does not always translate into better outcomes. As a result, attention is shifting toward inference architectures built around shared, active data that can be reused and coordinated across systems.
VAST’s approach, particularly in partnership with NVIDIA, reflects this system-level shift. By bringing data services closer to the inference data path and enabling shared access to context across hosts, the architecture addresses challenges that surface only when inference becomes sustained, multi-user, and operationally critical. This direction also aligns with the NVIDIA AI Factory framing, which treats AI infrastructure as an integrated production environment where compute, networking, and data platforms are designed together to support continuous, large-scale AI operations.
From a storage perspective, this progression echoes earlier transitions driven by virtualization and database platforms. Shared access to active datasets improved utilization, reduced duplication, and simplified operations. AI inference is now applying similar pressure to infrastructure, extending familiar storage principles into the inference runtime environment.
Operational considerations are equally important. As AI systems influence business processes and decision-making, organizations require predictability, visibility, and control. Elevating inference working data into the platform layer enables policy enforcement, isolation, and auditability, aligning AI infrastructure with enterprise operating expectations.
In my view, the significance of this announcement extends beyond a single implementation. It signals a structural change in how inference infrastructure is evaluated. Performance remains important, but efficiency, governance, and system-level behavior under sustained load are becoming decisive factors as AI moves deeper into enterprise operations.
What Was Announced
VAST Data announced new capabilities that extend its AI Operating System to support what it describes as memory-centric inference architectures, developed in close collaboration with NVIDIA and designed to run natively on NVIDIA BlueField-4 DPUs. The announcement positions inference context, the active working data used by AI models during execution, as a first-class infrastructure resource that is shared and persistent across systems.
At the center of the announcement is VAST’s ability to provide shared access to inference context across hosts, allowing AI inference services to reuse and reference active working data without repeatedly reconstructing it. This capability targets inference workloads defined by long-lived sessions, multi-step execution, and agent-driven workflows, where continuity of context directly influences performance, cost, and operational behavior. VAST frames inference memory as part of a globally accessible data platform that supports coordination across services and execution environments.
The solution is architected to run directly on NVIDIA BlueField-4 DPUs, placing data services closer to GPU execution paths and reducing the number of hops, copies, and control layers involved in serving inference data. By collapsing traditional storage and data services into the inference runtime environment, VAST seeks to minimize unnecessary data movement while maintaining the parallel access characteristics required for high-concurrency inference. This design aligns with NVIDIA’s broader direction toward tightly integrated, AI-native infrastructure spanning compute, networking, and data services.
The announcement also highlights enterprise operational requirements tied to large-scale inference. By elevating inference working data into the platform layer, VAST enables policy enforcement, isolation, and auditability for data used during inference. These capabilities support production environments where predictability, governance, and trust are essential as AI systems become embedded in business processes and operational decision-making.
Taken together, the announcement underscores VAST’s intent to extend its data platform deeper into the AI runtime environment. The collaboration with NVIDIA reflects a shared view that future AI infrastructure will be defined by how effectively systems manage active data during inference, alongside continued advances in model performance and acceleration.
Looking Ahead
Looking toward 2026, I believe memory-centric inference architectures are likely to move from early adoption into broader production use, particularly in environments running sustained, multi-user AI inference services. As this shift unfolds, storage and infrastructure teams will increasingly be asked to engage in architectural decisions that extend beyond traditional capacity planning.
Several patterns are becoming clearer. Active inference data will increasingly be treated as a managed resource, not an internal implementation detail. Storage and data platform teams will play a more direct role in AI platform design, even when they are not responsible for models or applications. Decisions about data locality, access paths, and lifecycle management will directly influence AI cost, predictability, and operational resilience.
At the same time, efficiency, predictability, and power consumption are gaining weight alongside performance. As organizations seek to control costs and operate AI systems at scale, architectures that reduce duplication, improve utilization, and simplify operations will gain favor. Inference is becoming a steady-state workload, and platforms will be evaluated on how well they behave under sustained load, not just peak benchmarks.
Governance expectations will continue to extend into inference workflows. As AI systems support regulated and mission-critical processes, enterprises increasingly expect visibility, policy enforcement, and auditability for data used during execution. Platforms that incorporate these capabilities natively will be better positioned for long-term adoption.
It is also important to recognize that memory-centric inference represents an architectural choice, not a universal prescription. Deep integration with accelerated computing ecosystems, including close alignment with partners such as NVIDIA, brings clear benefits in performance, efficiency, and ecosystem credibility. It also introduces design decisions that shape how platforms behave over time and should be evaluated deliberately as part of long-term infrastructure planning.
Organizations will need to consider how tightly inference data services are coupled to execution environments, how operational boundaries and failure domains are defined, and how platforms evolve as hardware, workloads, and deployment models change. These are architectural decisions, not necessarily compromises, and the right balance will vary based on workload mix, operational priorities, and desired flexibility over time.
In this context, memory-centric inference is best viewed as a set of design principles guiding platform selection. Data platforms that make these decisions transparent, define operational boundaries clearly, and support governance and control will be better positioned to support scalable, trusted AI services as inference becomes central to enterprise operations.
Don Gentile | Analyst-in-Residence -- Storage & Data Resiliency
Don Gentile brings three decades of experience turning complex enterprise technologies into clear, differentiated narratives that drive competitive relevance and market leadership. He has helped shape iconic infrastructure platforms including IBM z16 and z17 mainframes, HPE ProLiant servers, and HPE GreenLake — guiding strategies that connect technology innovation with customer needs and fast-moving market dynamics.
His current focus spans flash storage, storage area networking, hyperconverged infrastructure (HCI), software-defined storage (SDS), hybrid cloud storage, Ceph/open source, cyber resiliency, and emerging models for integrating AI workloads across storage and compute. By applying deep knowledge of infrastructure technologies with proven skills in positioning, content strategy, and thought leadership, Don helps vendors sharpen their story, differentiate their offerings, and achieve stronger competitive standing across business, media, and technical audiences.