DDN: Keep Your GPUS Working Productively and Profitably. You Get More Value From Your AI Investment

Eric Wright and Scott Ekstrom from DDN emphasize that DDN’s core goal is to ensure GPUs operate productively and profitably, allowing customers to extract maximum value from their AI data investments. They approach AI infrastructure with a “first principles” mindset, focusing on understanding actual business outcomes rather than just hero metrics. This involves deep dives into data skills, ML Ops, governance, and analytics to ensure systems are built to meet evolving needs for enterprise and sovereign customers, as well as service partners.

The presentation outlines key problems DDN solves, particularly related to GPU underutilization in training and checkpointing due to slow data movement. For shared clusters, they address the complexities of varied tenant profiles, priorities, and consumption patterns, which can lead to unpredictable job performance and nonlinear expansion costs. In inference, a critical focus is optimizing the KV (Key-Value) cache to avoid costly context recomputation. DDN employs two distinct data engines: Exascaler, which is robust for ingestion, data preparation, training, and fine-tuning, leveraging its HPC origins; and Infinia, specifically designed for inference, particularly for agentic and long-context edge inference, with a strong emphasis on KV cache optimization.

DDN’s strategy for performance and profitability centers on a shared KV cache tier, ensuring cross-node and cross-cluster cache coherency. They cite a case where using Infinia for KV cache reduced a critical turn time from nearly 32 seconds to under 4 seconds. The adaptive AI infrastructure supports optimized inference workflows by splitting pre-fill and decode operations, using zero-copy RDMA for efficient KV block movement between GPUs without CPU involvement. Crucially, multi-tenancy is built into the KV cache layer, ensuring isolation and security by attaching tenant keys to data blocks. This flexible architecture allows for dynamic allocation of resources, such as creating distinct embedding and generation pools, enabling efficient and scalable AI operations across various workloads.