CoreWeave – The Essential Cloud for Al

CoreWeave is the essential cloud for AI, distinguishing itself by building infrastructure specifically optimized for the unique and demanding requirements of AI workloads, unlike traditional data center deployments or general-purpose clouds. Sean Derrington introduced CoreWeave’s portfolio, operations, compute, storage, and networking infrastructure, emphasizing how they approach these elements differently to address AI’s specific needs. He also highlighted CoreWeave Forge, an end-to-end platform designed as a continuous loop for running, evaluating, measuring, improving, and curating AI models, whether open or closed, with this particular presentation focusing on the underlying infrastructure. Mission Control, another CoreWeave offering, provides operational insights and an agent for natural language interaction with models to optimize GPU storage and performance.

On the compute front, CoreWeave showcased its liquid-cooled data centers featuring GB200 and Vera Rubin racks, and holds patents for its liquid-cooling control system designed to prevent large spills. They were the first to deliver NVIDIA Vera Rubin systems, validate the architecture with Dell, release the DeepSeek R1 benchmark, and have a customer, Cognition, use Vera Rubin in production, reporting 3-5x performance improvements. For storage, CoreWeave AI Object Storage (CAIOS) is S3-compliant, offers unlimited capacity, and is optimized for AI workloads with multi-cloud capabilities. A key innovation is LODA (Local Object Transport Accelerator), a node-level NVMe caching mechanism providing 7 GB/s throughput and dramatically reducing read latency by up to 8x, with a leading model provider achieving 99.7% cache hit rates across 15,000 GPUs. All CoreWeave object storage tiers (hot, warm, cold) are SSD-based with intelligent auto-tiering by age of access, plus an optional, lower-cost hard drive-based archiving tier. CoreWeave’s billing model for object storage is also unique, featuring no egress, operation, or early deletion fees, and offering programs to offset migration costs from other clouds.

CoreWeave’s networking infrastructure is purpose-built for AI, utilizing InfiniBand and RDMA for front-end connectivity to minimize latency and ensure rapid data transfer to GPUs. The back-end network is non-blocking and non-oversubscribed, facilitating high-throughput communication between GPUs, particularly with NVIDIA’s NVLink capabilities. Dedicated NICs for application and storage traffic prevent resource contention, ensuring optimal performance for AI-specific machine types. Security is enhanced with DPUs (Bluefield 3 or 4) on every node, offloading networking and security operations to free up CPU cycles. Globally, CoreWeave operates over 51 data centers with 1.5 gigawatts of power in North America and Europe, and has expanded its global AI network to Asia Pacific and the Middle East with announced plans in India and Indonesia, and network locations in Japan, Australia, and Dubai. This global backbone features dual 400 gig links and MACSec encryption, designed to handle “elephant type flows” by providing substantial bandwidth for rapid data movement between regions. Users can experience this infrastructure through “Arena,” a 30-day trial featuring the latest GPUs, and CoreWeaveForge also offers a free sign-up for model testing.