Scaling the Agentic Era with CoreWeave Compute

Harshdeep Singh Banwait, CoreWeave’s Senior Director of Compute, works with customers to understand their evolving workloads and collaborates with data center teams on infrastructure operations for all CPU and GPU instances. He highlighted recent innovations, noting that CoreWeave was one of the first cloud providers to bring up and rigorously validate an entire Vera Rubin rack. This involved comprehensive checks of individual servers, backend networking, power shelves, and NVLink switches to ensure proper system function. CoreWeave released results on DeepSeek R1, showing a 10x improvement in tokens-per-second throughput between GB200 and Vera Rubin instances, then partnered with Cognition as the first customer on Vera Rubin, achieving 4-5x performance improvements for their custom reinforcement learning and inference models.

The Vera Rubin rack ecosystem is complex, integrating six different chips: the new Ruby GPU, NVIDIA’s second-generation ARM CPUs (VR CP), and new NVLink six switches, which enable GPUs within a rack to communicate at 3 terabytes per second, effectively making the entire rack operate as a single brain. CoreWeave also upgraded its backend networking with ConnectX9 SuperNICS and new Spectrum X. A key innovation is the liquid-cooled Bluefield 4 DPU on every compute instance, managing the Kubernetes service to give customers full, bare-metal access and avoid virtualization overhead. Furthermore, CoreWeave custom-designed “Valvey” (unofficially Valvey McValveTrain), a patent-pending valve assembly for liquid-cooled racks that enables per-rack cooling control and a rapid 7-second emergency shutdown. Complementing this is “Racky,” an in-rack manager that collects temperature, leak, and flow data, allowing independent rack management even if compute is down. The company also developed a rack lifecycle controller to streamline provisioning and ongoing management of rack-scale compute, automating tests for components ranging from cooling units and power to firmware and multi-node functionality, ensuring high performance and reliability.

CoreWeave provides customers with a comprehensive dashboard, dubbed “Mission Control,” offering cluster-wide and rack-level visibility into performance, GPU utilization, temperature, and NVLink utilization. This allows customers to identify if performance bottlenecks stem from their application code or potential hardware issues, which CoreWeave’s teams actively manage and resolve. The platform’s bare-metal isolation and 100 Gbps front-end networking are crucial for agentic AI workloads, where CPUs and GPUs engage in continuous back-and-forth communication: GPUs for “thinking” and CPUs for “executing” actions like API calls or opening browser windows. CoreWeave supports CPU-only instances with Intel Emerald Rapids, AMD Turin and Genoa, and upcoming NVIDIA Vera, all built to rigorous performance and reliability standards. While generally providing optimized “best recipes” for firmware and drivers, CoreWeave can accommodate specific versions for sufficiently large customer clusters.