HPE Networking Data Center Evolving AI data Center Architectures with Praful Lalchandani

AI infrastructure is rapidly evolving from centralized training environments into complex inference-driven systems in which a single workload can traverse front-end, back-end, storage, and security fabrics. HPE Networking’s strategy is to deliver open, secure Ethernet for every AI fabric (scale-up, scale-out, and scale-across), resulting in high performance, resilience, operational consistency, and strong customer momentum. Praful Lalchandani, Vice President of Product Management for Data Center Networking at HPE Networking, opened the company’s Networking Field Day 41 presentation by citing deployments at a tier-one model provider’s million-GPU cluster, Oracle, neocloud Vultr, and Oak Ridge National Laboratory’s next supercomputer, Lux.

Lalchandani argued that HPE Networking is the only OEM with a product for every AI fabric. Its lineup includes switch trays for AMD’s Helios rack-scale scale-up design, which will ship within months, the QFX family for scale-out, storage, and front-end networks, MX and PTX platforms for data center interconnect and scale-across, and SRX for security. He tied that breadth to rising GPU bandwidth demands. Hopper-class systems needed 400 Gbps per GPU in 2024, Blackwell B300 needed 800 Gbps by the end of 2025, and the next step is 1.6 Tbps roughly 16 months later, with rack-scale designs reaching 14.4 Tbps per GPU. He also highlighted the shift from single-connection rail-optimized designs to multiplane fabrics with two to eight planes. These fabrics enable larger two-tier clusters and let jobs keep running at reduced capacity after a link failure instead of rolling back to a checkpoint.

Lalchandani identified inference, more than training, as the main driver of architectural change, pointing to mixture-of-experts models, chatty agentic workloads, multimodal traffic, disaggregated prefill and decode, and remote KV caching. He acknowledged that nanosecond-scale switch latency is minor next to WAN and GPU time, but said congestion stalls still hurt inter-token latency and tokens-per-second throughput. He traced a single query through the front end, a RAG vector database, prefill nodes, a KV cache store, and decode nodes. That walkthrough showed how caching turns a compute problem into a high-performance data access problem on the storage network. The result is five fabrics with distinct requirements, from multi-tenant isolation at the front end to ZR optics and encryption for scale-across, and each is a point where a stall can degrade the user experience.