Upscale Presents at AI Infrastructure Field Day 6

Purpose Built for AI-Scale with Upscale

Watch on YouTube
Watch on Vimeo

Upscale AI’s mission is pure-play AI networking: a networking infrastructure company that designs, builds, and engineers systems in-house. The company emerged from stealth mode in Q3 2025, having secured $50 million in funding and achieving a valuation of $2 billion, which enables it to hire top talent. Its 300-strong team comprises experts across silicon, systems, software, operations, and manufacturing, drawing experience from leading networking vendors, ASIC builders, and hyperscalers, all dedicated to solving AI challenges. The presentation agenda includes an introduction to Upscale by Mayuresh, followed by Aninda detailing the evolution of data center designs from class fabric to AI clusters and the role of inference, and finally, Nicholas will demonstrate SkyD, an in-house tool for designing network fabrics from scratch.

Upscale offers two primary product lines tailored for AI use cases: “scale up” and “scale out.” The “scale up” product line involves in-house designed and built silicon for systems, complemented by an operating system and orchestration software. For “scale out” applications, Upscale has a strong partnership with Nvidia, using Spectrum X to build systems suited to backend and front-end requirements. A core tenet of their development is adherence to open standards, demonstrated by their participation in organizations like UEC and OCP forums, and their role as a premium member of the SONiC technical committee, with their operating system built on SONiC.

Personnel: Mayuresh Gangal

Network Engineer Meet AI Cluster with Upscale AI

Watch on YouTube
Watch on Vimeo

The session provides a worldview of AI clusters and the importance of the network infrastructure behind them. Aninda Chatterjee of Upscale AI tackles the challenge network engineers face in understanding complex AI clusters. It begins by contrasting traditional data center networking, characterized by north-south traffic patterns and three-tier architectures with spanning tree, against the evolution driven by virtualization, microservices, and distributed storage, which shifted traffic predominantly east-west, leading to the adoption of the Clos architecture. The core shift from CPU-centric computing, often bottlenecked by memory channels, to GPU-centric architectures with thousands of tensor cores and high-bandwidth memory (HBM) is highlighted. This shift creates new demands: GPU memory is fast but small, so models must be split across GPUs, requiring substantial inter-GPU communication to exchange partial results and reductions, which places considerable strain on the network infrastructure.

The discussion then delves into the three tiers of AI backend cluster networking: “scale-up” (intra-rack communication with sub-microsecond latency, acting as one large GPU), “scale-out” (inter-rack communication using Leaf-Spine architectures, Rocky V2, Ultra Ethernet, or InfiniBand, with microsecond latency), and “scale-across” (data center interconnects for stitching sites into a single logical AI factory). A key architectural concept introduced is “rails,” where specific NICs from different GPUs connect to the same leaf switch to optimize communication for certain collective operations, minimizing hops for intra-rail traffic while using spines for cross-rail communication. Beyond the backend, the crucial roles of the front-end and storage networks are emphasized, handling long-lived streaming connections, pulling large model weights (often hundreds of gigabytes), storing training checkpoints, and managing KV cache offloading, which involves different traffic patterns and resiliency requirements compared to the backend.

The inference process, broken into pre-fill (compute-bound, generating the first token) and decode (memory bandwidth-bound, generating subsequent tokens iteratively), further illustrates network demands. The KV cache, storing keys and values of earlier tokens to avoid recomputation, can grow significantly, necessitating offloading to storage or other memory pools, thereby increasing network traffic between GPUs and storage. The presentation concludes by underscoring that effectively building and scaling AI infrastructure requires a deep understanding of what data is moving, how often, and who is waiting, whether through tensor, pipeline, or expert parallelism, or even disaggregated serving where GPUs specialize in pre-fill or decode, creating handoff traffic for the KV cache across the network.

Personnel: Aninda Chatterjee

Designing and sizing an Al cluster with SkyD by Upscale

Watch on YouTube
Watch on Vimeo

Designing and sizing an Al cluster with SkyD by Upscale. Nicolas Fevrier introduces SkyD (Sky Design), a new tool built by Upscale AI, to address the inherent difficulties in designing and sizing AI clusters. The complexity arises not from the concept itself, but from the immense scale of elements involved, such as thousands of GPUs, hundreds of switches, and thousands of cables and fibers, which become impossible to manage manually. The tool was initially developed to create illustrations and design topologies for Upscale’s internal validated designs, evolving to encompass broader functionalities for comprehensive cluster planning.

SkyD facilitates the design process by focusing on the fundamental components: the ports on compute servers (like Nvidia B300 or GB300 systems) and the characteristics of the network switches. It applies a series of constraints and rules to derive precise sizing for the backend, frontend, and management networks. The tool also helps anticipate potential failures, considering how to prepare for switch, optic, or fiber outages, and translates abstract claw designs (leaf-spine, superspine) into practical, real-world site maps. Key considerations include avoiding oversubscription on the backend network while allowing it on the frontend, and leveraging multiple network planes to enhance redundancy and significantly expand the number of GPUs supported within a two-tier network.

Beyond logical network design, SkyD extends to the physical world, enabling users to optimize fiber distances and simplify deployment. It generates floor maps, organizes racks, and defines rack elevations, providing detailed visualizations and optimizing cable routing to reduce overall fiber lengths. The tool includes a resiliency map that calculates the impact of potential outages on bandwidth, helping users plan for fault tolerance. Nicolas Fevrier announced that SkyD will be publicly available on upscale.com very soon, requiring a one-time registration, and will be continuously improved. It serves as a crucial “Day minus one” tool for architects preparing Upscale’s validated designs.

Personnel: Nicolas Fevrier