Arrcus Presents at Networking Field Day 41

Delivering Modern Networking for the Age of AI with Arrcus

Watch on YouTube
Watch on Vimeo

Arrcus outlined its approach to networking for AI, presenting a unified, programmable fabric designed to span training and inferencing workloads from the edge to the data center and public cloud. Sanjay Kumar and Keyur Patel described the company’s hardware-agnostic network operating system, its broad merchant silicon and DPU support, the Arrcus Inference Network Fabric, and the automation and observability capabilities built around it. The session took place at Networking Field Day 41, where Kumar, who leads product management and marketing, was joined by CTO and co-founder Keyur Patel and AI solutions chief architect Nalin Pai.

Kumar framed the problem around AI’s need for deterministic, low-latency networking that never leaves expensive GPUs waiting, along with security built into the fabric, end-to-end telemetry, and automation across the full lifecycle. Arrcus defines programmability as network-as-code, with OpenConfig, YANG, and NETCONF northbound and support for any merchant silicon southbound. The hardware list includes Broadcom’s XGS and DNX families, NVIDIA Spectrum switches, BlueField-3 DPUs with BlueField-4 now being onboarded, Intel’s NetSec Accelerator, and Fujitsu’s ARM-based Monaka CPU, plus certified images for AWS, Azure, GCP, and OCI and container deployment under Kubernetes. Asked how Arrcus keeps pace with new silicon, Kumar credited its data plane abstraction layer, and Patel added that the layer targets routing on deep-buffer platforms, covering IPv6, SRv6, and MPLS rather than switching alone. Pensando support is on the roadmap.

For training, Arrcus offers scale-out and scale-across designs built on a tuned lossless fabric with adaptive routing, multi-tenancy for GPU-as-a-service operators, and standards support that extends past RoCEv2 and PFC into Ultra Ethernet Consortium work, with scale-up support to follow as those standards solidify. For inferencing, the company introduced the Arrcus Inference Network Fabric, which Kumar described as an inferencing router. Delegates also probed the multi-cloud overlay, and Arrcus confirmed that a single policy, including IPsec encryption between clouds, can be defined once and applied across private and public infrastructure. Kumar closed by claiming total cost of ownership reductions of 40% or more across capex and opex, positioning one hardware-agnostic fabric as the way operators move AI into its commercial phase.

Personnel: Keyur Patel, Nalin Pai, Sanjay Kumar

Arrcus Inference Network Fabric (AINF)

Watch on YouTube
Watch on Vimeo

Keyur Patel of Arrcus laid out the problem space behind the Arrcus Inference Network Fabric, explaining how agentic and physical AI applications are driving distributed inference traffic that behaves very differently from training or web traffic. He covered the latency, sovereignty, model diversity, and capacity requirements this places on the network and the operator segments a dedicated inference fabric could serve. Patel, the company’s founder and CTO, presented at Networking Field Day 41 ahead of chief architect Nalin Pai’s walkthrough of the solution itself.

Patel split inference into physical AI, such as drones, patient monitoring, and robot-assisted surgery, and agentic AI, such as applications handling 911 dispatch or drafting medical notes. He cited OpenRouter data showing year-over-year growth in token consumption and Cisco reports showing inference flows lasting longer than web flows, with upstream traffic rising as each request carries more context. KV cache transfer dominates inference traffic, and the metrics that matter are time to first token, time per output token, and p90 and p99 latency. Patel argued that as GPUs and memory shrink end-to-end token latency, network latency will become the visible bottleneck, just as it did in training. Responding to a delegate, Arrcus noted that human-facing applications prioritize time to first token while others favor throughput or the cheapest GPU, and pointed to a public safety proof of concept with TELUS in Canada as a highly latency-sensitive case.

When a delegate questioned how this differs from back-end congestion control, Arrcus answered that distributed inference shifts the problem to the front end, where the task is steering each request to the right site with the right model and capacity. Patel added data growth, geofenced sovereignty requirements, model diversity and auto-discovery, and uneven site power and capacity to that list, concluding that the answer is a holistic fabric that keeps request and response latency bounded rather than a component-level fix. He identified neoclouds steering traffic to GPUs, interconnect providers selling SLA-backed inference, sovereignty-focused enterprises, GPU-as-a-service operators, and colocation providers converting facilities into inference sites as the segments a single inference router could serve.

Personnel: Keyur Patel, Nalin Pai, Sanjay Kumar

Introducing Arrcus Inference Network – AINF Demo

Watch on YouTube
Watch on Vimeo

Nalin Pai of Arrcus introduced the Arrcus Inference Network Fabric, a layer 7 to layer 3 routing fabric that steers inference requests across distributed sites based on model availability, load, and policy. The session covered the AINF router and orchestrator, runtime integrations, network slice support for data sovereignty, and a demo of the management portal, followed by delegate discussion of latency, cost, and target customers. Pai, Arrcus’s chief architect for AI solutions, presented at Networking Field Day 41 with additional input from Keyur Patel and Sanjay Kumar.

Pai positioned AINF as a front-end distribution layer for operators running their own models, sitting behind existing API gateways and alongside runtimes such as vLLM, Dynamo, and SGLang rather than replacing them. The AINF router is a Kubernetes-deployed software service that uses plugins for llm-d and Dynamo to gather KV cache usage, queue depth, worker counts, and GPU class, then shares that data with peer routers so each can pick the best site while the in-site load balancers pick the GPU. A central orchestrator handles lifecycle, policy, and telemetry but stays out of the request path, and a tiered control plane distributes load updates well under a minute. Policies combine match-action routing rules on layer 7 metadata, including PII flagged by lightweight LLM classifiers or third-party tools, with a scored load-balancing step. Operators can also feed WAN slices into AINF, and the demo showed a US-only slice built on SRv6 Flex-Algo, with traffic steered through DSCP or BGP coloring.

Asked whether inspection adds latency, Pai argued that time to first token is measured in seconds, so spending a few milliseconds choosing a site with headroom lowers overall job latency. Arrcus confirmed AINF can run purely as an overlay, keeping load balancing and compute-level sovereignty without network slicing. Kumar tied AINF’s value to faster time to first token, higher tokens per second, and power-aware site selection, and when a delegate questioned whether buyers can span network, cloud, and serving teams, he cited proofs of concept with Lightstorm and TELUS, two others underway, and an unnamed enterprise with distributed factories that needs in-country inference and lacks the power to feed every site.

Personnel: Keyur Patel, Nalin Pai, Sanjay Kumar