Watch on YouTube
Watch on Vimeo
Jonathan Bryce argued that AI agents will force an infrastructure rebuild on the scale of the shift that followed the iPhone. He recalled a 2016 OpenStack Summit talk by the future president of AT&T’s network business, who described data traffic growing 1,500 times in a few years after the iPhone turned phones from devices for calls and texts into data devices, including HD video. That shift pushed carriers to rebuild mobile networks worldwide in under a decade, and Bryce said agents will change the purpose of today’s systems the same way. Those systems are built for human speed, while agents consume whole websites at once, follow links in parallel, and return repeatedly. Bryce cited figures showing that 53% of web traffic last year was machine-generated, the first time machine traffic exceeded human traffic, and that user-triggered AI crawling, where an agent browses on a person’s behalf, grew 15 times. Unattended autonomous actions by AI systems more than doubled this year, and a single agent task can generate dozens to thousands of inference requests with fewer chances for a human to intervene. When delegates asked how representative these early-adopter figures are, Bryce said most of the morning keynote audience of about 6,000 raised their hands for using coding agents regularly, but fewer than a quarter used agents for other work, and he estimated current usage at 10% to 15% of what he expects within five years.
Delegates noted that enterprises are adopting agents far more slowly than the wider internet, and Bryce attributed the gap to a lack of trusted patterns and tools to keep non-deterministic systems within safe limits. He pointed to new CNCF projects such as NVIDIA’s OpenShell, announced about six months ago, and Agent Substrate, alongside sandboxing tools such as gVisor and Kata Containers, as the toolkit that will close that gap. Because 82% of enterprises that run containers run Kubernetes in production, teams can add these tools to infrastructure and skills they already have, including OpenTelemetry and Prometheus, instead of starting over. Bryce said agents put outsized pressure on scale, efficiency, control, and governance, and that inference is the main constraint on growth because agents multiply the prompts behind each request by 10 to 100 times. He added that inference combined with reinforcement learning, which he called on-the-job training, will become a key workload for improving agents.
Bryce used LLM-D as an example of extending Kubernetes for inference. The project, which entered the CNCF in March and recently reached version 1.0, takes the routing and key-value (KV) cache functions of the vLLM model serving engine and spreads them across a cloud native architecture, using Kubernetes dynamic resource allocation (DRA) to load models onto GPUs. Bryce said this kind of front-end inference routing can reach cache hit rates of 80% to 90%, and that the largest efficiency gains in AI are coming from the operations stack, where cutting even a few percentage points of GPU waste saves millions of dollars. When a delegate raised concerns about overlap among projects such as Envoy AI Gateway and LLM-D, Bryce described the field as fast-moving and said parallel efforts will settle into standard patterns, as they have in open source before. He highlighted the CNCF’s Kubernetes AI Conformance program, which defines a baseline for running inference so workloads can move between providers while vendors still compete on components like network gateways, and he noted collaborations such as LLM-D routing to GPU slices exposed by HAMi and Ray, and the PyTorch Foundation integrating LLM-D. Bryce closed by arguing that open source has to win in AI, because technology this powerful and this widely adopted needs to be open to inspection and shared problem solving.
Personnel: Jonathan Bryce
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!