Accelerating AI In the Enterprise with Traefik Labs

Sudeep Goswami from Traefik Labs presented at Tech Field Day at KubeCon North America 2025, focusing on accelerating AI in the enterprise using Traefik Labs’ runtime gateway. Given the rapid proliferation of AI models, decoupling applications from specific models becomes crucial to avoid constant refactoring. Traefik Labs advocates for decoupling at the gateway layer, providing operational freedom and leverage.

The gateway’s role extends beyond simple routing, encompassing critical functions like authentication, rate limiting, and implementing guardrails to ensure AI usage aligns with enterprise policies. These guardrails prevent misuse, such as finance agents answering legal questions. Caching at the gateway also optimizes token consumption. Sudeep emphasized that while such logic can be embedded in applications, production environments benefit from consolidating it at the gateway for scalability, performance, unified control, and observability.

The presentation introduced the concept of a “triple gate pattern” for agentic workflows involving interactions with LLMs, MCP resources, and backend APIs. This necessitates AI gateways, MCP gateways, and traditional API gateways, ideally within a single binary to simplify deployment and management. Decoupling the API runtime from the model runtime is crucial, acknowledging the rapid evolution of AI models. Sudeep emphasized that no single model will ultimately dominate.