CoreWeave Agent Lens Intelligent Observability at Enterprise Scale

CoreWeave Agent Lens provides intelligent observability purpose-built for enterprise scale, monitoring agents to diagnose and fix issues. It is part of Forge, CoreWeave’s development layer for building and improving models and agents, specifically focusing on the agent development side. Forge unifies model training, experiment tracking, evaluation, and agent development into one connected environment, eliminating the need to stitch together disconnected workflows. Agent Lens offers open integration across model frameworks and clouds, supporting popular large language models and coding agents, providing developers flexibility in their choice of tools and streamlining workflows by combining various services.

The need for Agent Lens stems from the inherent challenges of modern agent development, which differ significantly from traditional software. Agents face high volumes of complex interactions and exhibit unpredictable behavior, making manual debugging and traditional offline evaluations insufficient for identifying “long tail” user-generated failures or rapidly evolving issues. CoreWeave advocates an “online first” approach to agent quality, asserting that production traces are the most representative and up to date data for discovering problems. Agent Lens facilitates this by observing, mining failures, and then distilling them into offline evaluations for regression testing.

Agent Lens is designed for cross functional teams, ensuring that observability is accessible not only to engineers but also to product managers and subject matter experts, each with a tailored user experience. It intelligently detects and prioritizes failures at scale by automatically analyzing vast numbers of production traces, identifying recurring issues, and grouping them semantically. This enables teams to quickly understand problems, their frequency, and their importance, and to drill down to root causes. Agent Lens supports user defined tagging with online large language model judges for continuous monitoring of agent behavior, validating judge accuracy by aligning with human assessments. Leveraging CoreWeave’s inference infrastructure, it provides insights into user intent and agent performance, identifying capability gaps. As an enterprise grade solution, Agent Lens integrates with OpenTelemetry, ensures governance and compliance, and offers a cost effective solution compared to general purpose large language models, with AI powered insights and online large language model judges currently offered free on CoreWeave Forge managed cloud until 2027.