CoreWeave Presents at AI Field Day 9

Connecting the AI Loop with CoreWeave Forge

Watch on YouTube
Watch on Vimeo

Shane Robbins from CoreWeave discusses the fundamental shift occurring as artificial intelligence initiatives transition from experimental pilots to production deployments. While pilot projects permit flexible timelines, limited users, and manual oversight, production applications demand rigorous reliability at scale, strong governance, security, and continuous improvement. Traditional software development operates linearly through building, deploying, and monitoring, whereas production artificial intelligence functions as a continuous cycle where deployment initiates an ongoing feedback loop. As a wider variety of businesses deploy commercial agents and customize models, relying on a patchwork of disconnected tools has become a critical liability that causes teams to lose vital signals and operational data between handoffs.

The continuous operating cycle is defined by five key stages comprising run, observe, curate, improve, and evaluate. During the run and observation phases, instrumentation and tracing capture operational data to monitor how models and agents behave under real customer loads. Without effective curation and tracing, evaluations rapidly grow stale and prevent developers from understanding how applications are performing. In the improvement and evaluation stages, maintaining lineage and reliable benchmarks ensures that every newly deployed iteration builds directly on prior learnings, allowing engineers to pinpoint anomalies, refine models through post-training, and avoid repeating previous errors.

To unify these fragmented processes, CoreWeave introduced CoreWeave Forge, an integrated environment designed to manage the full development and deployment loop. Built using technology from acquisitions and internal development, Forge brings together evaluation, tracing, post-training, and inference services while remaining completely agnostic to specific clouds, frameworks, or hardware models. Targeted at developers building or customizing models and agents, the platform features tools such as Aria to detect interactions that violate operational guardrails. Forge provides workflow automation and an available thirty-day free trial, allowing enterprises to eliminate manual development overhead and iterate on workloads without being locked into a single compute provider.

Personnel: Shane Robbins

Post-Training on CoreWeave Forge

Watch on YouTube
Watch on Vimeo

David Corbitt introduces CoreWeave Forge post-training, explaining how his background co-founding OpenPipe led to CoreWeave acquiring the company to bring accessible fine-tuning to enterprises. Forge provides an integrated environment where observability, post-training, and inference work together in a continuous improvement cycle. Teams observe production behavior, extract traces to form evaluation or training datasets, alter the weights of open-source models, and redeploy. The goal of the managed platform is to eliminate the need for in-house machine learning experts or deep PyTorch knowledge by handling the underlying complexities of training runs and quickly catching errors through integrated evaluations and metrics.

The presentation covers three supported post-training methods: supervised fine-tuning, reinforcement learning, and model distillation. Supervised fine-tuning relies on high-quality input and output examples to enforce style or consistency, as seen with startup Willow using it to clean up voice-to-text transcriptions containing self-corrections. Reinforcement learning uses reward functions to improve model performance, evidenced by a case study with You.com and Nvidia that increased web search accuracy while reducing tool calls. Corbitt emphasizes that before attempting post-training, users should always consider faster and cheaper alternatives, such as optimizing prompts, repairing environment tools, or evaluating larger models.

CoreWeave manages this process on its own GPU infrastructure, where base models remain pre-loaded to avoid spin-up latency and out-of-memory issues. Instead of retraining full base weights, the platform trains small LoRA adapters that run alongside shared models on serverless inference, allowing customers to pay standard token rates for customized performance. For distillation, an automated background agent gathers production data, relabels it with a stronger model, conducts training sweeps on a smaller model, and evaluates the results against production standards. This architecture establishes an automated, repeatable lifecycle that reduces deployment costs and operational complexity while allowing models to adapt to changes over time.

Personnel: David Corbitt

CoreWeave Inference From Model Training to Production Scale Serving

Watch on YouTube
Watch on Vimeo

Nisha Nadkarni, a specialist field engineer at CoreWeave, delivered a presentation on CoreWeave inference, detailing the process from model training to production scale serving. She introduced CoreWeave Forge as an additional development layer built upon CoreWeave’s core compute offerings, designed to help users construct AI models and agents with familiar frameworks. A significant part of her discussion focused on the operational shift required when moving from a model training workload, characterized by predictable batch jobs, to an inference workload, which behaves more like a live service that dynamically scales to meet incoming requests.

Production inference presents unique challenges, involving a delicate balance between competing objectives such as latency, throughput, scale, and cost, all constrained by GPU memory. Nadkarni explained that optimizing for one goal often necessitates trade-offs in others, referencing the Pareto frontier concept. To navigate these challenges, CoreWeave provides customers with three distinct inference options, differing in the level of operational responsibility they entail. These options include serverless inference for a fully managed API endpoint, dedicated inference for those seeking more control over model weights and engine configurations, and full control inference on CKS for users who want to manage the entire stack, including custom runtimes.

Delving deeper into dedicated inference, Nadkarni outlined its architecture, which comprises a gateway handling authentication, authorization, and intelligent routing, along with a deployment where users configure their models. This includes specifying model weights from CoreWeave AI object storage, setting VLLM engine parameters, choosing GPU classes like H100s, and defining replica scaling. CoreWeave also prioritizes observability, offering pre-populated Grafana dashboards that display crucial metrics such as request rates, VLLM specific performance, and GPU utilization. Furthermore, they feature a mission control agent that can assist users in proactively optimizing their models based on specific targets, such as reducing time to first token or overall costs.

Personnel: Nisha Nadkarni

CoreWeave Agent Lens Intelligent Observability at Enterprise Scale

Watch on YouTube
Watch on Vimeo

CoreWeave Agent Lens provides intelligent observability purpose-built for enterprise scale, monitoring agents to diagnose and fix issues. It is part of Forge, CoreWeave’s development layer for building and improving models and agents, specifically focusing on the agent development side. Forge unifies model training, experiment tracking, evaluation, and agent development into one connected environment, eliminating the need to stitch together disconnected workflows. Agent Lens offers open integration across model frameworks and clouds, supporting popular large language models and coding agents, providing developers flexibility in their choice of tools and streamlining workflows by combining various services.

The need for Agent Lens stems from the inherent challenges of modern agent development, which differ significantly from traditional software. Agents face high volumes of complex interactions and exhibit unpredictable behavior, making manual debugging and traditional offline evaluations insufficient for identifying “long tail” user-generated failures or rapidly evolving issues. CoreWeave advocates an “online first” approach to agent quality, asserting that production traces are the most representative and up to date data for discovering problems. Agent Lens facilitates this by observing, mining failures, and then distilling them into offline evaluations for regression testing.

Agent Lens is designed for cross functional teams, ensuring that observability is accessible not only to engineers but also to product managers and subject matter experts, each with a tailored user experience. It intelligently detects and prioritizes failures at scale by automatically analyzing vast numbers of production traces, identifying recurring issues, and grouping them semantically. This enables teams to quickly understand problems, their frequency, and their importance, and to drill down to root causes. Agent Lens supports user defined tagging with online large language model judges for continuous monitoring of agent behavior, validating judge accuracy by aligning with human assessments. Leveraging CoreWeave’s inference infrastructure, it provides insights into user intent and agent performance, identifying capability gaps. As an enterprise grade solution, Agent Lens integrates with OpenTelemetry, ensures governance and compliance, and offers a cost effective solution compared to general purpose large language models, with AI powered insights and online large language model judges currently offered free on CoreWeave Forge managed cloud until 2027.

Personnel: Angela Samples

Connecting the AI Loop with CoreWeave Forge Live Demo

Watch on YouTube
Watch on Vimeo

Russ Ratshin from CoreWeave’s product marketing team presented a live demo demonstrating how to connect the AI loop using CoreWeave Forge. He explained that the AI loop is a recognized workflow for developing AI agents, and CoreWeave Forge serves as the environment to facilitate this process. The presentation took the audience through each stage of the AI loop with a practical help desk use case, showcasing the product in action and highlighting its real world applicability for agent building.

The presentation detailed the five stages of the AI loop: run, observe, curate, improve, and evaluate. The ‘run’ stage involves deploying the model for inference and exposing the agent to users. Next, ‘observe’ captures all interaction traces and stores them in Agent Lens, allowing product managers, domain experts, and AI engineers to review the agent’s performance. In the ‘curate’ stage, tagging is used to organize and filter this invaluable production data, making it actionable for further model enhancements and evaluations. The ‘improve’ stage covers various post-training methods, and ‘evaluate’ confirms that the agent model has indeed been significantly enhanced, ultimately leading back to the ‘run’ stage with an improved model.

The demo illustrated these steps using a help desk agent named Harness, which utilizes an LLM served by CoreWeave Inference. It began by instrumenting the agent with ARIA to gather traces for observability within Agent Lens. The Agent Lens dashboard provided a comprehensive view of the agent’s activity, including conversation counts, errors, tags, cost, latency, and model and tool usage. The speaker showed how tags could be created manually or automatically by an LLM judge, with a built-in alignment feature to ensure the judge accurately applies tags. Automated insights were presented, clustering conversations by agent failures and user intents. For the ‘improve’ phase, serverless reinforcement learning was applied, running numerous rollouts against CoreWeave sandboxes to train the agent to make better decisions. Finally, the evaluation phase compared the improved model to the base model, demonstrating superior performance across key metrics before providing the URI of the best model version for redeployment, thus completing the AI loop in CoreWeave Forge, which offers a unified and open environment.

Personnel: Russ Ratshin