Watch on YouTube
Watch on Vimeo
Elias Alagna, a Distinguished Technologist at HPE, presented this segment with Jason Lee of Equinix. Alagna argued that AI bills are surprising enterprises the same way cloud and software bills once did, because the way AI is consumed and priced has changed. He showed OpenAI data on users of its coding agent, priced at the per-token rates OpenAI charges customers. Over a couple of months, median users went from about $1 a day in token spend to around $600 to $700, and 90th-percentile power users reached about $7,000 a day, roughly doubling spend every month. The driver is agents. A simple prompt and answer might use eight tokens. An agentic loop that checks its own work can turn the same request into 20,000 tokens, and orchestrated agents that split a task across tools for parsing, answering, writing, and checking use more still. Alagna said software development is now the largest category of AI spend, ahead of chatbots, ranging from code completion in the IDE to fleets of coding agents.
Alagna’s central point was that enterprises are measuring the wrong things. Citing MIT Sloan research, he said AI tools increase the lines of code developers generate by 200% to 700%. Commits back to version control rise only 30% to 100%, and software releases rise just 10% to 20%. When a delegate suggested the gap reflects human review acting as a bottleneck, Alagna agreed in part but said most of what agents produce is still thrown away, and every discarded token was paid for. He proposed four questions for enterprise AI work: whether current AI can do the task, whether it should, where it should run, and what outcomes it achieved. Measuring what actually ships, rather than tokens or lines of code, helps organizations identify the overhead they paid for but got no value from.
Alagna used HPE’s own operations as evidence. HPE runs about 150 internal AI applications, with work split among frontier models, privately hosted models, and developer experiments. By applying those four questions and matching each task to the right model in the right place, HPE has cut its own token usage by 30% to 50%. Alagna said moving work from heavyweight models to specialized, task-specific small language models produces about 450 times the savings. He runs an internal system of about 30 models, each built for one job. Asked whether the comparison accounts for the price difference between frontier and small-model tokens, he said it does, because the savings are measured in cost and most of HPE’s work no longer goes to a frontier model.
Personnel:
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!