Watch on YouTube
Watch on Vimeo
Elias Alagna, a Distinguished Technologist at HPE, opened by conceding that owning an AI system doesn’t make sense. Using Deloitte data, he showed that an organization needing 10 billion tokens in its first year with no longer planning horizon will find HPE’s approach the most expensive option and is better served by a neocloud or a frontier model provider. The economics change as usage grows. In the second year of Deloitte’s scenario, token consumption rises 30 times, the cost of the owned system only doubles, neocloud costs rise about 5.5 times, and frontier model costs rise about 26 times, because per-token pricing scales directly with usage. Alagna said the comparison shows neoclouds breaking even at about 49 billion tokens a year, an owned AI factory at about 67 billion, and the owned system becoming cheaper than neoclouds at about 84 billion. Spread across an average S&P 500 company of 61,000 employees, the tokens needed per employee to break even are modest. He noted that Lily, the Equinix solution architect from the previous segment, uses about 32 billion tokens a year on her own.
Jason Lee of Equinix added that the per-token bill leaves out other costs, such as cloud egress fees. Cloud providers discount egress over private interconnects such as AWS Direct Connect and Azure ExpressRoute, and Lee said Equinix hosts about 35% of the world’s native cloud on-ramps. When a delegate asked whether the systems are actually available, Lee said HPE Private Cloud AI test drives are already running in Equinix data centers. Alagna added that production orders take weeks rather than the months or years of the past, and that a system will be in production about eight hours after the equipment arrives. He summed up the strategy as “buy the base, rent the burst,” and claimed HPE can deliver up to 90% lower cost per token than the public cloud.
Alagna listed five levers that owning the system provides. Customers decide where each workload runs, choose the right model for each task, fine-tune open-weight models, and use a KV cache to reuse context they have already generated rather than paying a provider for it again. They also keep the system busy, since an owned system running at 1% utilization is an expensive 1%. Lee said that when customers do burst to rented capacity, they should do it over private connections. Some models are hundreds of gigabytes to terabytes in size. A customer that started on a 1 Gbps link needed two to two and a half hours to download a terabyte, while a 100 Gbps Equinix Fabric connection, set up in minutes and removed afterward, cuts that to a minute or two and keeps GPUs from sitting idle. Hosted trials of HPE Private Cloud AI are available at Equinix sites in Toronto, Ashburn, and London, so customers can test the system before choosing a location and connecting it through Fabric.
Personnel: Elias Alagna, Jason lee
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!