Watch on YouTube
Watch on Vimeo
Elias Alagna, a Distinguished Technologist at HPE, argued that enterprises should run most AI work on open-weight models they control rather than sending everything to closed frontier models. Using a chart from Epoch AI, he showed that frontier models lead in capability but open-weight models usually catch up within a few months. When a delegate pointed out that vendor tooling and coding harnesses make frontier models hard to leave, Alagna agreed. He said tooling makes those platforms sticky, but argued that the learning curve of a more open approach pays off over time, helped by NVIDIA’s work to optimize open-weight models. On governance, he said a single-tenant system in a caged Equinix space on a private network can be fully isolated. A proxy routes each workload to the right model, and fine-tuning with proprietary data stays private rather than helping a vendor retrain its models. Alagna showed that most open-weight models fit on a single eight-GPU NVIDIA H200 server, with the largest needing two. He then made the cost concrete with Lily, an Equinix solution architect who uses about 620 million tokens a week, or 32 billion a year. Running mostly on Claude Opus, her usage costs about $1,500 a week, and she is one of roughly 13,000 Equinix employees.
Affan Syed, who leads technical marketing for HPE Private Cloud AI, presented a cost-per-task model based on MIT research. That research tested eight frontier and open-weight models on about 500 real GitHub merge conflicts. On average, an agentic coding task consumed about 4.17 million tokens, costing $1.86 at frontier prices. Syed compared that all-frontier approach with a hybrid that sends most work to smaller dense open-weight models and some to frontier models. He noted that frontier models now help fine-tune small models, citing a new paper in which a frontier model raised an 8-billion-parameter model’s math accuracy from 7.9% to 48%. His core argument was that rented tokens cost the same per token forever, while owned capacity gets cheaper per task as utilization grows. He acknowledged that at low utilization, public cloud pricing makes more sense. Jason Lee of Equinix described a customer that installed 32 GPUs, planned to double capacity in 12 months, and doubled it in four months as users filled the capacity. Alagna described HPE Private Cloud AI as an engineered system co-developed with NVIDIA, combining servers, storage with KV cache offload, networking, open source software, and NVIDIA software. Because HPE spreads the engineering cost across many customers, it costs less than a system an enterprise builds itself. He said HPE sizes systems on three factors: the task to be done, the number of users, and their throughput.
Lee then addressed a more basic problem: many enterprise data centers are out of space and power. HPE Private Cloud AI is air-cooled at about 20 kilowatts per rack, beyond what most legacy facilities can support but well below the liquid-cooled racks NVIDIA’s future products will require. Many IT teams also lack experience with liquid cooling infrastructure such as coolant distribution units. Under the partnership, HPE pre-positions systems at Equinix sites, where they are certified, installed, and connected. That shortens time to value, avoiding the permitting, utility, and seismic delays and the backlogs that come with liquid-cooled builds. Lee closed with the Equinix Distributed AI Hub. Equinix places the stack in whichever of its 77 metros best fits the customer’s data, users, and latency needs. It then uses Equinix Fabric to connect privately to public clouds, neoclouds, frontier model providers, and the customer’s own network.
Personnel: Affan Syed, Elias Alagna, Jason lee
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!