Real-World Customer Journey with VMware Private AI from Broadcom

Event: AI Field Day 6

Appearance: VMware by Broadcom Presents at AI Field Day 6

Company: VMware by Broadcom

Video Links:

Personnel: Alex Fanous

Broadcom is actively engaged with customers on proof of concepts and production deployments of VMware Private AI Foundation. This session details a composite example of a typical customer journey, drawing from real-world scenarios encountered during customer engagements. The presentation focuses on the infrastructure aspects often overlooked, emphasizing the importance of a robust foundation for data scientists and AI engineers to effectively utilize AI tools. It highlights the iterative process of deploying and refining a private AI solution, starting with a simple Retrieve Augmented Generation (RAG) application built on VMware Private AI Foundation.

The customer journey begins with a high-level mandate from senior leadership to implement AI, often without specific technical details. A common starting point is a simple application, such as a chat app, using readily available data such as HR policies. This initial deployment allows for a gradual learning curve, introducing the use of vector databases for similarity searches and leveraging the VMware Private AI Foundation console for easy deployment. The presentation showcases how customers typically customize the initial templates, often adopting open-source tools like OpenWebUI for a more familiar user interface. The iterative process involves continual refinement, adjusting parameters, testing various LLMs, and ultimately scaling the infrastructure as needed using load balancers and multiple nodes.

Throughout the customer journey, the presentation stresses the importance of iterative development and feedback. The process emphasizes starting with a functional prototype, gathering feedback, and then progressively improving performance and scalability. This approach involves close collaboration between the infrastructure team, data scientists, and developers. The use of VMware’s existing infrastructure, such as vCenter and Data Services Manager, is emphasized as a key advantage, minimizing the need for retraining staff or adopting new vendor-specific tools. The session concludes by highlighting the flexibility and adaptability of the VMware Private AI Foundation platform, its ability to accommodate evolving AI architectures and future-proof investments in AI infrastructure.


VMware Private AI Foundation Capabilities and Features Update from Broadcom

Event: AI Field Day 6

Appearance: VMware by Broadcom Presents at AI Field Day 6

Company: VMware by Broadcom

Video Links:

Personnel: Justin Murray

A technical review of the generally available VMware Private AI Foundation with NVIDIA product, an advanced service on VMware Cloud Foundation, was presented. The presentation focused on the architecture of VMware Private AI Foundation (VPF), highlighting its reliance on VMware Cloud Foundation (VCF) 5.2.1 as its base. The speaker, Justin Murray, explained the layered architecture, distinguishing between the infrastructure provisioning layer (VMware’s intellectual property) and the data science layer, which includes containers running inference servers from NVIDIA and other open-source options. Significant advancements since the product’s minimum viable product launch in May 2024 were emphasized, including enhanced model governance capabilities for safe model testing and deployment.

The presentation delved into the rationale behind using VCF and VPF for managing AI/ML workloads. The speaker argued that the increasing complexity of model selection, infrastructure setup (including GPU selection), and the need for RAG (Retrieve, Augment, Generate) applications necessitates a robust and manageable infrastructure. VMware Cloud Foundation, with its virtualization capabilities, provides this solution by enabling isolated deep learning VMs and Kubernetes clusters for different teams and projects, preventing management nightmares and optimizing resource utilization. A key element is the self-service automation, allowing data scientists to request resources (like GPUs) with minimal IT interaction, streamlining the process and enabling faster model deployment.

A significant portion of the presentation covered GPU management and sharing, emphasizing the role of NVIDIA drivers in enabling virtual GPU (VGPU) profiles that allow for efficient resource allocation and isolation. The speaker highlighted the advancements in VMotion for GPUs, enabling rapid migration of workloads, and the integration of tools for monitoring GPU utilization within the VCF operations console. The discussion touched on model version control, the role of Harbor as a repository for models and containers, and the availability of a service catalog for deploying various AI components. The presentation concluded with a demo showing the quick and easy deployment of a Kubernetes cluster for a RAG application, showcasing the self-service capabilities and simplified infrastructure management offered by VMware Private AI Foundation.


Three Reasons Customers Choose VMware Private AI from Broadcom

Event: AI Field Day 6

Appearance: VMware by Broadcom Presents at AI Field Day 6

Company: VMware by Broadcom

Video Links:

Personnel: Tasha Drew

Overview of why customers are choosing VMware Private AI and popular enterprise use cases we are seeing in the field. Tasha Drew’s presentation at AI Field Day 6 highlighted three key reasons driving the adoption of VMware Private AI. First, she addressed the often-overlooked issue of GPU underutilization. Data from UC Berkeley’s Sky Computing Lab, corroborated by VMware’s internal findings, demonstrated that current deployment practices, such as dedicating one GPU per model, lead to significant inefficiency due to inconsistent inference workload patterns. This underutilization is exacerbated by a phenomenon Drew termed “GPU hoarding,” where teams within organizations hoard their allocated GPUs, fearing resource sharing. VMware Private AI addresses this through intelligent workload scheduling and resource pooling, maximizing GPU utilization and enabling resource sharing across different teams and priorities.

The second driver for private AI adoption is cost. Drew presented data indicating a dramatic increase in cloud spending driven by AI workloads, often leading to budget reallocation and project cancellations. This high cost is attributed to various factors including platform fees, data security, infrastructure expenses, and the cost of upskilling staff to handle cloud-based AI tools. In contrast, VMware Private AI offers a more predictable and potentially lower total cost of ownership (TCO) by optimizing resource usage within the enterprise’s existing infrastructure. The presentation referenced an IDC white paper showing that a significant percentage of enterprises perceive on-premise AI solutions as equally or less expensive than cloud-based alternatives, primarily due to the decreased infrastructure and service costs.

Finally, Drew emphasized the critical role of model governance in driving the shift towards private AI. As enterprises embrace generative AI and train models on proprietary data and intellectual property (IP), concerns around data sensitivity and security become paramount. VMware Private AI tackles these concerns by providing robust control mechanisms such as role-based access control (RBAC) to regulate model access and ensure compliance with data protection regulations. While the technical complexities of managing access control within embedding and vector databases are acknowledged, Drew highlighted ongoing development efforts to integrate comprehensive security measures at both the database and application output levels. Overall, the presentation positioned VMware Private AI as a comprehensive solution addressing the challenges of cost, efficiency, and security in deploying and managing enterprise AI workloads.


MLCommons MLPerf Storage

Event: AI Field Day 6

Appearance: ML Commons Presents at AI Field Day 6

Company: ML Commons

Video Links:

Personnel: Curtis Anderson, David Kanter

MLCommons’ MLPerf Storage benchmark addresses the rapidly growing need for high-performance storage in AI training. Driven by the exponential increase in data volume and the even faster growth in data access demands, the benchmark aims to provide a standardized way to compare storage systems’ capabilities for AI workloads. This benefits purchasers seeking informed decisions, researchers developing better storage technologies, and vendors optimizing their products for AI’s unique data access patterns, which are characterized by random reads and massive data volume exceeding the capacity of most on-node storage solutions.

The benchmark currently supports three training workloads (UNET 3D, ResNet-50, and CosmoFlow) using PyTorch and TensorFlow, each imposing distinct demands on storage systems. Future versions will incorporate additional workloads, including a RAG (Retrieval Augmented Generation) pipeline with a vector database, reflecting the evolving needs of large language model training and inference. A key aspect is the focus on maintaining high accelerator utilization (aiming for 95%), making the storage system’s speed crucial for avoiding costly GPU idle time. The benchmark offers both “closed” (apples-to-apples comparisons) and “open” (allowing for vendor-specific optimizations) categories to foster innovation.

MLPerf Storage has seen significant adoption since its initial release, with a substantial increase in the number of submissions and participating organizations. This reflects the growing importance of AI in the market and the need for a standardized benchmark for evaluating storage solutions designed for these unique demands. The benchmark’s community-driven nature and transparency are enabling more informed purchasing decisions, moving beyond arbitrary vendor claims and providing a more objective way to assess the performance of storage systems in the critical context of modern AI applications.


MLCommons MLPerf Client Overview

Event: AI Field Day 6

Appearance: ML Commons Presents at AI Field Day 6

Company: ML Commons

Video Links:

Personnel: David Kanter

MLCommons presented MLPerf Client, a new benchmark designed to measure the performance of PC-class systems, including laptops and desktops, on large language model (LLM) tasks. Released in December 2024, it’s an installable, open-source application (available on GitHub) that allows users to easily test their systems and provides early access for feedback and improvement. The initial release focuses on a single large language model, LLaMA 2.7 billion, using the Open Orca dataset, and includes four tests simulating different LLM usage scenarios like content generation and summarization. The benchmark prioritizes response latency as its primary metric, mirroring real-world user experience.

A key aspect of MLPerf Client is its emphasis on accuracy. While prioritizing performance, it incorporates the MMLU (Massive Multitask Language Understanding) benchmark to ensure the measured performance is achieved with acceptable accuracy. This prevents optimizations that might drastically improve speed but severely compromise the quality of the LLM’s output. The presenters emphasized that this is not intended to evaluate production-ready LLMs, but rather to provide a standardized and impartial way to compare the performance of different hardware and software configurations on common LLM tasks.

The benchmark utilizes a single-stream approach, feeding queries one at a time, and supports multiple GPU acceleration paths via ONNX Runtime and Intel OpenVINO. The presenters highlighted the flexibility of allowing hardware vendors to optimize the model (LLaMA 2.7B) for their specific devices, even down to 4-bit integer quantization, while maintaining sufficient accuracy as judged by the MMLU threshold. Future plans include expanding hardware support, adding more tests and models, and implementing a graphical user interface (GUI) to improve usability.


MLCommons and MLPerf – An Introduction

Event: AI Field Day 6

Appearance: ML Commons Presents at AI Field Day 6

Company: ML Commons

Video Links:

Personnel: David Kanter

MLCommons is a non-profit industry consortium dedicated to improving AI for everyone by focusing on accuracy, safety, speed, and power efficiency. The organization boasts over 125 members across six continents and leverages community participation to achieve its goals. A key project is MLPerf, an open industry standard benchmark suite for measuring the performance and efficiency of AI systems, providing a common framework for comparison and progress tracking. This transparency fosters collaboration among researchers, vendors, and customers, driving innovation and preventing inflated claims.

The presentation highlights the crucial relationship between big data, big models, and big compute in achieving AI breakthroughs. A key chart illustrates how AI model performance significantly improves with increased data, but eventually plateaus. This necessitates larger models and more powerful computing resources, leading to an insatiable demand for compute power. MLPerf benchmarks help navigate this landscape by providing a standardized method of measuring performance across various factors including hardware, algorithms, software optimization, and scale, ensuring that improvements are verifiable and reproducible.

MLPerf offers a range of benchmarks covering diverse AI applications, including training, inference (data center, edge, mobile, tiny, and automotive), storage, and client systems. The benchmarks are designed to be representative of real-world use cases and are regularly updated to reflect technological advancements and evolving industry practices. While acknowledging the limitations of any benchmark, the presenter emphasizes MLPerf’s commitment to transparency and accountability through open-source results, peer review, and audits, ensuring that reported results are not merely flukes but can be validated and replicated. This approach promotes a collaborative, data-driven approach to developing more efficient and impactful AI solutions.


Enabling AI Ready Data Products with Qlik Talend Cloud

Event:

Appearance: Qlik Tech Field Day Showcase

Company: Qlik

Video Links:

Personnel: Sharad Kumar

In this video, Sharad Kumar, Field CTO of Data at Qlik, discusses how Qlik is revolutionizing how organizations create, manage, and consume data products, bridging the gap between data producers and business users. Qlik’s platform enables teams to deliver modular, trusted, and easily consumable data that’s packed with business semantics, quality rules, and access policies. With Qlik, data ownership, transparency, and collaboration are simplified, empowering organizations to leverage data for advanced analytics, machine learning, and AI at scale. Unlock faster decision-making, reduced costs, and impactful insights with Qlik’s data product marketplace and powerful federated architecture.


Transforming Data Architecture – Qlik’s Approach to Open Table Lakehouses

Event:

Appearance: Qlik Tech Field Day Showcase

Company: Qlik

Video Links:

Personnel: Sharad Kumar

In this video, Sharad Kumar, Field CTO of Data at Qlik, discusses the future of data architecture with Open Table-based Lakehouses. Learn how formats like Apache Iceberg are transforming the way businesses store and manage data, offering unparalleled flexibility by decoupling compute from storage. Sharad highlights how Qlik’s integration with Iceberg enables seamless data transformations, empowering customers to optimize performance and costs using engines like Spark, Trino, and Snowflake. Discover how Qlik simplifies building modern data lakes with Iceberg, providing the scalability, control, and efficiency needed to drive business success.


Driving AI Adoption with Qlik – Key Market Trends in Data and AI

Event:

Appearance: Qlik Tech Field Day Showcase

Company: Qlik

Video Links:

Personnel: Sharad Kumar

In this video, Sharad Kumar, Field CTO of Data at Qlik, dives into the latest market trends, including the rise of generative AI and its impact on data integration and analytics. Explore how organizations can unlock the full potential of AI by preparing data for real-time, AI-ready consumption. Qlik Talend Cloud empowers businesses to ensure data quality, enhance security, and make data more accessible and actionable. See how Qlik is building a trusted data foundation that drives smarter decision-making and sustainable AI success.


The Future of Wireless-as-a-Service

Event: Mobility Field Day 12

Appearance: Mobility Field Day 12 Delegate Roundtable

Company: Tech Field Day

Video Links:

Personnel: Tom Hollingsworth

Network-as-a-Service appears to be the future of operations. Executives love the idea of consistent costs and predictable upgrades. Operations teams have questions about the way that the solutions will be deployed and maintained. It’s almost like the two discussions going on aren’t speaking to the same audience. In this Mobility Field Day delegate roundtable, Tom Hollingsworth moderates a discussion between the traditional network engineering and operations teams as well as the people seeing the changes on the horizon. Hear the challenges that Network-as-a-Service might face in the wireless realm as well as the unease that IT teams feel when confronted with this new operational model.


Nile Access Service AI Network Optimization and Automated Day N Operations

Event: Mobility Field Day 12

Appearance: Nile Presents at Mobility Field Day 12

Company: Nile

Video Links:

Personnel: Ebrahim Safavi

Nile’s purpose-built AI network foundation eliminates common challenges like siloed data, fragmented feedback, and scalability limits seen in traditional AI solutions. Ebrahim Safavi, Head of AI Engineering, discusses how Nile’s advanced AI framework produces high-quality data and client-level insights, empowering faster deployment of generative AI enhancements specifically designed for streamlined network management.


Wi-Fi Optimization with the Nile AI Automation Center

Event: Mobility Field Day 12

Appearance: Nile Presents at Mobility Field Day 12

Company: Nile

Video Links:

Personnel: Dipen Vardhe

Optimizing wireless network access begins with a standardized architecture and an efficient data store. Dipen Vardhe, Head of Wireless Service & AI Automation Center, shares how the Nile AI Automation Center uses real-time network telemetry from the Nile Access Service to deliver AI-driven insights and automated optimizations. This approach enhances wired and wireless experiences while enabling zero-touch network administration.


Nile Access Service Network Planning, Design and Deployment

Event: Mobility Field Day 12

Appearance: Nile Presents at Mobility Field Day 12

Company: Nile

Video Links:

Personnel: Shiv Mehra

Shiv Mehra, VP of Service and Solutions, highlights how outdated approaches to campus networks often result in subpar designs and deployments, causing poor wired and wireless experiences. Explore how Nile leverages AI-driven automation to transform campus network design and deployment, delivering deterministic performance and ensuring exceptional end-user experiences.


Introduction to the Nile Access Service

Event: Mobility Field Day 12

Appearance: Nile Presents at Mobility Field Day 12

Company: Nile

Video Links:

Personnel: Suresh Katukam

Discover Nile’s revolutionary Campus Network-as-a-Service (NaaS), the industry’s first solution offering guaranteed wired and wireless performance, integrated Zero Trust security, and AI-driven self-healing technology—all available through a subscription model. Co-founder and CPO, Suresh Katukam discusses how wired and wireless networks can now be delivered as simply, securely, and reliably as electricity.


Cisco Amplified NetOps with AI

Event: Mobility Field Day 12

Appearance: Cisco Presents at Mobility Field Day 12

Company: Cisco

Video Links:

Personnel: Karthik Iyer, Minse Kim

Learn how Cisco AI-Enhanced Radio Resource Management (AI-RRM) improves wireless network performance and user experience. Never miss wireless connection issue using Intelligent Capture with Proactive PCAP.


Cisco Ultra-Reliable Wireless Backhaul (URWB) Update

Event: Mobility Field Day 12

Appearance: Cisco Presents at Mobility Field Day 12

Company: Cisco

Video Links:

Personnel: Dave Benham

Dive into one of the latest advancements in the Cisco Wireless portfolio, where Cisco simultaneously combines the Ultra-reliable Wireless Backhaul solution with Wi-Fi on the same AP.


Cisco Wi-Fi 7 – What You Need to Know

Event: Mobility Field Day 12

Appearance: Cisco Presents at Mobility Field Day 12

Company: Cisco

Video Links:

Personnel: Ameya Ahir, Nicholas Swiatecki

Wi-Fi 7 from Cisco is here! Get an overview of not only the APs that are now fully unified, but also insights into client ecosystem readiness and pitfalls. Cisco furthermore explores the real world impliciations of deploying a Wi-Fi 7 network, and how you can prepare for it.


Demonstrating the Codiac Value-Driven Engineering Platform

Event: AppDev Field Day 2

Appearance: Introducing Codiac at AppDev Field Day 2

Company: Codiac

Video Links:

Personnel: Michael Levan

Michael Levan demonstrated the Codiac Value-Driven Engineering Platform, focusing on simplifying the deployment and management of applications across Kubernetes clusters. Levan emphasized the challenges developers face when dealing with infrastructure and configuration management, particularly the repetitive nature of writing YAML or Infrastructure as Code (IAC) configurations. He highlighted how Codiac addresses these pain points by allowing users to deploy applications without worrying about the underlying infrastructure. Through a drag-and-drop interface, users can deploy containerized applications, such as a Go web app, without needing to manually manage Kubernetes manifests or other complex configurations. The platform abstracts much of the infrastructure management, making it easier for developers to focus on their applications rather than the environment they are running in.

Levan also demonstrated how Codiac allows for dynamic configuration management across different environments, such as development, staging, and production. Users can easily adjust parameters like replica counts for different environments without needing to maintain multiple Kubernetes manifests or use tools like Helm or Kustomize. The platform provides a central configuration system that can be modified per environment or per “cabinet,” which Levan likened to a Linux namespace or a mini-environment. This flexibility allows for more efficient application management, as users can make changes to configurations and redeploy applications either through the graphical interface or via the command line interface.

Additionally, Levan introduced the concept of “snapshots” within Codiac, which allows users to capture the state of their application stacks and easily redeploy them across different environments or clusters. This feature is particularly useful for scenarios like blue-green or canary deployments, where different versions of an application need to be tested or rolled out incrementally. The platform also supports cluster migrations, enabling users to move applications between clusters with minimal effort. Codiac abstracts much of the complexity of managing Kubernetes clusters, allowing developers to treat clusters as ephemeral resources that can be easily replaced or upgraded without manual intervention. Overall, the platform aims to streamline the deployment process, reduce the need for manual configuration, and provide a more efficient way to manage applications across multiple environments.


Value-Driven Engineering for Everyone with Codiac

Event: AppDev Field Day 2

Appearance: Introducing Codiac at AppDev Field Day 2

Company: Codiac

Video Links:

Personnel: Mark Freydl

Mark Freydl, CEO and Co-Founder, introduces Codiac, a platform designed to streamline the build and release process for SREs and development teams by addressing the friction and complexity that often arise in modern DevOps workflows. The platform focuses on simplifying the communication and coordination between different team members involved in the software development lifecycle (SDLC). By providing a common language and intuitive interface, Codiac aims to reduce the manual overhead and miscommunication that can occur when managing infrastructure and deployments. The platform offers both a CLI and GUI, allowing users to interact with it in various ways, whether through a browser, console, or pipeline, ensuring that all team members, from developers to project managers, can understand and contribute to the process.

One of the key features of Codiac is its “build once, configure on deploy” approach, which allows teams to build a container once and then configure it dynamically as it moves through different environments, such as development, QA, and production. This eliminates the need for manual configuration changes and reduces the risk of errors during deployment. The platform also supports snapshot deployments, where multiple services can be deployed together as a collective version, ensuring consistency across environments. Additionally, Codiac automates tasks like ingress management and environment scaling, further reducing the burden on SREs and allowing them to focus on higher-level discussions around performance and utilization rather than getting bogged down in the minutiae of YAML files and configuration management.

The motivation behind Codiac stems from the founders’ frustration with the growing complexity of modern infrastructure and the inefficiencies it creates for development teams. They recognized that while tools like Kubernetes offer powerful capabilities, they also introduce significant overhead, making it difficult for teams to move quickly and efficiently. By abstracting away much of the complexity and providing a more user-friendly interface, Codiac enables teams to focus on delivering value to the business rather than getting stuck in the technical weeds. The platform is designed to be extensible and adaptable to different workflows, making it a valuable tool for organizations looking to improve their DevOps processes and reduce the friction that often accompanies large-scale software development.


What’s Next for Heroku from Salesforce

Event: AppDev Field Day 2

Appearance: Heroku Presents at AppDev Field Day 2

Company: Heroku

Video Links:

Personnel: Chris Peterson

In this presentation, Chris Peterson, Senior Director of Product Management at Heroku, discusses the future of Heroku and its foundational principles, particularly the 12-Factor App Manifesto. This manifesto, created by Heroku co-founder Adam Wiggins, outlines best practices for building scalable and maintainable applications. These principles have guided Heroku’s development, ensuring that apps built on the platform can scale horizontally and integrate seamlessly with services like databases through environment variables. Heroku is now revisiting and modernizing the 12-Factor App Manifesto to address contemporary needs such as app identity, logging, and telemetry, and is actively seeking feedback from the developer community through open-source discussions.

Peterson also highlights recent advancements in Heroku’s scalability and language support. In 2024, Heroku introduced new features to ensure that customers can scale both horizontally and vertically, offering larger dynos with up to 128GB of memory and smaller options for enterprise customers. The platform has also modernized its language support, adding faster package managers like PNPM and new tools like Poetry for Python. Additionally, Heroku has expanded its Postgres offerings, providing larger database plans and new versions to accommodate growing customer needs. The platform has also integrated with other Salesforce services, such as Mulesoft Flex Gateway, to enhance API management and security within private spaces.

Looking ahead, Heroku is focusing on enhanced networking, including HTTP2 and HTTP3 support, and expanding its language ecosystem with the addition of .NET support. The platform is also working on deeper integrations with Salesforce through event-driven and API-driven solutions, allowing developers to easily connect Heroku apps with Salesforce events and APIs. Heroku is also embracing open standards, particularly in the Kubernetes ecosystem, and is collaborating with AWS to leverage new services and technologies. These efforts are part of a broader strategy to re-platform Heroku and refresh its core values, ensuring it remains a leading platform for developers in the cloud-native era.


« Previous Page
Next Page »