Getting Started with VMware Private AI Foundation with NVIDIA

In this session, we will take a practitioner’s point of view of Private AI, walking through the value of not trying to create a do it yourself AI infrastructure, how to pick the right GPU for your organization, and delivering your AI use case.

In this presentation, Chris Gully from VMware by Broadcom discusses the challenges and solutions for organizations embarking on their AI journey, particularly focusing on the importance of not attempting to build AI infrastructure from scratch. He emphasizes that simply acquiring GPUs and installing them in servers is not enough to create a functional AI environment. There are numerous logistical considerations, such as power, airflow, and compatibility, that need to be addressed. Gully advocates for purchasing pre-validated and certified solutions to ensure a smoother experience, better support, and faster deployment of AI services. He also highlights the importance of selecting the right GPU for specific AI use cases, as different GPUs offer varying levels of performance and functionality.

Gully also delves into the complexities of GPU virtualization and the benefits of using technologies like VGPU and MIG (Multi-Instance GPU) to optimize resource utilization. These technologies allow organizations to slice and share GPU resources more effectively, ensuring that expensive hardware is used efficiently across multiple business units. He shares real-world examples of customers who faced challenges when their AI models did not fit the GPUs they had selected, underscoring the importance of understanding the technical requirements of AI workloads before making hardware decisions. Gully also discusses how VMware works closely with OEMs and NVIDIA to ensure that their solutions are fully certified and supported, providing customers with confidence that their AI infrastructure will work as expected.

The presentation further explores VMware’s Private AI Foundation, which integrates NVIDIA’s AI technologies into VMware’s Cloud Foundation (VCF) platform. This solution provides a streamlined, automated approach to deploying AI workloads, allowing organizations to quickly roll out AI use cases without the need for extensive manual configuration. Gully explains how VMware’s automation tools, such as the VCF Quick Start, enable rapid deployment of AI environments, reducing the time it takes to get AI models up and running. He also highlights the flexibility of the platform, which allows customers to customize their AI environments and add proprietary models to their catalogs. Overall, the session emphasizes the importance of simplifying AI infrastructure deployment and management to help organizations realize the value of AI more quickly and efficiently.