VMware Private AI Foundation with NVIDIA Overview

VMware Private AI Foundation with NVIDIA is a fully integrated solution featuring generative AI software and accelerated computing from NVIDIA, built on VMware Cloud Foundation and optimized for AI. The solution includes integrated AI tools to empower enterprises to customize models and run generative AI applications adjacent to their data while addressing corporate data privacy, security and control concerns. The platform will feature NVIDIA NeMo, which combines customization frameworks, guardrail toolkits, data curation tools and pretrained models to offer enterprises an easy, cost-effective and fast way to adopt generative AI.

In this presentation, Justin Murray, Product Marketing Engineer at VMware by Broadcom, discusses the VMware Private AI Foundation with NVIDIA, which is a solution designed to run generative AI applications with a focus on privacy, security, and control for enterprises. The platform is built on VMware Cloud Foundation and optimized for AI, featuring NVIDIA NeMo for customization and generative AI model deployment.

Murray explains the architecture of the solution, which includes a self-service catalog for data scientists to easily access their tools, GPU monitoring in the vCenter interface, and deep learning VMs pre-packaged with data science toolkits. He emphasizes the importance of vector databases, particularly PG vector, which is central to retrieval-augmented generation (RAG). RAG combines database technology with large language models to provide up-to-date and private responses to queries.

He also touches on the GPU operator and Triton inference server from NVIDIA for managing GPU drivers and scalable model inference. Murray notes that the solution is designed to be user-friendly for data scientists and administrators serving them, with a focus on simplifying the deployment and management of AI applications.

Murray mentions that the solution is compatible with various vector databases and is capable of being used with private data, making it suitable for industries like banking. He also indicates that there is substantial demand for this architecture across different industries, with over 60 customers globally interested in it before the product’s general availability.

The presentation aims to provide technical details about the VMware Private AI Foundation with NVIDIA, including its components, use cases, and the benefits it offers to enterprises looking to adopt generative AI while maintaining control over their data.