The Art of the Possible with VMware Private AI

This session will discuss how VMware’s Private AI architectural approach enables the flexibility to run a range of GenAI solutions for your environment. We’ll explore how customers can achieve business value by running applications on a Private AI that offers unique advantages in privacy, compliance, and control. We will demo the “VMware Expert” app, built on VMware Private AI. Join us to learn how your organization can maximize its data strategy with this powerful platform.

In this session, Ramesh Radhakrishnan from VMware by Broadcom discusses the potential of VMware’s Private AI platform in enabling organizations to run generative AI (GenAI) solutions while maintaining control over privacy, compliance, and data management. He emphasizes that once customers have deployed VMware Private AI in their environments, the next challenge is demonstrating business value. The platform provides a flexible infrastructure that allows developers, data scientists, and software engineers to leverage GPUs for various AI applications. Radhakrishnan’s team, which includes infrastructure experts, software developers, and data scientists, has been working internally to build services on top of this platform, such as Jupyter notebooks and Visual Studio IDE environments, which allow users to access GPUs and AI capabilities for tasks like code completion and large language model (LLM) development.

One of the key services highlighted is the LLM service, which functions similarly to OpenAI but is designed for regulated industries that require strict control over data. This service allows organizations to run LLMs on their private infrastructure, ensuring that sensitive information is not exposed to third-party providers. Additionally, Radhakrishnan introduces the “VMware Expert” app, an internal tool that leverages AI to improve documentation search and provide expert Q&A capabilities. The app has evolved from a basic search tool using embedding models to a more advanced system that integrates retrieval-augmented generation (RAG) techniques, allowing users to interact with large language models that are fine-tuned with VMware-specific knowledge. This tool has shown significant improvements in search accuracy, with results being five to six times better than traditional keyword searches.

Radhakrishnan also discusses the challenges of ensuring that AI-generated answers are accurate and not prone to hallucination, a common issue when the LLM is not provided with the correct documents. To address this, VMware is exploring corrective RAG techniques and post-training methods to embed domain-specific knowledge directly into the models. This approach, which involves fine-tuning large language models on VMware’s internal documentation, has shown promising results and can be replicated by other organizations using VMware Private AI. The session concludes with a demonstration of the “VMware Expert” app and a discussion on how organizations can use VMware’s platform to build their own AI-driven solutions, maximizing the value of their data and infrastructure.