Falling Down a LLM Rabbit Hole with Ben Young

In this Edge Field Day Delegate Ignite Talk, Ben Young shares his journey into building a chatbot web interface, with a specific focus on creating a chatbot for the Veeam knowledge base articles using OpenAI models. Along the way, he delves into the concept of retrieval augmented generation (RAG) and the challenges of relying solely on large language models, such as limited access to private data and high retraining costs. To overcome these challenges, Young emphasizes the importance of understanding the chatbot’s inner workings, from document retrieval to interpretation and answer generation. He describes the process of building the knowledge base dataset and the API, highlighting the role of embeddings in matching queries to relevant information within the database. Young also explores different options for data storage, including vector databases and traditional databases with vector support, while noting the use of tools like Superbase and Langchain to manage queries and interactions with the data. Ultimately, he underscores the power of language models like Vani GPT in providing accurate and targeted responses, emphasizing the significance of crafting effective queries and prompts for optimal results.