Watch on YouTube
Watch on Vimeo
Solidigm focused on the evolving role of storage in AI, specifically highlighting its significance in the AI data pipeline through 2025. The presentation emphasizes that the AI workflow involves a series of distinct tasks, each with unique demands on hardware, and that it’s often distributed among different organizations. The presentation identifies six primary stages in this pipeline: ingestion, data preparation, model training, fine-tuning, inference, and archiving. Solidigm’s perspective underscores the increasing data intensity of these steps, especially during data ingestion, inference, and archiving.
Solidigm explores how storage choices impact these stages, distinguishing between direct-attached storage within GPU servers and network-attached storage. Direct-attached storage is optimized for performance-intensive tasks, while network-attached storage is used for larger datasets and offers capacity and cost-effectiveness. Ace Stryker highlights the rising importance of network storage, driven by advances in network bandwidth and the growing size and complexity of AI models and larger datasets for RAG. The key takeaway is that great storage facilitates larger models, longer interactions, and improved outputs on the inference side.
Finally, the presentation showcases a collaboration with Metrum AI, presenting a real-world demo that addresses the challenges of data-intensive inference by offloading model weights and RAG data to SSDs. This allows running 70 billion parameter models on less powerful hardware, which would have been impossible without storage-based offloading, saving on costs and hardware requirements. The demo emphasizes the potential of SSDs to enhance performance in AI applications by reducing GPU memory usage. The collaboration offers insights into the benefits of leveraging storage in AI, as the partnership also showed that offloading to SSD could provide the same or better performance as DRAM in this application of retrieval augmented generation.
Personnel: Ace Stryker, Steen Graham
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!
