Watch on YouTube
Watch on Vimeo
With aiDAPTIV+, Phison makes on-premises AI processing more accessible and affordable, especially for small and medium-sized businesses, government entities, and educational institutions. CTO Sebastien Jean explained that the primary goal of Phison’s solution is to facilitate fine-tuning of large language models (LLMs) on-site. Fine-tuning often demands significantly more memory than inference, making it expensive and difficult for those without massive budgets or a lot of resources. The presentation highlighted the massive memory requirements for fine-tuning, which can be up to 20 times the memory needed to run an LLM, driving up costs and making it impossible for some organizations to begin with this approach.
Phison’s solution addresses this challenge by decoupling compute and memory. Sebastien Jean, Phison’s CTO, focused on how Phison’s technology, with its AI-optimized SSDs and middleware, enables on-site LLM training and inference. The product uses a combination of their proprietary middleware, Adaptive Link, and custom-built ProSuite software to manage and extend the memory available to PyTorch, effectively turning an SSD into an extended memory pool. This architecture allows for training large models using fewer GPUs. The system uses a software layer within PyTorch that intercepts calls and then offloads slices of the model to the SSD, which helps in memory management.
By leveraging SSDs and their proprietary controller technology, Phison offers a cost-effective alternative to expensive GPU-intensive setups and targets the SMB, government, and education markets with this solution. The presentation concluded with a focus on the financial benefits and the sustainability of the solution. By allowing for more efficient hardware utilization, Phison provides not just a financially smart solution but one with power and cooling benefits as well. Also, by using repurposed NAND, the solution can increase the lifespan of hardware, reduce electronic waste, and extend the useful life of data center infrastructure.
Personnel: Sebastien Jean
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!
