Intel Optane Memory Requirements of Meta AI Workloads

Complexity of AI workloads scales rapidly and entails faster scaling of memory requirement than scaling of the memory technology. In this talk, we break down the memory requirement for recommendation models, as one of the main drivers of AI platforms, and make the case for a new tier of memory, optimized for capacity, to help cope with workload scaling. Using Intel Optane SSD with software-defined memory sales power and allows for a simpler serving paradigm. While the talk argues that CXL-attached memory is a viable solution, as a precursor, it presents empirical results of leveraging additional memory tier using currently available NVMe-attached Storage Class memory. Presented by Manoj Wadekar, Hardware Systems Technologist at Facebook, Ehsan Ardestani, Research Scientist and Technical Lead Manager, Meta, and Chris Petersen, Hardware Systems Technologist, Intel.