Watch on YouTube
Watch on Vimeo
As large language models scale, new challenges emerge – not only in maximizing GPU performance but also in validating the infrastructure that fuels the data pipeline used for training. On the front end, this includes securely ingesting user data from distributed cloud and customer environments into centralized AI data centers and ensuring high-speed, low-latency data transfer between VMs and hosts within the data centers. This session focuses on testing products that evaluate the performance, scalability, latency, reliability, and security of critical data pathways.
The presentation addresses the critical role of front-end networks in ensuring efficient data flow to data-hungry GPUs for LLM training. It highlights the two primary data movement patterns: north-south, involving data ingestion from sources like cloud providers and user environments, and east-west, which focuses on data transfer between virtual machines within the data center. Each of these patterns has unique testing demands, with east-west requiring ultra-low latency and high line rates, while north-south necessitates a balance of minimal latency and strong security to protect user data in transit.
Keysight’s solution, Cyperf, is a software-based traffic generator designed to emulate various application traffic types and measure key performance indicators like bandwidth, latency, connections per second, and security. Cyperf ensures that GPUs receive data with low latency, security, and at the promised rate of the network infrastructure. The presenter emphasized the importance of thoroughly testing all layers of the OSI model, especially the application layer, to avoid relying on end-users as beta testers. The presenter lauded Crusoe for proactively testing their front-end infrastructure to ensure optimal performance.
Personnel: Amritam Putatunda
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!
