Maximizing your GPU’s with a Tuned OS with CIQ

Brian Dawson from CIQ discusses how to leverage the operating system to optimize GPU performance for AI workloads, particularly for inference. CIQ introduces its AI stack, emphasizing the OS layer, and highlights its RLC Pro family of products, including RLC Pro AI, a vendor-backed version of Rocky Linux specifically tuned for GPUs and performance.

RLC Pro AI is designed for production inference, demonstrating significant performance improvements. In some cases, it achieved 32% higher throughput for image segmentation on Ampere-generation hardware. The solution also boasts 9x faster time to first inference, reducing setup time from a clean OS install to first token output from 30 minutes to 3 minutes and 43 seconds, primarily due to pre-baked and tuned configurations. CIQ provides an 18-month kernel advantage with vendor-backed feature commits and improvements not found in traditional enterprise Linux. The offering includes pre-tuned and validated frameworks such as PyTorch, Torch Vision, Ollama, and VLLM, along with optimized NVIDIA and AMD (ROCm) drivers, and kernel-level tunings for memory allocation and transparent huge pages.

CIQ aims to provide a consistent environment across bare metal, on-premises, Kubernetes, and major cloud platforms (AWS, Azure, GCP). The product addresses challenges like post-installation driver assembly, per-instance driver installation across diverse fleets, and compatibility drift. CIQ’s continuous validation process includes regular benchmarks to ensure performance and prevent degradation, with plans for continuous benchmarking with every build. The offering also features enterprise-grade commercial backing, including CVE patching and remediation with SLAs, secure boot, and FIPS readiness. This comprehensive approach reduces the need for customers to build and maintain custom images, enabling engineers to focus on higher-level tasks and maximizing the return on GPU investments, especially for organizations where even minor performance gains translate into significant value.