Build a composable AI infrastructure Stack with RLC Pro AI and CIQ
- Why an AI Kit vs an AI Factory
- Maximimzing your GPU’s with a Tuned OS
- Distributed AI Workflows with Fuzzball
Build a composable AI infrastructure Stack with RLC Pro AI and CIQ
Watch on YouTube
Watch on Vimeo
Brian Dawson from CIQ discusses the need for a composable AI infrastructure stack, advocating for an “AI kit” approach over fixed “AI factories.” He highlights that the AI technical evolution cycle is moving faster than ever, with every layer of the stack, including hardware, evolving on rapid cycles (one to three months or within a year). This rapid pace pressures organizations to quickly move from research to production, often leading them to adopt rigid “AI factory” solutions. However, Dawson warns that tying into a fixed stack risks locking into something that could become obsolete or antiquated in a matter of months, stressing the importance of building for flexibility.
Brian Dawson, leading product strategy at CIQ, explains that the company is behind Rocky Linux, founded by Gregory Kurtzer, who also co-founded CentOS and created products like Fuzzball and AppTainer. CIQ’s expertise is rooted in HPC and Linux, providing a unique approach to assembling the AI stack. Their core thesis is that existing infrastructure is unprepared for AI’s demands, necessitating a modern IT stack. Challenges include rising enterprise Linux costs (despite Linux running 95% of AI workloads), increased security criticality for valuable AI data, and immense spending on GPU hardware that often sits idle due to difficulties in setting up the surrounding stack. Operations are fragmented, with separate solutions for traditional fleets and GPU nodes. CIQ proposes a new hybrid-first integrated stack, built on a “homogeneous foundation that can be deployed to support heterogeneous use cases,” to address these issues, allowing flexibility and avoiding vendor lock-in through open-source components and workload-tuned variants.
The presentation emphasizes the critical, often forgotten, role of the OS layer beneath the GPUs. Dawson explains that the OS, particularly the kernel, controls access to all expensive resources, enabling the workload layer to tap into the compute layer. While some control is shifting towards the GPU, the kernel remains fundamental for tasks like turning devices on and off, configuring them, and managing data streaming from storage. An “AI kit” from CIQ is designed to be built from necessary layers, evolving with changing compute (bare metal to Kubernetes) and supporting diverse performance-intensive computing workloads and heterogeneous infrastructure, including on-prem, cloud, VPC, and various processor architectures (XPUs, NPUs, DPUs, TPUs, NVIDIA, AMD). This flexible approach ensures that the infrastructure can adapt to the fast-paced, diverse demands of the AI landscape, providing a stable base while supporting underlying changes.
Personnel: Brian Dawson
Watch on YouTube
Watch on Vimeo
Brian Dawson from CIQ discusses how to leverage the operating system to optimize GPU performance for AI workloads, particularly for inference. CIQ introduces its AI stack, emphasizing the OS layer, and highlights its RLC Pro family of products, including RLC Pro AI, a vendor-backed version of Rocky Linux specifically tuned for GPUs and performance.
RLC Pro AI is designed for production inference, demonstrating significant performance improvements. In some cases, it achieved 32% higher throughput for image segmentation on Ampere-generation hardware. The solution also boasts 9x faster time to first inference, reducing setup time from a clean OS install to first token output from 30 minutes to 3 minutes and 43 seconds, primarily due to pre-baked and tuned configurations. CIQ provides an 18-month kernel advantage with vendor-backed feature commits and improvements not found in traditional enterprise Linux. The offering includes pre-tuned and validated frameworks such as PyTorch, Torch Vision, Ollama, and VLLM, along with optimized NVIDIA and AMD (ROCm) drivers, and kernel-level tunings for memory allocation and transparent huge pages.
CIQ aims to provide a consistent environment across bare metal, on-premises, Kubernetes, and major cloud platforms (AWS, Azure, GCP). The product addresses challenges like post-installation driver assembly, per-instance driver installation across diverse fleets, and compatibility drift. CIQ’s continuous validation process includes regular benchmarks to ensure performance and prevent degradation, with plans for continuous benchmarking with every build. The offering also features enterprise-grade commercial backing, including CVE patching and remediation with SLAs, secure boot, and FIPS readiness. This comprehensive approach reduces the need for customers to build and maintain custom images, enabling engineers to focus on higher-level tasks and maximizing the return on GPU investments, especially for organizations where even minor performance gains translate into significant value.
Personnel: Brian Dawson
Watch on YouTube
Watch on Vimeo
This presentation by CIQ at AI Infrastructure Field Day 6 introduces FuzzBall as a solution for orchestrating distributed AI workflows. Brian Dawson from CIQ explains that modern compute stacks, especially for AI, require addressing multiple layers from hardware capitalization to reliable workload support, including orchestration, operation, and automation. FuzzBall sits in the orchestration layer, running on top of Kubernetes, to modernize traditional HPC organizations’ stacks and accelerate distributed workload deployment.
FuzzBall’s core capability is to let users run the same workflow on-premises, in public cloud, or on a GPU cloud without rewriting it, enabling federation across clusters and compute resources. Initially designed for batch HPC and stateless jobs, it has evolved to support both stateful and stateless AI workloads, addressing the challenge of unifying diverse fleets that may include HPC, general Kubernetes, and Gen AI demands. A key feature is the ability to describe a job, including compute, images, and storage, in a single workload file, making it portable. The solution also supports checkpointing, allowing for resilience and continuous tracking of jobs, and offers a workflow catalog for easy deployment of jobs via a “run” button.
Operationally, FuzzBall provides fault-tolerant scheduling, the ability to restart failed jobs, multi-tenancy, resource accounting, and support for both AMD and NVIDIA GPUs under a single control plane. It emphasizes operational hardening, including federation, auto-scaling, and managing Kubernetes certificates. CIQ aims to deliver a complete infrastructure stack that allows organizations to maximize Linux and GPU access, ensures built-in security, and provides an automated stack across diverse fleets, without demanding a full “AI factory” adoption. They encourage exploring their products, including Rocky Linux from CIQ Pro and CIQ Stratum Kubernetes Engine.
Personnel: Brian Dawson
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!