Warm Flash for AI Context Storage using the Open Flash Platform with Xsight Labs and Hammerspace

The Open Flash Platform, co-founded by Xsight Labs and Hammerspace, introduces a new approach to “warm flash storage” for AI context, promising enhanced efficiency and performance. This collaborative effort leverages Hammerspace’s software, built on a Linux-based NFS file system that supports distributed management. By separating metadata from the data path and decentralizing storage, the platform eliminates traditional x86 servers, drastically reducing total cost of ownership, power consumption, and system complexity. This streamlined architecture also minimizes data hops, improving performance for large AI clusters by enabling direct data access to storage targets while managing metadata out of band.

Xsight Labs contributes its E1 DPU, forming the core of a unique cartridge design developed with Lumineer. Each cartridge is a self-contained “server on a cartridge,” featuring two 400 GbE ports for NVMe-style fabric and eight flash drives. These cartridges offer exceptional density, allowing for exabyte-scale memory within a standard seven-foot rack. The E1 chip’s ability to run a full Linux operating system makes it more powerful than a simple network interface card, supporting additional use cases beyond mere data transport. Data protection is managed through erasure coding across multiple blades rather than within individual SSDs, embracing a model where the blade is the consumable unit, further simplifying infrastructure and reducing failure points.

The concept of “warm flash” is vital for AI, as these applications require data that is always ready and accessible, rather than relying on traditional cold storage. This aligns with a growing customer demand to move away from disk drives towards all-flash environments, even for archival purposes, as flash longevity has significantly improved. The Xsight Labs E1 chip is precisely balanced for this, delivering optimal throughput without overprovisioning and ensuring quick data extraction even from very large flash capacities per blade. With significant market interest, the product is moving towards production, with initial releases expected soon and full production anticipated by year-end, underscoring a successful partnership focused on software-defined efficiency.