Cohesity Data Platform Deep Dive

In this presentation at Storage Field Day 8, Johnny Chen provides a detailed overview of the Cohesity Data Platform, focusing on its architecture and capabilities aimed at addressing issues related to secondary storage, which include fragmentation, silos, and challenges in copy data management. The talk is segmented into several parts, beginning with discussions on the scale-out distributed architecture of the Cohesity file system, which is designed to handle mixed workloads effectively and includes an adaptive self-healer for system maintenance and operations. The system’s hardware comprises a 2U chassis with four nodes, each equipped with dual CPUs, memory, SSDs, and hard drives, allowing substantial flexibility and scalability for various enterprise needs, including data protection, DevOps workflows, and analytics.

Chen delves into the specifics of the Cohesity OASIS architecture, highlighting elements such as the distributed lock manager, a strongly consistent NoSQL store, and the intelligent coordination required to ensure seamless integration and operation of multiple nodes within the cluster. Particularly noteworthy is their method of metadata management, including creating, managing, and ensuring the transactional integrity of file operations through a distributed NoSQL store and a two-phase commit process. The platform also employs innovative approaches to manage and optimize data storage through methods like global deduplication and adaptive tiering, which dynamically moves data between SSDs and HDDs based on access patterns, ensuring efficient utilization of storage resources and maintaining performance.

Additionally, Cohesity’s approach to mixed workloads and performance isolation is geared toward maintaining high efficiency and preventing heavy operations, such as large backup jobs, from affecting the performance of other tasks within the system. This is achieved through a user-defined quality of service (QoS) management system, which allocates resources proportionally based on predefined priorities. The self-healing capabilities of the system, running continuously at a low-priority backdrop, ensure that the system remains optimized and fault-tolerant, capable of handling tasks like garbage collection, disk rebalancing, and data replication seamlessly without disrupting primary operations. This continuous background process underscores the platform’s resilience and ability to operate efficiently even under diverse and demanding workload conditions.