Scality Presents at Cloud Field Day 23

How we Help Our Customers to Build Exabyte-Scale Clouds with Scality

Watch on YouTube
Watch on Vimeo

Paul Speciale’s presentation at Cloud Field Day 23 highlighted Scality’s approach to helping customers build exabyte-scale clouds. The presentation opened by addressing the shift towards private cloud computing driven by AI workloads and data sovereignty concerns. Scality RING, already managing a significant amount of data, is chosen by market leaders, including major telcos and banks, to maintain control and achieve cloud-scale performance. The presentation’s core message establishes the business imperative that is driving enterprises away from public cloud dependencies and towards hybrid architectures for both compliance and competitive advantages.

Scality’s presentation focuses on its RING product as the primary solution for cloud infrastructure, data lakes, and data protection. It emphasized the RING’s S3-compatibility, deep support for advanced APIs, and a cloud-like identity and access management system. Furthermore, the presentation highlighted the RING’s distributed data protection, geo-stretched capabilities for high availability, utilization tracking, and the Core 5 initiative that focuses on cyber resiliency. The presentation emphasized the importance of multiscale architecture in a cloud environment due to the varying workload patterns and I/O needs.

The presentation showcased Scality’s market entry in 2010, coinciding with the rise of cloud services. Scality aimed to provide scale-out storage solutions with their Ring product, which has been in the market since 2010 and has been adopted by major telcos and financial institutions. Scality’s presentation also includes customer stories involving U.S. and European banks, the space industry, and Iron Mountain, highlighting the versatility of Ring for various applications and deployment sizes. Scality’s response to questions highlighted backup and automated tiering capabilities within the RING system, underscoring its design for high-capacity use cases.

Personnel: Paul Speciale

S3 Object Storage for Real-Time Compliance Analytics at a Large US Global Bank with Scality

Watch on YouTube
Watch on Vimeo

Ben Morge, VP of Customer Success at Scality, presented a deployment of Scality RING for a large US bank that needed to store 40 petabytes of Splunk SmartStore data across two sites.  The bank required active-active replication and a one-year data retention period. RING’s S3 compatibility enabled seamless integration with Splunk, allowing the indexers to tier data from hot, fast flash storage to the warm Scality RING, which consisted of 80 servers.  The data is immutable on the ring for one year, with the Splunk application handling data deletion.

The Scality deployment leverages S3 for data storage. The solution employs a two-site architecture, where each site features Splunk indexers and a hot storage cluster. The indexers are responsible for replicating indexes and references to objects stored on the ring. Scality manages object replication, utilizing cross-region replication, which is the S3 standard. The system addresses network issues by employing infinite replay to ensure reliable replication and decoupling storage from compute.

The presentation highlighted a couple of challenges and their solutions. The initial ingest exceeding firewall throughput was resolved with infinite replay. The initial difficulties with the read traffic exceeding CPU capabilities, leading to overwhelmed flash caches, were addressed by decoupling the architecture and adding compute resources to handle the metadata layer and S3 stateless services, resulting in simultaneous 75 gigabytes throughput on both sites and a fully replicated cluster with all the objects replicated in under two minutes.  The customer’s active production has been running successfully for over five years.

Personnel: Ben Morge

Leading Space Agency’s Long-Term Scientific Storage at Scale with Scality

Watch on YouTube
Watch on Vimeo

Scality’s presentation at Cloud Field Day 23 focused on their collaboration with a major European space agency, which utilizes Scality RING to manage massive scientific datasets in a hybrid storage model. The agency, dealing with data from approximately 200 satellites, faced challenges with legacy storage solutions and the need for a cost-effective, scalable, and easily accessible system for both live and archival data. Scality’s solution utilizes S3 as a unified namespace, providing a single access point for data regardless of its location, whether on hot or cold storage.

The solution employs a multi-tiered approach, where live data is stored on the RING for active analysis and subsequently moved to tape for long-term archiving after six months. This vast amount of cold data, representing hundreds of petabytes, is managed using a custom-built API called TLP (Tape Library Protocol) to integrate with HSMs from partners such as HP, Atempo, IBM, and Spectra. TLP handles the retrieval and storage of data to tape, providing transparent access for users through the S3 Glacier API. This provides cost savings and energy efficiency by moving data to tape when it is not frequently accessed.

This architecture offers several advantages, including data durability through a three-site stretch ring, with data replicated across two tape libraries for enhanced resilience. The agency’s users and applications interact with the data via a single namespace using S3, unaware of the underlying complexity of the hybrid storage system. This transparency, combined with the cost-effectiveness of the solution and security features like object lock, has made Scality’s solution a key enabler for long-term data access and efficient workflows for the space agency.

Personnel: Nicolas Sayer

Major European Bank’s Enterprise Cloud Built on RING & S3 APIs with Scality

Watch on YouTube
Watch on Vimeo

Aurelien Gelbart’s presentation at Cloud Field Day 23 highlighted a major European bank’s successful deployment of a private cloud built on Scality RING and S3 APIs. The bank sought to consolidate disparate storage solutions, aiming for a lower cost per terabyte and easier adoption for its users. Leveraging Scality’s offerings, the bank achieved these goals, fostering widespread adoption of the new platform and significantly reducing costs.

The bank’s private cloud architecture comprises six independent RING clusters across three geographical regions, each with a production and disaster recovery platform. Utilizing Scality’s native S3 API support, the bank implemented multi-site replication and object lifecycle policies to meet stringent financial compliance requirements. The implementation allows the bank to run hundreds of production-grade applications, including database backups, financial market data storage, and document management.

The success of this deployment is evident in substantial growth, with the platform scaling from one petabyte to 50 petabytes of usable storage within seven years. This growth brought new challenges related to performance and resource management across the many different applications. Scality addressed these challenges by improving software performance through reconfiguration and architectural improvements. The results are impressive: the bank now operates 100 petabytes of usable storage, manages 200,000 S3 buckets, and processes 300 billion client objects, achieving a 75% reduction in the cost per terabyte per year compared to its previous solutions.

Personnel: Aurelien Gelbart

Learn About Scality RING’s Exabyte Scale, Multidimensional Architecture with Scality

Watch on YouTube
Watch on Vimeo

Scality’s Giorgio Regni presented at Cloud Field Day 23, focusing on the Scality RING’s exabyte-scale, multidimensional architecture. Scality’s origin story stems from addressing storage challenges for early cloud providers, such as Comcast. They found that existing solutions weren’t meeting the demands of petabyte-scale data and the need to compete with large providers. The company’s core concept is “scale,” and their system is designed to expand seamlessly across all crucial dimensions. This includes capacity, metadata, and throughput, allowing them to scale each of these components independently.

Regni emphasized the RING’s disaggregated design, highlighting its ability to overcome common storage bottlenecks. The architecture separates storage nodes, I/O daemons, and a connector layer, enabling independent scaling of each component. He shared impressive numbers, including 12 exabytes of data currently in production and 6 trillion objects stored, with customers having billions of objects and applications using the system. The presentation also contrasted Scality’s approach to that of competitors like Ceph and MinIO, highlighting differences in metadata handling, bucket limits, and the flexibility of the architecture’s scaling capabilities.

Finally, the presentation covered the multi-layered architecture that supports various protocols, including S3, a custom REST protocol, and file system connectors. The architecture is based on a peer-to-peer distributed system with no single point of failure, supporting high availability and replication across multiple sites and tiers. It can manage different tiers, such as Ring XP, the all-flash configuration, and long-term storage. Scality RING also offers multi-tenancy and supports usage tracking, allowing customers to build their billing systems, with the overall goal of the system being an infinitely scalable storage solution.

Personnel: Giorgio Regni