Scality Presents at AI Field Day 8

Why Data Infrastructure Needs to Become Autonomous with Scality ADI

Watch on YouTube
Watch on Vimeo

Scality unveiled its new product, Scality ADI, or Autonomous Data Infrastructure, which aims to address significant market shifts and customer challenges. Paul Speciale, the Chief Marketing Officer, introduced the solution, highlighting Scality’s 15 years of experience in storage software, with a focus on AI workloads, cyber resilience, and cloud infrastructure. Scality operates at an impressive scale, managing 12 exabytes of customer data globally across 70 countries, with individual deployments reaching hundreds of petabytes and delivering exceptional performance, including terabytes per second throughput and microsecond latency.

The presentation emphasized that the data infrastructure industry is at a crucial inflection point, driven by the demands of the AI era. While previous cycles involved web scale unstructured data and cloud native containerized applications using object storage and S3, AI introduces a new level of data volume, diversity in workloads, and varying performance and resiliency requirements across the data pipeline. Critical challenges include price spikes and extended lead times for storage media like NAND flash and hard disk drives, along with escalating power consumption and data center construction costs. These issues stem from inflexible, single-tiered storage architectures.

Scality ADI offers a systemic solution by providing a more flexible and dynamic approach to data management, moving away from inefficient storage silos. An illustrative customer example is the European Space Agency (Kness), which benefits from a single S3 endpoint across multiple storage tiers, enabling smart data lifecycle management for petabytes of imaging data. Scality is uniquely positioned to deliver this, leveraging its extensive experience with large-scale systems, hardware independence allowing customer choice, and an innovative open code model for transparency and collaboration. Furthermore, Scality is embracing the NVIDIA ecosystem with S3 over RDMA, facilitating direct and high-speed GPU access for demanding AI applications.

Personnel: Paul Speciale

How Do You Build a UI for an Exabyte-Scale Distributed Storage System with Scality

Watch on YouTube
Watch on Vimeo

Scality CTO Giorgio Regni opened by addressing the complex challenge of managing exabyte-scale distributed storage systems, which involve thousands of drives, intricate networking, and microservices architecture. He recounted Scality’s twelve-year journey in UI and UX design, starting with early customers like internet service providers who, coming from traditional SAN environments, demanded extensive control and visibility. This led to an initial design phase characterized by numerous manual controls and detailed dashboards, reflecting a desire to manage every aspect of the system, even when processes like data repair and balancing were automated.

However, this approach soon revealed its limitations. The proliferation of dashboards and alerts led to “dashboard fatigue,” where operators became overwhelmed by information and desensitized to notifications, missing critical issues until it was too late. Similarly, providing an excessive number of configuration variables, often requested by customers influenced by systems like Kubernetes, was deemed counterproductive. Regni highlighted that expecting customers to tune intricate distributed system parameters was impractical, as even engineers struggled with such granularity. This era solidified the realization that relying on humans as the primary control plane for such vast systems was inefficient and prone to error.

Scality’s new direction, embodied in their ADI design, aims to reinvent system management. It shifts from an imperative “tell the system to do things” model to a declarative one, where users specify desired outcomes, and the system autonomously delivers and monitors them. This modern UX leverages AI and expert agents to continuously observe the system’s state, act within defined guardrails, explain its actions, and escalate to human operators only when complex judgment or high risk is involved. The goal is a learning system that minimizes human login frequency, as happier customers are those who implicitly trust the system to manage itself effectively, allowing human intervention to be reserved for truly exceptional circumstances.

Personnel: Giorgio Regni

Introducing Scality ADI – Autonomous Data Infrastructure

Watch on YouTube
Watch on Vimeo

Scality introduced its Autonomous Data Infrastructure, Scality ADI, at AI Field Day 8, positioning it as a necessary evolution beyond traditional storage models that are no longer adequate for the demands of artificial intelligence. Billy Cashwell, Director of Product Marketing, explained that past storage system designs failed to account for the operational scale and performance requirements of modern AI. Scality ADI offers a new operational framework that integrates AI, cyber resilience, cloud capabilities, and sovereign control into a single unified platform. Leveraging Scality’s 15 years of expertise in operating at extreme scale and delivering high performance, ADI is designed to overcome these architectural limitations.

The Scality ADI platform autonomously matches workloads to appropriate storage, protection, and economic models throughout the data lifecycle, addressing key use cases. For enterprise AI, it supports the entire data pipeline from ingestion and preparation to training, inference, and long term retention, including bidirectional data flow between core and edge environments. Cyber resilience is built in with features like API object lock, encryption, multi factor authentication, distributed erasure coding, and geographic distribution. The platform also facilitates cloud infrastructure and sovereign control, supporting private, hyperscale, and sovereign clouds with true multi tenancy and deployability across various environments, enhanced by open code inspection for regulatory compliance. These capabilities are underpinned by Scality’s core pillars of multi scale architecture, Core V for data safety, AI Connect for AI toolchain integration, and Guardian for AI assisted monitoring.

Scality ADI is delivered as an enterprise software appliance featuring the Guardian AI assistant for intuitive interaction, the Ring engine for performance and scale, and an integrated operating system to simplify management. A crucial element of the offering is outcome and SLA based pricing, which intelligently aligns different storage tiers, from GPU direct flash to warm and cold hard drive tiers, with the specific performance and cost needs at each stage of the data’s lifecycle. This dynamic tiering aims to prevent over provisioning and reduce power consumption, providing substantial cost savings. Ultimately, Scality ADI helps organizations maximize GPU productivity, ensure data recoverability, maintain data sovereignty, operate within power and budget constraints, and scale operations without increasing headcount, thereby streamlining complex data management tasks through AI assistance.

Personnel: William Cashwell

AI-Assisted Operations with Scality ADI

Watch on YouTube
Watch on Vimeo

Scality presented its AI-Assisted Operations with Scality ADI, addressing the challenge of exponentially growing storage demands versus static operational teams. The company aims to bridge this gap with AI, not just as a dashboard, but as a new way of operating its platform. Scality’s solution, Guardian, observes the system and assists users in managing their storage, specifically for its Artesca edge deployments and larger Ring environments. A key benefit is reducing customer support tickets and optimizing Scality’s own support organization by empowering customers to handle more issues with AI guidance.

Guardian functions by collecting data on customer sites, which is then sent to a Scality-operated cloud service for processing, with future plans for on-premises versions for disconnected environments. Critically, it operates within strict boundaries, never making complex, destructive decisions independently. Instead, it acts as another auditable user within the system, adhering to existing authentication, audit trails, and compliance frameworks. Scality ensures that Guardian uses established APIs through a UI proxy, preventing backdoors and maintaining robust control. For example, deleting a bucket requires multiple human confirmations and checks for existing objects, and the system intelligently applies best practices, such as utilizing server-side copies for efficient data movement.

The system allows users to express desired outcomes declaratively, rather than specifying exact commands, such as requesting a file to be accessible without knowing the specific S3 features. Guardian then intelligently leverages S3 functionalities like pre-signed URLs for temporary access or lifecycle policies for object expiration, even suggesting improvements like tagging for better data management. Scality emphasizes multiple layers of protection, including UI and S3 level authentication, with Guardian filtering commands and blocking unsafe actions. All AI commands are tagged for traceability, integrating seamlessly with existing security and audit infrastructure to enhance operational efficiency and user safety.

Personnel: Giorgio Regni