Chris Evans highlights the new king of HDD capacity, the 15TB Ultrastar DC HC620 from Western Digital. While only representing a 7% increase in capacity from 14TB drives, the real innovation here is these drives use host-managed Shingled Magnetic Recording, as opposed to being managed directly on the drives. This has big implications for hyperscalers, who can see greatly improved performance. Dropbox is one of the initial customers to use the drives, and Chris got a deep dive into their “Magic Pocket” architecture at Storage Field Day earlier this year.
Storage Field Day 15: Dropbox the high availability in a “pocket”
At Storage Field Day, Lino Telera got to hear from Dropbox. They presented about their Magic Pocket architecture, which allowed them to transition from depending on AWS, saved them $75 million over two years, and increased performance. Lino gives a great overview of the architecture here. Perhaps most surprising to him was the work Dropbox put into optimizing their networking across their entire platform.
The Challenge of Scale
Josh De Jong go his idea of scale adjusted after hearing Dropbox present at Storage Field Day. In his previous SMB experience, 20-30TB had been a large amount of storage. Moving into the enterprise, multiple petabytes became more of the norm. But at Dropbox, they referred to 30PB as a “toy system.” Josh overviews many of the problems and solutions Dropbox showed to operate at such scale.
Dropbox – It’s Scale Jim, But Not As We Know It
Dropbox’s recent presentation at Storage Field Day certainly left an impression with the delegates, and Dan Frith is no exception. The company reviewed their Magic Pocket architecture, which enabled them to move infrastructure back on-premises from the public cloud. For Dan, the big takeaway here is that this was an innovative solution for Dropbox, but the level of investment and development is definitely not for every organization with massive scalable storage needs.
Magic happening
Since they’ve launched, Chin-Fah Heoh has seen Dropbox change from an innovative Enterprise File Synchronization and Sharing service to a full fledged collaboration platform. At Storage Field Day, he got a architectural overview of how the company transitioned away from AWS to their own infrastructure. This required moving more than 500PB of customer data, and saved the company over $75 million.
Dropbox’s Magic Pocket: Power of Software Defined Storage
Chan Ekanayake was “ecstatic” when he found out that he’d get a technical deep dive on Dropbox’s infrastructure at Storage Field Day. This piece reviews the company’s Magic Pocket, a custom built, internally hosted, software defined, cloud storage infrastructure. This was designed so the company could move from storing actual customer data from Amazon S3 to get better performance and reduce raw storage costs.
Storage dinosaurs evolving too
In the age of cloud, the traditional storage admin are often seen as relics of the past. But for Chin-Fah Heoh, these old “dinosaurs” are actually evolving to become vital to the modern enterprise. This is because they have a deep understanding of data. At Storage Field Day, he’s looking forward to hearing from companies like Hedvig and WekaIO on how they solve the problems of persistent storage in distributed systems. With these difficult data question more important than ever in the enterprise, Chin-Fah thinks its the best time to be a “storage dinosaur”.
