Since Twitter sucks when the number of participants in any conversation increases to more than three I thought I’d break my conversation out as a blog post.
VMware’s Scott Lowe asked an interesting question on Twitter which I thought was limited by the transmission channel.
Scott’s discussion point was ‘Cloud environments are fond of DAS because it’s simple and inexpensive, both of which are important in cloud environments. Discuss!’
First off, what’s DAS? Direct Attached Storage is one or more dedicated storage devices connected to one or more servers. That’s DAS. There’s nothing more to it.
Do the massive cloud providers use DAS, yes, but they don’t offer it to you and what you’re getting is in no way simple. There are layers and layers of storage software between anyone and what they think is a disk or other unit of storage. This is where all the storage magic happens. Maybe it’s object storage with objects numbering in the trillions, maybe it’s block storage with volumes in the millions. Some ephemeral, others persistent.
If we look at Amazon Web Services, Google Compute Engine or Microsoft Azure when you request block storage from any who provide such, you’re not getting DAS. You’re getting server based distributed block storage where read and write I/O is distributed amongst servers which could number in their thousands or tens of thousands.
Why would you want to do anything like that? Taking protection and fault domain mitigation out of the picture, which you shouldn’t but we will, management and the simple elimination of I/O contention to individual disk drives requires it. If you want to drive down cost by using fragile components, you may or may not be using the fastest hard drives priced like their materials were mined from the foothills of Mount Olympus, while driving up margins by getting the utmost utilization at an immense scale, you have to use distributed storage.
We haven’t discovered a better answer yet.
So yes, cloud providers do use DAS, the same way they use RAM DIMMs, but how they give it to you when you hand over your credit card and click a button on the control panel is the product of hard nosed storage software development.
And that DAS is just a component in a massively distributed storage array.
Because it’s in the storage software that all the magic happens.