On a day where EMC has performed a refresh across it's platform business Tony P is whining about how mean everyone is for not bowing down before IBM's benchmarks.
Yes my friends there he goes again banging on about the IBM strategy for reclaiming the storage market, it boils down to SVC in one hand and a bunch of worthless SPC benchmarks in the other. You'll notice it's now so common a theme in his posts that he's even mentioned that he's starting to sound like a broken record.
I tend to stay away from anything even remotely to do with legal stuff as I've been around long enough to know that not all lawyers are as good humored as the characters in Boston Legal but the Anarchist points out that the following assertion is false.
What can EMC do if you violate this? They can yank away your firmware license and support, leaving you exposed by running on unsupported hardware, a situation to avoid. (I don't know how often EMC does this, or threatens this, or how often they put this kind of language into their contracts, but we heard it often enough that IBM made sure that EMC would always support their gear behind the SVC, They do, and we have that in writing, just in case you need it.)
Tony is mistaken on that point, we know this as we've checked. I'm told that he, IBM, or anybody else can buy an EMC system benchmark the hell out of it and shout the numbers from the rooftops if they so choose. If someone tells you otherwise they too are mistaken.
That's as far into the contract stuff as I'm willing to go for fear of stirring the rabid pitbulls at EMC Legal so back to the technology we go.
So what's so wrong with SPC that it sets the teeth of EMC's MIT educated mathematical nerds on edge?
Well to be honest a lot, and if anyone has ever read the disclaimers on the tests they wouldn't put too much stock in them either. Lets look at this perfectly formed little snow flake..
“Actual system performance is highly dependent upon specific workload characteristics, platform configuration, and application-specific tuning. Relative system performance will vary as a result of these and other factors. Thus, SPC-2 should not be used as a substitute for customer application benchmarking when critical performance requirements are called for.”
Emphasis mine as it means that the people who most require accurate benchmarks are the same people least likely to get them from looking at SPC numbers. SPC which by it's very design eliminates the benefit of caching algorithms, is incapable of factoring in the overhead from local or remote replication operations, and favors JBOD at every turn. This suits IBM just fine since that's what SVC does to those intelligent arrays you've bought. It turns them into JBOD.
EMC performs probably more array benchmarking than anyone but it does so using it's own internal tools (Which by the way are also used to measure competitors system performance. It's no good deluding yourself when it comes to your competitors capabilities as to beat them you need to know exactly what they've got.), and with customer specific application workloads.
This is an incredibly expensive and resource intensive way of providing customers with benchmarking & planning information, I know this since I've worked with people who gather such data, but it's the only way you can be sure to provide accurate performance numbers tailored to what the customer is actually going to be doing with the gear.
Now I'll start to sound like a broken record as I've said this before but while a lot of people would love to be able to get a score out of ten for whatever hardware they're looking at, just like the scores PC mags give test systems, in our business things just aren't that easy.
SPC won't cut it when you're planning that big new application roll out or you want to see the effect of consolidating a lot of different application servers on a smaller number of frames, or you want to discover the performance impact of array based point in time copies or remote replication operations.
For numbers which really matter it's a case of doing the research rolling up the sleeves getting real numbers and then doing all the mathematics required to ensure that the customer will see the exact same numbers when they roll it out in their data center.
It's not as easy as quoting a makey-uppy SPC number then shrugging when someone gets nowhere near to achieving those when they deploy it but it's orders of magnitude more accurate. And for the record FeS2 (Iron Disulfide) is commonly referred to as Fools Gold.
While it may look like the real thing it's incredibly less valuable.
