Wednesday, February 26, 2014

Reviewing SPC-1 Performance Results

If you're trying to keep up with developments in Information Technology then we live in a wonderful time.  Twitter, Wikibon, Vendor Blogs, Analyst Blogs - there's a wealth of information available.  The real challenge is separating the signal from the noise and making sure that you don't fall victim to groupthink.


One of the impartial tools I use to keep up with developments in the storage arena is the SPC-1 benchmark, managed by the Storage Performance Council and published at http://www.storageperformance.org/results/benchmark_results_spc1.  About once a quarter I'll go through the benchmark executive summaries gathering price and performance information and charting it out.

It's an admittedly imperfect exercise - performance isn't everything after all (otherwise we'd all be driving Bugatti Veyron Super Sports), pricing is constantly changing (meaning that the SPC-1 Price-Performance data is of questionable utility), and not all vendors participate.  In addition, there is lag between when a technology is introduced and when benchmark results are available (meaning that it can be a trailing indicator).  With that said, it does provide insight into price and performance trends and helps understand how the vendors that do participate are positioning their offerings.

Looking solely at performance, here is a scatter plot of SPC-1 IOPs over time:
Here's the raw data for the top 10:

Vendor Report Product Date SPC-1 IOPS
Kaminario A00137 K2 (K2F00000700) 10/17/2013 1,239,898.00
Kaminario A00118 K2-D (1875K-1.1) 7/30/2012 1,219,973.91
Huawei  A00140 OceanStor 18800 1/24/2014 1,005,893.43
IBM A00130 IBM Power 780 server (with SSDs) 4/11/2013 780,081.02
HDS A00136 VSP w/accelerated flash 7/30/2013 602,019.47
HP A00138 StorageWorks XP P9500 (w/Performance Accelerator) 12/4/2013 602,019.47
Huawei A00119 OceanStor Dorado5100 8/13/2012 600,052.49
IBM A00113 IBM SVC v6.2 w Storwize V7000 1/30/2012 520,043.99
IBM A00120 IBM System Storage DS8870 10/3/2012 451,082.27
HP A00109 P10000 3PAR V800 10/17/2011 450,212.66

Some interesting points about the top 10 results:
  • In terms of age:
    • 1 is from 2011 (HP P10000 3PAR V800)
    • 4 are from 2012 (Huawei OceanStor Dorado5100, IBM DS8870, IBM SVC v6.2 w/Storwize V7000, and Kaminario K2-D)
    • 4 are from 2013 (HDS VSP w/accelerated flash, HP StorageWorks XP P9500, IBM Power 780 Server, and Kaminario K2)
    • 1 is from 2014 (Huawei OceanStor 18800)
  • In terms of technology:
    • 6 are Flash-only solutions (HDS VSP w/accelerated flash, HP StorageWorks Xp P9500, Huawei OceanStor Dorado5100, IBM Power 780 server, Kaminario K2, and Kaminario K2-D)
    • 1 is a hybrid Flash/SAS solution (Huawei OceanStor 18800)
    • 3 are traditional "lots o' disk" solutions (HP P10000 3PAR v800, IBM DS8870, and IBM SVC v6.2 w/Storwize V7000)
Other comments:
  • Kaminario is leading both the SPC-1 and SPC-2 benchmarks right now
  • The IBM benchmark for the Power system is a head scratcher.  It shows that they can drive a lot of I/O, but isn't benchmarking a storage array in the traditional sense.
  • The HDS VSP and HP XP P9500 have the same results because they're using the same technology

Share/Save/Bookmark

No comments:

Post a Comment