Monday, June 29, 2009

More on node scalability

I'm at SIGMOD this week, so I'll make this blog post quick:

A week and a half ago, I blogged on analytical database scalability. The basic thesis was:
(1) People are overly focused on raw data managed by the DBMS as an indicator of scalability.
(2) If you store a lot of data but don't have the CPU processing power to process it at the speed it can be read off of disk, this is a potential indication of a scalability problem not a scalability success.

I insinuated that:
(1) Scalability in terms of number of nodes might be a better measure than scalability in size of data (assuming of course, you can't get true scalability experiments as specified by the academic definition of scalability)
(2) Scaling the number of nodes is harder than people realize.

I am not so presumptuous as to assume that my blog carries enough weight to affect the actions of Aster Data's marketing department, but I was nonetheless pleased to see Aster's new marketing material (which I believe was just put online today) which specifically shows how their new appliance has more CPU processing power per usable storage space than a subset of their competitors (I'm assuming they either only showed the competitors they can beat in this metric, or their other competitors don't publish this number).

I do want to point out one interesting thing, however, about Aster Data's scalability (at least what limited knowledge we can deduce from their marketing material). For 6.25TB, 12.5TB, 25TB, 50TB, and 100TB, Aster Data's appliance has 8, 16, 32, 63, and 125 worker nodes, and 64, 128, 256, 504, and 1000 processing cores respectively. So basically, it's completely linear: double the amount of data, then double the amount of nodes and processing cores respectively. But then going from 100TB to 1PB of data (10X more data) they increase the number of worker nodes by less than 2X (to 330) and processing cores by a little less than 3X (2640). So after 100TB/100 nodes, their processing power per storage space drops off a cliff. (Aster has a note for the 1PB saying they are assuming 3.75X compression of user data at 1PB, but compression at 1PB shouldn't be any different than compression at 100TB; and if they are assuming uncompressed data at 100TB, then the comparison in processing power per storage space relative to other vendors is misleading since everyone else compresses data.)

Bottom line: scaling to 100 nodes is one thing (I know that Teradata, Vertica, and Greenplum can do this in addition to Aster Data, and I'm probably forgetting some other vendors). Scaling to 1000 nodes (and keeping performance constant) is much harder. This might explain why Aster Data only doubles the number of nodes/cores for 10X more data after they hit 100 nodes.

I will have more on why it is difficult to scale to 1000 nodes in future posts.


  1. Hi Daniel,

    Thanks for the post. We certainly agree with your points 1 and 2 re:scalability. Scaling to a large number of nodes is hard, but provides much more compute power for analytic workloads, which is the focus of the Aster nCluster architecture.

    To clarify re:compression - all configs we list are based on uncompressed data (except for our 1 petabyte appliance).

    Compressed data size is a misleading metric since compression swings wildly depending on the data. Our internal compression tests have shown compression in the range of 2X to 50X, depending on the kind of data. It would be misleading to claim that our compute power increased by 50X as we compressed data! :-)

    Looking forward to reading your future posts.

  2. Hi "Aster",

    First, I want to say that I've been chatting with John Cieslewicz (recent Aster recruit) at SIGMOD this week and a) he's doing a great job evangelizing your product b) you guys are super lucky to have him.

    Second, I want to reiterate that I really like the way Aster is talking about processing power per Terabyte storage. To me this metric will get increasingly important as more analysis is pushed into the data management layer.

    Third, you have admitted that advertising numbers with assumed data compression rates is misleading due to high variability in actual results. Yet that's exactly what you do for the 1PB numbers. You admit it's misleading, yet do it anyway. I wish Aster would play it a little more straight and not stoop to such unnecessary marketing tricks.

  3. Kickfire, SenSage, Aster Data... Wow, I didn't realize that database researchers had so much interest in current database vendors and their marketing practices!