We have made an Amazon EC2 deployment of Virtuoso 7 Commercial Edition, configured to use the Elastic Cluster Module with TPC-H preconfigured, similar to the recently published OpenLink Virtuoso Benchmark AMI running the Open Source Edition. The details of the new Elastic Cluster AMI and steps to use it will be published in a forthcoming post. Here we will simply look at results of running TPC-H 100G scale on two machines, and 1000G scale on four machines. This shows how Virtuoso provides great performance on a cloud platform. The extremely fast bulk load — 33 minutes for a terabyte! — means that you can get straight to work even with on-demand infrastructure.

In the following, the Amazon instance type is R3.8xlarge, each with dual Xeon E5-2670 v2, 244G RAM, and 2 x 300G SSD. The image is made from the Amazon Linux with built-in network optimization. We first tried a RedHat image without network optimization and had considerable trouble with the interconnect. Using network-optimized Amazon Linux images inside a virtual private cloud has resolved all these problems.

The network optimized 10GE interconnect at Amazon offers throughput close to the QDR InfiniBand running TCP-IP; thus the Amazon platform is suitable for running cluster databases. The execution that we have seen is not seriously network bound.

100G on 2 machines, with a total of 32 cores, 64 threads, 488 GB RAM, 4 x 300 GB SSD

1000G on 4 machines, with a total of 64 cores, 128 threads, 976 GB RAM, 8 x 300 GB SSD

For the larger scale we did 3 sets of power + throughput tests to measure consistency of performance. By the TPC-H rules, the worst (first) score should be reported. Even after bulk load, this is markedly less than the next power score due to working set effects. This is seen to a lesser degree with the first throughput score also.

The numerical quantities summaries are available in a report.zip file, or individually --

• report-100-1.txt
• report-100-2.txt
• report-1000-1.txt
• report-1000-2.txt
• report-1000-3.txt

Subsequent posts will explain how to deploy Virtuoso Elastic Clusters on AWS.

In Hoc Signo Vinces (TPC-H) Series

• In Hoc Signo Vinces (part 1): Virtuoso meets TPC-H
• In Hoc Signo Vinces (part 2): TPC-H Schema Choices
• In Hoc Signo Vinces (part 3): Benchmark Configuration Settings
• In Hoc Signo Vinces (part 4): Bulk Load and Refresh
• In Hoc Signo Vinces (part 5): The Return of SQL Federation
• In Hoc Signo Vinces (part 6): TPC-H Q1 and Q3: An Introduction to Query Plans
• In Hoc Signo Vinces (part 7): TPC-H Q13: The Good and the Bad Plans
• In Hoc Signo Vinces (part 8): TPC-H: INs, Expressions, ORs
• In Hoc Signo Vinces (part 9): TPC-H Q18, Ordered Aggregation, and Top K
• In Hoc Signo Vinces (part 10): TPC-H Q9, Q17, Q20 - Predicate Games
• In Hoc Signo Vinces (part 11): TPC-H Q2, Q10 - Late Projection
• In Hoc Signo Vinces (part 12): TPC-H: Result Preview
• In Hoc Signo Vinces (part 13): Virtuoso TPC-H Kit Now on V7 Fast Track
• In Hoc Signo Vinces (part 14): Virtuoso TPC-H Implementation Analysis
• In Hoc Signo Vinces (part 15): TPC-H and the Science of Hash
• In Hoc Signo Vinces (part 16): Introduction to Scale-Out
• In Hoc Signo Vinces (part 17): 100G and 300G Runs on Dual Xeon E5 2650v2
• In Hoc Signo Vinces (part 18): Cluster Dynamics
• In Hoc Signo Vinces (part 19): Scalability, 1000G, and 3000G
• In Hoc Signo Vinces (part 20): 100G and 1000G With Cluster; When is Cluster Worthwhile; Effects of I/O
• In Hoc Signo Vinces (part 21): Running TPC-H on Virtuoso Cluster on Amazon EC2 (this post)