The Portland Group
Oakridge Top Right
HPCwire

Since 1986 - Covering the Fastest Computers
in the World and the People Who Run Them

Language Flags

Visit additional Tabor Communication Publications

Enterprise Tech
Datanami
HPCwire Japan

Amazon EC2 Versus the Competition


Independent cloud computing analyst Cloud Spectator released a report today comparing Amazon EC2 cloud servers to that of nine other IaaS providers with regard to performance and stability.

Cloud Spectator's CloudSpecs Performance System uses industry-standard open-source benchmarks to measure virtual server performance. The Cloud Performance and Stability Report examined four aspects of the EC2 cloud: CPU, RAM, disk, and internal network (VPN connections).

Over a consecutive 30-day period, the CloudSpecs System performed a series of 19 tests on servers located within the Amazon EC2 East (1a) datacenter: 3 server system tests, 5 CPU tests, 2 disk tests, 2 RAM tests, and 7 VPN network tests.

Test Results (compared with industry average)

 Metric Performance Stability
 CPU  2% above average  15% more stable than average
 RAM  36% above average 10% more stable than average 
 Disk  68% below average  91% more stable than average
VPN Connection  49% below average   5% less stable than average

Details from Cloud Spectator report

The performance of Amazon's virtual CPUs was on par with the IaaS industry standard, while RAM performance came in above average. VPN and disk performance, on the other hand, were below average.

As for stability, disk performance did very well, with virtual CPU and RAM coming in slightly above average. VPN was again lacking.

Overall, CPU and RAM were winners with above average performance and decent stability. On the flip side is Amazon's VPN; performance and stability were below average. The report authors say that this could help "explain performance issues with more complex, multi-server applications."

"There has been a lot of talk among the industry about Amazon EC2 and its stable performance," noted Kenny Li, founder and CEO at Cloud Spectator. "Our analysis validates that indeed there are inconsistencies in service performance. But more than that, we provide detailed data that shows the highs and lows of the service. This level of research will help not only Amazon, but companies who are evaluating Amazon. Inconsistency doesn't have to be bad, it depends on what you are using the service to accomplish."

Although it did so anonymously, Cloud Spectator signed up for an Amazon account using the same process as a typical AWS EC2 client. The CloudSpecs system automatically runs tests 4 times a day, 365 days a year, and this particular report was based on a 30-day consecutive period period from August 18, 2012, to September 16, 2012.

All tests were run on Amazon's EC2 Extra Large server, with the VPN connection test also using the EC2 Medium instance as a dummy server. Server benchmarks include the byte-Unixbench (http://code.google.com/p/byte-unixbench/) and Phoronix Test Suite (http://www.phoronix-test-suite.com/). The complete list of benchmarks and the significance of each is detailed in the full report.

To compile the industry averages, Cloud Spectator measured the performance of nine cloud IaaS providers over 30 days, using the same benchmark tests. Stability metrics were obtained by "calculating the standard deviation of the performance results of the CloudSpecs tests over a period of 30 days, also compared to an industry average."

In a separate report, also released today, Cloud Spectator compares the pricing structure of various IaaS offerings. The analysis is based on data from 20 cloud providers, including Amazon, Microsoft Azure, SoftLayer, PEER1 Zunicore, Rackspace, CloudSigma, GoGrid and others.

These reports come on the heels of another AWS outage. Although Amazon held up pretty well against Super Storm/Hurricane Sandy, the service suffered some downtime the week before. Github, Pinterest, Reddit and Netflix were among the high-profile companies that were affected.

Most Read Features

Most Read Around the Web

Most Read This Just In

Most Read Blogs


Sponsored Whitepapers

Breaking I/O Bottlenecks

10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.

A New Ultra-Dense Hyper-Scale x86 Server Design

10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.

Sponsored Multimedia

Xyratex, presents ClusterStor at the Vendor Showdown at ISC13

Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.

HPCwire Live! Atlanta's Big Data Kick Off Week Meets HPC

Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?

Newsletters

Stay informed! Subscribe to HPCwire email Newsletters.

HPCwire Weekly Update
HPC in the Cloud Update
Digital Manufacturing Report
Datanami
HPCwire Conferences & Events
Job Bank
HPCwire Product Showcases


Xyratex

HPC Job Bank


Featured Events


HPCwire Events