December 11, 2012
As 2012 winds down, it's a good time to reflect on the major cloud computing themes and stories of last 12 months. With the "What is cloud?" question finally laid to rest, thanks in large part to the official NIST Cloud Computing Definition, published Oct. 25, 2011, everyone was free to move on with the business of actually building, implementing and managing cloud platforms. Without further ado, let's look at the stories that shaped this year.
1. Amazon Web Services
Ahead of their time, but not too far (unlike Sun), Amazon still owns a large portion of the cloud, measured in datacenter space and in mindshare. Just as the Web-era vendor revolutionized the bookselling business, they've redefined computing. Starting in 2006, with the launch of Amazon Simple Storage Service (S3) and Elastic Compute Cloud (EC2), Amazon Web Services has evolved its cloud business into dozens of services, from compute to storage, and data management.
The year culminated into Amazon's first annual partner and customer event: Amazon re: Invent, which took place the last week of November. During his keynote speech, Andy Jassy, AWS senior vice president, said the public cloud offers a different model than what has been available for the past 30 years. He called out traditional enterprise platforms, like IBM, HP and Oracle, saying:
"When we launched AWS, and the cloud started to get some traction, what we started hearing from old guard technology companies was, 'There's nothing new here. This is just virtualization.' Then they abandoned that line of reasoning and said, 'Okay, fine, the cloud is different, it's not virtualization, but you can get the benefits of AWS in a private cloud in your own datacenter.'"
"The [old guard technology] companies started taking all their existing products and affixing the word 'cloud' to them. The industry started referring to these people as cloud washers. You have to be careful about who is telling you what. For a few companies who have implemented a private cloud, they aren't accomplishing the objectives they set out to achieve. The economics of what we're doing are extremely disruptive for old guard companies, companies who have lived on 60-80% gross margins for many many years."
If anything could put a pin in Amazon's ballooning success, it was a series of much publicized cloud outages, going back to April 2011. Two outages in June occurred just weeks apart. And when a AWS datacenter suffered downtime in October, it affected popular sites like Reddit, Foursquare, Heroku and Pinterest.
Cloud outages are nothing new, but the sheer number of AWS customers makes them a target. For the many benefits that cloud offers (scalability, lower cost, faster deployment times) cloud providers are unable to guarantee 100 percent uptime. An on-premise system (or private cloud) also has outage risks, but the user has a higher degree of control or at least the illusion of control. Just as with on-premise, there are ways to mitigate risk by designing for failure. This is something AWS customer Netflix has done with promising, though not perfect, results.
3. Microsoft Azure
Microsoft has shown it is a serious contender to AWS by upping its Azure cloud game. Originally offered only as a Platform-as-a-Service (PaaS), in June Microsoft Azure moved into Infrastucture-as-a-Service (IaaS) territory by allowing users to create virtual machines with Linux images.
In November at SC12, Microsoft issued an announcement that further underscored its intentions to compete with AWS on the high-end. The Redmond giant debuted a set of "big compute" capabilities for its Windows Azure offering. Designed to offer an alternative to Amazon's High-Memory Instances, the entry-level HPC instance offers 8 cores and 60 GB RAM, while the higher-end option doubles these specs for a total of 16 cores and 120 GB of RAM. The new servers are outfitted with dual Xeon E5-2670s (2.6 GHz) with DDR3 1600 MHz RAM. Even more significantly, the nodes are linked via a 40 Gbps InfiniBand network with RDMA.
4. Google Compute Engine
Google also expanded its cloud stack when it added an IaaS offering in July a month after Microsoft did. Google Compute Engine lets users run workloads virtually inside the search giant's massive datacenters, just like Amazon does with its Elastic Compute Cloud (EC2). The most significant question at the time was why it didn't happen sooner. Google is one of the few companies with the kind of uber-scale datacenter space that can pose a threat to Amazon. It's hard to compete with purpose-built infrastructure, but Google (and perhaps Facebook and eBay) can do it.
While Google launched with only four virtual machine configurations, the company will be rolling out 36 additional instance types in the weeks ahead (including high memory instances, high compute instances and diskless configurations - a lower cost option for applications that can exclusively utilize persistent disk). The utility computing services are still only available in "limited preview" and no official product release has been announced yet.
5. Open Cloud / API Wars
There's no doubt about it, 2012 was the year of open cloud platforms. The big four are, of course OpenStack, CloudStack, Eucalyptus Systems and the European-based OpenNebula. While OpenStack and CloudStack are focused on enabling public clouds, Eucalyptus Systems and OpenNebula are oriented toward private cloud. Eucalyptus has a licensing agreement in place with Amazon Web Services, strengthening their appeal as hybrid solution partners.
The arrival of all these cloud platforms created a lot of discussion, controversy even, over cloud APIs. OpenStack has held firm to its own unique standard, while the other three are basically in favor of supporting a de facto standard, which is Amazon's API.
In August, the CloudOpen Conference kicked off its first meeting this year in San Diego, colocated with LinuxCon.
6. OpenStack Foundation Takes Off
OpenStack got a lot of the glory this year. This open source cloud computing platform, launched by Rackspace and NASA in 2010, was turned over to the independent OpenStack Foundation in September 2012. The foundation has the support of over 150 companies and some 6,695 individuals. Platinum level OpenStack sponsors include AT&T, Canonical, HP, IBM, Nebula, Rackspace, Red Hat and SUSE Linux.
Dell, Cisco, CloudScaling, Piston Cloud and Rackspace all offer OpenStack distros, as do Linux vendors Red Hat, SUSE and Canonical.
Last week, HP announced the general release of HP Cloud Compute, a public cloud based on the OpenStack platform. As HP wrote in a blog entry: "Delivering an OpenStack distribution is one thing; building an enterprise-grade cloud environment that addresses the needs of our existing and future customers is something else." Until now the only other commercial implementation of OpenStack has been Rackspace's Open Cloud service.
While it's been a banner year for OpenStack, a recent report from Gartner's Lydia Leong took some of the air out of OpenStack's sails. The analyst warns that "the hype around OpenStack has led to a set of dangerous myths," and even though it's open source, it's "dominated by commercial interests." She proffers this takeaway: perform due diligence when it comes to weighing alternatives and do not sign anything without a technical evaluation.
NASA had some very visible cloud stories this year. One of the founders of the OpenStack project (along with Rackspace), NASA ceded its developer role at the same time as it announced it would begin using public cloud services from Amazon and Microsoft - controversy ensued. Some took this to mean the space giant was abandoning OpenStack, but NASA CIO Linda Cureton explained that wasn't the case.
In an interview with HPC in the Cloud, she said:
It's not NASA's role to develop software, especially software that has a commercial application. That's the way we do other technology, like our space technology. We may work in the early points of the lifecycle to develop the technology, but then at some point when it becomes commercially viable, it's not really appropriate for NASA to continue development. So many other entities can do it much better than NASA, it's their business.
In an earlier blog entry, dated June 8th, Cureton reported that the move to cloud-based enterprise architecture would save NASA nearly a million dollars yearly.
The Martian rover Curiosity mission subsequently received a lot of press for its sheer awesomeness, but it also received a lot of attention for its reliance on public cloud services. NASA's Jet Propulsion Laboratory used AWS to handle image and video processing for the Martian rover Curiosity mission. The mars.jpl.nasa.gov website that conveyed the images to the public is also powered by Amazon EC2.
8. Helix Nebula
In March, three of Europe's most prominent research centers, CERN, the European Space Agency (ESA), and the European Molecular Biology Laboratory (EMBL) launched a massive cloud computing project, called Helix Nebula – the Science Cloud. The project is exploring whether virtualization and cloud technologies will provide a better return on investment over current grid resources. The supply side is comprised of both public and private cloud providers, and includes vendors Atos, CloudSigma and T-Systems.
In July, the partners announced that they were transitioning from the initial proof of concept phase to the start of the two-year pilot phase, which involves expanding the proofs of concept and refining the commercial terms. One of the major challenges of this multi-vendor, multi-cloud endeavor is operability. Where federation is an essential characteristic of grid computing, cloud federation still has a ways go to.
9. Cloud Price Wars
From a future vantage point, 2012 will go down in history as marking the beginning of the cloud price wars, aka the race to the bottom. At Amazon re: Invent, AWS announced that it was reducing its cloud storage pricing for the 24th time!
Amazon's extensive datacenter ecosystem provides it with the economies of scale to compete in this race by outbidding the competition, but Google wrote the book on scale-out architecture. The day before the AWS re: Invent conference started, Google lowered its storage pricing by 20 percent. And then, on Thursday, presumably in response to the AWS decrease, Google lowered its prices again, by an additional 10 percent. Last I checked, Google's storage was 10 cents cheaper than Amazon S3.
There is speculation that Amazon may be pushing at the limits of its infrastructure, and thus can only go so low on price... If that is true, this would be the time for a competitor to push the lower price edge hard and fast to see if they could force Amazon to their limit and win some customer share in the process, but price is only a meaningful metric, all else being equal, so any challenger must also have a proven service record.
China is going all kinds of cloud crazy, with multiple high-end cloud centers, a cloud operating system, and even a "valley" devoted to cloud technology. Last month, Microsoft brokered an agreement with Shanghai and with ISP 21Vianet to offer Office 365 and Windows Azure services in China.
According to the figures put out by CCID Consulting, China's cloud computing market is estimated to grow from about $2.62 billion (US) in 2010 to $18.6 billion (US) in 2013. Despite all this activity, a number of experts, including the IDC analyst group, believe that China does not place enough emphasis on software development. It will be interesting to see how these plans take hold in the year ahead.
10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.
10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.
Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?