August 28, 2012
Amazon has just announced a new service called Glacier. The cloud giant describes its new offering as a secure, extremely low cost storage technology for data archiving and backup. Like the name implies, the service is slow, cold and remote. As with Amazon Simple Storage Service (S3), Glacier allows users to host their data on Amazon's infrastructure, but overall functionality and use cases are expected to vary quite a bit.
Amazon Evangelist Jeff Barr explains in a blog that Glacier's main advantage is low-cost storage, which is advertised at $.01/GB/month. Users only pay for the amount of storage they actually utilize, as opposed to subscribing for a certain data limit. The service is also set up to handle petascale volumes "and beyond."
Given these characteristics, Amazon believes Glacier will provide a decent home for digital media archives, financial records, genomic data, healthcare records, long-term backups and data held for regulatory purposes. These uses are different from S3 implementations, which focus on Web applications and content distribution. Both services claim similar reliability rates with four nines availability and eleven nines durability. this means that if a company uses it to hold 100 billion objects, it can expect to lose one each year.
Amazon explains that Glacier's lower price point is enabled through slower processing times. When a user wants to retrieve an archive, their request gets queued. In roughly 3-5 hours, the data is made available for download and remains available over a 24-hour period.
Keeping in mind that S3 was designed for rapid retrieval of data: it costs $0.125/GB/month to store up to 1 TB using the simple storage service. As data levels increase, the price can drop as low as $0.055/GB/month but this requires the user to store over 5 petabytes on Amazon's infrastructure. On the other hand, Glacier runs $0.01/GB/month regardless of how little or much a user decides to store.
To incentivize customers to choose the appropriate storage service, Amazon charges for retrieval of data on Glacier. Users can download up to 5 percent of their average monthly storage for free. Beyond that, Amazon charges retrieval fees to the tune of $0.01/GB.
Setting up a Glacier account appears to be a relatively simple process. As Barr explains:
To store data in Glacier, you start by creating a named vault. You can have up to 1000 vaults per region in your AWS account. Once you have created the vault, you simply upload your data (an archive in Glacier terminology). Each archive can contain up to 40 Terabytes of data and you can use multipart uploading or AWS Import/Export to optimize the upload process. Glacier will encrypt your data using AES-256 and will store it durably in an immutable form. Glacier will acknowledge your storage request as soon as your data has been stored in multiple facilities.
Initiating the process will require client software or familiarity with Amazon's SDKs and APIs. FastGlacier is a free client for Windows users, although as of today, no Mac clients have been developed. This is unfortunate, as the writer of this article considers the cost and functionality of Glacier as good reasons to store old media archives on Amazon. The only roadblock to migration is the lack of a Mac client.
The tutorial video below explains how to setup Glacier for the first time.
While the new service offers a low-cost solution for archiving and backing up data, Amazon is not the only player in this field. Yesterday, Venture Beat covered another storage provider named Quantum. Their Q-Cloud service aims to handle similar workloads as Glacier, but combines on-premise and cloud storage to enable faster access to saved datasets. Like S3, the service has a variable pricing structure based on how much data is stored by the user. While initial costs offer little advantage to Glacier, when data stored consumes more than 72 TB, the price drops to $.15/GB/mo. This assumes Glacier users chose a 15:1 deduplication ratio.
Enterprise IT typically dictates archiving with off-site tape backups. With this development, Amazon is looking to change the norm with their Glacier service. Whether the project will pick up steam is yet to be seen, but it appears to be a solid use of cloud technology without overplaying its hand.
Amazon Glacier is available now across the US, and in Europe and Japan. The Amazon Glacier detail page offers additional details and pricing information.
10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.
10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.
Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?