October 29, 2012
Martian rover Curiosity successfully performed an on-site validation of an application essential to the next generation of Martian missions. Cloud computing, specifically Amazon Web Services, played an important role.
An SUV on Mars
Launched from Cape Canaveral on November 26, Curiosity became the latest guest of the Martian surface when it made an epic touchdown – considering the complex landing procedure – on Gale Crater on August 6.
As a part of NASA's Mars Science Laboratory mission (MSL), the rover's goals include researching the climate and geology of the Red Planet. This mission represents another giant leap for humanity on its road to a manned expedition.
Last September 13, Curiosity put the breaks on its 1,980 lb mass and pointed its Mast Camera (MastCam) to the Sun in order to take a set of truly awesome pictures.
Curiosity's famous self-portrait
Dozens of Martian meteorological stations
Our story begins with the Mars MetNet Mission, which aims to go where no other Mars missions have gone before, at least in terms of the way it will gather and process data. This mission to Mars will be based on the power of a new type of dandelion seed-shaped, semi-hard landing vehicle called the MetNet Lander.
The main idea behind these vehicles is that by using a state-of-the-art inflatable entry and descent systems (instead of rigid heat shields and parachutes like those from the earlier semi-hard landing devices) the ratio of the payload mass to the overall mass is optimized. This means that more mass and volume resources are spared for the science payload and more probes (dozens) are to be deployed.
The scientific payload of the Mars MetNet Mission encompasses separate instrument packages for the Martian surface operation phase. At the Martian surface, the lander will take panoramic pictures and will also perform observations of pressure, temperature, humidity, magnetism, as well as atmospheric optical depth.
The network of MetNet landers will provide valuable scientific data, indispensable for studying the Martian atmosphere and its phenomena. Leading the project are the countries of Finland (Finnish Meteorological Institute - FMI), Russia (Lavochkin Space Association and Russian Space Institute) and Spain (Instituto Nacional de Técnica Aerospacial).
The Mars MetNet mission logo
The Mars MetNet collaboration also includes the Distributed Systems Architecture Research Group of the Universidad Complutense de Madrid. Led by Prof. Ignacio M. Llorente, the group is involved in using cloud computing to support scientific research. The MetNet effort is also dedicated to using cloud computing for boosting all possible applications pertaining to the Mars mission, as will be explained in greater detail.
The important role of a non-so-distant moon
One challenge of this mission is that the specified landing area is not known until two hours before touchdown. Working toward a solution, the Meiga-MetNet team developed an application for tracing Phobos (Mars' biggest moon). The resulting Phobos cyclogram, as its known, describes the trajectory of the Martian moon using coordinates, dates and time intervals as an input . The MetNet lander would achieve its exact location on the Martian surface by comparing the position of Phobos and the cyclogram, which is sent to the probe in advance of the landing procedure. This solution will spare not only time but also the cost of using a Martian orbiter for downlinking the calibration.
If you are complaining about the cost of your cell phone data plan, think about the cost of relying on other agencies' communication satellites to transmit data in outer space.
We performed an initial parallelization of the application so that the complete set of coordinates pertaining to the approximated landing area can be processed with a desired grain. This process of profiling brought us to the conclusion that the needed hardware could be too expensive for executing this HPC application only twice a year. We had no way of even knowing if there would be other uses for this costly hardware either.
For this reason we turned to Amazon EC2, the de facto standard public cloud, attractive for its on-demand deployment and its "pay-as-you-go" basis. From all the possible setups that Amazon EC2 offered, we crafted and validated an execution model for the application, taking into account time, cost and a metric involving both. This way, the optimal infrastructure could be obtained given a problem size.
The setup we looked at would be akin to getting 37 nodes of the latest HP Proliant DL170 G6 Server (as of year 2010). With a retail price of $4,909 per node, purchasing such machines outright would cost $181,633 without considering any other expenses like shipping or insurance. And what about electricity? Administrator's salary? Startup time? Even more, would we be able to use this infrastructure at full power in a 24x7 fashion? Probably not.
On the other hand and according to our model, Amazon EC2 provides the needed infrastructure for only $7.50!
Results of the execution model by means of performance, cost and a metric which interrelates both for different instance types offered by Amazon
And, by the way, we turned over a public cloud for Martian business just one year before NASA did. By December 2010, NASA was processing large satellite images for the ATLETHE vehicle (to be used in future manned missions) while we were presenting our first paper in June of the same year . The first executions of our application took place exactly six months before.
Martian Computational Archeology
Data coming from the Mars MetNet probes is to be processed using different applications. The more probes that will be literally nailed onto the Martian surface, the more data will be generated.
Although this process is to be done in a regular basis, it won't be continuous. For this reason, cloud computing on public infrastructures appears again as a valid solution.
Not having data from the Mars MetNet probes yet, the Finnish Meteorological Institute (FMI) started an interesting approach: use legacy data from NASA's Mars Viking missions.
Legacy data, indeed! The first Viking was launched in 1975 and all the data and programs for sorting and analysis (including the processing environment) were optimized for a PRIME computer built in the late 70s.
The great astronomer Carl Sagan with a full-size model of a Viking lander
The FMI conducted a great effort for analyzing this data again in order to identify instrument failures and instrument calibration changes. When this analysis is ready the full Viking meteorological data set will be available for the scientific community for the first time.
The MEIGA-MetNet project developed the Phobos eclipse application in order to help explain certain uncommon readings. The final computing framework will also be used for processing fresh Martian data, so the Viking data represents a great tuning opportunity .
Thank you, Curiosity
Returning to September 13 (on the mission's 37th sol), Curiosity made a stop in its path to take pictures of a beautiful Phobos eclipse. The objective was to confirm that our predictions of the eclipse were accurate.
These predictions, based on the execution of the application described before, have been just published by the Monthly Notices of the Royal Astronomical Society journal in a paper entitled "Opportunities to observe solar eclipses by Phobos with the Mars Science Laboratory" .
The precision achieved was within 1 second. This was also the first on-site validation of our application.
The Mars Science Laboratory represents the newest milestone in the exploration of humanity's closest "final frontier." Curiosity is not only bringing us incredible data from the Red Planet but it's also paving the way for the next generation of Martian missions thanks to cloud computing, an essential tool for space exploration.
 P. Romero, G. Barderas, J.L. Vázquez-Poletti and I.M. Llorente: Chronogram to detect Phobos Eclipses on Mars with the MetNet Precursor Lander. Planetary and Space Science, vol. 59, n. 13, 2011, pp. 1542-1550.
 J.L. Vázquez-Poletti, G. Barderas, I.M. Llorente and P. Romero: A Model for Efficient Onboard Actualization of an Instrumental Cyclogram for the Mars MetNet Mission on a Public Cloud Infrastructure. PARA2010: State of the Art in Scientific and Parallel Computing, Reykjavík (Iceland), June 2010. Proceedings published in Lecture Notes in Computer Science (LNCS). Volume 7133, pp. 33-42, 2012. Springer Verlag.
 A.-M. Harri, W. Schmidt, P. Romero, L. Vazquez, G. Barderas, O. Kemppinen, C. Aguirre, J.L. Vazquez-Poletti, I.M. Llorente and H. Haukka: Phobos Eclipse Detection on Mars, Theory and Practice. Finnish Meteorological Institute Research Report 2012:2, Finland, 2012.
 G. Barderas, P. Romero, L. Vazquez, J.L. Vazquez-Poletti and I.M. Llorente: Opportunities to observe solar eclipses by Phobos with the Mars Science Laboratory. Monthly Notices of the Royal Astronomical Society, 2012, Volume 426, Number 4, pp. 3195–3200. Wiley.
About the Author
Dr. Jose Luis Vazquez-Poletti is Assistant Professor in Computer Architecture at Complutense University of Madrid (UCM, Spain), and a Cloud Computing Researcher at the Distributed Systems Architecture Research Group. He is (and has been) directly involved in EU funded projects, such as EGEE (grid computing) and 4CaaSt (PaaS Cloud), as well as many Spanish national initiatives.
From 2005 to 2009 his research focused in application porting onto grid computing infrastructures, activity that let him be "where the real action was." These applications pertained to a wide range of areas, from fusion physics to bioinformatics. During this period he achieved the abilities needed for profiling applications and making them benefit of distributed computing infrastructures. Additionally, he shared these abilities in many training events organized within the EGEE Project and similar initiatives.
Since 2010 his research interests lie in different aspects of cloud computing, but always having real life applications in mind, especially those pertaining to the high Performance computing domain.
The author with Curiosity's soviet grandfather, Lunokhod (Луноход, moon walker in Russian), at the Russian Space Research Institute (IKI).
10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.
10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.
Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?