August 22, 2013
Companies of all stripes occasionally run into trouble when trying to send large files over the public Internet. But for life sciences organizations, the file-transfer bottleneck is a pain in the neck on a daily basis. Now, thanks to work between Aspera and Intel in the development of the Fast Adaptive Secure Protocol (FASP), life sciences firms can now enjoy LAN-like I/O speeds over a WAN.
Good old FTP is a great tool for transferring small and medium-sized files over the Internet, because it's ubiquitous and easy to use. It's not even remotely secure (at least with the plain vanilla version), but for some types of files, that's not a big concern.
Similarly, cloud-based file storage solutions that work over HTTP have become a popular way for people to share files over the Internet. However, both FTP- and HTTP-based file transfer solutions suffer from one big flaw: they are horrible at moving large files over long distances.
"The simple fact is, there is a fundamental issue that file transfer applications that rely on TCP transport layer will experience performance degradation as file transfer distances increase," Jay Migliaccio, director of cloud software and services for Aspera, said in a recent presentation. "Traditional ways of sharing do not scale."
That led Aspera to work with Intel on the development of FASP, which the companies unveiled in April. FASP was built using a combination of Non-Uniform Memory Access (NUMA) and Intel Data Direct I/O (DDIO) technology that Intel built into the Xeon processor E5-2600 product family, which was also the first Xeon processor to support the new, faster PCI 3.0 bus standard.
The DDIO technology allows the Xeon chip--which also features an integrated 10 Gigabit Ethernet controller (also a first for Intel Xeon)--to route I/O traffic directly to the processor cache. "Previously, data had to be moved to and from main memory prior to integration with cache," Chris Gough, Intel's lead healthcare solutions architect, said in the presentation with Aspera. "It further reduces I/O latency."
The end result is that FASP and DDIO work together to remove longtime bottlenecks to I/O throughput. "The key capabilities of FASP are to enable users to transfer data as fast as the infrastructure will allow," Migliaccio said. "With FASP the transfer software is no longer the bottleneck. It allows you to utilize the full bandwidth" of your network.
This means that FASP will enable users to utilize the full 10 Gigabit capability of the integrated Ethernet controller, even when sending files to the other side of the world. FASP can also take advantage of the new Single-Root I/O Virtualization (SR-IOV) that Intel put on the E5-2600 product, which virtualizes the controller. FASP suffers very little degradation in performance when using multiple virtualized Ethernet controllers, according to Aspera.
Aspera put these tall WAN claims to the test with the help of three life sciences organizations, including the Beijing Genomics Institute, the National Center for Biotechnology Information, and the University of California, Davis.
The organizations used the FASP technology to send a 24 GB file from China to the US. According to Aspera, the large file was transferred to UC Davis in less than 30 seconds. The same file took 26 hours to transfer using traditional FTP.
Full details of the test and other information on FASP are available in a white paper on the Aspera website.
10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.
10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.
Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?