July 19, 2010
La Mirada and San Diego, Calif., July 15, 2010 — When a national network of ocean observatories begins streaming environmental sensor data in March 2012, researchers throughout California will be able to use the state’s high-speed academic network to transmit some of that data to storage and computing clouds operated by Amazon Web Services.
With new ultra-high-performance peering connections, members of CENIC and PNWGP can take full advantage of Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), whether for K-12 education or for university-based research.
CENIC and PNWGP today announced two 10 Gigabit per second (Gbps) connections to Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2) for the use of CENIC’s members in California, as well as PNWGP’s multistate K-20 research and education community. With these new ultra-high-performance peering connections, members of CENIC and PNWGP can take full advantage of those services, whether for K-12 education or for university-based research.
“The ability to use cost-effective cloud services to store and process large amounts of data is vital for researchers in many of the most active areas of research,” says CENIC President and CEO Jim Dolgonas. “Collaborative research is dependent on the ability to share, and seamlessly access and manipulate data. With these new peering connections to Amazon Web Services, both the CENIC and PNWGP communities can obtain maximum speed and benefit from AWS cloud services.”
CENIC owns, operates, and manages the ultra-high-performance California Research & Education Network (CalREN), an Internet-based network comprising nearly 3,000 miles of fiber-optic cable that stretches throughout California and to which the state’s entire K-20 public education system and a significant number of private institutions connect. The 10 Gbps connection that PNWGP and CENIC have collaborated on to connect to Amazon S3 is now joined by a 10 Gbps connection linking CalREN and PNWGP to Amazon EC2. The connections enable out-of-the-box, server-based computation from any computer on the CalREN network in California.
Members of CENIC include the California K-12 system, all 114 campuses of California’s Community Colleges, all 23 campuses of the California State University, all 10 campuses of the University of California, and prestigious private universities including Caltech, USC, Stanford, and others. With this type of fiber-optic path connecting California’s research and educational institutions, data exchanges can take place in a fraction of a second – enabling research and collaboration previously within the realm of science fiction.
California is not the only state that will benefit, however. Thanks to PNWGP’s multi-state, high-performance research and education network connecting nearly all of the major research institutions and many other schools and colleges in Washington, Alaska, Hawaii, Montana, Idaho and Hawaii, will automatically get direct high-performance access to the full range of cloud services provided by Amazon Web Services.
Among the first users to benefit from access to Amazon Web Services will be researchers participating in the National Science Foundation-funded Ocean Observatories Initiative (OOI) Cyberinfrastructure (CI) project. The University of California, San Diego is building the information technology and telecommunications infrastructure that will bring ocean and atmospheric sensor data from the observatories and make them available to environmental researchers around the country and throughout the world. Research teams in California will have the option to do their storage and computing on their own machines, or remotely on Amazon S3 or Amazon EC2.
As far back as 2006, OOI CI researchers recognized that given the scale of the initiative, it would be too expensive and unwieldy to manage all storage and computation via the traditional model; i.e., by building large, dedicated data centers.
“We always envisioned cloud computing as a key component of our implementation strategy,” said John Orcutt, OOI CI Principal Investigator and a professor of geophysics at UCSD’s Scripps Institution of Oceanography. “At our core we are a sensor network, and with streaming data from sensors, we need both continual and periodic computation, including elasticity to deal with a highly variable demand. So we focused on core measurement processing as well as distribution and presentation of the data. This led to a shared model where some work could be managed by academic computing and other work by commercial clouds.”
"Twenty-first century discovery will be driven by the automated analysis of massive amounts of sensor data captured from the world around us,” said Ed Lazowska, the Bill & Melinda Gates Chair in Computer Science & Engineering and Director of the eScience Institute at UW. “The focus is on data, more than cycles. Cloud resources are an essential component. The direct connections from to the Amazon Web Services cloud from CENIC and PNWGP provides scientists with the bandwidth they need to utilize these resources.”
CENIC officials identified a set of research opportunities where Amazon Web Services could be of substantial use in the academic world. In addition to K-12 storage and computation, one such class of opportunities was environmental observation. Both UW and UC San Diego are involved in the $400 million Ocean Observatories Initiative, and its Cyberinfrastructure group – led by Scripps Institution of Oceanography and based in the California Institute for Telecommunications and Information Technology (Calit2) – determined that cloud computing and remote storage would reduce the need for capital expenditures.
“We view cloud computing as providing two core benefits,” said OOI CI project manager Matthew Arrott. “With cloud computing you buy computing resources on demand, as you need them. It’s also different from the standard, fixed data center model, with more emphasis on agility of deployment.”
The OOI will monitor and forecast environmental changes in the oceans on global, regional and coastal scales. Scientists will be able to extrapolate from data gathered by an array of more than 50 diverse sensor types and other scientific instruments that will communicate through permanently installed seafloor cables and satellite telemetry. Scientists will then be able to share data with their colleagues around the world via OOI's networked cyberinfrastructure, which is being implemented by a team of computer scientists, engineers and geophysicists.
In the long run, hundreds or thousands of researchers will benefit from the new high-bandwidth connections between CalREN and PNWGP’s networks and Amazon Web Services. “We anticipate that many OOI users will need Amazon’s cloud,” said OOI CI’s Matthew Arrott.
Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. Like Amazon S3, Amazon EC2 is a web service. It provides resizable compute capacity in the cloud, and it is designed to make web-scale computing easier for developers. Amazon EC2 features a simple web service interface that allows a researcher to obtain and configure the massive compute resources that innovative research requires.
CENIC and PNWGP member researchers who are interested in making use of the new 10 Gbps connections to Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Cloud Compute (Amazon EC2) can contact CENIC at firstname.lastname@example.org mailto:email@example.com for more information.
California’s education and research communities leverage their networking resources under CENIC, the Corporation for Education Network Initiatives in California, in order to obtain cost-effective, high-bandwidth networking to support their missions and answer the needs of their faculty, staff, and students. CENIC designs, implements, and operates CalREN, the California Research and Education Network, a high-bandwidth, high-capacity Internet network specially designed to meet the unique requirements of these communities, and to which the vast majority of the state’s K-20 educational institutions are connected. In order to facilitate collaboration in education and research, CENIC also provides connectivity to non-California institutions and industry research organizations with which CENIC’s Associate researchers and educators are engaged.
About the Pacific NorthWest GigaPoP (PNWGP)
PNWGP is a not-for-profit, advanced networking organization whose roots go back to the creation of the original Internet. PNWGP provides state-of-the-art broadband, optical, and other networking as well as offering direct peering and exchange capabilities. These offerings serve to interconnect nearly all of the major research institutions and many other schools and colleges in Washington, Alaska, Hawaii, Montana and Idaho to one another and all of the USA national research and education networks. Further international connectivity is facilitated via the Pacific Wave distributed international peering facility which PNWGP operates with CENIC, and in which most of the research and education networks in countries around the Pacific Rim participate.
About UC San Diego
Since its founding nearly 50 years ago, the University of California, San Diego – one of the ten campuses in the world-renowned University of California system – has rapidly achieved a reputation as one of the top institutions in the nation for higher education and research. With annual research funding of more than $650 million, UC San Diego ranks fifth in the nation and first in the UC system in federal R&D expenditures. UC San Diego faculty and graduate programs have been ranked 10th best in the nation by the National Research Council. The campus, which enrolls more than 25,000 students, has one of the nation's highest percentages of faculty elected to the prestigious national academies. Among UCSD’s best-known units are the Scripps Institution of Oceanography (which pre-dates the university’s founding), and the California Institute for Telecommunications and Information Technology (Calit2), which turns 10 years old in December 2010.
Source: UC San Diego
10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.
10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.
Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?