December 12, 2011
At Microsoft® Bing™ Maps, technology imitates art. The Bing Imagery Technologies team takes diverse satellite, aerial, bird’s eye, and street-side imagery and stitch them together to form visual mosaics for Bing Maps.
Given its compute- and data-intensive mission, and the huge scale of its operations, the team requires enormous amounts of processing power, data storage capacity, and memory. It measures its storage needs in petabytes, its memory needs in terabytes, and its processing needs in tens of thousands of cores. At the same time, the team must work continuously to drive down the cost of processing this data.
When it came time to launch a new imagery-processing site at the edge of the Colorado Rockies, the Bing Imagery Technologies team (BITs) studied its options carefully. It wanted to find a no-frills data center infrastructure that was highly efficient, highly reliable, extremely scalable, and quick to deploy. It found its answer in Dell™ Modular Data Centers, which are helping the team build a highly efficient, state of the art imagery processing microsite.
The route to the Modular Data Center
The team’s road to a more efficient microsite began when the organization was making plans to move its legacy containerized data-processing infrastructure to a new site that could accommodate additional capacity and ongoing growth.
The team found a workable, eight-acre patch of ground in Longmont, Colorado, northwest of Denver. The requirements for the microsite were aggressive. Bing Maps needed an extremely cost-effective, highly efficient solution to power high-performance compute-and storage-intensive geospatial applications running a customized parallel processing system built on Windows Server 2008 and SQL Server 2008. The solution needed to be up to the challenges that come with processing and storing terabytes of data on an hourly basis.
“We produce hundreds of terabytes each day,” notes Ryan Tracy, IT operations lead for Bing Imagery Technologies. “We generate about ten terabytes per hour. Our record is three petabytes in four days. So we have a significant amount of processing going on.”
The requirements didn’t stop there. The team needed a modular solution that could be quickly deployed to add capacity to support growth and new partner needs. In addition to creating imagery for Bing Maps, the organization produces geospatial imagery for various strategic partners.
On the power and cooling front, the team wanted a modular solution that could make maximum use of free air cooling to capitalize on Colorado’s climate. What’s more, it wanted to achieve an extremely low power usage effectiveness (PUE) ratio—a key measure of data center energy efficiency.
Technology at Work
Choosing the Dell solution
Bing Maps personnel talked to multiple vendors before settling on Dell. At a product level, they liked the modularity, superior PUE, low overall power utilization, and high density of the Dell Modular Data Center, according to Clark. At an engineering level, they were impressed with the expertise of the Dell Data Center Solutions team who designed the Modular Data Center and customized the solution, tailoring it to the needs of Bing Maps.
“Dell had a solution that was designed for high efficiency, density, and low cost,” Clark says. “Their container allowed us to have a flexible IT unit, which gave us room to customize or modify to meet our needs. It was a clean and simple solution, and it was obvious that we were not going to be paying for items within the solution that we didn’t need.”
The Dell DCS engineers on the project went to great lengths to accommodate the specific needs of Bing Maps and the custom geospatial applications it runs, Clark says. For example, they changed the server design to meet the unique storage and I/O needs of the BITs processing environment and they integrated InfiniBand networking infrastructure into the solution.
“Dell closely partnered with us for the duration of the project on all aspects,” he says. “We really challenged everything to keep costs low. The Dell guys were cut from the same cloth. We just made sure that everything that was needed was really needed. Dell realized that we were unorthodox, and they were willing to jump in with us.”
Fast deployment speed
Rapid deployment was another over-arching need for the team. The organization wanted the ability to bring on new IT capacity quickly to support new products, new partnerships, and ongoing growth.
In its Longmont experience, the team learned that it can now have a microsite up and running within 60 days, assuming all government permits are in place, according to Clark.
“We have the design now, and it can be put anywhere, as long as you can find land that is relatively flat and has power and networking,” he says. “We now have a reference site, we have the design, and we have a great partner in Dell that can produce the IT gear and the containers in the time-frame that we put in place. And we’ve proven that we can turn those containers on in three weeks, from dropping them from a truck to turning them live. So everything’s up and running within 60 days. That’s a pretty amazing feat.”
“In working with Dell, we were able to build a microsite that’s cost-effective, reliable, agile, and rugged, that could be located in extreme environments within various countries around the globe,” he notes.
‘Better all the way around’
“When I first looked at the design of the Dell Modular Data Center, I saw that it’s very clean and it’s very elegant,” Clark says. “It’s very well thought-out. And when you get inside the container, it’s very quiet.”
The quiet environment is made possible by the Dell Modular Data Center’s use of a central fan system that cools the environment, rather than fans in each of the servers.
“I thought that was a smart and clever approach,” Clark says of the central fan. “And having the evaporative cooling system in a dry climate like the Colorado Front Range is perfect. It makes sense. It drives lower operating costs. It’s just better all the way around.”
Ultimately, “better all the way around” describes the feeling of the Bing Maps’ team when it comes to working with the Dell Data Center Solutions team to design and deploy Modular Data Centers at the Bing Maps Imagery microsite in Colorado.
“They had the best solution and the best team, which was well aligned with our site philosophy,” Clark says. “The whole Dell team was just fantastic on all fronts.”
Watch the time-lapse video of the site being built!
Watch a quick walkthrough video on a Dell Modular Data Center.
10/30/2013 | Cray, DDN, Mellanox, NetApp, ScaleMP, Supermicro, Xyratex | Creating data is easy… the challenge is getting it to the right place to make use of it. This paper discusses fresh solutions that can directly increase I/O efficiency, and the applications of these solutions to current, and new technology infrastructures.
10/01/2013 | IBM | A new trend is developing in the HPC space that is also affecting enterprise computing productivity with the arrival of “ultra-dense” hyper-scale servers.
Ken Claffey, SVP and General Manager at Xyratex, presents ClusterStor at the Vendor Showdown at ISC13 in Leipzig, Germany.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?