By Jamie Beckett, July 2008
Researchers will gain access to a vast store of computing power to test and advance new types of cloud-computing software, data center management and hardware issues as a result of a new collaboration involving HP, Intel, Yahoo! and several partners.
The test bed will provide an open, global network of computing clusters, allowing scientists to pursue research on the software, data center management and hardware issues associated with cloud computing. It is designed to promote open and collaborative research by removing the financial and logistical barriers to research in Internet-scale computing.
"It's all about making it easy for researchers to access the resources they need to push the envelope," says Prith Banerjee, senior vice president for research at HP and director of HP Labs, "The test bed furthers our commitment to the global, collaborative research community that is advancing the new sciences of the Internet.”
Computing resources will initially be hosted at six centers of excellence – the Infocomm Development Authority (IDA) in Singapore, the University of Illinois at Urbana Champaign; Karlsruhe University in Karlsruhe, Germany; HP Labs, Intel Research and Yahoo!. More centers will be added in the future.
Each center will offer 1,000 – 4,000 processor cores and up to several petabytes of storage. The facilities will consist of HP hardware and software, and will run Yahoo's Apache Hadoop, an open source platform that lets one easily write and run applications that process vast amounts of data.
For more in-depth technical information on the test bed, please see »The Technical Overview
Although cloud computing is still relatively new, large-scale, distributed computing environments are regularly used for such data-intensive tasks as predicting climate change or analyzing risk.
Russ Daniels, vice president and chief technology officer for cloud computing services at HP, says he expects innovations in cloud services to eventually lead to an "intuitive Internet" that is focused on users rather than on technologies.
He sees computing moving toward a model where services hosted in the cloud collaborate with a broad range of devices to determine a person's intentions and offer assistance.
"Today we leave it to the user to figure out when and how technology might help, limiting the industry’s addressable market to the technology-savvy," Daniels says. "The cloud provides a platform to attack this complexity, capturing context and analytics that enable technology to behave intuitively, make suggestions and take initiative."
“This environment is unique in that it is open and enables research on systems and data center management – giving researchers the ability to ‘look under the hood’ to better understand an infrastructure shared across multiple continents and data centers,” says Kumar Goswami, director of strategic commercial collaborations at HP Labs.
Access to this level of distributed computing power allows scientists to conduct "a very rich form of research," he adds.
Access to multiple interconnected data centers could, for example, enable research on loosely coupled federated computing, disaster recovery and follow-me functionally for mobility applications.
For HP Labs, the shared large-scale cloud infrastructure will provide a vehicle to further work around cloud computing and services.
Specific areas to be explored
» Cells as a service
» Exascale data centers
» Scalable storage
» Service lifecycle management (SLiM)
» Sustainable data centers
"The test bed is aimed at researchers who want to try things out at large enough scale that they experience real problems," says HP Fellow John Wilkes, one of the researchers on the project. "It's not enough to find solutions that work in a research lab under controlled conditions – we need to find a set of simple solutions that can be applied en masse to the next generation of cloud-based services."
Organizations interested in joining the collaborative research initiative, please contact us.