The Future is Grid

by Simon Firth

For years, Grid computing has been the province of academic and government researchers, who have been hooking together individual computers around the world to perform compute-intensive tasks.

These virtual supercomputers, which can quickly process vast amounts of information, have helped produce breakthroughs in meteorology, physics, medicine and other fields. Scientists have even harnessed the power of the Grid in the SETI@home search for extraterrestrial life.

Now researchers at HP are working to bring the benefits of Grid computing to the corporate world.

Grid computing will let enterprises use their IT resources more efficiently. It will allow collaboration within a company on any kind of compute task of any size or duration. It will make it possible to set up ad hoc, secure IT projects between geographically disparate units, and even across organizational boundaries.

"Companies want to better utilize the technology they have," says Greg Astfalk, HP's Chief Scientist and a leading proponent of grid computing within HP. "They want to interoperate and collaborate better and more efficiently and securely with partners. Grid addresses those needs."

But even though the promise of Grid computing makes it one of the most talked-about ideas in corporate IT today, the barriers to creating an enterprise Grid are considerable.

An enterprise grid must be smart enough to navigate boundaries such as corporate firewalls and networks, and at the same time, must be reliable enough to process mission-critical applications such as company payrolls and credit card transactions.

"Scientific applications can be scheduled in 'batch' mode or are deadline-driven," says Sharad Singhal, a principal scientist in HP Labs. "By contrast, commercial applications run continuously over long periods of time and have more stringent requirements on reliability, security and accountability."

The idea of the enterprise Grid has galvanized scientific institutions and major corporations such as HP into forming a standards body, the Global Grid Forum (GGF). Since its inception in 2001, GGF has overseen creation of an Open Grid Services Infrastructure.

"The Grid has the potential to solve real business problems by simplifying global access to enterprise computing services," says Shane Robison, HP executive vice president and chief strategy and technology officer.

HP already delivers many Grid capabilities to commercial customers via its Utility Data Center. The UDC, based in part on research from HP Labs, is a self-adapting, self-healing and policy-driven system in which computing assets are wired once, then provisioned virtually and automatically. The UDC lets organizations allocate and reallocate resources on the fly.

Hard technical problems remain before enterprises can make full use of Grid capabilities. One problem being tackled by HP Labs is determining how to connect multiple data centers to the Grid.

To address that, researchers designed the Grid Resource Topology Designer, a simple, graphical user interface for designing Grids of UDCs or data centers. Topology Designer allows users to easily and securely "draw" resource needs, then submit the requirements to the Grid for fulfillment. Working with the data center, it automatically decides on the appropriate resources to deploy to fulfill the service-level request.

"Say you wanted to deploy an application in two data centers," says Sven Graupner of HP Labs, one of the creators of Topology Designer. "The description is submitted using Grid protocols to the Grid, then to the data centers, or a broker between data centers, and it will come back and say if the job's been accepted or not."

Although still at the prototype stage, Topology Designer illustrates that it's possible to build application environments that work across data centers. "This is exactly the vision the Grid summarizes under its notion of a virtual organization or collaboration," Graupner notes.

A related technology being developed by HP Labs, the Adaptive Control Interface, uses standard Grid infrastructure in the UDC to create an adaptive control system for adding or subtracting servers from an application.

Another hurdle for the enterprise grid is the lack of a broadly accepted way of deploying and configuring software and hardware to run over it. A team in HP's Bristol, UK, lab built a framework -- the Smart Framework for Object Groups, or SmartFrog - that provides rules for how resources on a Grid are configured, run and managed through their operative life cycle.

"Configurability and deployment are critical aspects of the technology today. Adaptive infrastructures, and utility computing in particular, are complex concepts that can be significantly simplified by automated deployment tools such as SmartFrog," says Patrick Goldsack, one of the designers of SmartFrog.

Researcher Dejan Milojicic is working with the Global Grid Forum to try to incorporate SmartFrog concepts into an industry standard. "We've had a lot of interest from the Grid community," he says. "Other companies have expressed interest in making reference implementations -- so we're very happy about that."

This year, HP Labs put SmartFrog to work configuring a service for rendering complex computer graphic images. A team created an interface to an HP Utility Data Center in HP Labs Bristol (UK) for a local animation company and commissioned a high-quality, animated short film, "The Painter," which thanks to HP technology, was rendered in a fraction of the usual time.

Another missing piece needed to bring Grid to the enterprise is a uniform way of managing the wide range of computing resources in an organization, which may include multiple operating systems, platforms, data formats and applications.

HP's Web Services Management Framework (WSMF) could be the answer, providing a management framework for all types of IT resources. Applied to the Grid, WSMF offers a common approach for managing all components of a Grid deployment, including resources and services. Because the framework is itself built on Web services, it can deal with such issues as federation, distribution, manageability and standards, according to Homayoun Pourheidari, one of the architects of WSMF.

"Grid wants to render everything in IT as a Grid service," explains Chief Scientist Astfalk. "It could be a computer, a storage array, a file, a database. It could be a record in that database. It could be an application program or it could even be a scientific instrument."

Using open protocols and interfaces, he says, makes it possible create a way "to describe all those services, to register them, provision them, bring them to life and tear them down -- all the types of manipulations you'd expect on a service."

Engineers in HP's OpenView team in the HP Software Global Business Unit and HP Labs researchers recently worked together to make WSMF Grid-compatible, integrating it with the Global Grid Forum's Open Grid Services Infrastructure standards.

"We're trying to create standard interfaces to manage and control all sorts of resources -- from printers to firewalls to applications," says Akhil Sahai, an HP Labs researcher involved in the project.

WSMF was recently submitted to the OASIS Web Service Distributed Management Technical Committee toward the goal of achieving a standard management solution for the industry.

Inside and outside the lab, HP is putting its energies toward Grid computing, working to Grid-enable all of the company's products. Products ranging from the smallest handhelds, printers and PCs to the most powerful storage arrays and supercomputers will be able to connect with and serve as resources on a Grid.

HP's Grid strategy has three primary thrusts. Most importantly, Grid computing enables HP's vision of the Adaptive Enterprise -- where information technology is a highly efficient, flexible service that is agile enough to change in line with a corporation's business and its business environment.

"If you think of the Adaptive Enterprise as set of bricks," says Astfalk, "and each brick being a solution piece, the mortar that we envision holding these pieces together is Grid.

Another benefit to Grid-enabling HP's systems is working with customers and partners already using Grid, primarily those in scientific and technical computing.

In the long term, Astfalk foresees the Grid making a huge contribution in a third area, the space where home networking meets entertainment media. In these cases, there's typically a content provider that allows access only to subscribers. The provider may also require that materials are viewed on a device with certain characteristics, such as high-grade security.

"All this could be done by virtual organizations in a loosely coupled service-based economy," says Astfalk. "And that's Grid."