The Client Utility project at HP Labs was started to explore ideas needed to build a scalable, secure, and manageable peer-to-peer computing environment. The results of that research formed the basis for e-speak, an open source project that HP also released as a fully supported product. This report is the most complete description of the Client Utility architecture at the time the research work was transferred to the newly formed Open Services (soon to be renamed E-speak) Operation. It is reproduced here for its historical interest, showing the thinking that led to e-speak and the concept of e-services.

* Present address Transmeta Corporation, Santa Clara, CA

Introduction

Before there was e-speak¹, there was the Client Utility project at HP Labs. Work began in late 1995, and a first prototype was ready by March 1996. That prototype was refined over the next 6 months, and a set of use cases were put together to demonstrate the vision. Once a commitment to go beyond the prototype stage was made by HP management, the entire system was rearchitected based on what we had learned from the prototype. Eventually, these ideas were implemented as the first prototype of e-speak and announced to the world in May 1999. The motivation for, and ideas behind, e-speak have been described elsewhere².

As you will see from reading this document, the architecture describes a peer-to-peer system, one of the earliest to be based on this concept. One of the other ideas that form the basis of the architecture is that everything can be thought of as a service, leading to the e-services vision introduced by HP and that has recently captured the imagination of the computing community as web services. The first figure, which we used as the logo for the project, shows examples of many of the features we were designing for. Today, we see the same features listed for peer-to-peer infrastructures and web services environments.

There are three principal ways that the Client Utility architecture anticipated the environment needed for peer-to-peer computing. The first was treating everything as a service or resource that could be linked into a dynamically composed value chain instead of using a specific web service from a client applet. The second was the need for a flexible way of describing services and a powerful means of finding them, vocabularies as discoverable resources. The third was recognizing the need for advertising services so that those looking for services could find them, even without knowing ahead of time who might be providing them.

As with many research projects, the Client Utility was rich in ideas but weak in documentation. This report is the most complete specification of the architecture. Most of the ideas were present in the open source releases through December 1999, although some pieces were left for later implementation. After that, the product was targeted at the B2B portal space, and substantive changes were made. However, many of the concepts presented here survived.

Since this document is to be treated as a historical document, only minor changes were made in producing this report. For example, the "Confidential" label was removed and dead web links were replaced with explicit references. No attempt has been made to finish any of the incomplete parts of the architecture or to update them with what we learned during the implementation. In particular, the issues of specifying security policy and interfacing to management systems are particularly weak. The failure limitation work is still a subject of active research. What follows is the document in its original form.

1. http://www.e-speak.net

2. "E-speak E-xplained", HP Labs Tech report HPL-2000-101.

The Client Utility

Table of Contents

1. 1. The Vision: Why the world needs the Client Utility
2. 2. Technology Overview: Describes virtualization, resource model, protection domains
3. 3. Core Architecture: Informal description of the architecture
4. 4. Repositories and Persistence: Mechanisms used to maintain state
5. 5. Security Model: How access to resources is controlled
6. 6. Managing and Monitoring a Utility: Management interface to dynamic state
7. 7. Intermachine Protocol: Sharing resources between machines
8. 8. Systems: What it means to make a Client Utility System
9. 9. Failure Limitation: How to report failures and limit their impact
10. 10. Scenarios: Examples of operation of the Core
11. 11. Related Work: Why the Client Utility is different
12. 12. Summary and Conclusions: Wrapping it all up

Why the world needs the Client Utility

• 1.1. Why Client Utility: What problems the Client Utility addresses
• 1.2. Enabling Infrastructure: What we need to build a Client Utility
• 1.3. Scenarios: How the Client Utility can be used

Today at our homes and work we do not think about power, how it is generated or where it comes from. We simply plug into the wall and use the facilities provided by our local power utility company. Client Utility computing intends to create a paradigm for end-user computing where users simply plug into the wall and use the facilities, services and applications provided by their local utility. This form of computing is realized by a middleware infrastructure that facilitates ubiquitous/pervasive computing.

For information technology to become truly pervasive, it must transcend being merely manufacturable and commonplace. It must become intuitively accessible to ordinary people and must deliver sufficient value to justify the large investment needed in the supporting computation infrastructure. The consumer's expectations from the computation infrastructure or utility will be substantial. Just as people pick up a phone and expect a dial tone, so too will people expect the infrastructure to be available, ready and waiting.

In a high bandwidth, digital, multimedia world -- in which clients (general-purpose computers or appliances) connect to a utility - people could pay for their computing by usage, modulated by guaranteed response requirements. This enormous paradigm shift, changes what is now a capital investment into a competitive service, like electricity and water. It will do for computing what the Web did for data. It is not the ultimate revenge of time-sharing, which was proprietary, fixed resource, and usually location-dependent. In this form of computing, called Client-Utility Computing, standards-based open resources, located arbitrarily, are combined as needed for a particular job. It will no longer matter where the computers operate or which manufacturer makes them.

The suicidal obsolescence schedule of today will be replaced by the capacity requirements of the service provider, with upgrades undetectable by the end-user within a given performance envelope. The trend towards such utilities is likely, since the end-user benefits from greatly decreased cost of ownership, the manufacturers gain a relaxation of insane time-to-market demands, and a new lucrative service industry should emerge. If this new model of computing becomes practical and reliable, history tells us that current manufacturers who do not adjust in time will not be able to satisfy customers and will suffer dire consequences.

Resource access does not mean owning the machine, configuring the resource (such as through file system mounting), or even managing the resource itself. All these activities can be automated and outsourced, thus opening up the possibilities for new types of service industries. It is important to note that such a paradigm does not require the users to give up control of their machines or have to compromise on the safety of their data. This fundamental change in end-user computing allows for the creation of paradigms where the users deal with computing and computing systems much like they deal with power utilities and paper. In short, users may pay for it and use it is as an integral part of their daily lives, but in general they do not notice or pay attention to its existence.

Gives examples of how the Client Utility changes the way people use computers.

• 1.3.1. Introduction to Scenarios: Why you should read this section
• 1.3.2. Enterprise Computing: The Client Utility as an enterprise IT infrastructure
• 1.3.3. Internet Service Provider: A new set of businesses for ISPs
• 1.3.4. Print Services: Giving a printer access to data and compute cycles
• 1.3.5. Commercial Services: Composing services and making them widely available
• 1.3.6. Web Management: How Client Utility addresses problems of large Web sites
• 1.3.7. Compute Center: Making money on surplus cycles and disk space

Client Utility computing makes possible many different scenarios which would otherwise only stay on our technology wish list. Many of these scenarios ameliorate existing ingrained problems, but quite a few of them open up the possibility of entirely new markets and businesses. In fact, there are few restrictions to the kind of utilities that may be built. For example, a printer manufacturer may want a printing utility to deploy and provide innovative printing solutions; a large enterprise such as a FedEx may want to deploy an enterprise utility to simplify (read lower the costs) of the management and administration of its IT infrastructure; a company like GM that needs large Web sites can significantly increase their flexibility and lower costs through a Web Utility and so on. The next few sections present utilities that could be of interest to various HP businesses and divisions.

A whole enterprise or simply parts of it may use a utility for the IT infrastructure. Users in an enterprise get simultaneous access to a myriad of applications from different operating systems, transparently through the utility. Type "winword" on a UNIX machine and the utility will find the application, run it on an appropriate machine and forward the display to the end user's machine, without any prompting. Incompatible file formats from different versions of software are seamlessly handled; the utility uses attribute matching to find and handle differences between file formats.

System administrators may reset and move storage around for best utilization of network disks. Thanks to the utility this does not affect active users or running applications. Rolling upgrades of operating systems and application software do not affect any users and applications in the enterprise; the utility virtualizes resource accesses so that name transformations, or application availability can be completely hidden. During peak processing requirements or heavy business seasons, compute resources from outside vendors are automatically and seamlessly brought on-line to ramp up capacity. In short, a utility environment in an enterprise drastically reduces the cost of ownership without rapid obsolescence or complete rewrite of application programs. Enterprise Intranets and systems work as they are supposed to - smoothly - not as a disjoint system requiring immense amounts of caring and feeding.

Internet service providers can change the very model of service that they provide to their users. ISPs provide users with a virtual personal computer. This customizable personal computer may contain applications from any number of different systems and the users may configure the environment, directories as they choose. No matter where in the world the users log in from they see the same set of files, directories, preferences, and resources. The ISP company dynamically maps this virtual personal computer to resources on a large number of different machines - changing this mapping as required by system loads, user location or network bandwidth.

None of this mapping and remapping affect the active users or running applications. So, a user may start a large computation on his virtual personal computer and fly to the next state, login and check on progress of his computation. Naive users do not have to worry about installing applications or configuring them. They simply boot their machines and use a pre-configured virtual computer provided seamlessly by their ISP utility. At the end of the month, users pay one additional bill - to their ISP utility. Simply, put none of this service model would be technologically feasible without having a utility infrastructure.

ISPs can also manage their plethora of machines more easily via the management infrastructure provided by the Client Utility. Centralized management is provided by the same mechanisms used to provide transparent access to users. All that is needed is to give the administrator a different set of permissions than those given other users. In addition, a hacker breaking into one of the ISP's machines will have a difficult time compromising more of the system.

Print Utilities extend the notion of what a printer can do, how tightly it may be integrated with other compute resources, and allows printer manufacturers to think beyond the printer hardware to alternate commercializable print solutions. A large rendering job that reaches a printer is automatically forwarded by the Print Utility to the nearest machine with adequate horsepower. Such an operation cannot be implemented simply through remote management of printers but needs a utility-like infrastructure to allow the printer to locate seamlessly remote compute cycles and subsequently access them.

A new non-standard format document reaches a printer, which upon not recognizing it uses the utility to find a resource on the Internet and to render automatically the document to a recognizable format. Travelers in an airport walk up to kiosk printers and identify themselves through a swipe of their smart credit-card. The printer utility contacts the traveler's home machine automatically, extracts all urgent email messages and displays them on a small display attached to the kiosk printer. The traveler points to one or more items and they are automatically printed out; charges being automatically sent to the traveler's credit-card. Here the print utility enables an entirely new business opportunity for printer manufacturers and resellers.

In a similar vein, a travelling executive could walk up to an Ethernet tap or phone line in an offsite office and hook up his portable. He opens up an appropriate document, suggests the kind of printing desired and prints it - without having the appropriate drivers or having to configure the appropriate printer queues. Here the utility locates the machine with the appropriate drivers, send over the document to it. That machine in turn locates the nearest appropriate printer and sends the print job to it.

Utilities that allow for the on-line representation of business processes, inter-company financial transactions and other Internet services are very much in the realm of short term possibilities. Of course, many companies are marketing business management solutions, but each is a point solution to a specific problem. The Client Utility leads to an environment where such solutions are constructed by composing a number of services instead of writing a specific program. It is the Client Utility infrastructure that makes such compositions feasible.

The change is significant. When in need of parts, a computer manufacturer's inventory management program puts out an on-line request for bids with appropriate specifications for both the parts as well as the supplier businesses. Supplier businesses with the required credentials (supplied by an online certification authority) bid for the part dynamically. The utility ensures that the bids are matched with what the supplier companies claim to provide, uses an on-line negotiation service to provide a fair price for both. Finally, the manufacturer's inventory management client contacts an online representation of a packaging service through the utility and arranges to have the parts delivered. Negotiation for services, contract maintenance, and payment support can be provided as available services, while the matching and mapping of bids and requirements, deployment of services, and real-time location of appropriate services is done by the utility itself.

This sample is only one possibility in the realm of commercial services utilities. It is also possible to use the same model to create new software businesses. A deployer of a new service in a commercial services (e-commerce) utility need only describe the components needed by the new service. If these components are provided by other providers in the Client Utility, the workflow completely specifies the service. When a customer request comes in, the service provider gets bids from its providers of the individual components, selects the appropriate ones to do the work, and returns the final result to the customer. In this way a software service is no different than a build to order computer made up from parts delivered by a just-in-time inventory management system.

The Client Utilty provides the infrastructure the provider needs to make money from the service; this includes service specification, dynamic installation and execution as a result of client accesses, and security. In each of the steps the utility plays a significant role. In the service specification step, the new service may use one or more pre-existing utility services; the utility here provides the basis for thinking and reasoning about value-added services. The glue for the installation step is the utility infrastructure's APIs, which automatically subsume advertising and informing clients if necessary. During the execution step, the utility actually acts as a composition service allowing for the interposition of payment and contract maintenance options, as well as allowing the new service to easily access/use pre-existing services. In every step of the process, the Client Utility protects the integrity of all the participants.

Large, complex websites of today can be reorganized into highly manageable web utilities. An image processing request comes into a large web site; the utility automatically finds the least loaded machine and runs the image processing application. In fact, if the requestor of the processing needs a print out, the utility could find an appropriate printer geographically near the requestor (say at the nearest Kinkos) and automatically send it there after informing the requestor.

As requests increase beyond a certain threshold, the utility automatically moves the overflow to the computer center utility. To enable this the administrator simply had to modify the capacity constraints at the utility management console for the website. Adding machines, changing software, and failures are handled without affecting incoming requests. As machines come up and go down the utility ensures continuity and resource visibility. To support better availability for a particular set of files, the web-site administrator simply changes the replication policy at the utility management console. The utility ensures that the appropriate replication module gets interposed.

Compute center utilities rent out hardware to other commercial concerns in very novel ways. They do not rent out disks; they rent out disk space. Even more, they rent out file services such as version control, backups, etc. They do not rent out computers; they rent out compute cycles. Even more, they rent out reliable access to cycles by rolling work over to new machines as others get overloaded. They do not rent out memory simms; they rent out the use of RAM. Even more, they can rent out non-volatile storage for a fee.

A package handling company during Christmas season could automatically offload overflow computation to machines in the compute center utility. To the running applications that need these extra cycles, this is completely transparent. Due to a contract with the compute center, the package handling company's machines can seamlessly access the cycles on the compute center machines. Interestingly enough, if the compute center wants to ensure that the package handling company only uses 100,000,000 cycles/second and no more; it can do so through the utility infrastructure's fine-grained resource access checks.

Through similar mechanisms, small businesses can rent out out disk space with varying constraints, such as storage that is backed up once a week or remote storage access times less than 1ms, from a compute center utility. Such on-line outsourcing of hardware resources is not practical with current technology. In fact, a Client Utility can evolve from a conventional client-server environment. Here the existing infrastructure itself becomes a utility and finally links with an external computer center utility to get additional computation resources.

It is important to note that these utilities are not carved out from a disparate and disjoint sets of technologies. In fact, designing and constructing such utilities would be impossible without a structured way of thinking about the technology for utility infrastructures. By solving such problems as naming, security, and location independence in the underlying infrastructure, the Client Utility makes a large number of new approaches feasible.

• 2.1. Introduction to the Technology: Structure of system
• 2.2. Abstractions: Key abstractions used
• 2.3. Core Services: Basic services provided on all systems
• 2.4. Support Software: Additional services for some systems
• 2.5. Intermachine Protocol: Sharing resources with other machines

At the core of our technology lies the ability to virtualize resource accesses on host systems (which could be appliances, workstations, or large servers) and map these accesses to the uniform resource abstraction presented by a Distributed Resource Interchange Protocol (DRIP). Virtualization of resource accesses allows for such characteristics as mobility, distribution, availability, manageability, security, and reliability to be added seamlessly to the resource being accessed, without affecting the client applications.

Virtualization paves the way for domain specific protocols such as file access interfaces and the X-protocol to be mapped to a generic interchange protocol like DRIP. The use of DRIP as a protocol creates a highly extensible software bus that can connect any set of resource provider-accessor pairs without extensive a priori arrangement. This software bus is programmable and may be associated with properties such as secure access, negotiation for services, attribute-based location of appropriate services and replication to name a few. This generic interchange protocol allows fine-grained manipulation of inter-module (or client-resource provider) interactions. It also forms the basis for creating higher level resource and management abstractions such as the virtual personal computer or the web management utility, to name a few.

An application may choose to be directly aware of this software bus or be completely unaware of it. In case it is aware of the Client Utility software bus, it can exercise fine-grained control over the properties associated with the software bus. On the other hand, an unaware application can still use the facilities provided by the software bus. However, control and management of the properties must be done through management APIs (interactive or programmable). The control of these properties allows an application to negotiate for the cheapest service, locate the nearest service, ensure that its data is safe, replicate the use of certain services to ensure reliability, etc. There is also a third possibility depending upon the success of such utility infrastructures. The infrastructure may be subsumed into the local operating system, thus extending the nature and functionality of operating systems. This merger is a possible future step, but in no means necessary for the use and success of the utility.

The Client Utility architecture is a framework that can be used to describe, control, manage, and match the interaction between any resource provider and a client of the resource. The resource provider could be a virtual memory manager or a search engine - the principles used in the framework still apply. The client could be an OS's process management subsystem, a legacy NT application, a newly minted Java applet or an Active-X control - the intention at least is to provide appropriate mappings for most of the important categories of applications. This framework not only acts a "software bus" for controlling resource interactions but also acts as a substrate that facilitates higher-level resource aggregations and management abstractions, such as information utilities and Web-site control for large distributed sites.

This approach allows for various ways of componentizing applications and services. The kinds of functionality that had to be built into clients or servers, such as conversion between two incompatible interfaces, can now be provided as separate intermediary services implemented by independent organizations. This degree of componentizing significantly changes how we think of application construction and service providers in distributed systems. All these drastically lower the barrier to creating, deploying, and using networked services in a large distributed environment - the fundamental goal of a utility.

     /usr/bin/winword,

     c:\winnt\system32\WinWord.exe.

The entities that the core services deal with are components of all application-specific (or domain-specific) interactions. These entities are dealt with by the core services in a generic manner, and consequently higher level management abstractions can reason about and manipulate client-service interactions uniformly though the use of introspection APIs. Also, the conversion to a generic interaction protocol relieves clients and services of dealing with naming mismatches, interface variations, and the like, that invariably occur in the interactions between independent components. In this sense, the OS not only acts as a substrate for building higher-level paradigms, such as utilities and virtual personal computers, but also simplifies the construction, deployment, and management of networked clients and services.

Support software in the Client Utility architecture falls into three specific categories. Emulation software allows existing legacy environments to benefit from the functionality provided by the client utility middleware implementation. For example, emulation may enable Microsoft Word '97 to access seamlessly remote UNIX files. Similarly, NTFS files may be seamlessly exported to the utility and made accessible to client applications on non-NT machines.

General form a small list of meta-services such as metadata repositories, directory, and authentication services which are necessary to build complete systems. They allow the client-utility systems to bootstrap and use existing industry-wide standard functionality as far as possible. Domain specific services include all the services and functionality needed to implement specific forms of utilities. Hence, a utility supporting e-commerce could include contract maintenance services.

The protocol used by Client Utility machines has several stages - connection, authentication, exchange of resource descriptions, use of remote resources, and failure limitation and recovery. Each of these components was designed to be consistent with the scalability, heterogeneity, and security requirements of a large, distributed system.

The first step in participation in the Client Utility is to find one or more machines to connect to. The list of machines could be provided by a system administrator, a list of participants could be kept by some directory service such as Yahoo, or the machine could broadcast a request. However it is obtained, the machine now has a list of other machines to talk to. Since all connections are pairwise, we need only describe a single connection.

The information needed to establish the connection is contained in a Connection object that was prepared by the machine being contacted. It specifies everything needed to communicate with this machine, such as communications mode (HTTP, TCP, UDP, ...) and contact port (if appropriate) as well as things about the machine, such as its endiannness.

A machine running the full Client Utility infrastructure will start a new task running in a very restricted protection domain, a sandbox that makes available only the resources the task needs. This task is only allowed to talk to the other machine over the agreed upon channel. All communication from a specific machine is mediated by its proxy. These proxies will negotiate a dialect of the Client Utility protocol that both machines understand. In this way machines running different versions of the protocol can still communicate. If they don't have a dialect in common, they will terminate the connection.

The first thing the machines will do once they have determined a common dialect is to authenticate each other. What constitutes authentication determines the context. Two machines within an enterprise can prove to each other that they do indeed belong to the same company. Once that is done, they can share resources they wouldn't want seen by outsiders. If they are communicating over an insecure line, they can also establish a session key so that all messages can be encrypted.

In a commercial setting, authentication may not require identifying the machine or owner of the machine. Instead, it may make more sense to establish an ad hoc contract. For example, a customer may supply a credit card number or prove possesion of a digital wallet acceptable to the vendor. In these cases, the service provider is not authenticating the buyer, only the party guaranteeing payment, a scheme that greatly simplifies the model. It is also reasonable to defer the authentication, allowing the other party to see only publically available resources such as advertising.

Once the machines know who they are talking to, they can decide what resources they are willing to make available to users on the other machine. This list constitutes a policy determined by the administrator of each machine. Few resources will be exposed to machines that haven't established a high degree of trust; more resources will be shared with those that have.

The proxy on the machine that owns the resources will ask the Core to add name associations for the resources to be exported to its protection domain. It will then send a description of each resource to the other machine. When it receives a description of a remote resource, the receiving proxy will validate whatever part of the resource description it understands. Next, the receiving proxy will ask its Core to register these resource naming itself as the resource proxy.

When the exchange is complete, the sending proxy will have a protection domain consisting of the minimum set of resources it needs to do its job, all the resources imported from the other machine, and all the resources exported to the other machine. The Core will have added entries for all the resources its proxy imported. Any use of these resources will be forwarded to the proxy as the designated resource proxy.

Tasks add resources to their protection domains by telling the Core the attributes of what they want. There is no need for the applications to know the source of these resources. They may be local, or they may be remote. If they are remote, any request to use one will be forwarded to the proxy for the machine that owns the resource. Even the Core is not aware that the request is for a remote resource. As far as the Core is concerned, the proxy is just another resource proxy. When a proxy executes a command on behalf of a task on the other machine, the Core sees a local request. This time the proxy looks like any other local client.

The proxy for another machine can be intelligent and partially process the request, or it can merely forward it across to its counterpart on the other machine. That proxy can also do some processing of the request, or it can attempt to execute the request. When it does, it talks to the thread that is acting as its client proxy for the Core. The request gets marshaled into the proper format and forwarded to the Core. At this point, the Core sees a normal request from a local task. It can do all the checking it normally does without knowing anything about another machine.

The proxies not only isolate the Cores from each other, but they also provide the opportunity to do some filtering. For example, it may be corporate policy to do certain functions only on machines within the enterprise. Since the proxy sees the payload, it can redirect the request to a proxy connected to an internal machine. In this way it acts like a firewall.

Some requests may involve resources from a variety of places. For example, if a task runs a word processor on another machine with a file from the first machine, we need some way for the word processor task to have permission to read and write the file. Clearly, the resource description was exported to the machine, but the task running the word processor needs to have a name for the file. We handle this situation by having the proxy transfer parts of the protection domain to the proxy on the other machine.

When a task wants to start a job on another machine, it transfers resources to the proxy for the other machine. (Of course, we use our security mechanisms to make sure that the proxy doesn't get access to resources that shouldn't be available to users on the other machine.) The proxy forwards the request across the wire along with the resources the requester transferred as part of the request. The receiving proxy registers these resources with its Core and adds them to its protection domain. Finally, it executes the command. If the program needs resources from both machines, the two proxies must impersonate the requesting task.

A single proxy can represent many tasks by getting a number of distinct protection domains from the Core. Each one will have the set of resources common to both the requester and the machine running the command. The Core need not be aware of this schizophrenia of the proxy or any other task, for that matter.

Components will fail. These may be hardware failures, such as network links and disk drives, but software failures, such as operating system kernel panics and memory leaks, are also common. It is important to discover such failures and limit their effects. Well-behaved components are expected to report failures they detect to a designated agent. This agent will make inferences about the cause of the failure using information collected from many sources. If this agent can determine the cause of the failures, it can direct components to take action to limit the spread of the failure. Well-behave components are expected to accept such instructions from their designated monitor.

As an example, consider a machine that provides a copy of a particular file that is used by a number of applications. If this machine fails, applications asking for the file will experience a failure. They may have an alternate source for the file, so they need not elevate the problem to the user. However, the extra overhead involved in identifying the failure may be substantial. For example, they may have to wait for a time-out to expire before going to the alternate source. If these applications report the failure, the monitor receiving the information can infer the source of the problem and tell other applications to go directly to the alternate source. Further, the monitor can tell the non-responsive machine to take corrective action such as rebooting.

The goal of the failure limitiation architecture is to use information from a number of sources to limit the affect of a failure in the system. In many cases, a failure that would be elevated to the user can be repaired without adversely affecting running components. At the very least, the number of components affected by the failure can be limited.

3. CORE ARCHITECTURE

Architectural principles that apply within a machine

• 3.1. Introduction to Core Architecture: Introduces the nomenclature used
• 3.2. Walk Through: Stepwise introduction to the concepts
• 3.3. Naming: How tasks can share a resource without having a common name
• 3.4. Resource Access Control: How access to resources is controlled
• 3.5. Messaging: How a request gets from an application to a resource proxy and back

The only services provided by the Client Utility Core are name resolution, extraction of resource specific data, message routing, and a means to monitor and manage the system. All other services are provided by resource proxies written to understand the management of specific kinds of resources, such as files, memory segments, processes, etc. These resource proxies can be separate user-space processes, but trusted services can run in the same address space as the Client Utility Core.

In order to avoid confusion with terms such as System, Machine, or Processor to designate the domain of control, we will use the designation Core. Resources within a Core have unique designations; each Core has a Repository listing all resources that it knows about; each Core has the 4 basic components.

Although it is most commonly the case that there is a single Client Utility Core running on a single machine, the architecture is more general. We can have one Core running across several machines, whether they form a multiprocessor or are network connected. We can also have more than one Core running on a single CPU. This latter configuration is useful when we need to repair a machine yet present the view to the users that it is running.

This overview provides an introduction to the Client Utility architecture. Many of the special cases have been ignored in order to keep the discussion straightforward. In particular, we will describe only a single Core running on a single machine with only one CPU.

Fundamental to the Client Utility is the fact that virtually everything in the system is treated as a named resource. These resources include such familiar things as files, processes, and memory segments, but also may include resources that are not usually named. For example, it is common to name a data base, but the Client Utility also allows naming of data base records. The Client Utility also names resources that don't exist in other systems, such as name spaces and key rings. The Client Utility Core maintains a Repository which lists all resources known to this Core. Each resource has a name unique to this Core, a Core Repository Handle (CRH). A Core repository handle is never used outside the Core.

The Core can only deal with resources having entries in its repository. However, the Core, as a policy decision, may delete repository entries any time it wishes. Hence, a small machine can continue running even if it is too small to hold all resource descriptions exported to it. The Client Utility makes provisions for external agents to provide resource descriptions that the Core has chosen not to keep. Hence, an application referring to a resource description that the Core has deleted can still continue running, although it may suffer a preformance penalty.

Each request from a task for a system resource is made via a message from the task to the Core. A message consists of an outbox envelope, which is read by the Core, and a payload, which is read by a resource proxy, a task which will process the request. The router component of the Core constructs an inbox envelope and forwards the rewritten message to the resource proxy which is responsible for interpreting the message payload. A task can grant access to a resource to the recipient by including it in the message envelope. Unless there is strong authorization on the resource, a corresponding entry will be put into the inbox envelope. In this way, the sender can give the receiver a place to send replies; it includes a resource naming the sender as resource proxy. The same mechanism allows the sender to pass parameters to the receiver even though they don't share names for the resources.

Since the payload is defined by a convention established between a task and a resource proxy, we say that each resource speaks a specific language. (A language in this context is simply an API plus Client Utility specific information.) Thus, a file resource proxy speaks the file language; a scheduler speaks the scheduler language; etc. There is also a Client Utility Core language used to manipulate Core data structures. The Core never looks into the message payload unless the message is in the Core language. The Core knows nothing about languages other than the fact that they are unique within a Core. Any task can register a resource that represents a language and give itself ownership rights. The Core will make sure that this resource can be uniquely identified.

The Client Utility separates the concepts of naming and permissions. Each task works within a name space that it is free to define. Each name in this name space is bound to zero or more Core repository handles and an optional attribute description. Permissions are defined by the set of keys held by the task. Thus, access to a resource is controlled both by associating a task specific name to a Core repository handle and by controlling which keys are transferred to the task.

Three kinds of errors can occur when a task attempts to manipulate a resource. If no name association exists in the task's name space for the specified name, the task receives a Does not exist error. If the name exists but can not be bound to a repository handle, the task receives a No resource with that name error. If the name association exists, but the task's permissions indicate that it shouldn't see the resource, a Does not exist or an Access denied error is returned depending on the reason for the denial. Of course, the proxy responsible for this resource can return a variety of errors, as well.

The messaging is straightforward. Consider an application that needs a service from a resource proxy, say to open a file. It sends a message to the Core. The Router examines the message envelope to find the named resources. For each, it asks the Name Manager to find the repository handle associated with the name in the application's designated name space. Next, the Core repository handler looks up the resource metadata in the repository, including permissions that correspond to keys on the application's designated key rings. If all has gone well, the Router constructs an inbox envelope which it forwards to the proxy for this resource along with the payload of the original message. Meanwhile, the Monitor has been logging what's been going on.

• 3.2.1. Introduction to Walk Through: How this tutorial is structured
• 3.2.2. Getting a Protection Domain: How a tasks initializes in the Client Utility
• 3.2.3. A Typical Request: The flow followed for every request
• 3.2.4. Structure of a Name Space: Giving a name space structure
• 3.2.5. Lookup Procedure: How the Core resolves the name association
• 3.2.6. Resource Discovery: Getting more resources into a protection domain
• 3.2.7. Advanced Access Control: Implementing enhanced security policies
• 3.2.8. Inheriting Resources: Making sure all needed resources are available
• 3.2.9. Positive and Negative Permissions: Controlling who can discover a resource
• 3.2.10.Bids: Declaring the cost of using a resource
• 3.2.11. Delegation: Getting help from another task
• 3.2.12. Resource Metadata: Core's view of a resource

A simple walk through of the features of the Client Utility architecture will make it easier to understand the details. We'll use the example of opening a file since it's familiar to most readers. However, any access of any resource of any type will be handled in the exact same way.

For pedagogical reasons, we're going to hide details at each step along the way. We'll introduce them as the proper context is established. Two facts are needed to get started.

1. Everything managed by the Core is a named resource.
2. Every named resource has an entry in the Core Repository that carries information about the resource needed by the Core.

In addition, every entry in the Core Repository has a handle which we call its Core Repository Handle (CRH). Note that repository handles are unique within a single Core. No coordination of repository handles is needed between Cores, whether on different machines or on the same machine. A Core never accepts a repository handle from the outside.

The Client Utility uses a mailbox metaphor to describe the way applications make requests to access resources. The actual implementation may use mailboxes or not. In one prototype implementation, the messages are actually constructed by a thread running in the Core based on data passed from the application.

A task makes a request to the Core by putting a message in the task's outbox. This message consists of an envelope, the outbox envelope and a payload. The outbox envelope is defined by the Client Utility protocol; the payload is a convention between the requester and supplier of the service. Because of this convention, we say that the message is in a specific language. The Core will never look at the payload of the message unless it is in one of the Core languages, languages used to manipulate resources owned by the Core, such as mailboxes.

Any task can run on a Client Utility machine as long as it can do its job with the default resources. Hence, a typical Java applet will work without interacting with the Core. However, any task that needs more resources can get them only from the Core.

Most operating systems have facilities that allow the Core to start all tasks. In this situation, the Core will establish a default protection domain for every task. For example, users will log on using a logon task that the Core has started with this default protection domain. Communications with the Core are via anonymous pipes that the Core created when it started the task. The Core uses the pipe it reads from as the task's unforgeable identity.

A two step process is needed on operating systems on which the Core can not start all tasks. The first thing the task needs to do is contact the Core by connecting to the Core's portmapper. The Core will respond by returning to the task a port to use to contact the thread acting as the task's client proxy. (Note that these ports need not be socket ports; they are merely a means to exchange messages.) The Core will also return a token for the task to present on each request. This token is an unguessable 64-bit number used as the unforgeable identity of the task for the task's lifetime. The Core will associate a protection domain with this token.

There appears to be a security hole here because there is a time interval between when the port is assigned and when the task first uses it. A malicious task could come along and steal the port at this time. However, nothing is gained in this attack because any task will be given the initial resources, malicious or not. Once the connection is established, the token is used to authenticate any future requests.

The differences between these two means of connecting to the Core is hidden in the library routines provided on different operating systems. The client always uses the API to connect to the Core, and the library routine uses the proper messaging scheme. In this way, a single application can run on Cores that implement different messaging layers.

The initial protection domain built by the Core will include exactly enough resources to allow the task to request more resources from the Core. The task will then begin by adding resources to its protection domain, either by making general requests or by authenticating itself and getting resources belonging to its account.

Let's look at a typical request, say to open a file, made by a task that has been running for a while. I call this file my.addresses, but I may not want anyone else to know the name. After all, my name might be the_boss_is_dumb. Hence, I want file system to see an alias, say the_boss_is_smart.

Back to reality. The task will construct a payload in a language known to the file system, say

open read address.book

When the message is delivered to the Core, it looks in the envelope to extract the task's identifier token and to see the name of the resource being accessed, my.addresses in this case. The task's identity, determined either by its communications channel or its token, is associated with a protection domain which, in turn, specifies a name space. The Core looks in this name space to find the repository handle associated with the task's name my.addresses. If no name association is found, an error is returned to the task stating that the resource doesn't exist.

If a name association is found, the Core looks in the Repository for the state information associated with this resource. One such piece of information is the resource proxy for the resource. This proxy is designated by a mailbox attached to a task that understands the language of the payload.

Another field in every repository entry contains resource specific data. This field has two parts, one private to the resource proxy and the other available to all tasks. The private data contains any data the resource proxy may need to deal with requests. It usually includes a set of permission fields made up of a lock and an associated permission. Each request contains a list of key rings, each of which holds a number of keys. The Core will match up the keys on this key ring with the locks in the permission field. Any permission associated with a lock that gets opened will be forwarded to the resource proxy. Note that the Core never looks at the resource specific data, it just passes it on to the resource proxy. In our example, the core might extract the strings read and write.

The public part of the resource specific data field contains any data that the task registering the resource feels tasks might need to know. For example, the public field might contain a digital certificate identifying the supplier of the resource. It might also contain a piece of code, like a Corba stub, that the task can use to invoke an operation on the resource.

The Core will now construct a message to be delivered to the resource proxy. This message consists of the unmodified payload and an inbox envelope. The Core will put a name for this resource in the inbox envelope. In addition, if the outbox envelope specified a label for the resource, say address.book, the Core will insert that value into the label field of the inbox envelope. This label can be used by the task and resource proxy to mutually identify the resource.

The inbox envelope will tell the resource proxy the resource specific data and the name for each of these resource. The association between these names and the fields in the payload is part of the language specification agreed on by the requester and resource proxy. The semantic content of the payload and the resource specific data is the business of the resource proxy, not the Core.

The resource proxy can now do its job, except for one thing. How does it know what file in the underlying system the requesting task is talking about? Fortunately, there is no problem if the repository entry is configured properly. The resource simply gets registered with this information encoded in the field containing resource specific data. In our example, this name could be /u/joey/addresses.

The resource proxy now knows that the request is to open file /u/joey/addresses and that the requester has the permissions associated with the strings read and write. It can now access the file system to open the file, but what does it do with the file handle? How does it get the handle back to the requester? It could ask the Core, but the Core doesn't keep any information about message-reply status. Instead, we use a different scheme.

Another field that can be included in the outbox envelope constructed by the requester includes a name association to be transferred to the message recipient. Except for being used to find a resource proxy for the message, this name field is identical to the primary one. It has the requester's name for the resource and an optional label.

In our example, the requester would specify a resource for which the requester is the resource proxy. By convention, the resource proxy knows that the second name field in its inbox envelope refers to a resource to be named to send a reply. The resource proxy can now put the file handle into the payload of a message and the outbox envelope can specify the name associated with the reply resource as the primary resource. The Core will then use the same procedure as for the original message to deliver the reply.

Since the Core decides what name goes into the recipient's name space, it can not be sure it won't specify a name already in the recipient's name space unless it searches the entire name space, too time consuming to do on every message. There is another problem. What if one task creates a file to be used by another task? Must one explicitly transfer a name for the resource to the other? The solution to both these dilemmas is the same - give the name space some structure.

MyNameSpace=(MyDefaultFrame,InboxFrame,FrameA,FrameB)

A name space consists of an ordered list of frames. Each frame contains the association between the task's name for a resource and a specification for the resource. Both name spaces and frames are named resources, so they can be manipulated in the same way as any other resource. Once a name association is put into a frame, the name is available to any task with permission to use the frame.

The sharing problem is solved by defining a frame that both tasks can use. This frame can be a global frame available to all tasks, which gives us the file visibility of conventional systems. A global frame is usually built during system start-up and put into the name space of every task. On the other hand, this frame could be shared by all tasks in a single login session. It is built as part of the login process and given to all tasks in the session. Finally, the frame can be constructed at the request of a task and shared by the usual means. Once both tasks share the frame, any name put into the frame is immediately available to both tasks.

We also use frames to avoid the name multiplicity problem. Each mailbox has a frame associated with it. When a message is delivered, the names of the resources being transferred are put into the frame attached to the mailbox. The recipient can use these names by including the frame in its name space, or it can copy the entries to another frame with the same names or different ones.

Frames are useful for handling other kinds of name multiplicities. If a task is collaborating with two users, say chuck and sally, it can decide which task's names take precedence. By putting sally's frame first in the name space, a task can assure that it will see a name association in chuck's frame only if it doesn't exist in sally's.

A task can construct many name spaces. An entry in each name field in the outbox envelope is used to tell the Core the name of the name space to use for that resource. Hence, a set of resources can be hidden for the purpose of a particular message by not including the frame containing their name associations in the name space.

Building name spaces can be quite onerous if the task has to list explicitly every frame to be included. In particular, we can envision getting a large set of frames from a name mapping service. Building a name space by listing all these frames is prone to error and makes it difficult for the provider to make changes. We solve this problem by having each frame specify an ordered list of child frames. A task can now build a name space by simply listing a modest number of frames and a set of traversal rules, say depth first, and stopping criteria, such as not including child frames from certain sources. Once built, the name space can be used many times.

All this is great, but where does the Core start the lookup? If everything is a named resource, where does the name recursion end? It turns out that this bootstrap is straightforward because each mailbox has an associated frame. We call the frame associated with a task's outbox its bootstrap frame. Some names are put into this frame when the task checks in. Among them are names for a mandatory key ring and a default name space. The task is free to change these names, but these resources will still be used to begin the name look-up. The mandatory key ring is presented to the Core on every request; the default name space is used whenever a name specification does not designate a name space.

When a message is delivered to the Core, it looks into the bootstrap frame associated with the mailbox. There it finds the RCHs for the mandatory key ring and default name space. This name space is used to find the list of frames that contain the name associations specified in the message envelope that don't designate a name space. The keys on the mandatory key ring are used to check the permissions to these resources. That's all there is to it. The Core can now search through the frames in the designated name spaces using the keys on the designated key rings.

We now see the only resource in the entire Client Utility that isn't accessed by specifying a name, the task's outbox. This resource can't have a name because the Core has no place to look for the name association. It is this feature that grounds the name lookup recursion.

So far we've acted as if the application's protection domain automatically contains names for all the resources the task will ever need. Clearly, this isn't the case. We need some way for a task to add name associations to its protection domain. One approach would be to ask another task to transfer the name association, which we allow, but we also allow a different approach.

Each entry in the repository has an optional set of attributes. A task can add name associations to its protection domain by asking the Core to give it associations for all the resources that have certain properties defined by their attributes. All resources that match the request get bound to a single name in the requester's name space.

Defining a single attribute vocabulary for all uses and all times is not a good idea, even if room is left open to extend it. Instead, we'd like to let the attributes, like the resource specific data and message payload, have a syntax and semantic content that the Core does not in general understand. We provide for this case by allowing any task to create a new attribute vocabulary that the Core will use when machine requests against resource attributes. These vocabularies are named resources and can have attributes of their own. The recursion is grounded by making a Core vocabulary available to all tasks.

A vocabulary consists of a set of rules. There are a set of name-value pairs. Associated with each name is a value type. For example, TITLE might be associated with a string while SIZE is associated with an integer. The vocabulary also contains a matching rule for each field. For example, a shoe size vocabulary might match a request for a 10C shoe with a 9D. It is also possible for a matching rule to denote other vocabularies that it can match. For example, a US shoe size vocabulary could match entries in a European shoe size vobabulary.

Since the Core will be using the vocabulary to match the attributes of resources registered in the Core repository, we can't let it run code provided by client tasks. Instead, we provide a toolkit consisting of basic data types, comparison operations, and logical combinations. A vocabulary specification language is part of the Client Utility specification.

There may be times where we want to do a look-up in a different vocabulary. In this case, we can provide an attribute vocabulary matching and/or translating service. The request takes the form of a message that is routed to the task doing the translation with a payload containing the request. The translator can now return a new look-up request in the new vocabulary.

It is important for security reasons that information not be leaked between the owner of the resource that defines its attributes and the requester. We don't want a malicious task to know all the attributes on a resource because the information can be used for an attack. We don't want a malicious provider to garner information on the task's resource requests. In addition, we want the requester to decide what constitutes a match to protect against an unethical supplier that gets its resource heavy usage by saying it matches every request. On the other hand, we want the resource owner to determine what constitutes a match in case one of the attributes specifies a password. The Core, the only component that sees both the attribute description in the repository and in the look-up request, does the matching. The matching rules require that both the matching rules of the look-up request and those of the resource specify that a match has occurred.

Attribute descriptions can be included in a name association along with repository handles. Hence, a name association can be

• Explicit: associated with one or more repository handles,
• Implicit: associated with an attribute description,
• Hybrid: associated with both repository handles and attributes,
• Unbound: associated with neither repository handles nor attributes,
• Partially bound: needs task to complete association.

What if we want some more advanced forms of security? For example, audit trails are one of the easiest and most critical steps in dealing with attackers. Also, you may have noticed that the Client Utility Core doesn't have the concept of authentication, not even something as simple as passwords.

We allow for these extensions by having a field in the repository entry that lists resources that act as authorizers. If the authorizer is just building an audit trail, the Core forwards a message to the authorizer when it puts a name association for the resource into the frame associated with the recipient's mailbox. We call this a notify authorizer. We can also designate the authorizer as being a grant authorizer. In this situation, a name association for the resource is not delivered. Instead, the recipient is told that delivery is pending. The partial association given the name does not allow the name to be used, but it does contain information on how to complete the association. Only a grant authorizer for such a resource can forward a name association for it to another task.

Authorizers let us implement some interesting functions. As noted, we can build an audit trail of what task was granted what name association when. We can also use a notify authorizer as an interface to a system monitoring tool. For example, if memory segments are treated as named resources, we can display the memory usage on a task by task basis by tracking the transfer of name associations to memory segments to the user tasks.

Here's a rather extreme possible use. Say that we make a time slice a named resource. When the scheduler swaps a task back in, it gives the task a time slice. If this task needs a service, say to do a message/reply with another task, it can transfer a name association for the time slice along with the message. Making the scheduler a notify authorizer allows it to know that the message recipient should be scheduled next. The reply can send back the name association for the time slice. When the time slice finally expires, the scheduler can create a new time slice and give it to some other task.

Grant authorizers also have many uses. If a resource is to be protected with an access control list, the grant authorizer can assure that the resource is only made available to tasks in the list. The requesting task's identity can be determined in many ways, one being authentication carried in the message payload. We also use a grant authorizer to handle password protected resources; the grant authorizer gets notified when the partial association for the resource is to be completed. The authorizer can then check the message payload for the proper password. Actually, the authorization can be arbitrarily complex, including challenge response sequences.

Grant authorizers can also be used to synchronize tasks in a parallel program. For example, the authorizer registers a barrier resource with a particular attribute description. When a task is ready to wait at the barrier, it asks the Core to deliver a name association for a resource with the designated attributes. Since this resource has a grant authorizer, the name association is delayed. The requester sends a request to complete the binding and waits for a reply. Once the final task has requested a name association for the barrier resource, the authorizer can transfer the name association in a reply to all waiting tasks. The tasks now know that they can proceed.

Some applications need a large number of resources to run. For example, a word processor might need a long list of font files for correct execution. The word processor might also use other applications for certain functions, such as graphics editing, that also need a large number of resources. It would be inconvenient to have to access all these resources individually. Instead, we can set up a field in the repository entry of a resource that lists the other resources that should be delivered to the message recipient. If this inheritance is recursive, the requester will end up with all the necessary resources just by requesting a single resource.

Inheriting a resource guarantees that the resource has a specific name in the frame containing the parent resource. Hence, the word processor not only knows that the task has a name for the resource, it knows the name of the resource. Hence, programmers can write applications with hard-wired names. Note that the inherited resource can be a name frame that contains the names, which makes it easier to deal with a large number of resources.

Our security was pretty tight until we got to resource discovery. After all, if there was no name for a resource in a name space, there was no way to access the resource. However, resource discovery opens a hole. How can we construct a set of attributes so that only authorized tasks can get a match? We could put a required field in the attributes that must be specified exactly. However, relying on secret information is dangerous (what if the secret gets exposed), so the Client Utility uses a different scheme.

Every entry in the repository has a visibility field that holds two collections of locks. One is the allow field, and the other is the deny field. Whenever a task attempts to access a resource, either for resource discovery or as part of a message envelope, these fields are checked. If the specified key rings have at least one key that opens a lock in the deny field, the Core behaves as if the name is not bound to this resource. Then, if the key ring does not have at least one key that opens a lock in the allow field, the Core acts as if the task does not have a name association for the resource. Denial takes precedence.

It is now a simple matter to keep unauthorized tasks from discovering resources that need protection. Simply put a lock in the allow field and control who gets the key. Any task that doesn't have this key never knows that the resource exists. There isn't even an attack based on guessing attributes.

Using the allow and deny fields in combination makes it simple to enforce compartmentalization. This form of security says that you can not see items from one compartment when you're in another. For example, you can't see the submarine blueprints while looking at the bomber blueprints. All that is needed is make the allow key for the submarine be the deny key for the bomber and vice versa. The mere fact of presenting the key needed to see one of them makes it impossible to see the other.

The allow and deny fields also give us a way to implement roles. A role is used when a single task has many different jobs. For example, sometimes an engineer is also a manager. When acting as an engineer, only engineer resources need be available; when acting as a manager, only manager resources are needed. While we can implement roles by having different key rings for each role, all the names for both roles would still be visible. However, it is possible to set up the allow fields of the engineer resources with one key and the allow fields of the manager resources with a different key. Now, when only the engineer key is presented, the manager names can not be accessed. In other words, we're dealing with just the engineer name space. On the very next message, the task can present just the manager key making all the engineer resources invisible.

The deny field has an important role in the security model. Recall that resource lookup begins by having the Core identify the task's mandatory key ring. This key ring is used on every resource lookup in addition to those specified in the message envelope. We can get very strong protection by putting a key on this key ring that opens a lock in the deny field of a resource we want to protect. By not giving the task a name for this key, we ensure that the key is presented on every access which, in turn, ensures that the task can never know that the protected resource exists.

There are times that one task needs to take on the abilities of another. For example, when a request involves a resource coming from the Core of another machine, the proxy on the machine that provides the resource must have the permissions of the original requester plus some of its own. We could have the requester transfer name associations for all of its resources, but the overhead would be large. Instead, we have the requester transfer a name association for one of its name spaces to the proxy on its machine. The proxy converts the exportable resources to resource descriptions which it sends to the proxy on the machine providing the resource. The receiving proxy registers these resources and executes the requested command in a name space containing the imported resources.

This scheme works quite well unless we want to be able to revoke the delegation, perhaps because the proxy isn't responding and we want to get the resource from another site. Revocation can work if the proxy trusts the requester enough to give it write permission to parts of its environment, but we can avoid this potential security problem.

When a requester wants to delegate authority to another task, it can set up a new frame containing a name space and a key ring. A name association for this frame is passed to the delegate as part of the request. The proxy can now use this name space and the key ring in making its own requests. The delegation can be revoked simply by unregistering the frame or removing one or more of the names in it.

We still have a problem. What if the delegate needs to use keys of its own for its requests? It could put these keys on the key ring supplied by the requester, but now the requester has access to the keys. The requester can include names for the keys on the key ring so the delegate can copy them to its own key ring, but now the access to the keys can't be revoked. The delegate can put these additional keys on its mandatory key ring since that key ring is implicitly used for all requests. However, this approach has too much overhead in moving keys on and off the mandatory key ring.

Our solution to this problem is to have each request include a list of key rings. The list can be empty, in which case only the mandatory key ring is searched. The delegate can now include the key ring specified by the requester as well as one or more of its own when making requests of the Core. Since any task can specify multiple key rings, the number of key ring manipulation requests to the Core will be reduced leading to more efficient operation.

Each resource managed by the Client Utility Core has an entry in the repository. If the resource managed by a Core is intended to be permanent across Core start-ups, such as a file, its repository entry is maintained in the persistent state of the Core. When the Core initializes, it first restores any existing entries fron its persistent store. Then, it creates a unique identifier, the Core Repository Handle (CRH), for each resource it manages. Resources supplied by other Cores also have repository entries. Some may be preserved across Core start-ups, but others may not be. Since the repository handle is arbitrary, it can be structured to make resource look-ups more efficient.

The preceding parts of this walk-through have introduced all the fields used by the Core to completely describe a resource. These fields are

• Language
• Resource proxy
• Resource specific data
• Attributes
• Security field
• Bid

The security field is made up of the

• Permissions
• Authorizers
• Visibility field

More detail is given in the detailed description of the Repository.

• 3.3.1. Introduction to Naming: Why names are local to a task
• 3.3.2. Name Spaces: Structure of a name space
• 3.3.3. Building a Name Space: How a name space is built
• 3.3.4. Name Association: What's in a name association
• 3.3.5. Name Multiplicity: Resolving name multiplicity
• 3.3.6. Name Visibility: Sharing a resource without sharing its name
• 3.3.7. Getting a Name Association: Adding name associationss for resources

The Client Utility is designed to work in a distributed environment in which a task can be started on a different Core from the one on which it was invoked. In addition, a task that is written to be mobile can be migrated from one Core to another. It is critical that the task be able to name the resources it needs in such an environment.

In the Client Utility, each task is free to assign any name it likes to any resource. This name is the only means the task has to refer to the resource. When a message containing this name is sent to the Core, the Name Manager looks in the task's name space to see what resource is associated to this name. If there is no name, an error is returned. Hence, there is no way for a task to tell the Core to do something with a resource that hasn't been put into its name space. Security is enhanced because there is no way to express an action against a resource a task is not allowed to see. You can't hurt what you can't name.

A task's name space consists of an ordered list of frames. Each frame contains a set of associations between the task's logical name for the resource and an association specification. The frames themselves are also named resources in a language owned by the Core which a task can connect into an ordering called a name space. Name spaces are also named resources in a language owned by the Core.

When a task requests an action on a resource it specifies its logical name for the resource. This logical name may optionally specify the logical name for a name space in which to resolve the name. If no name space is specified in the logical name, the name manager looks up the name in the task's default name space found in the task's bootstrap frame. It then searches the frames in the designated order looking for a name association. The name manager will stop when it finds the first association for the designated name. If this name is associated to more than one resource, the Core will invoke a mechanism for resolving name multiplicity.

Tasks can share names by agreeing on a convention, but the Client Utility provides a more flexible scheme. Since frames are named resources, one task can create a set of name associations in a named frame and give a name association for that frame to another task. That task can insert this frame into its name space and thereby share the name associations. The standard user naming scheme allows all tasks belonging to a given user to share a common set of names.

We can now see how these name spaces can be used. If Task A wants to create a new resource that only it can see, say a temporary file, it will put it into its Task frame. If it creates a resource that it wants to be seen by a sibling in the same session, say a shared memory segment, it puts the name into its Session frame, frame 1. This resource is now visible to both Task A and Task B. Those resources common to all tasks running on behalf of a given user appear in the User frame. This hierarchy can be extended to groups and global resources, but this extension is by convention only. Each task decides on its own name space.

Here we see that a task, say D, has created a new frame X, which has been put into task C's name space between its Task frame and its Session frame. Now, when C uses a name that is in frame X and its Session and/or User frames, the association in frame X will be used. More examples will be given when we walk through more scenarios.

A parent can control a child's name space by putting names for certain resources into the child's frames. In our example, task D might not want task C to access accidentally a particular resource in the User frame. One approach is to put the name appearing in the User frame into frame X but not associate the name with an association object. If task C doesn't have permission to change its name space, it won't be able to see the resource its parent wants to hide.

A name space is an ordered list of frames. It would be inconvenient to have to list all the frames in order to create a name space. Such a strategy would make it difficult to compose name spaces from components collected from others. We could compose name spaces from other name spaces, but, as we'll see, this approach is too static.

We make name spaces composable out of smaller components by having each frame specify an ordered list of frames as its children. A task can now construct a name space by specifying a list of frames and the traversal rules through its children. Traversal rules can include instructions like "depth first" or "don't look at children of FrameZ". Once defined, this name space can be used many times.

This approach lets someone provide name associations as a service using name associations provided by others as components. For example, Service A provides a set of name associations in a particular frame, call it FrameA; service B provides its associations in FrameB. Service X can now provide a unified set of associations by providing FrameX with children FrameA and FrameB. FrameX could contain any number of name associationss, but it need only contain name references to FrameA and FrameB. Any task that builds a name space that includes FrameX, either directly or as a child of another frame, will also see the name associations provided by Services A and B. Service X can later change the order of the child frames, add more, etc, without having to inform potential users of the internal structure and allowing different users to traverse the tree differently. Simply sharing name spaces would lose this last bit of flexibility.

What's in a name? Or more precisely, what's in a name association? The Client Utility supports 6 states for a name. It is an error to request an operation on a resource for which there is no name association.

1. Nonexistent: The name does not appear in the task's name space.
2. None: The name is not associated with any resource or description.
3. Implicit: The name is associated with one or more resource specifications but no CRHs.
4. Explicit: The name is associated with one or more CRHs but not a resource specification.
5. Hybrid: The name is associated with both one or more resource specifications and one or more CRHs.
6. Partial: Further action is needed to complete the name association.

An explicit name association is connected to one or more resouces by specifying an ordered list of repository handles. This type of association is used when the task wants to connect to a specific resource.

There are times when it makes sense to associate a name with a specification of the desired resource rather than to a specific resource. One such case is when the name is used to access a service provided by one of several different providers. As machines go down and back up, the application doesn't have to identify a specific resource on a specific machine. When the name is resolved, the Core will find any resources that match the specification.

The hybrid association connects the name to both a collection of repository handles and a resource description. Such an association allows the task to specify that the Core should first try to find a valid resource among those specified by repository handles. Should none be valid, the Core will then treat the name as if it were an implicit association. A hybrid association can also be used to tell the Core to update the collection of repository handles.

A primary resource must be bound to a single repository handle or a set of repository handles that all have the same resource proxy. If it isn't, the task will have to ask the Core to resolve the name before it can be used. Requests to resolve the name carry a flag telling the Core how to interpret the name association. Recognized options are

• Pick one of the CRHs if available, description otherwise,
• Pick one of the CRHs (ignore descriptiong),
• Look up using attributes (ignore CRHs).

A second option indicates whether the Core should update the repository handles with resources that match the descriptions when doing a look up.

There are times when a task needs to know that a single logical name refers to more than one resource. For example, the task may be trying to decide among a number of providers of a given service. We allow the task to see this multiplicity by associating a single logical name to more than one resource, both repository handles and resource descriptions.

Name multiplicity will be a common occurrence. For example, a user might well use the same name for a machine and the display attached to that machine. If the name is for a secondary resource, the resource proxy handling the message will have to resolve the name. However, the resource proxy can't be identified unless we know whether the payload is intended for the machine resource proxy or the display resource proxy.

It is the programmer's responsibility to make sure that the name association used as the primay resource in a request correctly identifies the desired resource or resources by asking the Core to produce a new name association specified by a designated arbitration policy. The allowed arbitrations are:

1. Use first resource,
2. Report an error if more than one resource,
3. Forward all to recipient,
4. Parameter field is a logic expression,
5. Parameter field is a task to be contacted.

The first choice is useful when a task wants to protect itself against accidental name multiplicity and makes the most sense for a explicitly bound name; the second, when it the name must be attached to a single resource. The third choice is used when the recipient is to decide what to do, as might be done with a mirrored file. In this case, it is an error if all the resources matched for the primary resource don't have the same resource proxy.

The last two choices allow the task to specify the selection via an algorithm. The logic expression option is used when the selection algorithm is very simple, such as making a random choice; the specification is built out of operations defined by the Client Utility. More general operations must be done by forwarding the request to a task, perhaps the sender itself, which will run code to make the choice. This method is used when we want multiple providers to bid for the work. In this case, the designated task gets a partial name association for the resource.

If a task has no name association for a resource in any of its frames, it can't access that resource in any way. (You can't hurt what you can't name.) However, unlike most security mechanisms, this one provides an additional benefit to the user. A utility with a very large number of machines has more resources than any individual user wants to see. (Imagine asking for a directory listing on a million machine utility.) The user wants to see the resources relevant to the task at hand and nothing more. The name scoping provided by the Client Utility naming scheme gives the user the desired degree of control.

An important question remains. How do two tasks that don't share a naming convention talk about a resource? The answer is to use the Core to provide the name translation.

When two tasks wish to take action on a specific resource, they use the message envelopes to make the connection between their names. The Name Manager looks in the sender's name space to find the resource's association. The router puts a name and the label field given by the sender into the receiver's inbox envelope. It also puts this name association into the name frame associated with the recipient's mailbox. The router then delivers the message to the recipient. In order to avoid the problem of a task being tricked into using a name association that was transferred from another task, the Core makes sure that the this frame is empty before delivering the message.

There are no naming conventions imposed by the Client Utility architecture, not even for tasks running on the same machine. How do tasks tell the Core what resources they want added to their protection domains? The answer is that it is done the same way you find a florist. Specify a set of attributes - close to my home, open at 6 PM, etc. - and look through a directory such as the Yellow Pages. The Core Repository serves the role of the Yellow Pages in the Client Utility.

The Core Repository is where a cache of the resource descriptions is held. Its internal structure is not part of the architecture. It must support attribute based look up, and we expect it to do fast look ups of resource metadata when given a repository handle.

Each entry in the Repository has a field containing a set of attributes. When a task asks the Core to add a resource to the task's name space, the task specifies the attributes. If a match is found, a name association for the requested resource will be added to the task's name space unless authorization is required or unless the name visibility restrictions are not met. In the first case, the name association will be partial until the authorizer does the transfer; in the latter, the lookup acts as if there was no attribute match. If more than one resource is identified that matches the request, the arbitration policy included with the request is used to decide with of these resources to bind to names for the requester.

Unlike most directory services, such as ORB Trader or LDAP, the Core Repository doesn't impose an attribute grammar. Instead, each request and each resource description include a designation of its grammar. The grammar is a named resource. Of course, there is a default grammar understood by the Core. If the request and description are in the same grammar, the grammar interpreter can decide if there is a match. If they are in different grammars, the request will fail to find a match unless it specifies a task that can translate the attributes to a common grammar.

• 3.4.1. Introduction to Permissions: Enforcing access rights without understanding them
• 3.4.2. Restricting Use of Names: Hiding names that exist in the name space
• 3.4.3. Authorization: Building audit trails and enforcing more strict access control
• 3.4.4. Key Management: Controlling access to keys
• 3.4.5. Security Level: Limiting which tasks can communicate

Having a name association for a particular resource is only part of the access control. The rest is what a task can do with the resource it has named. The Client Utility uses keys (not cryptographic keys) and locks to determine the permissions a given task has for a resource. Each message passed to the Core designates a set of keys, described by a set of key rings, that are used to determine the permissions.

Keys are named resources that are kept on key rings, which are also named resources. Keys are different from other resources in one important respect; mere possession of the key is sufficient to use the key. All other resources need an additional lookup to determine permissions. Of course, keys do have permissions for other operations such as unregistering, copying, etc.

Keys are used differently from other resources in that it often makes sense to put a key onto a task's key ring without giving the task a name for the key. This feature makes it possible to grant a task a set of permissions without letting the task forward them to another task. (You can't manipulate what you can't name.) Control of resources other than keys, including key rings, is managed with permissions.

Each resource in the repository has some resource specific data, including permissions, that may get passed to the resource proxy. This information has been inserted into the repository by the Core on behalf of the task that registered the resource. Some of this data is always passed; other pieces are passed only if the requester has a key that opens the lock associated with that permission. The Core interprets these parameters only if the resource is in a Core language.

Consider the entry for a file for a Core running on top of a conventional operating system. The name of the file in the underlying file system is a parameter that is always passed to the resource proxy for the file system. There will also be permissions for different kinds of access, each with a different lock. For example, the permission that the file system will interpret as granting read access will have one lock, while the permission for write access may have a different lock.

Once the Name Manager has determined the Core Repository Handle for the resource specified in the message, it forwards the request to the Permission Manager. This component looks at the repository entry for this resource. First, it validates that the visibility rules are enforced. Next, it finds permissions with locks that match keys on the designated key rings. The resource specific data, including all the permissions with matching keys, get passed to the resource proxy in the order they appear in the resource specific data field.

In the most general case, each resource can have an arbitrarily large amount of resource specific data. Fortunately, there will be very little defined most of the time. For example, to provide Unix or NT file access semantics we need the name in the underlying file system, at most 3 permissions (read, write, execute) for the world, 3 permissions for each group, and 3 permissions for each user, independent of the number of files in the system. More commonly we'll need even fewer permissions. For example, a Unix file with access code 664 (rw-rw-r--) needs only 2 permissions.

Permissions are split into 2 sets. One set contains permissions associated with the resource; these permissions are meaningful to the resource proxy. The other set contains permissions associated with the resource metadata in the repository; these permissions are meaningful to the repository manager. They include such things as permission to change various fields in the description.

Keys can be used to hide names when it is convenient. The security field of each resource description has a visibility field made up of a set of allow and deny locks. A name will appear to be undefined unless the request includes keys that open at least one lock in the allow field and opens no locks in the deny field. This behavior is in addition to the limitations imposed on lookup requests.

The allow and deny fields make it possible to control who discovers a resource. These visibility fields also make it possible for a task to control what resources are visible on a given request. For example, all resources related to the production version of a piece of software can have a particular lock in their allow fields. A test version can be run without this key which guarantees that none of the production resources are included by accident.

Another application is protecting critical resources, such as system configuration files. In this case, a particular key is put on the mandatory key ring of all general users. If this key opens a lock in the deny field of system critical resources, general users can never find out that the resource exists. If the general user does not have a name for this key, there is no way to remove it from the mandatory key ring.

Using both allow and deny fields makes it easy to implement compartments. If a lock appears in the allow field of one resource and the deny field of another, a task can never see the two in the same request. Either the allow key is absent, which hides the former resource, or it is present, which hides the latter.

There are times when we need other kinds of control. For example, we may need an audit trail to track which tasks have had access to certain resources. Very secure systems may also want to control granting of keys and/or individual resources, perhaps to enforce access control lists or to implement military Orange Book type security.

A field in the Repository entry for a resource designates authorizers. For each, there is a bit to indicate whether the authorizer is to be notified on the transfer of a name association, a notify authorizer, or if only the authorizer can transfer a name association for the resource, a grant authorizer. The former is used when all we want is an audit trail; the latter, when we want tighter control over resource distribution.

When a name association of a resource with one or more notify authorizers is about to be put into a message recipient's mailbox frame, either as the result of a look up or because the resource is named in an outbox envelope, the message is delivered to both the appropriate resource proxy and to the notify authorizers. The name association is inserted into the frame associated with the receiving mailbox.

If the resource has one or more grant authorizers, the message is delivered, but only a partial association is put into the mailbox's frame and this fact is noted in the inbox envelope. This partial association contains a point of contact to complete the name association. Until then, it is treated as if the name does not exists. It is the responsibility of the message recipient to complete the name association before specifying the resource in a message envelope.

The authorization mechanism is general enough to support some functions that are normally part of an OS kernel. For example, some resources are password protected, but the Core doesn't know anything about passwords. Instead, a language owner can list an authorizer that will look in the payload for a password and decide whether or not to complete a name association for the resource. (An alternative approach is to start a task that registers itself as the resource proxy for password resources and send explicit messages to it.) Notice that this scheme also allows us to use an arbitrary challenge/response protocol to authenticate the requester because the names of the sender's and receiver's mailboxes can be exchanged.

The authorization mechanism can also be used to implement some useful functions not found in conventional operating systems. One such function is inheritance of resources. Say an application, such as a word processor, needs the user to have access to a number of files. We don't want to require the user to add explicitly each and every resource the application needs. If the names don't depend on the state of the system, we can use the inheritance field of the resource description to give the application names for all the resources it needs. Sometimes, though, the resources to be inherited depend on recently created resources. In this case, we can point the user to a single resource, an inheritance resource. When a partial name association for that resource is to be completed to the task's name space, the