DIGITAL Storage Technical Director
Welcome to the winter 1997-98 issue of the DIGITAL Technical Journal. This issue
does not have a single theme; it contains a potpourri of papers on a wide range of
technical topics. This provides the foreword writer with a small gift and a not-so-small
The gift is the opportunity to tout the continuing fecundity of DIGITALs
engineering community. All the papers in this issue of the Journal come from
product development groups in DIGITAL, and all the technology described herein is directly
applicable to the problems of using computers in the real world. The papers themselves
cover a wide range of topics: designing storage buses and their infrastructure; building
IP routers that reduce network delays caused by link or router failure; sharing 3-D
graphical and audio data across networks of computers with different windowing systems;
and debugging programs written in languages that incorporate data parallelism.
The headache, of course, stems from this very diversity. Any attempt to derive some set
of common underlying principles other than "make better stuff" from this
collection is doomed to sophistry. And my technical background is too narrow to provide
any significant embellishment to any of the papers outside the domain of storage systems.
So, with apologies to the other authors, I am forced to restrict my comments to what I
know - the background and impact of Bill Hams work on advances in parallel SCSI
which are presented in his paper in this Journal.
Bill Hams paper not only describes a significant technical achievement; it
illustrates DIGITALs shift from engineering proprietary storage systems to
engineering open storage systems.
The SCSI bus was developed during the early 1980s as one of many attempts to
standardize the interface to storage devices. It succeeded beyond the expectations of its
developers, largely because it supported a device model that was abstract enough to be
extensible but inexpensive enough to be implemented in the technology of the time. For all
its advantages, however, SCSI suffered from poor engineering at the physical level. This
was a direct result of the way it was developed. The diverse corporate representatives
that defined SCSI did not have the time or money to specify and build custom bus
infrastructure components (transceivers, cables, terminators, etc.), so they used commonly
available parts. A lack of sophistication in specifying physical interface parameters
resulted in a specification that allowed too much component variation. As a result, it was
difficult to build reliable, multi-box systems using SCSI.
DIGITALs attitude towards SCSI during this period was to ignore it and hope it
would go away. We had designed our own proprietary Digital Storage Architecture (DSA),
which utilized an abstract and extensible device model and also incorporated many large
system features, including a robust physical interconnect. We controlled the design and
manufacture of all DSA components and could thus guarantee that they all met tight
architectural specifications. Moreover, DSA was a key enabling technology for VMS
Clusters, the individual DSA components were competitive with their counterparts from the
proprietary storage architectures of other large systems companies, our customers were
happy, and the storage business was profitable. We were feeling quite pleased with
ourselves - and we were profoundly ignorant of the power of a successful open market
standard, since one had never existed in the storage world.
During the latter half of the 1980s, SCSI grew steadily in popularity until it
dominated the workstation and small-server markets. These systems had at most a few disk
drives on them, and SCSIs signal integrity problems were manageable in that context.
They were not manageable in the larger and more demanding data center systems, and so SCSI
was not used there. The SCSI standards group was aware of the buss deficiencies,
however, and as the decade progressed, the group made amendments to the standard to
eliminate many of them. By the turn of the decade, several independent subsystem vendors
were selling subsystems utilizing SCSI devices as storage for large DIGITAL systems. These
subsystems did not, in general, have the features, performance, or robustness of our
subsystems, but they were significantly cheaper and improving all the time. By 1991, it
had become obvious to us that we would not be able to compete with these systems in the
long run. They were leveraging an entire industrys investment and talent and were
reaping the cost benefits of high-volume manufacturing; whereas we had to design and
manufacture (at relatively low volume) every component of every DSA system ourselves.
Our position was untenable. We had to change our strategy and embrace the bus that we
had so studiously ignored.
We designed a modular packaging architecture for SCSI devices (known commercially as
StorageWorks) and a set of storage array controllers that interfaced these devices to our
systems (and systems from other major vendors as well). We also became active participants
in the SCSI standards process. Where DIGITAL had previously sent one or two engineers to
SCSI standards meetings strictly to gather information, we started to send up to
half-a-dozen engineers to listen, learn, participate in debate, help with the grunt work
of the standards process, and make proposals to amend or extend the standard in directions
useful to us and our customers.
Our new modular packaging design allowed our customers to install and remove storage
devices themselves and to migrate storage devices between systems, even between systems
built by different system vendors. This modularity proved to be a very valuable feature to
our customers. However, it required us to build a physical infrastructure for the SCSI bus
that had the robustness needed by our large systems and that could accommodate a great
deal of variability in configuration, and to use a bus that was known to have residual
signal integrity problems in its physical interconnect. We were understandably worried
about this, worried enough to charter a small group of engineers as a SCSI Bus Technical
Office (SBTO) within the storage group, and to develop short-term configuration guidelines
for our packaging architecture and long-term technical proposals for the SCSI physical bus
architecture. Bill Ham has been the head of SBTO since its inception and has also been our
representative to the SCSI committee on all matters relating to the physical bus
In the summer of 1993, Bill completed a study of the signal integrity issues
surrounding parallel SCSI. His conclusions were startling. The SCSI standards committee
had, over the years, made enough improvements in the basic transmission line
characteristics of the SCSI bus that most of the remaining signal integrity problems were
due to the variations in component parameters allowed by the SCSI specification.
Exercising tighter control over component variation - through building selected components
or through purchase specifications with our suppliers - would not only produce excellent
signal integrity in our packaging but would allow the maximum clock rate of the bus to be
doubled while maintaining excellent signal integrity and backwards compatibility with
existing SCSI devices. Bills results also indicated that the maximum clock rate
could be increased even further, with more work.
This discovery came at a critical time in the evolution of the SCSI standard. Much of
the SCSI standard committees effort in the early part of the 1990s was being spent
in modifying the SCSI standard so that serial buses could carry the higher level SCSI bus
protocols. The committee had started this work under the assumption that parallel SCSI was
"out of gas" in performance, and the new serial bus variants would supplant it
by mid-decade. However, by 1993 not only was the definition and implementation of the
serial bus going slower than expected, but there were three independent and incompatible
serial bus proposals, each with unique useful features and unique drawbacks, each with a
cadre of supporters among the industry representatives. The market would ultimately choose
which serial buses would thrive; but it was highly unlikely that all three would thrive.
Storage vendors that made the wrong bus choice would suffer for it. Most galling to the
technophiles among us, the markets choice could not be predicted from the technical
merits of the contenders. If it could, wed all have Betamax VCRs in our homes today.
So, DIGITAL decided to have Bill present his results to the SCSI committee at its
November 1993 meeting and recommend that the committee extend the SCSI specification to
allow the bus to run at up to twice its old maximum clock rate if the components in the
physical interconnect met the tighter specifications. Our motive in doing this was purely
selfish: we were not ready to choose among the serial bus proposals, yet we would soon
need more performance than parallel SCSI could offer. A higher performance parallel SCSI
would allow us to improve our storage subsystem performance without having to stake our
fortunes on a potential Betamax.
Bills presentation at the SCSI committee meeting was met with enthusiastic
approval. It turned out - surprise! - that other system vendors were feeling as uneasy as
we were about the serial SCSI buses. The proposal, christened UltraSCSI, was adopted as an
extension to the parallel SCSI standard. Bill Ham and the SBTO then worked with component
vendors and the SCSI committee to develop the thinner cables, smaller connectors, and SCSI
expander circuits described in his paper, all with the aim of keeping parallel SCSI as a
desirable alternative to the serial SCSI buses. Today, four years after its committee
debut, UltraSCSI is solidly entrenched in the storage market. In fact, storage market
analysts are now projecting that the combined volume of devices on all serial SCSI buses
(yes, there are still three, but the market has already picked one, Fibre Channel, as the
winner) will not exceed parallel SCSI device volumes until early in the next century. And
the SCSI committee has finished extending the parallel SCSI specification to achieve a
second doubling of maximum bus clock and is in the midst of defining a third doubling.
Without hyperbole it can be said that the technology embodied in Bill Hams paper
has directly affected the course of the computer storage industry, and it continues to
affect positively DIGITALs position in that industry. Enjoy reading the paper and
those that follow it in this issue.