Hence, to understand the factors which influenced the design of TOPS-20, we have to look back to the late 1960s and the environment in which TENEX was created. Some of the ideas that went into TENEX, and the process which ultimately brought it into DEC, had roots going back to the early 1960's. In this section, we will trace these aspects of TOPS-20's history.
I was an undergraduate at MIT from 1961 to 1965. After receiving my degree in 1965, I went to work for BBN doing various system programming activities and, in particular, supporting the LISP system then in use for AI research. I was involved in the design of TENEX and implemented a number of its kernel modules from 1968 to 1972 and then came to DEC along with TENEX in 1973.
DEC bought TENEX in 1973, believing that it represented a good base for a new, state-of-the-art commercial system. However, as with other "overnight sensations", this was just the final step in a flirtation between DEC and BBN that had gone on over many years. In 1961, BBN was one of the first sites external to DEC to install and use a PDP-1 (DEC's first computer), and did some early experiments in timesharing using this machine and some locally designed modifications. This is generally recognized as the first practical timesharing system. During the subsequent years, many discussion took place between the two companies about the best way to design and build computers.
One such area of discussion was paging and memory management. BBN was pursuing a number of AI research projects in LISP, and these typically required a large amount of random access memory. Few computers of that day (and none that BBN could afford) had the size of memory that was required, so various alternative techniques had been sought to work around the limitation[2]. Paging and "virtual memory" was one in which there was a great deal of interest. There was a LISP system running on the PDP-1 at BBN, and I had converted it to use a software paging system that I had developed earlier at MIT.
This system used a high speed drum as backing store for main memory, and divided both main memory and the drum into fixed size units (pages) that could be copied back and forth as necessary. The LISP system itself used 18-bit pointers and was built as if there were an 18-bit (262 kword) memory on the machine. However, with software paging, each reference to the target of one of these pointers was changed to a subroutine call (or in-line sequence). The subroutine would take the high order few bits of the pointer as an index into an array (the page table) which would contain the actual current location of the page. If the page were then in main memory, the page number bits from the table would replace those which had been used as the index into the table, and the reference would be completed. If the desired page were not in main memory, a call would be made to a "page manager" routine to rearrange things as necessary. That meant selecting a page in main memory to be moved (copied) back to the drum, and then reading into this page the desired page from the drum. Finally, the table would be adjusted to reflect these changes, and the reference sequence begun again.
In actual use, this system was fairly effective. The reference patterns of the typical LISP programs that we were running were such that the system was not dominated by waits for drum IO. That is, most pointer references were to pages then in main memory, and only a small fraction were to pages which had to be read in from the disk*.
[* - Several studies, some by us at BBN, had been done to investigate memory reference behaviour and gather statistics from various programs[3].]
However, the actual number of references was sufficiently high that a great deal of time was spent in the software address translation sequence, and we realized that, ultimately, this translation must be done in hardware if a truly effective paged virtual memory system were to be built.
The PDP-6 had been announced in 1964 and was immediately the machine of choice for LISP programming. A single word holding two addresses (a CAR and CDR in LISP terms) seemed designed especially for LISP (this wasn't mere coincidence). The big (for the time) address space of 2_^18 words offered the promise of fast, efficient execution of large LISP programs. Although the basic instruction set didn't include a CONS instruction (the LISP primitive that builds lists), one early PDP-6 had a special modification installed to provide that operation. This was the machine that was installed at the AI Lab of Stanford University, headed by one of the inventors of LISP, John McCarthy.
The group at BBN had every intention of acquiring a PDP-6 to improve our LISP capabilities, but of course, we wanted it to have hardware paging so that we could carry over the paging techniques from our PDP-1 LISP. Several discussions were held between BBN and DEC, including Win Hindle and Alan Kotok on the DEC side. BBN was lobbying for paging to be included in a subsequent PDP-6 model, or possibly as an add-on, but these discussion came to an end when DEC announced that there would be no further PDP-6 models -- that is, that DEC was going out of the 36-bit business. This (1966) was the first of several such occasions.
Taking DEC at its word, BBN turned its attention to selecting another machine, hopefully one with an expanding future, that we could use to support our various research projects. The result of this selection process was the purchase of an SDS-940. SDS was Scientific Data Systems in El Segundo, California, later acquired by Xerox and known as XDS. The SDS-940 was a 24-bit word machine with a modest virtual address space, but it did have hardware paging capabilities. Also, SDS was touting as a follow-on, the Sigma-7 then under development, which would be larger, faster, and do all the things we could ever possibly want. If it did, we never found out because by the time it came out, the DEC KA-10 had been announced (or at least leaked), and we happily switched back to the architecture we really wanted.
It was also a factor that the Sigma-7 suffered a bit of schedule slippage. Yes, this actually happened despite firm assurances from SDS that no such thing was possible. SDS had even published several ads in trade magazines touting their forthcoming system as "the best timesharing system not yet available". Another memorable ad read "they said we must be crazy to publish a firm software development schedule, but here it is." Apparently, "they" were right because that firm schedule went the way of most "firm" software development schedules of the period.
The former ad also inspired someone at DEC to create a counter ad for TOPS-10 which read "announcing the best timesharing system very much available."
BBN did use the SDS-940 for a couple of years however, and we ran on it an operating system developed by a group at U.C. Berkeley. That was significant because a number of features in the Berkeley timesharing system were later modified and adopted into TENEX.
When BBN found out the DEC had secretly gone back into the 36-bit business (or at least was developing a new machine of that architecture), we once again lobbied for the inclusion of a hardware paging system. And once again, this was to no avail. One advancement in the KA-10 was the dual protection and relocation registers (the PDP-6 had only a single pair). This allowed programs to be divided into two segments: a reentrant, read-only portion, and a read-write data area. That, however, was as far as DEC was willing to go at that time in advancing the state of the art of operating system memory management support.
Another factor was DEC's firm intent to keep down the price of this new large machine. DEC was doing well with small machines (PDP-5 and PDP-8 in particular), but the PDP-6 had suffered numerous engineering and reliability problems, was hard to sell, and, some say, almost wrecked the company. The KA-10 had a design constraint that a minimal machine could be configured and sold for under $100,000. This configuration was limited to 16 kwords of main memory and had no solid-state registers for the 16 accumulators (all AC references went to main memory at considerable cost in speed). To support this, a "tiny" build of TOPS-10 was designed which would fit in 8 kwords, and various utilities were similarly squeezed: the 1K PIP, etc. Whether the $99,999 entry price was a good marketing ploy is hard to say. In any case, none of that configuration were ever sold.
Then too, paging and "virtual memory" were still rather new concepts at that time. Significant commercial systems had yet to adopt these techniques, and the idea of pretending to have more memory that you really had was viewed very skeptically in many quarters.
Undaunted by DEC's refusal to see the wisdom of our approach, BBN nonetheless planned the purchase of several KA10's and set about figuring out how to turn them into the system we really wanted. In addition to the 36-bit architecture we favored, the KA10 system had retained much of the hardware modularity of the PDP-6 including in particular the memory bus with independent memory units. This led us to conceive of the idea of putting a mapping device between the processor and the memories which would perform the address translation as we wanted it done.
Another decision we had to make was whether or not to attempt to modify TOPS-10 to support demand paging. The alternative was to build a new system from scratch, an ambitious undertaking even in those days. Obviously, we decided to build a new system -- the system which was later named TENEX. This decision was justified largely on the grounds that major surgery would be required to adopt TOPS-10 as we desired, and even then we probably wouldn't be able to solve all the problems. In retrospect, this view was probably justified since, although TOPS-10 development continued for nearly 20 years after we started TENEX, TOPS-10 was never modified to have all of the virtual memory features of TENEX/TOPS-20. TOPS-10 did ultimately support paging and virtual memory, but not the various other features.
Beyond that, there were a number of other features not related to paging that we wanted in an operating system. This further tilted the decision toward implementation of a new system. On the other hand, we had no desire or ability to implement new versions of the many language compilers and utilities then available under TOPS-10, so a method of running these images "as is" was needed. We decided it would be possible to emulate the operating system calls of TOPS-10 with a separate module which would translate the requests into equivalent native services. This is described in more detail later. The plan then, would be to implement a new system from the operating system interface level down plus particular major systems such as LISP that were key to our research work.
Virtual memory through paging was the fundamental need that led to the decision to build TENEX. Had TOPS-10 or another system been available which supported only that requirement, we probably would have used it and not undertaken the construction of a new kernel. The other desires we had for our system would not have been sufficient to warrant such an effort, and of course, we did not foresee all of the capabilities that it would eventually have.
TENEX development was motivated largely by the needs of various research programs at BBN, Artificial Intelligence in particular. These needs were shared by a number of other labs supported by the Advanced Research Projects Agency (ARPA) of the DOD, and in the period from 1970 to 72, a number of them installed TENEX and made it the base for their research programs. As noted before, a large virtual memory was foremost among these needs because of the popularity and size of the LISP programs for speech recognition, image processing, natural language processing, and other AI projects.
During this same period, the ARPA network was being developed and came on line. TENEX was one of the first systems to be connected to the ARPANET and to have OS support for the network as a general system capability. This further increased its popularity.
Like most other software systems, TENEX was an amalgam of ideas from a number of sources. Most of the features which came to be prized in TOPS-20 had roots in other systems. Some were taken largely unchanged, some were modified in ways that proved to be critical, and others served merely as the germ of an idea hardly recognizable in the final system. Among those that we can trace to one or more previous systems are: "escape recognition", virtual memory structure, process structure and its uses, and timesharing process scheduling techniques.
Three system most directly affected the design of TENEX -- the MULTICS system at MIT, the DEC TOPS-10 system, and the Berkeley timesharing system for the SDS 940 computer. MULTICS was the newest and most "state of the art" system of that time, and it incorporated the latest ideas in operating system structure. In fact, it was popular in some circles to say that, with the implementation of MULTICS, "the operating system problem had been solved". Several members of the MIT faculty and staff who had worked on MULTICS provided valuable review and comment on the emerging TENEX design.
Many of the paging concepts came from our own previous work, the PDP-1 LISP system in particular. Other ideas had recently appeared in the operating system literature, including the "working set" model of program behaviour by Peter J. Denning[4]. MULTICS had developed the concept of segmentation and of file-process mapping, that is, using the virtual address space as a "window" on data which permanently resides in a file.
The Berkeley 940 system had a multi-process structure, with processes that were relatively inexpensive to create. That in turn allowed such things as a system command language interpreter which ran in unprivileged mode and a debugger which did not share address space with the program under test and therefore was not subject to being destroyed by a runaway program.
These two concepts, virtual memory and multiple processes, were fundamental to the design of TENEX, and ultimately were key to the power and flexibility of TOPS-20. The two abstractions also worked well together. The concept of a process included a virtual address space of 262 kwords (18-bit addresses) -- the maximum then possible under the 36-bit instruction set design. Extended addressing was still many years away, and the possibility that more than 262k might ever be needed rarely occurred to anyone.
We merely wanted the full virtual address space to be available to every process, with no need for the program itself to be aware of demand paging or to "go virtual". We believed that mechanisms could be built into the paging hardware and operating system to track and determine process working sets, and it would do as good a job at managing memory efficiently as if the program explicitly provided information (which might be wrong). It might happen, of course, that a particular program had too large a working set to run efficiently in the available physical memory, but the system would still do the best it could and the program would make some progress at the cost of considerable drum thrashing. In any event, the implementation of virtual memory was to be transparent to user programs.
To support transparent demand-paged virtual memory, we had learned that we had to maintain history information about the behaviour of programs and the use of pages. This led to the design and inclusion of the "core status table" in the hardware paging interface. Other tables (i.e. page tables) would hold information that provided the virtual-to-physical address translation; the core status table was specified to hold information related to the dynamic use and movement of pages. For each physical page in use for paging, the core status table would record: (1) a time-stamp of the last reference, and (2) an identification of the processes that had referenced the page. The latter was included because we expected that there would often be many processes sharing use of a given page. The need for sharing was represented in several other aspects of the paging design as well[5].
The perceived need for sharing pages of memory and the segmentation ideas we had seen in Multics led to the design of the file-to-process mapping capabilities. In the paging hardware, this was supported by two types of page table entries, "shared" and "indirect", in addition to the "private" type that would contain a direct virtual-to-physical mapping. These allowed the physical address of a shared page to be kept in one place even though the page was used in many places, including in the address space of processes that were swapped out. The "indirect" pointer eventually allowed another capability that we didn't initially foresee -- the ability to make two or more process address spaces equivalent, not only with regard to the data contained therein, but with regard to the mapping as well*.
[* - This allows, for example, a Unix-type "fork" operation to create the child address space very cheaply whether or not it is later replaced by an EXEC.]
In software, we wanted to achieve some of the benefits of Multics "segmentation", but the PDP-10 address space wasn't large enough to use the Multics approach directly. Consequently, we decided to extend the idea of named memory down to the page level and to allow mapping to be done on a page-by-page basis. The result of this was the fundamental design of the TENEX/TOPS-20 virtual memory architecture, specifically:
We knew that most sharing would be of read-only data, especially of program text and constant data, but that any program might contain initialized variables which would eventually be written. Beyond that, almost any normally read-only data might be modified under some circumstances, e.g. insertion of a break point for debugging. In the LISP system, vast amounts of program were represented as LISP structures, and these structures could be edited and modified by the user.
Hence, it wasn't acceptable to enforce read-only references on the bulk of this storage, nor would it be efficient to copy it each time a process used or referenced it. That would defeat much of the sharing objective. The solution we devised was the "copy-on-write" page. This meant that the page would remain shared so long as only read or execute references were made to it, but if a process tried to write it, a copy would be made and the process map adjusted to contain the copy. This would happen transparently to the process. True read-only enforcement was available of course, and the actual file contents (e.g. of a system program executable file) were appropriately protected against modification, but the user could map any such pages copy-on-write, and a write reference would then merely result in a modified copy being used.
Multics may be said to have contributed more than just the ideas for virtual memory organization and other specific capabilities. During the design of TENEX, we invited some of the Multics designers and implementors to review our progress and decisions. As is often the case, we had fallen into the trap of trying to do too much in a number of areas and had produced some designs that were quite convoluted and complex. Several people from Multics beat us up on those occasions, saying "this is too complicated -- simplify it! Throw this out! Get rid of that!" We took much of that advice (and could probably have taken more), and I credit these reviews with making the core capabilities significantly easier to understand and program, as well as to implement and debug.
This was one of the earliest occasions when I really began to appreciate the fact that a complex, complicated design for some facility in a software or hardware product is usually not an indication of great skill and maturity on the part of the designer. Rather, it is typically evidence of lack of maturity, lack of insight, lack of understanding of the costs of complexity, and failure to see the problem in its larger context.
The time from when the first word of TENEX code was written to when the system was up and serving regular users was about six months -- remarkable even for that era. The system was far from complete at that point, but BBN internal users were making effective use of it.
One of the things that contributed to such speedy development was our concern for debugging support. DDT was the standard symbolic debugger on the PDP-10, and it had been evolved from predecessor versions on the PDP-6, PDP-1, and TX-0 (an experimental machine at MIT that preceded the PDP-1). DDT had a lot of power for its time, and even today remains the choice of some experienced users despite the availability of screen- and window-oriented debugging systems. DDT ran in both exec (kernel) mode and user mode under TOPS-10, and we supported both on TENEX as well.
In addition, we provided a way for DDT to run under timesharing while looking at the system code and data structures. This allowed several developers to be debugging system code at the same time, at least to a limited extent. Even breakpoints could be handled, provided that no other process would execute the code while the breakpoint was in place. In all modes, DDT had the full set of symbols available to it, so we were able to debug without constant referral to listings and typing of numeric values.
In user mode, DDT could be added to any program even if it were already running. Previous systems required that debugging be anticipated by linking the debugger with the program. In TENEX, DDT was merely mapped when needed at a high unused address in the address space well out of the way of most programs. An even more powerful version of DDT, called IDDT (for invisible) ran in a separate process from the program being debugged. Thus, it was impervious to memory corruption by the program under test, and it could intercept any and all illegal actions. Unfortunately, this version of DDT diverged from the mainstream DEC DDT, and IDDT was never brought up to date and made the normal debugger, even under TOPS-20. Nonetheless, the easy and extensive use of DDT surely made development more effective, both for user and system programmers.
Another area where TENEX, and later TOPS-20, excelled was in the user interface, described in detail below. A key reason that we were able to extend the user interface so significantly is that the code which implemented it ran an an ordinary user program. Hence, it wasn't constrained by size or limited by functions which could not be performed in kernel mode. Beyond that, debugging it was much simpler -- it didn't require stand-alone use of the machine as did kernel debugging. All of this allowed a great deal of experimentation and evolution based on tuning and feedback from real use and real users.
A piece of system design "philosophy" had emerged at BBN and among some of the ARPA research sites that was to have a large impact on the overall feel of TENEX and, ultimately, TOPS-20. At the time, we called this "human engineering" -- we wanted the system to be easy to learn and easy to use, and we wanted the system to take care of as many grungy details as possible so that the programmer or user did not have to deal with them. Beyond that, we were willing to spend real machine cycles and real (or at least virtual) memory to make this happen.
This philosophy led initially to the human interface features of the EXEC, including "escape recognition", the question-mark help facility, optional subcommands, and "noise" words. Few people now argue against the need to provide effective human interfaces, but at that time there were many detractors who felt that it was a waste of cycles to do such things as command recognition. These kinds of things, they said, would "slow the system down" and prevent "useful work" from getting done. Other contemporary systems used short, often one-letter, commands and command arguments, provided no on-line help, and did not give any response to the user other than the "answer" (if any) to the command that had been entered.
Many such systems fell into the "by experts, for experts" category. That is, they were built by experts and intended to be used by other experts. An "expert" would obviously not need frivolous features like noise words or command recognition. Experts would know, not only all the commands, but all the legal abbreviations as well. Experts may have read the manual once, but would always remember everything needed to interact with the system. So went the implicit assumptions about users -- either you were an expert, or you were an inferior being not really entitled to use the Computer anyway.
The TENEX team took a different view. It was clear that the power of computers was increasing every year and so one should expect the computer to do more and more interesting things. Most people would agree that a major purpose of computer automation is to relieve people of boring, repetitive tasks; we believed that purpose extended to computer system development as well, and that it was wrong to require a person to learn some new set of boring, arcane tasks in order to use the computer. The machine should adapt to man, not man to the machine.
The view was probably reinforced by the artificial intelligence research being done in the environment where TENEX was designed. In such areas as speech recognition, pattern recognition, and natural language comprehension, massive computation power was being applied to make computers interact in ways that would be more human. These were long-term efforts, but we wanted our computer systems to be more human-oriented in the sort term as well.
One of the ideas to come out of the AI research was that the computer should "do what I mean", even if I didn't say it quite right or maybe made a spelling error. Warren Teitelman implemented extensive capabilities in LISP to correct errors automatically or with minor user intervention, and this came to be called the DWIM package. With this kind of research effort going on, we were sensitive to the restrictions and obtuseness of most existing systems.
Hence, we took (now obvious) decisions like, (1) identifying users by alphanumeric name rather than number; (2) allowing file names to be long enough to represent meaningful (to the human) information; (3) typing or displaying positive confirmation of actions requested by the user; (4) allowing for a range of individual preferences among users in regard to style of command input and output.
We specifically looked for ways of interaction that would seem natural when judged in a human-to-human perspective. On that basis, many typical computer interface behaviours are quite unfriendly. Terse, cryptic, and/or abrupt responses; inflexible and obscure input syntax; use of words for meanings that are counter-intuitive -- these things are all judged as rude or worse if seen in a human-to-human interaction.
This desire to make the computer do the grunge work motivated design in areas other as well. Demand paging and virtual memory were seen as ways to reduce the burden and difficulty of writing large programs. With a large virtual memory, the programmer would not have to be concerned with segmenting and overlaying his program. Rather, he could leave memory management to the system and concentrate on the particulars of the task at hand. Other features such as process structure, system utility calls, and standardized file parsing were intended to ease the job of system programming by providing standard, powerful functions to the programmer.
One of the most favored features among TOPS-20 users, and one most identified with TOPS-20 itself, is "escape recognition". With this, the user can often get the system to, in effect, type most of a command or symbolic name. The feature is more easily used than described; nonetheless, a brief description follows to aid in understanding the development of it.
Typing the escape key says to the system, "if you know what I mean from what I've typed up to this point, type whatever comes next just as if I had typed it". What is displayed on the screen or typescript looks just as if the user typed it, but of course, the system types it much faster. For example, if the user types DIR and escape, the system will continue the line to make it read DIRECTORY.
TOPS-20 also accepts just the abbreviation DIR (without escape), and the expert user who wants to enter the command in abbreviated form can do so without delay. For the novice user, typing escape serves several purposes:
DIRECTORY (OF FILE)
This prompts the user that files are being dealt with in this command, and that a file may be given as the next input. In a command with several parameters, this kind of interaction may take place several times. It has been clearly shown in this and other environments that frequent interaction and feedback such as this is of great benefit in giving the user confidence that he is going down the right path and that the computer is not waiting to spring some terrible trap if he says something wrong. While it may take somewhat longer to enter a command this way than if it were entered by an expert using the shortest abbreviations, that cost is small compared to the penalty of entering a wrong command. A wrong command means at least that the time spent typing the command line has been wasted. If it results in some erroneous action (as opposed to no action) being taken, the cost may be much greater.
This is a key underlying reason that the TOPS-20 interface is perceived as friendly: it significantly reduces the number of large negative feedback events which occur to the user, and instead provides many more small but positive (i.e. successful) interactions. This positive reinforcement would be considered quite obvious if viewed in human-to-human interaction terms, but through most of the history of computers, we have ignored the need of the human user to have the computer be a positive and encouraging member of the dialog.
Typing escape is only a request. If your input so far is ambiguous, the system merely signals (with a bell or beep) and waits again for more input. Also, the escape recognition is available for symbolic names (e.g. files) as well as command verbs. This means that a user may use long, descriptive file names in order to help keep track of what the files contain, yet not have to type these long names on every reference. For example, if my directory contains:
BIG_PROGRAM_FILE_SOURCE VERY_LONG_MANUAL_TEXT
I need only type B or V to unambiguously identify one of those files. Typing
extra letters before the escape doesn't hurt, so I don't have to think about the
minimum abbreviation; I can type VER
Finally, if the user still isn't sure what input comes next in a command, he
types question-mark, and the system will provide a list of choices that are
legal at that point. The list includes specific keywords (e.g. FILE, DIRECTORY)
and generic descriptions (e.g. "input file") Most importantly, the question-mark
request does not destroy the previous command input. After the list of
alternatives is displayed, the partial command is redisplayed, and input
continues from the point just before where the user typed question mark.
As a result of this feature:
As menu-oriented interfaces have become more widely used, the advantage of
having all choices visible to the user has become obvious. The question-mark
help feature can, in retrospect, be seen as a "menu on demand" kind of approach,
and it was one that worked even on terminals too slow to support full menu-based
interfaces.
The Berkeley Timesharing system for the SDS-940 had an earlier form of
recognition. It didn't use escape however. On that system, recognition was
automatic; whenever the user had typed enough to unambiguously identify a
command or symbolic name, the system would spring to life and type the rest of
it. This made for the minimum keystrokes in the ideal case, but had one major,
and ultimately fatal, problem: if the user typed too much in one field, i.e.
more than the amount necessary for the system to recognize that field, the input
would go into the next field where it wasn't intended. For example, the system
would recognize COP as sufficient for COPY and supply the "Y". But if you typed
the whole verb, you would get:
* COPY Y
|
|- typed by the computer
Then, you would at least have to erase the extra "Y". If you didn't notice
what happened and continued to type the remainder of the command, what you had
intended as:
* COPY OLDFIL NEWFIL
would come out as
* COPY Y OLDFIL NEWFIL
This would at least produce an error, and in pathological cases could do
major damage. [In the foregoing example, note that the old file name winds up in
the new file parameter field.]
With experience, users would come to know the abbreviations for the reserved
words and so would rarely fall into the trap of this example. However, this
automatic recognition also worked on files in file directories, and there, the
minimum abbreviation could change from minute to minute as files were added and
deleted. Hence, even an experienced user would have to type carefully.
Interestingly, this system favored slow typists who benefited from having to
type the minimum amount of input, but it was a source of maddening frustration
to fast typists who would often outrun the system. And when the system became
loaded and response decreased, it was a disaster for everybody.
This experience convinced us that we didn't want to perpetuate automatic
recognition in that form in TENEX. We spent a lot of time thinking about ways
that the "overtyping" problem might be solved, including rules like "if there is
extra input and it matches the last keyword after the point of recognition,
apply it to that", etc., but ultimately could find no method that didn't leave
open a large class of time-dependent errors. We considered a mode which would
enable or disable automatic recognition, but decided that would not make the
feature any more usable.
Hence, we decided to make recognition happen only on explicit request, even
though an extra keystroke would be required. We selected the escape key for this
purpose.
In general, it became our objective to avoid modes. We believed that defining
"expert", "novice", etc. modes would be to ignore the fact that one person can
be both novice and expert at the same time, and may have preferences that don't
fit either definition. Even an experienced user is likely to be unfamiliar (and
therefore a "novice") with some parts of the system, some commands, etc., and
would want the additional prompting when working in those areas. Over time, we
observed that some very experienced users continue to use the prompting
extensively because of the positive feedback it provides.
The circumstances under which TENEX moved to DEC and became TOPS20 seem in
retrospect to have included a number of fortuitous events. As noted above, by
1972, TENEX had achieved a certain amount of popularity among researchers on the
ARPA Network. The reborn 36-bit machine, newly christened the DECsystem-10 was
enjoying reasonable success in a number of other markets as well. The prestige
of being the choice of leading edge researchers was worth advertising though, so
DEC ran an ad in a number of trade publications headlined "ARPA has a network of
Supercomputers" and pointing out what a large fraction of those were
DECsystem-10s. In fact, most of those were running TENEX. By April, 1972, there
were seven sites in addition to BBN running TENEX.
BBN had a modest business building the outboard paging hardware which, with
the technology of that day, required an entire tall 19-inch wide cabinet of
logic. DEC, meanwhile, had begun work on a successor machine to be known as the
KI10 (the "I" at least suggesting the integrated circuit technology that was to
be used). As early as June of 1970, meetings were held where BBN people
attempted to persuade DEC to include paging hardware similar to the design of
the BBN pager. Eventually, DEC decided to include paging in the KI10, but it was
based on a much simpler architecture. DEC engineers were not convinced that the
several forms of pointers (private, shared, indirect) and the core status table
would be worth the amount of hardware required to support them. Nonetheless,
they did choose the same page size, 512 words, which at least left open the door
to some sort of later accommodation.
When the KI10 came out, DEC was disappointed (to say the least) by the lack
of interest among the research community which had helped spark KA10 sales. The
problem was that the machine would not run TENEX. It was almost twice as fast as
the KA10, but the paging was different from what TENEX required. It was also
quite evident that it would not be practical to modify the structure of TENEX to
use the more limited KI10 paging. As a further irony, the version of TOPS-10
initially shipped with the KI10 used the paging hardware only to simulate the
protection and relocation hardware of the KA10 and realized no benefit from it.
During the summer of 1972, I had decided to look for new opportunities
outside of BBN. Not surprisingly, one company I talked to was DEC. In the course
of those discussions, I was asked about the possibility of putting TENEX on the
KI10. This was not a desire that was widespread within DEC in general or within
DEC software engineering in particular, but it was of great interest to Allan
Titcomb who was the marketing manager covering the scientific and research
markets. Titcomb wanted very much to sell some KI10's to the sites that were
running TENEX.
The outcome of this was that I went to work for DEC -- as a contractor. I
contracted with DEC for a fixed-time (3 months), fixed-price contract to make
TENEX run on the KI10[6]. Clearly, the DEC internal processes were not about to
create a real project around TENEX at this point, but a one-man 3-month contract
must have seemed an acceptably small risk.
I also received an offer to join DEC as a regular employee after the contract
was finished. This permanent job was not initially intended to have any
continuing involvement with TENEX however. In fact, it was to work in the
TOPS-10 group.
The contract with me for KI-TENEX was negotiated through DEC's software
engineering department, led at the time by Dave Stone whom I knew previously.
During the latter part of the contract discussions, Dave asked someone else in
to consult on the technical aspects of my proposed method for dealing with the
different paging hardware on the KI10. This turned out to be Peter Hurley, then
doing special software engineering projects as part of the marketing group
headed by Allan Titcomb, and it marks the first of Peter's long involvement with
TOPS-20.
So, as of the beginning of October, 1972, I took my leave from BBN and
settled into an office on 3-5 (building 3, floor 5) in DEC's original buildings
in Maynard. Being at the time still rather youthful in outlook, the idea of a
3-month, fixed-price, one-man contract to port a base operating system to a new
processor in a family didn't strike me as particularly scary.
As part of my send-off from BBN, a couple of co-workers who had previously
worked at DEC gave me some farewell presents that, I was assured, would prove
very useful at DEC: a flyswatter and a can of bug spray. DEC's facilities at the
time lacked some of the aseptic uniformity generally expected of hi-tech offices
and labs. Because of their age, history, and proximity to a mill pond and
stream, the buildings were well supplied with various insect life and spiders,
and my friends at BBN wanted to be sure I knew what I was getting into.
Ultimately, I spent only a little over a year working in the Maynard mill
buildings, but there were numerous occasions late in the evening when the
possibility of further concentration on code was nil and I found myself watching
at length as a particularly skillful spider spun a nearly perfect circular web
among the posts of my partition.
The technique I had come up with did not involve major alterations in the
paging structure of TENEX in order to deal with the simpler pager of the KI10.
Rather, the idea was to simulate most of the logic of the BBN pager in software
and use a KI10 page table only as a software cache or translation buffer for
current virtual-to-physical page mappings. Logically, this was much like the
design used in many later processors where the logic would be realized in
microcode and the storage in RAM.
Implementation of PDP-10 code to simulate the BBN pager was not a large or
difficult task and took probably less than half the time of the project. In
addition to paging, it was necessary to write drivers for the file and swapping
devices then being shipped by DEC, neither of which had been used at BBN.
Checkout of TENEX on the KI10 did, however, encounter one new and rather obscure
logic error in the paging hardware.
Well before the end of the three month contract period, TENEX was running
well enough on the KI10 to support building new versions of itself. During this
period, the system was up and running for several hours each day on a machine in
the basement of the mill, and a few curious employees came around to try it. One
such person was Dave Braithwaite, then in the -10 benchmark group, who brought
over various benchmarks and tests to try.
The contract nominally ended (successfully) when I delivered an official set
of tapes containing the TENEX sources and a bootable system at the contractually
appointed time. This was somewhat academic at that point however, since it was
not by any means the end of TENEX related activity at DEC.
During the time I was working on KI-TENEX, the KL10 processor was under
active development in hardware engineering. The "L" in KL10 was originally
intended to mean "low cost", since the KI10 was perceived as being somewhat
expensive. However, technology was providing opportunities to make a significant
jump in performance, and that ultimately was to be the salient feature of the
KL10. The product line managers were seeing opportunities to grow in the high
end, so the stage was set to consider some major changes in capabilities.
Around this same time, the "Super Foonly" project was in progress in the AI
lab at Stanford. The Foonly designers were projecting a factor of 10 speedup
over the KA10, and DEC engineers were invited out to review these plans. Certain
ideas, such as the E-box and M-box functional split, were instrumental in the
ultimate design of the KL10.
Stanford wasn't the only other place where a PDP-10 architecture machine was
being designed and built. The newly opened Xerox PARC (Palo Alto Research
Center) included a number of people from BBN and other sites familiar with
TENEX, and they were quite interested in pursuing their work on that system.
Since Xerox was then in the computer business, PARC couldn't quite manage to
just buy a PDP-10 from DEC. Consequently, they built their own. It was a
single-processor machine compatible with the KA10 and BBN paging and was
designed and built in a relatively short time. It ran TENEX for a number of
years.
Quite possibly, the final fortuitous event involved in the DEC decision to
take TENEX as the base for a DEC product happened not at DEC but at IBM. It was
during this period in the latter part of 1972 that IBM announced "virtual
memory" systems for the 360 family. While this was not entirely unexpected, it
provided a major shot of "legitimacy" for the concept of virtual memory in
computer system products. I (and other TENEX proponents) had been actively
promoting the virtual memory capabilities of TENEX, but it took the IBM
announcement to prove that such capabilities could be a significant factor in
large systems markets. This is rather ironic since the memory management
architectures in TENEX/TOPS-20 and the IBM systems are quite different.
Soon, discussions were being held around the idea that the KL10 would be, not
just a new CPU for the existing DECsystem-10 but the cornerstone of a new VM
product family including both hardware and software architectures. Hardware
architecture departures were to include internal main memory (as opposed to
external bus memory as on the KA and KI), the Massbus interconnect for tapes and
disks, and the use of a PDP 11/40 as a system console. Software would take a
comparable step with the capabilities of TENEX: virtual memory, process
structure, easy-to-use command language.
Although I was part of only a few of those discussions, I still look back
with amazement at the speed and confidence with which the decision was made to
undertake such a major departure. The key individuals included Allan Titcomb who
had initiated the KI-TENEX project, Fred Wilhelm, the engineering manager of the
KL10 project, Bill Kiesewetter, the marketing manager of the DECsystem-10
product line, and John Leng, the product line manager. We didn't convene task
forces or study committees or waffle on the idea for a year or two. We met,
considered the issues, and decided.
Thus, by the end of the three-month KI-TENEX contract, I had a second offer
from DEC: to join the KL10 group as project leader for a new operating system
for the KL10 based on TENEX. By the time I started work at DEC as an employee on
January 2, 1973, one additional engineer had been recruited to form the nucleus
of a development group: Peter Hurley. The two of us set up offices on the fifth
floor of Maynard mill building 5, in a group with the hardware engineers, the
product line marketing and management people, and vice president Win Hindle.
There were a number of areas in which we felt that work was necessary on
TENEX to make it suitable for release as a product. Some of these were areas in
which TOPS-10 had received sigificant work during the preceding years, and
others involved general maintainability, support of hardware reliability and
service, and overall robustness. TENEX did not, at the time, have such features
as batch operation or mountable disk structures.
We also planned to significantly increase the support for "TOPS-10
compatibility" -- the system's ability to support existing TOPS-10 programs and
media. This support took several forms: the operating system service call
interface, the command language, disk structure formats, and other external
interfaces. TENEX had implemented provisions for the OS call interface only, and
that was limited by the lack of knowledge at BBN about the details and
undocumented aspects of the TOPS-10 calls. Also, the call interface
compatibility was only as complete as needed to run the particular set of
TOPS-10 layered software that BBN needed at the time.
One of the reasons for pursuing TOPS-10 compatibility features was the
possibility of re-converging on one operating system for all 36-bit machines.
That was, and continued to be, a controversial idea however, and the destiny of
these features is largely the result of continuing indecision around how much
effort should be invested in tools to help users move from TOPS-10 to TOPS-20.
At times, it was clearly the intent that TOPS-20 would eventually become the
sole 36-bit operating system. It appeared to have the capabilities to support
any desired level of TOPS-10 compatibility, and it offered features that many
TOPS-10 customers had been requesting but which appeared unlikely ever to be
realizable in TOPS-10. Most of the time, however, TOPS-10 (and enhancements to
it) appeared to be needed in the short term to maintain some portion of the
36-bit business, and so the product line as a whole was never able to achieve
sufficient resolve to move the business back to one operating system.
In the early life of the -20, the product line made some overtures to -10
customers about migrating to the -20. Migration aids weren't in place, TOPS-20
didn't have all the needed capabilities, and the -10 customers weren't about to
accept the limits on configurations that applied to the -20. Hence, the
customers reacted quite negatively and scared the product line out of doing
anything that even suggested a migration strategy.
Ultimately, both operating systems continued to be developed and shipped
until the corporation made the much bigger decision to offer no new 36-bit
hardware products and to wind down all 36-bit software development. For these
and other reasons, replacing TOPS-10 was mostly not a high priority during the
life of the DECSYSTEM-20 and so capabilities like a TOPS-10 command language
were not seen as important. Subsequent ups and downs of interest in
re-converging on one operating system are described further below.
With the structure of TENEX, it had been possible to implement the TOPS-10
call interface compatibility in a relatively modular form. This piece of software
was called PA1050, after the DEC 10/50 system which was the top of the -10 line
at the time. To be able to support both the old (TOPS-10) and new (TENEX) OS
call interfaces, we had decided to use a native call mechanism completely
different from that used by TOPS-10. TOPS-10 used the Monitor UUOs, a group of
instructions which transferred control to the kernal through a transfer vector.
TENEX took a previously unassigned opcode (104 octal) and gave it the mnemonic
JSYS for Jump to SYStem. (The Berekely 940 system had used a BRS, Branch to
System.)
Within TENEX, execution of any of the TOPS-10 service calls went to a small
piece of kernel code that would invoke PA1050. On the first such call, it would
map the PA1050 image into a high area of the user virtual memory and transfer
control to it. On subsequent calls, it would merely effect the transfer. Hence,
only those processes that actually made TOPS-10 calls would get the
compatibility code, and each such process had its own incarnation. The virtual
memory implementation of TENEX meant that code or data could be put anywhere,
and there was no penalty for sparse use of the address space. At the time, it
was quite safe to usurp a range of high addresses in a TOPS-10 program, since to
use such addresses on a real (non-VM) TOPS-10 system would have required the
full 256 Kwords of memory or more to run. Later on, as TOPS-10 programs became
larger, we had trouble squeezing the program, PA1050, and DDT (which also was
mapped into a high range of addresses) all together.
PA1050, then, was simply unprivileged user-mode code. It interpreted the
arguments of the TOPS-10 call that had been intercepted and performed the
requested action, using JSYS calls where necessary. It maintained its own
database of TOPS-10 job state in local process memory.
The idea of alternate command languages was one of the attractive features of
TENEX. The TENEX command language (the "EXEC") was simply a user-mode program
that operated within a process and performed any system actions via defined JSYS
calls. Hence, a similar program to realize any other desired user interface
could be written and executed in the same manner by any user without impacting
others on the same system. The marketing folks had visions of offering
interfaces that were compatible with various competing systems in order to lure
customers away from those systems. We also envisioned writing a user interface
that would look like the TOPS-10 command language so as to facilitate users
moving over from TOPS-10.
Such a program was in fact written during the first year of development in
DEC by the third member of the new OS group, Arnold Miller. This interface
worked well enough to support execution of TOPS-10 batch files as well as most
normal user activity. However, it was never shipped as part of TOPS-20, largely
because users who came to TOPS-20 from TOPS-10 or any other system quickly began
to use and prefer the TOPS-20 command language. Even for users who were
reluctant to try something different, the advantages of the TOPS-20 Exec were
sufficiently apparent that few wanted to continue working in their old
environment. Similarly, no other interfaces to mimic competing systems were ever
written.
Interest in the TOPS-10 Exec was revived on one or two later occasions when
the possibility of migrating TOPS-10 users to TOPS-20 was considered, but such
thoughts never lasted long enough to result in a product.
One other compatibility feature developed during early TOPS-20 development
but never shipped was TOPS-10 disk structure I/O. This allowed a TOPS-10 disk
structure to be mounted on a TOPS-20 system and be read and written by a program
running in TOPS-10 compatibility mode. This was achieved by taking a large part
of the actual file system code from TOPS-10 and encapsulating it with an
interface to PA1050 on one side and direct block-level IO on the other. Again,
this proved to be a feature that was used briefly, if at all, and so never
received much additional effort. When the next version of the TOPS-10 file
system code was released, it proved more difficult to encapsulate and run in
this way, and the capability was abandoned even for internal use.
It's time to discuss the history of names of the operating system that came
to be known as TOPS-20. As part of undertaking this new operating system
development, DEC purchased the commercial rights to TENEX from BBN early in
1973. However, with a major marketing thrust anticipated for the new KL10-based
system, a new name for the operating system would be necessary. Also, the
development group in DEC intended to be independent of the group at BBN which
was continuing to develop new versions of TENEX for the KA10 ARPA users. Within
the first few months, various names were considered. Finally, the KL10
engineering manager, Fred Wilhelm, declared that the new operating system would
be called VIROS. It seemed perfect for the time: it suggested "virtual", which
was to be a key feature of the new system, and it had a strong, virile sound.
Unfortunately, John Leng (and others) tended to pronounce it as "virus" and so
it probably wouldn't have survived even if other factors hadn't pushed it aside.
Later on during that first year, it was decided to change the name to confuse
the competition. This was part of a policy dictated from "on high" to keep the
marketplace from knowing too much about unannounced developments. We knew that
preventing all leaks was impossible, so the idea was to confuse the world with
conflicting information. The new name chosen for the operating system then was
SNARK, from a Lewis Carol story about the hunting of the SNARK, where "they
pursued it with forks and hope". (We had lots of both.) That name almost had a
short lifetime, since it was further declared that internal code names would be
changed every six months to keep the enemy guessing. When time came for the
first six month change, the group resisted the hassle of yet another perturbation
but eventually yielded and put forth KRANS (SNARK spelled
backwards) as the new name. Unfortunately, it was soon discovered that this was
a word that meant "burial vault" in Swedish. The name was quickly changed back
to SNARK, and even our management then declared that we would have nothing
further to do with the semi-annual name change folly.
Hence, the name remained SNARK until it was time to announce the product. The
various interim names didn't vanish completely however; they were all used for
other internal purposes, mostly as names for disk volumes in the TOPS-20 group.
In fact, the SNARK pack (pronounced with a Boston accent as "snak pack")
remained the home of the TOPS-20 sources for the entire life of the system.
SNARK and KRANS are currently nodes on the DEC internal network but are not in
the 36-bit engineering group.
As time for product announcement approached, very high-level consideration
was being given to the product name. Mere engineers were not a party to this
process, so we waited with anticipation for the new and final system name to be
handed down. Would they stay with SNARK (unlikely)? Go back to VIROS ("V" names
were getting more popular all the time)? Or come up with something new and
dynamic. The result was, to put it mildly, underwhelming. Although TOPS-20 as a
name became just a fact of life over the years, we originally saw it as about
the least exciting, least imaginative name one could possibly have chosen for
such a neat, new system. It was just the same as the old system, TOPS-10, except
for one digit. About as creative as "Wednesday Night at the Movies" for a new TV
show of the same era. But that was the story: the new product would be the
"DECSYSTEM-20" (somebody forgot to lower-case the "system" as in DECsystem-10)
and the operating system would be TOPS-20.
So, although the name TOPS-20 wasn't chosen until just before the first
system ship, I have used TOPS-20 to refer to the operating system for the entire
time it was under development at DEC, that is, from January 1973 onward.
The TOPS-20 group grew to four people during its first year. In addition to
myself, Peter Hurley, and Arnold Miller mentioned previously, Len Bosack joined
the group within the first few months. Tom Hastings was also involved, serving
briefly as the supervisor. Although it started as part of KL10 hardware
engineering, this new group made the transition to its more logical place in
software engineering after the first year.
Several others served as supervisor of the group between its formation and
first ship. On several occasions, the group hired its new supervisor. To save
time, an interview would be set up with several members of the group at once who
would fire questions at the hapless candidate. If the candidate couldn't survive
this, he clearly wouldn't last in the day-to-day functioning of the group.
Managers weren't the only candidates who were given practical scrutiny. Judy
Hall, the fifth member hired into the group, and others who joined during that
period, were typically asked to bring samples of code they had previously
written when they came for interviews.
The DECSYSTEM-20 (and TOPS-20) was first shipped in January, 1976, just three
years after DEC internal development began. This was, of course, longer than we
had anticipated. Both the hardware and the software schedules were extended, and
a few other changes in plans occurred. As mentioned above, the KL10 had already
departed somewhat from its original conception as a "low-cost" KI10 replacement
by the time TOPS-20 development started. As TOPS-20 development proceeded, we
provided a couple more "1-plusses" to the machine.
One of these was extended addressing. TOPS-20 didn't force this as an issue,
but it seemed to provide the opportunity to eventually take advantage of it. The
possibility of extending the address space of the PDP-10 architecture was first
suggested to me by Tom Hastings during the 3-month period when I was working on
KI-TENEX. At that time, it seemed a pretty radical idea that 256 Kwords of
address space might not be enough for most applications, but the PDP-11 was
already running into trouble in that regard and it seemed prudent to think about
it for the -10 while we were building this new processor.
There were two primary problems to be solved in extending the address space
of the PDP-10. First, a way would be needed to generate effective addresses of
greater than 18 bits; second, a means would be needed to map this larger virtual
address space. For the hardware designers, the second problem appeared more
formidable. They planned to use a table-lookup RAM to hold virtual-to-physical
address translations, and a larger address space appeared to require a
prohibitively large translation RAM. The KI10 had used an associative memory for
this purpose, but the integrated circuits available in the technology of the
KL10 didn't provide a way to make a practical associative memory.
As it happened, the solution to this problem resulted from a bit of synergy.
Alan Kotok had been looking for a way to make the paging RAM smaller even than
that which would appear to be required for the basic PDP-10 address space. With
512 pages in each of user and exec mode address spaces, 1024 entries would be
necessary for a direct mapping scheme. Alan was thinking about using the same
512 entries for both user and exec by storing the user-mode bit in the entry and
comparing it on a lookup. If the comparison failed, the entry would be
considered invalid and would be reloaded from memory. Thus, the same entry would
be used for a given user and exec mode address and would work as well as a
full-size table provided there were not too frequent references to the same user
and exec addresses.
While this was going on, I realized that there was nothing unique about the
user-mode bit in this scheme, and that it could be extended to an arbitrarily
large number of address bits. That is, a given number of bits for translation
could be divided into two parts: one part to index into the RAM, and the other
to be stored and compared. For purposes of extended addressing, that meant we
could have a number of spaces each as large as the original 256 Kwords all
mapped into the same translation RAM. The smaller the RAM, the greater the
possibility of conflict, but that could be somewhat mitigated by "hashing" the
address bits so that the different address spaces were offset differently in the
RAM.
As for modifications to the basic instruction set, we initially decided on an
extension of the address space from 18 to 23 bits. This number was chosen
because it was the number of bits available by combining the Y, I, and X fields
(address, indirect, and index) of the PDP-10 instruction. We were fortunate in
that the PDP-10 instruction set had a very clean and consistent design -- every
instruction computed its operand address in the same way. If this algorithm
could be modified to yield a larger address, the problem would be solved for
essentially all instructions. The PC could easily be made as large as necessary
in hardware, and the various word formats in which it appeared generally had the
additional 5 bits available.
All that notwithstanding, any modification to the instruction set to generate
larger addresses would be incompatible with existing code to some extent. To
avoid introducing such incompatibilities, the original design had a special way
of executing an instruction when an extended operand address was desired. An
instruction executed normally would generate a compatible 18-bit address, but
the same instruction executed as the operand of a "special execute" instruction
(SXCT) could generate a larger address. It did this by using additional bits of
either an indirect word or index specified in the instruction.
As it turned out, our initial extended addressing design was too
conservative. Before the first DECSYSTEM-20 even shipped, we had occasion to
revisit and greatly modify the design of extended addressing in connection with
the UNICORN project described later.
Another perturbation to the KL10 design was in the area of paging. Because of
the success of running TENEX on the KI10 using a software simulation of the
TENEX paging design, it was initially assumed that the same could be done on the
KL10. However, we knew that the software simulation was consuming some amount of
processor time because the speedup of TENEX programs moving from the KA10 to the
KI10 was typically not as much as expected. Some studies using PC-sampling
techniques revealed that the paging simulation code was often using over 30% of
the processor under a typical timesharing load.
This brought up yet again the thorny issue of putting the complex TENEX
paging algorithms into hardware -- the thing that had been resisted for both the
KA10 and KI10. Devoting that much hardware to paging still seemed like a bad
idea to the hardware engineers, but fortunately there was another possibility for
the KL10: microcode. Just as the KI-TENEX implementation had shown that the page
refill algorithm could be executed in software with the result handed to the
hardware, so could it be executed by microcode. The basic instruction set was to
be implemented in microcode on the KL10 (the first PDP-10 architecture processor
so designed), so, the thinking went, we could add "just a little more microcode"
for the paging algorithm.
Some changes were necessary to the hardware design of the KL10 as it existed
at that point in order to allow the microcode to get control when an attempted
address translation failed, compute the translation and load it into the
translation buffer, and continue with the instruction then in progress. That was
taken care of with little additional hardware, and it seemed we had finally
achieve full support of the paging architecture. With a bit of myopia typical of
that time, we figured we had solved all our paging performance problems since
anything done in microcode had to be incredibly fast. ("Micro madness" as Len
Bosack often called it.)
Thus the KL10 had both extended addressing and TOPS-20 paging capabilities
when it first shipped. It is just possible that those enhancements did extend
the development schedule somewhat, although we argued at the time each of these
was being considered that the additional work was insignificant in the overall
picture. I'm not quite sure how we finally got these things approved. I recall a
rather angry Win Hindle practically throwing us out of his office the first time
we came in to tell him about adding extended addressing to the machine.
While we were bothering the hardware group with changes around paging and
extended addressing, we were busy with quite a number of software projects as
well. Typical of projects to make the system more robust in commercial
environments was the one to modify the system's handling of disk writes and disk
allocation tables. In TENEX and TOPS-20, essentially all disk IO is initiated
via paging. File pages are mapped, whether into user or exec address space, and
read or written with ordinary instructions. In TENEX as it came to DEC, all file
pages were treated equally, including those in directories, page tables, and
disk allocation tables. This worked just fine providing the system never
crashed. If the system were taken down intentionally, its last act would be to
write all modified pages out to the disk and thus make the disk consistent.
If the system crashed however (as occasionally did happen), it could easily
be that an inconsistent set of pages had been written to the disk. A file
directory might be pointing to a file for which the page table had not been
written, or pages might be in use for which the allocation table had not been
updated. To take care of this, the practice on TENEX had been to run a disk
consistency check program called CHECKDSK. In particular, this would mark the
allocation table for any pages found in use so as to prevent multiple
assignment. This had been satisfactory at first, but with the increase in the
amount of of file storage, it had begun to take an objectionably long time. The
solution was to force writing of the allocation tables, page tables, and
directories at certain points and in a certain order using techniques now well
understood. It's interesting to note that Unix had similar problems in its early
versions, and the same techniques have had to be applied in more recent
versions.
Another interesting step was the development of the COMND JSYS. This system
call provides a set of services to implement the TOPS-20 style of command
interface (escape recognition, question-mark help, etc. as discussed
previously). It was not, however, part of TENEX as it came from BBN. Rather,
those interface characteristics were individually coded in each program that
supported them, and in fact, only the EXEC supported them in general. Other
programs typically supported only recognition of file names, since that was
provided by the GTJFN system call. Even basic command line editing (rubout to
delete a character, control-U to erase the line, etc.) was not centralized and
was provided more or less or not at all by various programs.
The need for centralized command line editing was addressed earlier in the
TOPS-20 development by the addition of the TEXTI JSYS (and related subsets RDTXT
and RDLINE). The increasing use of video terminals had made the ad hoc
provisions of the various utility programs particularly glaring. With TEXTI, we
undertook to both centralize the line editing functions and to make them
sensitive to the type of terminal being used. At that time, taking advantage of
the video terminal's ability to erase a character or line was still impressive.
Some systems had been able to add at least a minimum of video terminal line
editing support because they only supported line input, and the line buffering
was done centrally. TENEX had been designed with a maximum of terminal
flexibility however and so had no general way of editing input. The TEXTI JSYS
provided many options so that it could be used as well by programs that wanted
more or less than line-at-a-time input.
Implementing a centralized command input library was not, however, an item
that appeared on our schedules. Within the last year before the system shipped,
we had identified and were working on a few utility programs that needed
specific improvements. One of these was DUMPER (the disk-to-tape backup and
restore utility), and it had a particularly bad command interface. I had the
task of improving DUMPER's user interface to meet some minimal level of
acceptability, but I had been procrastinating doing anything about it because it
would be a rather routine and boring task. In the meantime, I had started
thinking about how this style of command interface could be centralized into a
set of functions and tables.
Because of the highly interactive nature of the interface, partial execution
of the program itself is necessary during input of the command. For example, if
file name recognition is attempted, the defaults and any other pertinent context
must be available so that failure or success can be determined. However, the
program must not take any irrevocable action until the command is confirmed
(typically with a carriage return) so that the user can erase all or part of it
with the line editing keys. Even the original TENEX Exec didn't get this quite
right. The user could erase the entire command line with control-U, but could
not delete backward more than the current field. In other words, the Exec knew
how to flush all state for the current line and start over, and it knew how to
delete characters from the current field before it was completed, but it didn't
know how to flush part of the state and backup into a field already completed.
My idea for solving the backup problem was not to backup at all but rather
save the command text and then rescan it from the beginning with the unwanted
part removed. With that idea and a list of functions to handle the various kinds
of fields that would occur in a command line, I started writing a library-type
facility instead of just point enhancements to DUMPER. Of course, DUMPER was
intended to be, and did become, the first program to use new facility, but by
the time of TOPS-20's first release, these command parsing routines had been
integrated into the monitor as the COMND JSYS. Most utilities had not been
modified to use it however, and the Exec still had its original parsing code.
Correcting all this was done in later releases.
Ultimately, most of the TOPS20 development done for the first release went
toward supporting the new hardware architectures and devices. A complete new
mass storage IO structure was necessary for the Massbus disks and tapes, and an
equally large change was necessary to handle communications IO through the
PDP-11 front-end rather than via direct line scanners.
Almost all system code for the 36-bit architecture machines had been written
in MACRO since the first boot routine of the PDP-6. Because of its regular
structure and powerful set of operations, 36-bit assembly language was
reasonably pleasant to write in, and so less pressure existed for a higher level
language than on most other machines. BLISS was gaining some adherents in some
parts of DEC, but its various peculiarities (e.g. the "dot" notation and use of
underscore for the assignment operator in the early -10 version) generated major
resistance elsewhere. Hence, new TOPS-20 code was written in MACRO, as TENEX had
been.
During the development of release 1 and continuing thereafter, various
features were added to the MACRO programming conventions used in TOPS-20. These
were mostly implemented by macros, and they gave certain higher-level language
capabilities to MACRO. The first of these involved mechanisms for representing
data structures in one place (i.e. a "header" file) such that the representation
would cause appropriate code to be generated for references. This started out as
a simple macro to select "left half" or "right half" of a 36-bit word (instead
of an explicit HLxx or HRxx). Next came macros to select the correct bit test
instruction (TLxx, TRxx, TDxx) for the mask at hand. Eventually, the macros were
capable of defining and representing complex multi-word record-type structures
with fields of various sizes.
Secondly, macros were used to provide procedure-call and automatic (stack)
storage using named variables. A procedure entry point could be declared using
symbolic names for the parameters, and these names would expand to the
appropriate stack reference when used in an instruction. Similarly, local stack
storage could be defined symbolically and local registers saved automatically.
These conventions greatly reduced the occurrence of explicit stack PUSH and POP
instructions and the bugs that often resulted from them.
Finally, macros were implemented to provide control structures without
explicit use of labels. Semantically, these resembled typical language IF/THEN
or IF/THEN/ELSE constructs, and arbitrary nesting was permitted. As a result, a
typical page of TOPS-20 code using these conventions often contained only one
label -- the procedure entry point.
All of this made for better programming, but it did not, of course, remove
dependence on the PDP-10 instruction set. Portability was not an objective we
thought to be concerned with until much too late.
The consequences of the ambitious set of new architectures were being felt by
the hardware engineers as well as by us in software. As schedules pushed out, it
wasn't too long before someone came up with the idea of shipping just the KL10
as a processor upgrade for existing DEC-10 systems. Dropping in new processors
was, after all, well established, and it looked like a source of significant
revenue well before the full DECSYSTEM-20 would be ready. As a result of this,
the "1080" project and product were born.
The 1080 was the KL10 processor with adapters to connect to the KI10 external
memories and IO bus. Neither external memory nor the IO bus were intended to be
part of the new architecture, but in the interest of generating some revenue
from the rather expensive KL10 development, the adapters were expedited. Since
the 1080 was to be sold to existing customers as upgrades to existing systems,
it would be delivered only with the existing operating system, TOPS-10. It
didn't take much work for TOPS-10 to run on the KL10; the machine even had a
mode which emulated the KI10 paging rather than the new microcode-based paging
designed for TOPS-20. So, after all the original excitement around the marriage
of TOPS-20 and the KL10 processor, it was TOPS-10 that shipped on the first
KL10s that were sold.
The first 1080 ship in August, 1975 was the occasion for a sizable
celebration in front of the Marlboro plant. The entire 36-bit product line had
moved to the Marlboro plant in early 1974, and this meant that we had
manufacturing as well as engineering, marketing, and all other groups in one
location. Later on, this self-contained nature of the 36-bit line, along with
the fact that we continued to build an architecture more and more out of the
mainstream of the company, led some to refer to the 10/20 product line as "The
Marlboro Computer Company".
For the 1080 ship, most of the Marlboro groups gathered for speeches in front
of a large Digital truck bound for the first 1080 customer. Actually the truck
was empty at the time due to a few last minute snags, but substantively the
product was ready and the party commenced.
Meanwhile, development continued on the DEC-20. With the new architectures
and the speed of the CPU, the system would be a great deal more cost effective
than previous systems. Because of this, the product line delayed announcing it
until shortly before it was actually ready to ship. This all took place in
January of 1976. The first half dozen or so DEC-20 customers were carefully
selected for what would really be a field test of the new system.
For the first two or three years after the DEC-20 began shipping, it enjoyed
a certain amount of favor in DEC. Even at the time of first ship however, there
were clear signs that the mainstream of DEC products would be elsewhere -- the
PDP-11 family was doing very well and was making a lot of money for the
corporation, and design was already underway for the machine that would later be
known as VAX. There was a time however, when, in retrospect, some significant
history hung in the balance.
By 1975, it had become clear that the PDP-11 architecture had topped out. In
particular, the address space was too limited to support many growing
applications. An effort was started to build a new machine that would sell at
the upper end of the PDP-11 price range and beyond and would be the growth path
for -11 users. This new machine, code-named the UNICORN, was to be based on the
36-bit architecture since there was already a suite of software available for
it. Several of the most senior engineers in the PDP-11 groups began coming to
Marlboro to talk about building a small -10 -- small to our way of thinking, but
large in comparison to the -11 line.
One upshot of this work was that the original design for extended addressing
came under review. With more ambitious goals for business and performance, and a
greater appreciation for the pain of running out of address space, the PDP-11
engineers insisted that the extended addressing design be enhanced to improve
ultimate effectiveness. They convinced us that performance should not be
compromised for reasons of compatibility or conversion as had been contemplated
in the original design.
A new extended addressing design emerged which was a big improvement in the
long run. It was, however, too late to be fully implemented in the initial KL10
product. The new design incorporated a 30-bit address rather than 23. It also
eliminated the "special execute" because that would have added an additional
cycle to every extended reference and impacted the performance of programs using
extended addressing heavily. Instead, compatibility was provided based on use of
the address space. The overall address space was divided into "sections" of 256
Kwords each. In section 0, dubbed the "KI memorial section", code would operate
exactly as it had on the KI10, i.e. with no extended addressing possible. If the
PC were in any non-0 section however, a different algorithm for address
calculation would be used such that an extended address (inter-section) could be
generated at any time. These differences meant that code was not strictly
compatible, but the overall style and feel was much the same. In any given body
of code, most instructions would not need to be changed to run in the extended
environment, so we felt we still had provided a good path for migration of
existing code.
Ironically, as you may have already realized, the UNICORN project never came
to fruition. Within a relatively short time, a conclusion was reached that, even
if a cheap 36-bit architecture machine could be built, it would not be
"culturally compatible" with the PDP-11 software and applications and so would
not meet the need. Instead, a core group was formed to design an architecture
which would be culturally compatible with the PDP-11 but would eliminate the
limitations of the PDP-11 address space. Eventually, the machine built as a
result of this effort was named VAX-11, for Virtual Address eXtension of the 11.
As we know however, the designers did a lot more than just extend the address
space. What is not widely known is that, for a while at least, the VAX was
planned to have 36 bits!
Extended addressing is not the only way in which VAX impacted the first
release of the -20. By the time the -20 shipped, design was already underway for
VAX software as well as hardware. In an effort to reduce the plethora of
different command interfaces in DEC products, a committee had been formed to
specify a DEC Standard Command Language, or DCL. At the time, existing system
command languages included TOPS-10, TOPS-20, and various PDP-11 systems: RSX,
RSTS, RT-11, IAS, and others. As part of the emerging "one system" strategy,
this new command language standard was intended to affect existing systems
wherever possible and especially to guide the implementation of all new VAX
software.
There were several controversial areas where the TOPS-20 example was finally persuasive
and was used as the basis for the standard. For example, the TOPS-20
ordering of COPY (COPY from to) was adopted rather than the assignment style
(COPY to=from) of TOPS-10 and others. In one area however, TOPS-20 agreed to a
change in response to the consensus of the committee. Up to that point, TOPS-20
had continued with the TENEX practice of using semicolon as punctuation for the
file generation number, e.g. NAME.EXT;3 for generation 3. Rather late in the
process, it was realized that the generation didn't require a punctuation
character different from that for the extension and that it would be more
economical to use the same one, e.g. NAME.EXT.3. We agreed with that, even
though it meant changing some code very close to the date for Release 1 of
TOPS-20. Nonetheless, we did it so as not to have to face the problem of making
an incompatible change in a subsequent release.
Unfortunately, the developers of the VMS software had already begun using
semicolon and resisted making the change. Ultimately, the change was never made,
and VMS software shipped using semicolon. The standard was amended to go back to
semicolon, but TOPS-20 had already shipped using period. Thus, one small
opportunity for 20/VAX compatibility was missed despite our best efforts to
achieve it.
Although the formal end of the 36-bit product family wasn't announced until
1983, life in 36-bit land seemed precarious for many years before that. The KS10
had been built as a low-end addition to the family and shipped as the "2020"
within a year or so after the introduction of the 20 line, but that would
ultimately be the last new processor to be introduced. Several other were
started however.
One was known as Dolphin and was to be a high-end machine with performance
well beyond the KL10. However, it was undertaken as a combined 36-bit/VAX
development. That is, most of the engineering was intended to apply equally to a
VAX or 36-bit processor, and only a little bit of architecture-specific work was
supposed to be needed to turn out each machine. This, along with the other
complexities of technology and performance goals, made for a project beyond the
capabilities then available. Costs increased and the project got bogged down. In
early 1979, the 36-bit part of the project was cancelled and only the VAX part,
then called VENUS, continued and ultimately produced the VAX 8600.
Another project cancelled at around the same time was a very small PDP-10
known as Minnow. This project, operating in a small backroom on the second floor
of the Marlboro plant, had built a breadboard which was running well enough to
demonstrate some exec mode programs. It consisted of three large PC boards: a
processor board including 8 UARTS for comm lines, a memory board, and a disk
controller board. These could sit on a desk top but were envisioned as residing
in a deskside type cabinet with one fixed and one removable disk. Had this
machine not been cancelled, I believe it would have been a significant entry in
what later became the workstation market. It was not a complete workstation,
since we had not anticipated the value in having an integrated display system,
but it would have been a "personal" machine and a product with cost/performance
several years ahead of the market.
It succumbed in part to the corporate determination to adhere to the VAX
one-architecture strategy. While the Minnow product seemed attractive, it was
decided that it would send the wrong message to introduce a non-VAX machine in
that price range. Also, the Minnow had limited support in the 10/20 groups as
well. Many people saw 36-bit machines as living only in computer rooms and
surrounded with big disks and tape drives, and couldn't figure out how a small
one in an office could possibly be useful. This seemed to be part of a larger
conservatism that prevented or discouraged significant attempts to sustain and
expand the 36-bit business.
Hence, by 1979, the 36-bit line had been essentially cancelled again [see
section 1.2]. However, the corporation wasn't quite ready to declare this to the
world. VAXes were doing well, but weren't quite ready to take over all of the
high-end business. It was decided in the wake of the Dolphin cancellation to
build one last PDP-10 architecture machine, to be called the "2080". This
project would be separate from any VAX development. It would be simple,
low-cost, predictable, and limited to producing a cost-effective successor to
the KL10. These limited objectives led to a going-out-of-business mindset among
the 36-bit groups. Ambitious or long-term plans were shelved, and projects were refocused
on short term deliverables. The only official purpose for building
the 2080 was "support the current customers", i.e. don't try to grow the
business.
One area affected by this was TOPS-10 and the prospects for a one-system
strategy. While Dolphin development was going on, some effort was invested in
considering what it would take to make TOPS-20 fully capable of supporting all
the 36-bit customers. Performance studies had been done to find areas in which
TOPS-20 performance was less than TOPS-10. Networking and multi-processing
issues were considered. However, with the cancellation of Dolphin and the
going-out-of-business strategy, there was clearly no point in working to move
TOPS-10 customers to TOPS-20, so those objectives were once again discarded.
Each system group continued with the development that seemed pertinent, in many
cases doing essentially duplicate projects for hardware support.
Another probable casualty of limited horizons in 36-bit land was TOPS-20 SMP.
The architecture of TOPS-20 was well suited for SMP because the needs of that
style of operation were considered when TENEX was designed. Not only would the
system have readily supported SMP timesharing, but the process structure would
also have allowed user programs to easily initiate parallel activity within a
single job.
The TOPS-20 product was initially not SMP because the DECSYSTEM-20 hardware
was inherently single-CPU. With external memories, such as were used with
TOPS-10 configurations, SMP could be supported. Indeed, TOPS-10 SMP is one of
the most successful and widely studied aspects of TOPS-10. As a product decision
however, the DECSYSTEM-20 would NOT be configured with external memory and so
TOPS-20 could not be run on a hardware configuration capable of SMP. The 2020
was also single-CPU. Dolphin was planned to be a multi-processor machine, and
SMP was included in initial TOPS-20 plans at the time, but the Dolphin
cancellation eliminated that possibility for a hardware platform.
Interestingly, TENEX was modified to support SMP, but not at DEC or BBN. A
two-processor KI10 configuration was assembled at the Stanford University
Medical Center, and KI-TENEX was modified to operate as a full SMP operating
system by Rainer Schultz of Stanford. This system was operational around 1977
and some discussions were held with DEC about adding the SMP modifications to
TOPS-20. Nothing ever came of this however, since the defined -20 hardware
product never did include a possible SMP configuration.
As the 2080 project continued, it followed the long 36-bit tradition of
picking up a few "1-plusses" here and there. Before long, it had developed some
very ambitious performance projections due in large part to planned use of
pipelining techniques. The PDP-10 instruction set, being relatively simple
(although not as simple as a RISC instruction set) seemed to lend itself to a
great deal of pipelining, particularly if compilers could generate optimal code
sequences. It began to look as if the KL10 experience might be repeated and give
the 36-bit architecture a new lease on life.
Another change to the original plan was the inclusion of the DEC interconnect
strategy then emerging. This called for mass storage devices to be connected via
the CI and HSC controllers, and other peripherals to be connected via the Ethernet. About two years into the project, the system was redefined to use
these new interconnects exclusively, discarding support for massbus, Unibus, and
other existing interconnects. The project was renamed JUPITER, the LCG product
line was reborn, and going-out-of-business thoughts had largely vanished from
the group.
Major software efforts were undertaken in connection with this. Beyond the
basic hardware support, we came up with the concept of the Common File System as
the means for supporting systems with multiple CPUs. Comparable work was going
on in the VAX/VMS group and came to be known as Clusters. Ultimately, both VMS
and TOPS-20 shipped this kind of capability. The implementations were different
and reflected the differences in the two systems. VMS used a cluster lock
manager as the basic method for regulating simultaneous access to files, while
TOPS-20 imbedded the locking in the basic file opening primitive routines. The
TOPS-20 implementation also continued to use virtual memory mapping as the
paradigm for referencing files, and it is possibly the only system ever to
implement distributed shared virtual memory access.
Once again, some ideas were generated around the possibility of moving to a
one-operating-system strategy for 36-bit machines. For a brief period, a system
somehow combining all the goodness of both TOPS-10 and TOPS-20 was envisioned.
This was being called TOPS-36 so as not to make partisans of either TOPS-10 or
TOPS-20 think that it was just the other one. How this might have come out is
impossible to say, since no planning or design beyond general scenarios was ever
done.
Most of these ambitious plans got scaled back or abandoned because it wasn't
the KL10 history that was repeated by Jupiter; rather it was more like the
Dolphin history. Whether the project was "1-plussed" too much or whether other
avoidable mistakes were made can never be known for sure. However, after having
received one major extension of time and additional infusion of development
funds, the project found that it was going to need a great deal more of both.
Further, some long-delayed analysis of the performance of certain common 36-bit
instructions led to the conclusion that overall performance would not be up to
expectations. In fact, it would hardly surpass the KL10 in some cases. It would
probably have been possible to fix those problems, but the project was already
way over its head and seemed mired in weekly schedule replans and indecisiveness.
The early popularity of TENEX on the ARPANET was certainly a key to its later
acceptance, transformation, and support as TOPS-20 by DEC. That popularity in
turn seems to have been based not only on its technical features but also on the
fact that it was developed within that community and was responsive to it. When
TOPS-20 became a DEC product, it became part of a much larger market and thus
less responsive to any particular segment of the market.
In addition to that, as interest in other computer architectures increased in
the late seventies, many research sites came to the conclusion that they did not
want to be dependent on any one vendor for either hardware or software and,
instead, wanted "open" systems which would be amenable to local modification and
evolution. This of course led to a rapid increase in the use and popularity of
Unix. The fact that Unix was implemented in a reasonably portable language (at
least as compared with 36-bit MACRO) also encouraged its spread to new chip
architectures and beyond. If I could have done just one thing differently in the
history of TENEX and TOPS-20, it would be to have coded it in a higher level
language. With that, it's probable that the system, or at least large parts of
it, would have spread to other architectures and ultimately survived the demise
of the 36-bit architecture.
Much of the success of TOPS-20 and other 36-bit software is the result of
qualities that came out of the interactive environments in which it was built
and used. Over the last 10 years, the driving force in interactive environments
has been moving computer power closer to the user via personal computers and
workstations. For TOPS-20 to have remained in the forefront of interactive
development and use, 36-bit hardware platforms would have had to be built that
were small and cheap enough to go into the office, or the software would have
had to be portable enough to move to other such platforms.
Although this book is about DEC's 36-bit architecture, it is clear now that
hardware CPU architectures are of declining importance in shaping software. For
a long time, instruction set architectures drove the creation of new software. A
new architecture would be introduced, and new software systems would be written
for it. The 36-bit architecture was large in comparison to most other systems
which permitted interactive use. It was also lower in cost than most other
systems of that size. These factors were important in the creation of the kind
of software for which the architecture is known.
Now, new architectures are coming along with increasing frequency, but they
are simply "slid in" under the software. The software systems are far too large
to be rewritten each time, and a system which cannot adapt to new architectures
will eventually suffer declining interest and loss of competitive hardware
platforms. TOPS-20 didn't pass that test, although it did contribute a number of
ideas to the technology of interactive systems. How far these ideas ultimately
spread is a story yet to be told.
Question-mark Help
Origin of Recognition
Avoid Modes
TENEX MOVES TO DEC
KI-TENEX
Paging Algorithm in Software
KL10 Processor in Development
IBM Makes Virtual Memory Legitimate
DECSYSTEM-20 is Born
TOPS-20 Development Begins
TOPS-10 Compatibility and Coexistence
One Operating System or Two for 36-bit Machines?
OS Call Interface
Command Language
Disk Structure Compatibility
The Name Game, or, "What are we working on again?"
The TOPS-20 Group
Architecture Enhancements in the DECSYSTEM-20
Extended Addressing
Paging Algorithm in Microcode
Software Enhancements
The COMND Service
Implemention Language
The KL10 Becomes a Product
DECSYSTEM-20 Announcement and First Ship
How the VAX Almost Had 36 Bits
Other Effects of VAX Development on TOPS-20
TOPS-20 During its Product Years
Dolphin and Minnow
The "Going-out-of-Business" Strategy
Ambitious Plans for Jupiter
Conclusion
REFERENCES