History of Symbolics lisp machines

This is an archive of Dan Weinreb’s comments on Symbolics and Lisp machines.

Rebuttal to Stallman’s Story About The Formation of Symbolics and LMI

Richard Stallman has been telling a story about the origins of the Lisp machine companies, and the effects on the M.I.T. Artificial Intelligence Lab, for many years. He has published it in a book, and in a widely-referenced paper, which you can find at http://www.gnu.org/gnu/rms-lisp.html.

His account is highly biased, and in many places just plain wrong. Here’s my own perspective on what really happened.

Richard Greenblatt’s proposal for a Lisp machine company had two premises. First, there should be no outside investment. This would have been totally unrealistic: a company manufacturing computer hardware needs capital. Second, Greenblatt himself would be the CEO. The other members of the Lisp machine project were extremely dubious of Greenblatt’s ability to run a company. So Greenblatt and the others went their separate ways and set up two companies.

Stallman’s characterization of this as “backstabbing”, and that Symbolics decided not “not have scruples”, is pure hogwash. There was no backstabbing whatsoever. Symbolics was extremely scrupulous. Stallman’s characterization of Symbolics as “looking for ways to destroy” LMI is pure fantasy.

Stallman claims that Symbolics “hired away all the hackers” and that “the AI lab was now helpless” and “nobody had envisioned that the AI lab’s hacker group would be wiped out, but it was” and that Symbolics “wiped out MIT”. First of all, had there been only one Lisp machine company as Stallman would have preferred, exactly the same people would have left the AI lab. Secondly, Symbolics only hired four full-time and one part-time person from the AI lab (see below).

Stallman goes on to say: “So Symbolics came up with a plan. They said to the lab, ‘We will continue making our changes to the system available for you to use, but you can’t put it into the MIT Lisp machine system. Instead, we’ll give you access to Symbolics’ Lisp machine system, and you can run it, but that’s all you can do.’” In other words, software that was developed at Symbolics was not given away for free to LMI. Is that so surprising? Anyway, that wasn’t Symbolics’s “plan”; it was part of the MIT licensing agreement, the very same one that LMI signed. LMI’s changes were all proprietary to LMI, too.

Next, he says: “After a while, I came to the conclusion that it would be best if I didn’t even look at their code. When they made a beta announcement that gave the release notes, I would see what the features were and then implement them. By the time they had a real release, I did too.” First of all, he really was looking at the Symbolics code; we caught him doing it several times. But secondly, even if he hadn’t, it’s a whole lot easier to copy what someone else has already designed than to design it yourself. What he copied were incremental improvements: a new editor command here, a new Lisp utility there. This was a very small fraction of the software development being done at Symbolics.

His characterization of this as “punishing” Symbolics is silly. What he did never made any difference to Symbolics. In real life, Symbolics was rarely competing with LMI for sales. LMI’s existence had very little to do with Symbolics’s bottom line.

And while I’m setting the record straight, the original (TECO-based) Emacs was created and designed by Guy L. Steele Jr. and David Moon. After they had it working, and it had become established as the standard text editor at the AI lab, Stallman took over its maintenance.

Here is the list of Symbolics founders. Note that Bruce Edwards and I had worked at the MIT AI Lab previously, but had already left to go to other jobs before Symbolics started. Henry Baker was not one of the “hackers” of which Stallman speaks.

Why Did Symbolics Fail?

In a comment on a previous blog entry, I was asked why Symbolics failed. The following is oversimplified but should be good enough. My old friends are very welcome to post comments with corrections or additions, and of course everyone is invited to post comments.

First, remember that at the time Symbolics started around 1980, serious computer users used timesharing systems. The very idea of a whole computer for one person was audacious, almost heretical. Every computer company (think Prime, Data General, DEC) did their own hardware and their own software suite. There were no PCs’, no Mac’s, no workstations. At the MIT Artificial Intelligence Lab, fifteen researchers shared a computer with a .001 GHz CPU and .002 GB of main memory.

Symbolics sold to two kinds of customers, which I’ll call primary and secondary. The primary customers used Lisp machines as software development environments. The original target market was the MIT AI Lab itself, followed by similar institutions: universities, corporate research labs, and so on. The secondary customers used Lisp machines to run applications that had been written by some other party.

We had great success amongst primary customers. I think we could have found a lot more of them if our marketing had been better. For example, did you know that Symbolics had a world-class software development environment for Fortran, C, Ada, and other popular languages, with amazing semantics-understanding in the editor, a powerful debugger, the ability for the languages to call each other, and so on? We put a lot of work into those, but they were never publicized or advertised.

But we knew that the only way to really succeed was to develop the secondary market. ICAD made an advanced constraint-based computer-aided design system that ran only on Symbolics machines. Sadly, they were the only company that ever did. Why?

The world changed out from under us very quickly. The new “workstation” category of computer appeared: the Suns and Apollos and so on. New technology for implementing Lisp was invented that allowed good Lisp implementations to run on conventional hardware; not quite as good as ours, but good enough for most purposes. So the real value-added of our special Lisp architecture was suddenly diminished. A large body of useful Unix software came to exist and was portable amongst the Unix workstations: no longer did each vendor have to develop a whole software suite. And the workstation vendors got to piggyback on the ever-faster, ever-cheaper CPU’s being made by Intel and Motorola and IBM, with whom it was hard for Symbolics to keep up. We at Symbolics were slow to acknowledge this. We believed our own “dogma” even as it became less true. It was embedded in our corporate culture. If you disputed it, your co-workers felt that you “just didn’t get it” and weren’t a member of the clan, so to speak. This stifled objective analysis. (This is a very easy problem to fall into — don’t let it happen to you!)

The secondary market often had reasons that they needed to use workstation (and, later, PC) hardware. Often they needed to interact with other software that didn’t run under Symbolics. Or they wanted to share the cost of the hardware with other applications that didn’t run on Symbolics. Symbolics machines came to be seen as “special-purpose hardware” as compared to “general-purpose” Unix workstations (and later Windows PCs). They cost a lot, but could not be used for the wider and wider range of available Unix software. Very few vendors wanted to make a product that could only run on “special-purpose hardware”. (Thanks, ICAD; we love you!)

Also, a lot of Symbolics sales were based on the promise of rule-based expert systems, of which the early examples were written in Lisp. Rule-based expert systems are a fine thing, and are widely used today (but often not in Lisp). But they were tremendously over-hyped by certain academics and by their industry, resulting in a huge backlash around 1988. “Artificial Intelligence” fell out of favor; the “AI Winter” had arrived.

(Symbolics did launch its own effort to produce a Lisp for the PC, called CLOE, and also partnered with other Lisp companies, particularly Gold Hill, so that customers could develop on a Symbolics and deploy on a conventional machine. We were not totally stupid. The bottom line is that interest in Lisp just declined too much.)

Meanwhile, back at Symbolics, there were huge internal management conflicts, leading to the resignation of much of top management, who were replaced by the board of directors with new CEO’s who did not do a good job, and did not have the vision to see what was happening. Symbolics signed long-term leases on big new offices and a new factory, anticipating growth that did not come, and were unable to sublease the properties due to office-space gluts, which drained a great deal of money. There were rounds of layoffs. More and more of us realized what was going on, and that Symbolics was not reacting. Having created an object-oriented database system for Lisp called Statice, I left in 1988 with several co-workers to form Object Design, Inc., to make an object-oriented database system for the brand-new mainstream object-oriented language, C++. (The company was very successful and currently exists as the ObjectStore division of Progress Software (www.objectstore.com). I’m looking forward to the 20th-year reunion party next summer.)

Symbolics did try to deal with the situation, first by making Lisp machines that were plug-in boards that could be connected to conventional computers. One problem is that they kept betting on the wrong horses. The MacIvory was a Symbolics Ivory chip (yes, we made our own CPU chips) that plugged into the NuBus (oops, long-since gone) on a Macintosh (oops, not the leading platform). Later, they finally gave up on competing with the big chip makers, and made a plug-in board using a fast chip from a major manufacturer: the DEC Alpha architecture (oops, killed by HP/Compaq, should have used the Intel). By this time it was all too little, too late.

The person who commented on the previous blog entry referred by to an MIT Masters thesis by one Eve Philips (see http://www.sts.tu-harburg.de/~r.f.moeller/symbolics-info/ai-business.pdf) called “If It Works, It’s Not AI: A Commercial Look at Artificial Intelligence Startups”. This is the first I’ve heard of it, but evidently she got help from Tom Knight, who is one of the other Symbolics co-founders and knows as much or more about Symbolics history than I. Let’s see what she says.

Hey, this looks great. Well worth reading! She definitely knows what she’s talking about, and it’s fun to read. It brings back a lot of old memories for me. If you ever want to start a company, you can learn a lot from reading “war stories” like the ones herein.

Here are some comments, as I read along. Much of the paper is about the AI software vendors, but their fate had a strong effect on Symbolics.

Oh, of course, the fact that DARPA cut funding in the late 80’s is very important. Many of the Symbolics primary-market customers had been ultimately funded by DARPA research grants.

Yes, there were some exciting successes with rule-based expert systems. Inference’s “Authorizer’s Assistant” for American Express, to help the people who talk to you on the phone to make sure you’re not using an AmEx card fraudulently, ran on Symbolics machines. I learn here that it was credited with a 45-67% internal rate of return on investment, which is very impressive.

The paper has an anachronism: “Few large software firms providing languages (namely Microsoft) provide any kind of Lisp support.” Microsoft’s dominance was years away when these events happened. For example, remember that the first viable Windows O/S, release 3.1, came out in in 1990. But her overall point is valid.

She says “There was a large amount of hubris, not completely unwarranted, by the AI community that Lisp would change the way computer systems everywhere ran.” That is absolutely true. It’s not as wrong as it sounds: many ideas from Lisp have become mainstream, particularly managed (garbage-collected) storage, and Lisp gets some of the credit for the acceptance of object-oriented programming. I have no question that Lisp was a huge influence on Java, and thence on C#. Note that the Microsoft Common Language Runtime technology is currently under the direction of the awesome Patrick Dussud, who was the major Lisp wizard from the third MIT-Lisp-machine company, Texas Instruments.

But back then we really believed in Lisp. We felt only scorn for anyone trying to write an expert system in C; that was part of our corporate culture. We really did think Lisp would “change the world” analogously to the way “sixties-era” people thought the world could be changed by “peace, love, and joy”. Sorry, it’s not that easy.

Which reminds me, I cannot recommend highly enough the book “Patterns of Software: Tales from the Software Community” by Richard Gabriel (http://www.dreamsongs.com/Files/PatternsOfSoftware.pdf) regarding the process by which technology moves from the lab to the market. Gabriel is one of the five main Common Lisp designers (along with Guy Steele, Scott Fahlman, David Moon, and myself), but the key points here go way beyond Lisp. This is the culmination of the series of papers by Gabriel starting with his original “Worse is Better”. Here the ideas are far more developed. His insights are unique and extremely persuasive.

OK, back to Eve Philips: at chapter 5 she describes “The AI Hardware Industry”, starting with the MIT Lisp machine. Does she get it right? Well, she says “14 AI lab hackers joined them”; see my previous post about this figure, but in context this is a very minor issue. The rest of the story is right on. (She even mentions the real-estate problems I pointed out above!) She amply demonstrates the weaknesses of Symbolics management and marketing, too. This is an excellent piece of work.

Symbolics was tremendously fun. We had a lot of success for a while, and went public. My colleagues were some of the skilled and likable technical people you could ever hope to work with. I learned a lot from them. I wouldn’t have missed it for the world.

After I left, I thought I’d never see Lisp again. But now I find myself at ITA Software, where we’re writing a huge, complex transaction-processing system (a new airline reservation system, initially for Air Canada), whose core is in Common Lisp. We almost certainly have the largest team of Common Lisp programmers in the world. Our development environment is OK, but I really wish I had a Lisp machine again.

More about Why Symbolics Failed

I just came across “Symbolics, Inc: A failure of heterogeneous engineering” by Alvin Graylin, Kari Anne Hoir Kjolaas, Jonathan Loflin, and Jimmie D. Walker III (it doesn’t say with whom they are affiliated, and there is no date), at http://www.sts.tu-harburg.de/~r.f.moeller/symbolics-info/Symbolics.pdf

This is an excellent paper, and if you are interested in what happened to Symbolics, it’s a must-read.

The paper’s thesis is based on a concept called “heterogeneous engineering”, but it’s hard to see what they mean by that other than “running a company well”. They have fancy ways of saying that you can’t just do technology, you have to do marketing and sales and finance and so on, which is rather obvious. They are quite right about the wide diversity of feelings about the long-term vision of Symbolics, and I should have mentioned that in my essay as being one of the biggest problems with Symbolics. The random directions of R&D, often not co-ordinated with the rest of the company, are well-described here (they had good sources, including lots of characteristically, harshly honest email from Dave Moon). The separation between the software part of the company in Cambridge, MA and the hardware part of the company in Woodland Hills (later Chatsworth) CA was also a real problem. They say “Once funds were available, Symbolics was spending money like a lottery winner with new-found riches” and that’s absolutely correct. Feature creep was indeed extremely rampant. The paper also has financial figures for Symbolics, which are quite interesting and revealing, showing a steady rise through 1986, followed by falling revenues and negative earnings from 1987 to 1989.

Here are some points I dispute. They say “During the years of growth Symbolics had been searching for a CEO”, leading up to the hiring of Brian Sear. I am pretty sure that only happened when the trouble started. I disagree with the statement by Brian Sear that we didn’t take care of our current customers; we really did work hard at that, and I think that’s one of the reasons so many former Symbolics customers are so nostalgic. I don’t think Russell is right that “many of the Symbolics machines were purchased by researchers funded through the Star Wars program”, a point which they repeat many times. However, many were funded through DARPA, and if you just substitute that for all the claims about “Star Wars”, then what they say is right. The claim that “the proliferation of LISP machines may have exceeded the proliferation of LISP programmers” is hyperbole. It’s not true that nobody thought about a broader market than the researchers; rather, we intended to sell to value-added resellers (VAR’s) and original equipment manufacturers (OEM’s). The phrase “VARs and OEMs” was practically a mantra. Unfortunately, we only managed to do it once (ICAD). While they are right that Sun machines “could be used for many other applications”, the interesting point is the reason for that: why did Sun’s have many applications available? The rise of Unix as a portable platform, which was a new concept at the time, had a lot to do with it, as well as Sun’s prices. They don’t consider why Apollo failed.

There’s plenty more. To the authors, wherever you are: thank you very much!