Hello, for those of you who are not (or no longer :-) subscribed to the Quasijarus list, you still want to read this. This is Michael Sokolov as we all love him: Passionate and knowledgeable about VAXen, and just that. The project sounds kinda cool, if he delivers I'll want to get it. regards -Gunther Hi there, It's me again, your faithful 4.3BSD-Quasijarus maintainer. Sorry that I've been silent for a while... but guess what, I've got something cool, something that may raise your hair... So, what is it? Well, after whining and whining to myself about being deprived of the past 10 years worth of technological advances (NVAX, the last thing from DEC as far as classical VAXen go, came out in 1992), I've decided it's time to stop whining and start doing something about it. We've got to bite the bullet and design and build a new VAX that would use today's technology, be comparable in performance to today's CPUs, and still be a fully compliant VAX. Design the CPU, the system bus, the console and other support facilities, etc. and build it all on a sparkling shiny board. Since for the past 10 months I've worked for a company building single-board computers with today's technology (PowerPC CPUs, system bus at 133 MHz, memory SDRAM at system bus speed, I/O the fastest PCI chips they could find) and was involved in most phases of board design and bring-up, I know now what's involved in building such a beast and I've decided I'll give it a try. The major difference of course will be having to design the VAX CPU instead of having a ready-made PowerPC or MIPS or Sexium or whatever chip. Not having six to seven digit cash to build an ASIC, I'll have to settle for the CPU on an FPGA. As those are field-programmable, there are no big bucks to pay someone to fab it, and there's plenty of room for debugging and experimentation. I'll just have to take the VAX Architecture Reference Manual and implement it, in VHDL or Verilog or somesuch. Scary, I know. But what the heck, I'll give it a try. Part of designing a CPU is designing the bus coming out of it. This brings us to the next part of our VAX design: what should its system bus be? As you may remember from discussions on this list a few years ago and from my writings about VAX hardware on the Quasijarus WWW pages and elsewhere, in order to have a true VAX, one must have a real VAXist system bus. It has to follow all the VARM rules as to memory and cache coherency, VAX native memory and I/O address spaces (512 MB each usually), interlocks, interrupt messaging, transaction ordering, nice and orderly machine check exceptions when accessing a non- existent address, etc. So, what bus are we going to use? I believe that in order to be a true VAX, we have to natively support all traditional DEC buses and peripherals with the top-level bus meeting the above requirements. Having just PCI or somesuch would make a very poor man's VAX. Our new VAX should of course be top of the line: completely supercede all earlier VAXen and support everything that DEC ever supported. This my friends means XMI. That's what DEC's last top-of-the-line VAXen (6600 and 9000) had. XMI gives you a 14-slot backplane which you can further expand with an XMI-to-VAXBI adapter and a VAXBI-to-UNIBUS adapter, and between UNIBUS, BI, and XMI you've got the whole DEC universe covered. This plus how nicely these buses work together plus the closeness of this design to the original VAX pillars of 780 and 8600 makes me believe that XMI is the way to go. So, should we use XMI as our top-level system bus? Not necessarily. While architecturally beautiful, XMI by its age alone will be a performance bottleneck in our SuperVAX. I think even on the 6600 (NVAX-based, the last VAX maybe why they went from 6600 to 7000 (Alpha bus, very ugly) so quickly. (The 7000's Alpha bus makes it so ugly because the software can't directly access XMI or any other I/O bus, just memory, as the Alpha bus doesn't support I/O transactions, just memory.) What we need then is our own top-level bus of new design with year 2002 performance characteristics with XMI and other buses (see below) attached to it. The top-level bus will have CPU(s) and memory on it. The memory should of course be modern SDRAM. To make this kosher by VAX standards this top-level bus will have to meet all VARM requirements with all downstream buses interfaced to it as proper VAX nexi. If it's done right, it'll be the same design as the last Ultimate VAX from DEC had. The last Ultimate VAX from DEC was the 9000, folks. We'll have basically the same design but on one board. Cool! I also mentioned having other buses besides XMI and its subordinates. I'm thinking about PCI (ideally 64-bit 66 MHz, the top) and VME. Yes, PCI is decidedly not a DEC bus and it's very substandard compared to real VAX buses like VAXBI or XMI, but I think we need to support it to be competitive and to be able to take advantage of all the devices that come in the form of PCI chips. Now support it doesn't mean make it our main bus. If it sits by the side, doesn't stand in the way of data flowing through real DEC buses, and its use is entirely optional, it should be OK I think. As for VME, while we should faithfully reproduce everything that DEC made that's holy and pure, we don't need to reproduce their screw-ups. In particular, their deliberate blocking of third parties doesn't need to be reproduced. There's nothing wrong with supporting an open standard multivendor bus. The last point applies not only to VME. It applies even more so to SCSI. DEC's stubborn refusal to support it except in low-end systems definitely doesn't need to be reproduced in our new VAX. DEC only supported SCSI on low-end BabyVAXen, but not on anything higher-end: not on Q-bus, not on BI, not on XMI. As there seems to be no place for SCSI on the XMI side of our new VAX, we can go for the obvious alternative: if we are going to have PCI in there, just stick your favorite PCI UltraSCSI chip on the board. So we are going to have our new fast system bus with XMI and PCI hanging off of it, and eventually VME too. So what should this system bus be? Well, I've been thinking about it for the past few days and I've come up with a design that I think is sound and I think I can implement. After working for 10 months with screaming fast PowerPCs, I've been spoiled by their 133 MHz system bus. I've been even more spoiled by some of the chips available for this bus. I've been particularly impressed by Galileo GT-64240 and GT-64260 system controllers for MIPS and PowerPC respectively. So far I've only worked with PowerPCs and GT-64260 and haven't looked at the 240 yet, but as I understand it the MIPS and PowerPC system buses are very similar and the 240 and the 260 are essentially the same chip with minor mods. These system controllers are not PeeCee stuff. They are specifically designed for high-end systems with heavy I/O requirements, which is exactly what we would want in our VAX. Each GT-642x0 system controller includes the CPU interface running at up to 133 MHz, an SDRAM controller running at full CPU bus speed, 2 64-bit PCI interfaces running at up to 66 MHz, 3 10/100 Mbps Ethernet ports, 2 high-speed serial ports running at up to I believe 55 Mbps (!), and support for all system glue logic one can think of: ROMs, random logic devices, I2C, you name it. All interfaces run simultaneously in parallel and are interconnected by a "pizza arbiter", a crossbar switch with a 64-bit data path running at 133 MHz. This is the kind of stuff I have in mind when I talk about using year 2002 technology. I believe the GT-642x0 is the most powerful chip of this kind available today. (The cost is a bit high by PeeCee standards but should be fine for a high-end SuperVAX: GT-64260A costs $170 IIRC.) After thinking about it for a while, I have come up with my VAX design based on either GT-64240 or GT-64260 (haven't yet decided which). It took some hard thinking, but I think I've got it right. The most challenging thing of course is to gerrymander the MIPS/PowerPC system bus into a VAX bus, i.e., make it meet all VARM requirements. OTOH, the benefits from using those chips are significant: using the same system bus used in today's fast CPUs would assure comparable performance for our VAX at least as far as the bus goes, the time to market is reduced as some important components (the SDRAM controller, the PCI interface, and some of the support logic) will be ready-made, the chip is among the best one can pick for this purpose, and we'll have the Ethernet and fast serial interfaces for free. To understand how I'm going to use the MIPS/PowerPC system bus as the VAX main bus and still call it a VAX, one needs to consider the board design. It'll have this system bus interconnecting the CPU, the GT-642x0, the XMI interface chip, and other stuff later. One nice thing about the GT-642x0 is that it's specifically designed to support being not the only system controller, but one of many. It doesn't mind seeing transactions on its interfaces that it doesn't understand and that are meant for someone else. As it turns out, the MIPS/ PowerPC system bus is not that alien to the VAX, it's basically a subset of what the VAX needs. Since it's a subset, it needs to be extended. As I'm going to implement the CPU and the XMI interface on FPGAs, I'll have the freedom to tweak it as I want. Of course the GT-642x0 won't understand my extensions, but it won't need to: we need strict VARM compliance when talking to XMI, but not necessarily to PCI, as PCI is alien to VAX anyway. Now the GT-642x0 will be the main memory controller, so it better be VARM-compliant as far as memory goes. At first I thought this was going to be a show-stopper, as I couldn't figure out how I would communicate the special bus transactions for the BBCCI, BBSSI, ADAWI, INSQHI, INSQTI, REMQHI, and REMQTI instructions. But then a solution struck me. It's hard to explain without going too much into the PowerPC cache coherency model, but basically the CPU can hoard a little chunk of memory (32- byte granularity) and not let anyone else touch it until it's done with whatever it wants to do with it. In my case the whatever will be implementing the above 7 VAX instructions as prescribed by VARM. Voila! (If you are reading this babbling about using a MIPS/PPC system bus in a VAX and wondering how is it different from VAX 7000 using an Alpha bus that I denounced so loudly, you're right, it isn't, except for one detail. That Alpha bus doesn't support transactions smaller than a full word (don't remember if that was 32 or 64 bits, but you get the idea), so if you wanted to dink an 8- bit or 16-bit register on an I/O bus, you would be out of luck. That's one of the main reasons they didn't support direct CPU access to XMI and other I/O on the 7000. The PowerPC system bus doesn't have this problem, it supports all transfer sizes, and the GT-642x0 chips move data of different sizes between their interfaces all the time.) Since we are going to have our own top-level bus with the CPU(s), memory, and bus adapters sitting above XMI, PCI, and VME, it'll need its own backplane of our own design to hold the CPUs, memory modules, and bus adapters. There are many possible designs, but here is the one I've chosen from considerations of ease of implementation, time to market, and cost, as well as some marketing considerations. The backplane will be active, and will have the GT-642x0 on it. The slots on it will be PCI, nexi, and SDRAM DIMMs. It will be mechanically compatible with an ATX motherboard, but past the same mechanics it'll have enough differences to assure anyone looking at it that it's not a PeeCee. Of the 7 slots allowed for by ATX mechanics, 3 will be PCI and 4 will be nexi. The nexus slot connectors will be of my own design. The first nexus slot will always have to have a CPU card in it, which will be the master CPU, and the other 3 nexus slots will support identical CPU cards (slave CPUs) and bus adapters such as XMI and VME. The active backplane will also provide most of the system support logic, including the console FEP (front end processor, a small cheap microprocessor which I haven't yet selected) and the console port. Each CPU card will have just the VAX CPU with minimum support logic on it, which will reduce the incremental cost of adding CPUs. The FPGA-based CPU will only implement the VAX architecture as a running processor, and will truly halt like a 780 rather than implement halt microcode. This will certainly make it cool in a VAX hacker's eyes and reduce the time to market by moving the halt complexity from the harder to design FPGA into the easy to write FEP software. The CPU FPGA will communicate halts and architectured console IPR accesses to the FEP via the Inter-IC bus (I2C) or a similar bus. The active backplane will also provide Ethernet (GT-based) and SCSI (PCI-based) interfaces. (The high-speed serial interfaces of the GT-642x0 may also be routed out to external 50-pin connectors of the type DEC has used on sync serial options.) Thus in the minimal configuration with only one CPU card and some SDRAM one will have a fully functional system. Since such a system will fit entirely in an ATX enclosure and offer ready PCI expansion, it can be used to introduce the VAX architecture and this high-end product into new markets not previously exposed to VAXen. We'll be able to build and sell these systems in any required quantity entirely from current parts, no dependence of exhaustible stockpiles of old DEC parts. OTOH, since when equipped with an XMI adapter the system will be a fully compliant VAX and will be highly compatible with top of the line systems from DEC, this system should be acceptable even to the most ardent DEC purist. In the future systems can be built in other form factors if ATX is unacceptable or if more than 4 nexus slots are needed. The backplane will still have to be active most probably, though, because such a high-speed bus cannot have more than a few slots, and if we want to offer something like the 14-slot XMI backplane for our new system bus, the only way I can think of to implement it is to have several independent bus segments each with no more than 4 slots and interconnect them with intelligent bridges. These bridges would be active and quite sophisticated. But that's for the future, I think for a start a single bus segment with 4 slots in the cheapest form factor (ATX) will be fine. Proper design of the system support logic on the active backplane will ensure that the memory and interrupt mappings are as close as possible to those of next of kin DEC systems. The GT-642x0 will play nicely with this as its memory map is completely programmable, and it'll have no problem with the physical address space being only 30 bits instead of 32 (VAX architecture limitation). That's what I have on my mind, folks. As you can see I've put a good deal of thought into it and I'm serious about it. The next step is to better explore FPGA technology to see if I can match or beat the NVAX in performance. On our side we have 10 more years of technological advances since the NVAX, but OTOH DEC had built the NVAX as a fully custom IC rather than an FPGA. We'll see. Now here comes the part where I'll need your help, folks. While I have everything I'll need to design the active backplane and the CPU card (for the CPU card I'll just need the VARM which I have and the backplane is all current technology, no DEC specs needed) and thus I think I'll be able to deliver the base system in the ATX box, in my mind to make the project worth doing and thus to maximise the chances of me actually doing it, we'll need an assurance that the XMI adapter would follow shortly. I am prepared to design the FPGA and the board, but it'll require the XMI spec. I know it conceptually which allowed me to come up with the conceptual design outlined here, but to actually build it I will of course need the full spec detailing each signal and all. So, to make this project worth doing and thus see to it that I actually do it, you folks with spare time on your hands may want to start looking into prying the XMI specs out of DECompaq. -- Michael Sokolov 786 E MISSION AVE APT F Programletarian Freedom Fighter ESCONDIDO CA 92025-2154 USA International Free Computing Task Force Phone: +1-760-480-4575 msokolov(a)ivan.Harhan.ORG (ARPA) Let the Source be with you Programletarians of the world, unite! P.S. To the folks I've Cc'ed this to besides the Quasijarus list: your input could be extremely helpful to this project and other great projects being discussed on our list, and I would very much like to have you on the Quasijarus list. To subscribe send a message to quasijarus-request(a)ivan.Harhan.ORG. TIA.