Writing emulators (was Re: VCF PNW 2018: Pictures!)

[Ray Arachelian]

On 02/19/18 19:36, Adrian Stoness via cctalk wrote:
> whats invovled in makin an emulator?
> i have a chunk of stuff for the phillips p1000

Quite a lot actually.  A single CPU system is difficult enough, but a
mainframe might be much, much harder.  The idea to use an existing
emulator framework, such as SIMH, is a great one.

An easy implementation is to implement an interpretive CPU emulation at
first, and then later on add other things such as JITs or caching.
Does this machine implement microcode?  Do you want to emulate it all
the way down to the microcode level?  Is microcode changeable by the
OS/applications?  If not maybe implement the higher level (assuming
CISC) CPU layer.

There's a lot to consider.  The CPU(s), any co-processors, I/O
devices/busses, peripherals/terminals, etc.  Are you going to emulate
every co-processor in software, or is the system documented enough so
you can emulate just the protocols that the main CPU(s) use to talk to
those devices?  For example, many systems have some sort of storage
processors.  You could emulate everything 100% in software, but for that
you'd need disk and firmware dumps of everything.  Or, if the firmware
on those is fairly fixed, just emulate the functionality.

Are you planning on emulating the whole machine, or just the userland? 
Might be easier to create a simulation of the OS in software on the host
side the way that Executor did with MacOS - Cliff implemented his own
version of MacOS 7.x, enough to be able to run most applications of that
era, but not all.  see: https://github.com/ctm/executor.git and
https://github.com/ctm/executor.git - some of this is called "High Level
Emulation" and is sort of what WINE does (though wine isn't an emulator).

You'll need to fully understand all aspects of the hardware of that
machine, every detail.  If you have schematics and can read them, they
can be an absolute gold mine as documentation sometimes is vague, and
looking at the schematics and resolve what actually happens.  If you
have source code for an OS, that too can be a great resource to help
understand what happens and how.

In terms of I/O. you'll be writing the reverse of what a driver will do
- so if you have docs on how to write drivers for that machine, you're
going to implement the hardware itself, in software, to match or connect
to drivers on the system.

Sometimes timing matters, sometimes it doesn't.  Where it doesn't, you
have the opportunity to improve execution speed by running at full speed
on the host.

You'll have to do lots of experimenting.  Queues tagged with future
events (i.e. at CPU_CYCLES+ONE_TENTH_THOUSANDTH_OF_A_SECOND, trigger IRQ
# 3 ), are a good way to go, or you could go the hard way and implement
a multi-process/thread emulator.

You'll have to implement more than 90% of a working emulator before
you'll even be able to see progress and know you're on the right track,
because more than likely you won't get far enough to get anything
starting to boot until that point.  Once you get there, you should
implement a debugger for yourself and write trace log mechanisms
throughout your code so you can turn on tracing at specific points to
see what your code is doing, then sit there and read megs and megs of
those logs until you figure out where it's going wrong vs actual hardware.

If it's possible to write a CPU tester (and device testers) on the real
thing, do that.  Run every CPU instruction through various edge cases,
look at the resulting status flags and log all registers in/out to a
database.  (This will easily be multi-gigabyte output).
For example. say you have a shift instruction on a 16 bit word.  What
happens when the register is 32 bits, and the carry flags are set and
you do a shift left by 18 times?  Yes, that doesn't make sense, I know. 
But an intel chip and 68000, and a SPARC, and a PPC, and ARM chip and a
POWER4 chip all produce totallly different results with such inputs, so
you can't assume it will just be handled by your host's CPU without a
lot of extra work.

And your emulator, if it's designed to be portable, must implement all
those weird edge cases.  And if you don't implement a portable emulator,
if it only runs on a current OS with a current chip, it will soon be
obsolete in a few years....

After you've built your test suite, run your emulated CPU against this
test frame, and flag any results that differ from what the physical
hardware generated.  Repeat that process with device hardware such as
tape drives, disk drives, etc.  Likely terminal emulation won't be too
hard if standard stuff like vt100 are supported as you can use existing
terminal emulators.  But other stuff will be hard.  Some, really, really
hard.  Other things will be easy.

Hopefully the documentation you have is complete enough to write a full
OS from scratch on that machine, if it isn't, then you may have to do a
lot of black box reverse engineering which is only possible if you have
access to working hardware and lots of time.

These are good resources to give you an idea of what you're in for:
 
 http://www.xsim.com/papers/Bario.2001.emubook.pdf
 https://fms.komkon.org/EMUL8/HOWTO.html
 https://www.reddit.com/r/EmuDev/

Good luck, you have a lot of work ahead of you.  If you can, I'd suggest
the first step would be to scan all the materials you have, find a bunch
of folks interested in your project and share it with them.  Be careful
as there's a lot of "emu scene" folks out there with tons of free time,
more than you might have, who will happily promise to help, but instead
take your documentation, firmware, OS images, etc. and compete with you
behind your back just to get there first, instead of actually helping
you with your project.  On this latter point, I sadly speak from experience.