PCIe/PCI I/O access (was: Re: VT340 Emulation)

[Maciej W. Rozycki]

On Sun, 27 Jun 2021, Kevin Bowling wrote:

> Thanks this was interesting learning.

 You are welcome!

 NB I think it is also worth noting that the presence of I/O instructions 
and corresponding bus cycles with x86 is most relevant for PCI as it is 
only *because* x86, or more importantly the PC/AT architecture, has had 
them in the first place that PCI (and consequently PCIe) has them as well. 
Likewise the interrupt acknowledge or the special cycle bus transactions 
-- they're both x86-specific, even if they may have been later repurposed 
in non-x86 PCI system designs.

 Intel was a key player in PCI design and surely their requirements were 
first-class citizens in the solution proposed.  While early PCI systems 
commonly had PCI and ISA/EISA buses arranged as peers on the CPU host bus, 
with the respective bridge chips being two of the three or more agents 
arbitrated there (the remaining ones being the CPU(s)), the ability to 
route x86-specific bus cycles over PCI eventually let the industry settle 
on PCI x86 systems of the common northbridge/southbridge architecture we 
saw so many specimens of, where I/O address space resources originating 
from the PC, PC/XT, and PC/AT architectures can be accessed behind a PCI 
bus segment just as if they were attached directly to the CPU, possibly 
via some bus buffers only, like with the original IBM design from 1981.

 Similarly an x86 CPU can send an interrupt acknowledge cycle over PCI to 
request an interrupt vector from an 8259A interrupt controller core 
present in the southbridge, or a shutdown special cycle, in response to a 
triple fault, so as to make the southbridge assert a CPU reset, just like 
the original discrete PC/AT circuitry did.  None of this is really needed 
by non-x86 systems, although again these features may have been repurposed 
now that they are present (Intel has since invented additional special CPU 
cycles too, used in power management and intepreted by the southbridge), 
and numerous host bridges used with non-x86 systems have dedicated CSRs in 
the memory address space for issuing these PCI bus cycles, which the CPU 
cannot natively produce.

 BTW it wasn't exactly obvious to me that a flood of:

LPC[000]: Got SYNC no-response error. Error address reg: 0xd0010014
IPMI: dropping non severe PEL event

messages produced by the system firmware to the console serial port was a 
symptom of a PCI device driver trying to poke at the port I/O address of 
0, correctly left by the OS in the relevant device's BAR (Base Address 
Register) along with the disabled status for I/O decoding in the device's 
PCI Command Register.  The driver, not platform-specific and unaware of 
the possibility of the inexistence of the PCI I/O address space in some 
systems, ignored the disabled status of I/O decoding with the device and 
just went ahead and poked at it, confusing the host bridge with the bus 
accesses that resulted (likely an OS bug too with port I/O address space 
accessors, though I wasn't inclined enough to chase it); of course the 
port I/O address of 0 itself has no particular meaning with non-x86 
systems and can be legitimately assigned and used with PCI devices.

 FWIW,

  Maciej