| Message ID | 1498117132-27139-1-git-send-email-bhupinder.thakur@linaro.org |
|---|---|
| Headers | show |
| Series | SBSA UART emulation support in Xen | expand |
Hi Bhupinder, On 22/06/17 08:38, Bhupinder Thakur wrote: > There are some TBD items which need to be looked at in the future: > > 1. Currently UEFI firmware logs the output to hvc console only. How can > UEFI firmware be made aware of pl011 console and how it can use it > as a console instead of hvc. Would it be possible to summarize the discussions we had a couple of weeks ago here? > 2. Linux seems to have hvc console as the default console i.e. if no > console is specified then it uses hvc as the console. How can an > option be provided in Linux to select either hvc or pl011 as the > default console. I am wondering what would happen if you use stdout-path in the device-tree. Does it override the default console? IHMO, the best way to select the default console would be using either the SPCR (for ACPI) or stdout-path (for DT). But the HVC console does not have any description in the firmware. It might be worth considering adding description. The drawback is the user would always have to specify the console on the command line. I think this is not too bad for a first approach. > > 3. ACPI support for pl011 device. I would be ok to defer this after PL011 series is merged. > Bhupinder Thakur (17): > xen/arm: vpl011: Move vgic register access functions to vreg.h > xen/arm: vpl011: Rename vgic_reg* functions definitions and calls to > vreg_reg* Stefano, would it be possible to commit those patches? They are both acked and it would avoid Bhupinder to carry them and rebase them if the vGIC code has changed. Cheers,
On Fri, 23 Jun 2017, Julien Grall wrote: > Hi Bhupinder, > > On 22/06/17 08:38, Bhupinder Thakur wrote: > > There are some TBD items which need to be looked at in the future: > > > > 1. Currently UEFI firmware logs the output to hvc console only. How can > > UEFI firmware be made aware of pl011 console and how it can use it > > as a console instead of hvc. > > Would it be possible to summarize the discussions we had a couple of weeks ago > here? > > > 2. Linux seems to have hvc console as the default console i.e. if no > > console is specified then it uses hvc as the console. How can an > > option be provided in Linux to select either hvc or pl011 as the > > default console. > > I am wondering what would happen if you use stdout-path in the device-tree. > Does it override the default console? > > IHMO, the best way to select the default console would be using either the > SPCR (for ACPI) or stdout-path (for DT). But the HVC console does not have any > description in the firmware. It might be worth considering adding description. > > The drawback is the user would always have to specify the console on the > command line. I think this is not too bad for a first approach. > > > > > 3. ACPI support for pl011 device. > > I would be ok to defer this after PL011 series is merged. > > > Bhupinder Thakur (17): > > xen/arm: vpl011: Move vgic register access functions to vreg.h > > xen/arm: vpl011: Rename vgic_reg* functions definitions and calls to > > vreg_reg* > > Stefano, would it be possible to commit those patches? They are both acked and > it would avoid Bhupinder to carry them and rebase them if the vGIC code has > changed. Sure
Hi Julien, On 23 June 2017 at 16:12, Julien Grall <julien.grall@arm.com> wrote: > Hi Bhupinder, > > On 22/06/17 08:38, Bhupinder Thakur wrote: >> >> There are some TBD items which need to be looked at in the future: >> >> 1. Currently UEFI firmware logs the output to hvc console only. How can >> UEFI firmware be made aware of pl011 console and how it can use it >> as a console instead of hvc. > > > Would it be possible to summarize the discussions we had a couple of weeks > ago here? Currently, UEFI firmware uses hvc as the console for input/output. Now with the support of SBSA UART in Xen, it is preferrable that UEFI firmware should be able to the uart as well. One option which was discussed was to use pl011 purely as a debug port. Currently the debug prints are intermixed with the normal console output. Now with uart port becoming available the debug prints can be redirected to pl011 thus cleaning up the console output. Other option is to output everything on both HVC and pl011 both but it takes away the advantage of separating out the debug and normal console prints. However, pl011 can be used as debug port based on a compile time flag. If this compile-time is off, then the output can be sent to both HVC and pl011. Based on this discussion I feel that: - the default behaviour should be writing the output to both HVC and pl011. - pl011 can be used as a pure debug port based on a compile-time flag. > >> 2. Linux seems to have hvc console as the default console i.e. if no >> console is specified then it uses hvc as the console. How can an >> option be provided in Linux to select either hvc or pl011 as the >> default console. > > > I am wondering what would happen if you use stdout-path in the device-tree. > Does it override the default console? > I tried adding a "chosen" node in the DT to select "sbsa-pl011" as the stdout-path. I added the following code in make_chosen_node() function: fdt_property_string(fdt, "stdout-path", "sbsa-pl011"); However, I still see the initial console output going to hvc only. > IHMO, the best way to select the default console would be using either the > SPCR (for ACPI) or stdout-path (for DT). But the HVC console does not have > any description in the firmware. It might be worth considering adding > description. > > The drawback is the user would always have to specify the console on the > command line. I think this is not too bad for a first approach. > Do we plan to address this requirement in the current patch series? >> >> 3. ACPI support for pl011 device. > > > I would be ok to defer this after PL011 series is merged. > Regards, Bhupinder
On 07/04/2017 08:31 AM, Bhupinder Thakur wrote: > Hi Julien, Hi Bhupinder, Thank you for the summary! [...] > > Currently, UEFI firmware uses hvc as the console for input/output. Now > with the support > of SBSA UART in Xen, it is preferrable that UEFI firmware should be > able to the uart > as well. > > One option which was discussed was to use pl011 purely as a debug > port. Currently the debug > prints are intermixed with the normal console output. Now with uart > port becoming available > the debug prints can be redirected to pl011 thus cleaning up the console output. > > Other option is to output everything on both HVC and pl011 both but it > takes away the advantage > of separating out the debug and normal console prints. However, pl011 > can be used as debug > port based on a compile time flag. If this compile-time is off, then > the output can be sent to both > HVC and pl011. > > Based on this discussion I feel that: > - the default behaviour should be writing the output to both HVC and pl011. Hmmm. If I remember correctly this was suggested but ruled out. It was considered that pl011 and PV console should not be treated equal. PL011 would be used for boot diagnostics (i.e imagine an Image with no Xen support). So we would continue to use Xen PV console for UEFI console and let the choice for the debug at compile to be either on PL011 or PV console. Stefano, any opinions? > - pl011 can be used as a pure debug port based on a compile-time flag. Agree. > >> >>> 2. Linux seems to have hvc console as the default console i.e. if no >>> console is specified then it uses hvc as the console. How can an >>> option be provided in Linux to select either hvc or pl011 as the >>> default console. >> >> >> I am wondering what would happen if you use stdout-path in the device-tree. >> Does it override the default console? >> > I tried adding a "chosen" node in the DT to select "sbsa-pl011" as the > stdout-path. I added the > following code in make_chosen_node() function: > > fdt_property_string(fdt, "stdout-path", "sbsa-pl011"); > > However, I still see the initial console output going to hvc only. What do you mean by inital console output? You mean the bootconsole? > >> IHMO, the best way to select the default console would be using either the >> SPCR (for ACPI) or stdout-path (for DT). But the HVC console does not have >> any description in the firmware. It might be worth considering adding >> description. >> >> The drawback is the user would always have to specify the console on the >> command line. I think this is not too bad for a first approach. >> > Do we plan to address this requirement in the current patch series? I am ok with this to be deferred. This is not highly critical. We should probably log it on xenproject.atlassian.net to avoid loosing the item. Cheers,
On Wed, 5 Jul 2017, Julien Grall wrote: > On 07/04/2017 08:31 AM, Bhupinder Thakur wrote: > > Hi Julien, > > Hi Bhupinder, > > Thank you for the summary! > > [...] > > > > Currently, UEFI firmware uses hvc as the console for input/output. Now > > with the support > > of SBSA UART in Xen, it is preferrable that UEFI firmware should be > > able to the uart > > as well. > > > > One option which was discussed was to use pl011 purely as a debug > > port. Currently the debug > > prints are intermixed with the normal console output. Now with uart > > port becoming available > > the debug prints can be redirected to pl011 thus cleaning up the console > > output. > > > > Other option is to output everything on both HVC and pl011 both but it > > takes away the advantage > > of separating out the debug and normal console prints. However, pl011 > > can be used as debug > > port based on a compile time flag. If this compile-time is off, then > > the output can be sent to both > > HVC and pl011. > > > > Based on this discussion I feel that: > > - the default behaviour should be writing the output to both HVC and pl011. > > Hmmm. If I remember correctly this was suggested but ruled out. It was > considered that pl011 and PV console should not be treated equal. PL011 would > be used for boot diagnostics (i.e imagine an Image with no Xen support). Actually I remember the opposite: afd2e931-706b-6e25-1f0e-feee16e83c88@redhat.com (this was a private reply though). > So we would continue to use Xen PV console for UEFI console and let the choice > for the debug at compile to be either on PL011 or PV console. > > Stefano, any opinions? I would approach this issue by asking the following question: "what behavior would be more beneficial to users?" Given the lack of concrete data, we have to make guesses. I think it would least suprise users if the output went to both consoles. Also, it would be more beneficial, because one could fully run operating systems without Xen support (think of a small baremetal app) without having to switch back and forth between PV console (for the firmware) and PL011 (for the app). However, as I wrote before on this topic, baremetal apps are unlikely to boot from UEFI in the near future, so I admit this is not a very strong use-case. > > - pl011 can be used as a pure debug port based on a compile-time flag. > > Agree. > > > > > > > > > > 2. Linux seems to have hvc console as the default console i.e. if no > > > > console is specified then it uses hvc as the console. How can an > > > > option be provided in Linux to select either hvc or pl011 as the > > > > default console. > > > > > > > > > I am wondering what would happen if you use stdout-path in the > > > device-tree. > > > Does it override the default console? > > > > > I tried adding a "chosen" node in the DT to select "sbsa-pl011" as the > > stdout-path. I added the > > following code in make_chosen_node() function: > > > > fdt_property_string(fdt, "stdout-path", "sbsa-pl011"); > > > > However, I still see the initial console output going to hvc only. > > What do you mean by inital console output? You mean the bootconsole? > > > > > > IHMO, the best way to select the default console would be using either the > > > SPCR (for ACPI) or stdout-path (for DT). But the HVC console does not have > > > any description in the firmware. It might be worth considering adding > > > description. > > > > > > The drawback is the user would always have to specify the console on the > > > command line. I think this is not too bad for a first approach. > > > > > Do we plan to address this requirement in the current patch series? > > I am ok with this to be deferred. This is not highly critical. We should > probably log it on xenproject.atlassian.net to avoid loosing the item. > > Cheers, > > -- > Julien Grall >
On 05/07/2017 20:06, Stefano Stabellini wrote: > On Wed, 5 Jul 2017, Julien Grall wrote: >> On 07/04/2017 08:31 AM, Bhupinder Thakur wrote: >>> Hi Julien, >> >> Hi Bhupinder, >> >> Thank you for the summary! >> >> [...] >>> >>> Currently, UEFI firmware uses hvc as the console for input/output. Now >>> with the support >>> of SBSA UART in Xen, it is preferrable that UEFI firmware should be >>> able to the uart >>> as well. >>> >>> One option which was discussed was to use pl011 purely as a debug >>> port. Currently the debug >>> prints are intermixed with the normal console output. Now with uart >>> port becoming available >>> the debug prints can be redirected to pl011 thus cleaning up the console >>> output. >>> >>> Other option is to output everything on both HVC and pl011 both but it >>> takes away the advantage >>> of separating out the debug and normal console prints. However, pl011 >>> can be used as debug >>> port based on a compile time flag. If this compile-time is off, then >>> the output can be sent to both >>> HVC and pl011. >>> >>> Based on this discussion I feel that: >>> - the default behaviour should be writing the output to both HVC and pl011. >> >> Hmmm. If I remember correctly this was suggested but ruled out. It was >> considered that pl011 and PV console should not be treated equal. PL011 would >> be used for boot diagnostics (i.e imagine an Image with no Xen support). > > Actually I remember the opposite: > afd2e931-706b-6e25-1f0e-feee16e83c88@redhat.com (this was a private > reply though). This was an answer to my question whether a user could select the serial by himself. To this reply, you asked whether it was feasible to output on all the serials console, but I don't see any yes/no answer. On the rest of the thread, it has been mentioned it was difficult to multiplex to serial console (I forwarded you the thread). Christoffer, Laszlo and Ard agreed that PL011 should only be used as boot diagnostics and debug (if selected at compile time). Cheers,
On 05/07/2017 20:43, Julien Grall wrote: > > > On 05/07/2017 20:06, Stefano Stabellini wrote: >> On Wed, 5 Jul 2017, Julien Grall wrote: >>> On 07/04/2017 08:31 AM, Bhupinder Thakur wrote: >>>> Hi Julien, >>> >>> Hi Bhupinder, >>> >>> Thank you for the summary! >>> >>> [...] >>>> >>>> Currently, UEFI firmware uses hvc as the console for input/output. Now >>>> with the support >>>> of SBSA UART in Xen, it is preferrable that UEFI firmware should be >>>> able to the uart >>>> as well. >>>> >>>> One option which was discussed was to use pl011 purely as a debug >>>> port. Currently the debug >>>> prints are intermixed with the normal console output. Now with uart >>>> port becoming available >>>> the debug prints can be redirected to pl011 thus cleaning up the >>>> console >>>> output. >>>> >>>> Other option is to output everything on both HVC and pl011 both but it >>>> takes away the advantage >>>> of separating out the debug and normal console prints. However, pl011 >>>> can be used as debug >>>> port based on a compile time flag. If this compile-time is off, then >>>> the output can be sent to both >>>> HVC and pl011. >>>> >>>> Based on this discussion I feel that: >>>> - the default behaviour should be writing the output to both HVC and >>>> pl011. >>> >>> Hmmm. If I remember correctly this was suggested but ruled out. It was >>> considered that pl011 and PV console should not be treated equal. >>> PL011 would >>> be used for boot diagnostics (i.e imagine an Image with no Xen support). >> >> Actually I remember the opposite: >> afd2e931-706b-6e25-1f0e-feee16e83c88@redhat.com (this was a private >> reply though). > > This was an answer to my question whether a user could select the serial > by himself. To this reply, you asked whether it was feasible to output > on all the serials console, but I don't see any yes/no answer. > > On the rest of the thread, it has been mentioned it was difficult to > multiplex to serial console (I forwarded you the thread). Christoffer, > Laszlo and Ard agreed that PL011 should only be used as boot diagnostics > and debug (if selected at compile time). Actually copying here as answer as Laszlo was happy to forward the answer on public list: (+CC Christoffer and Ard) * So, first of all, the debug port must be a super dumb device, available to the earliest firmware phases. It basically has to be a platform device, whose location and attributes can be figured out without hw discovery or enumeration. (Scanning the DTB is fine, albeit already quite laborious in the earliest phases.) PCI, virtio, or XenPV devices are unsuitable for this. * In OVMF (x86), we use the QEMU debug port for this purpose (hard-coding the accesses to IO port 0x402). For this, we have a specialized DebugLib instance, under "OvmfPkg/Library/PlatformDebugLibIoPort". * The serial port ("COM1") is used equivalently with the graphical display and the USB and PS/2 keyboard(s), for console purposes. The console(s) become available much later during firmware boot (only in the BDS phase). * In order for a device to be usable as a console, the driver stack must (recursively) provide the following two "higher level abstractions" on top of the device: - EfiSimpleTextInProtocol OR EfiSimpleTextInputExProtocol, - AND EfiSimpleTextOutProtocol If these are provided, then the console splitter / multiplexer mentioned by Ard will take care of the rest. * In OVMF (x86), this is the relevant protocol and driver stack for the "COM1" serial port (note that the necessary "high level abstractions" I mentioned above are at the bottom): MdeModulePkg/Bus/Pci/PciBusDxe ^ | [EfiPciIoProtocol] ^ | PcAtChipsetPkg/IsaAcpiDxe ^ | [EfiIsaAcpiProtocol] ^ | IntelFrameworkModulePkg/Bus/Isa/IsaBusDxe ^ | [EfiIsaIoProtocol] ^ | IntelFrameworkModulePkg/Bus/Isa/IsaSerialDxe ^ | [EfiSerialIoProtocol] ^ | Universal/Console/TerminalDxe ^ | [EfiSimpleTextInProtocol, EfiSimpleTextInputExProtocol, EfiSimpleTextOutProtocol] Most of this stack is platform dependent (up to and including IsaSerialDxe, which produces [EfiSerialIoProtocol]). The universal part starts only with TerminalDxe, which consumes [EfiSerialIoProtocol], and produces the needed higher level abstractions on top. * OVMF can be built (with -D DEBUG_ON_SERIAL_PORT) to ignore the QEMU debug port and to direct debug messages to the serial port instead. In that case, we use the following library instances, for directing debug messages to the serial port: - SerialPortLib: PcAtChipsetPkg/Library/SerialIoLib - DebugLib: MdePkg/Library/BaseDebugLibSerialPort The DebugLib instance uses SerialPortLib interfaces to print characters, and the SerialPortLib instance mentioned above hardcodes 0x3F8 as the "COM1" UART base address. This debug-on-serial build works, but once we're in the BDS phase, the serial output will be a mixture of console stuff and debug messages, because the two separate paths described above dump output to the exact same device. So the recommended (and default) build is to send DEBUGs to the QEMU debug port (redirecting them to a host side file), and to use "COM1" only for console purposes. * qemu-system-aarch64 has no "debug port", so in ArmVirtQemu we have something that can be compared to the (sub-optimal) "-D DEBUG_ON_SERIAL_PORT" build of OVMF. Namely, debug messages and console output are intermixed. * In particular, for the DEBUG messages, we use the following library instances: - PL011UartLib: ArmPlatformPkg/Drivers/PL011Uart/PL011Uart.inf - SerialPortLib [1]: ArmVirtPkg/Library/FdtPL011SerialPortLib/EarlyFdtPL011SerialPortLib.inf (in early firmware phases with no writeable RAM) - SerialPortLib [2]: ArmVirtPkg/Library/FdtPL011SerialPortLib/FdtPL011SerialPortLib.inf (later firmware phases with writeable RAM) - DebugLib: MdePkg/Library/BaseDebugLibSerialPort The DebugLib instance is the same as in OVMF's -D DEBUG_ON_SERIAL_PORT build, but the serial port APIs are filled in by different library instances. For the early firmware phases, we use EarlyFdtPL011SerialPortLib, which parses the PL011 location from the DTB on every single serial port API invocation. In later firmware phases, we use FdtPL011SerialPortLib, which can cache the parsed location in static variables. Finally, both SerialPortLib instances ask the same PL011UartLib to perform the actual PL011 accesses. * Now that we got the DEBUG message printing for ArmVirtQemu out of the way, let's look at how PL011 is used for console purposes. (Again, sharing the same PL011 between console and DEBUG messages is not optimal, but there is no separate debug port for the aarch64 target.) MdeModulePkg/Universal/SerialDxe ^ | [EfiSerialIoProtocol] ^ | Universal/Console/TerminalDxe ^ | [EfiSimpleTextInProtocol, EfiSimpleTextInputExProtocol, EfiSimpleTextOutProtocol] All the drivers in this stack are universal, and SerialDxe produces exactly one [EfiSerialIoProtocol] instance without consuming other protocol instances. The trick is that it delegates the actual work to the platform's SerialPortLib instance (which is linked into the SerialDxe executable). In ArmVirtQemu's case, this instance is the one marked above as "SerialPortLib [2]", which in turn pulls in PL011UartLib as well. * Considering the ArmVirtXen case. Both paths (DEBUG messages and console IO) are identical to those in ArmVirtQemu, except we use the following serial port library instance: - SerialPortLib: OvmfPkg/Library/XenConsoleSerialPortLib You might notice that we don't have two SerialPortLib instances here, one for "early" (RAM-less) phases and another for "late" (RAM-ful) phases, like we have in ArmVirtQemu. As far as I understand, the reason is that Xen guests lack the "early" (RAM-less) phases totally, and so we can get away with just a SerialPortLib instance that requires writeable RAM. This library instance performs Xen hypercalls. But, that makes no difference that the same SerialPortLib instance is used for *both* paths, namely console IO and DEBUG messages. * Now, assuming Xen gets another serial port, i.e. it'll have both emulated PL011 and the paravirt console device. Based on my experience with OVMF (where DEBUGs and console IO use different devices), I'd recommend to dedicate one device to DEBUG messages, and another to console IO. I would *not* recommend multiplexing UEFI console IO to both PL011 and the XenPV console -- simply because separating DEBUGs from console IO is much more important than that. So if you gain another, primitive enough serial port-like device, definitely dedicate it to this separation, IMHO! Whether you assign PL011 to DEBUGs and keep UEFI console IO (including grub2 IO, for example) on XenPV, or assign them the other way around, is a matter of taste (or standards), I guess. - For keeping the UEFI console IO on XenPV, and moving the DEBUG messages to the new PL011: implement a new DebugLib instance that grabs the PL011 location in a manner that is specific to Xen guests (my guess: simply open-code it?) and then delegates the transmit work to PL011UartLib. That's all. - For the inverse: add a new DebugLib instance that embeds XenConsoleSerialPortLib's functionality, and add a SerialPortLib instace -- a variant of FdtPL011SerialPortLib.inf -- that grabs the PL011 location in a Xen-specific way, and delegates the transmit work to PL011UartLib. Both of these involve the introduction of a DebugLib instance that does *not* depend on the SerialPortLib class. * Off the top of my head, I can't say how (and *whether*) this division of log devices in UEFI should be mirrored to the Linux guest OS as well. Earlier I made an effort to understand how Linux handled DBG2 versus SPCR versus... whatever, but I've forgotten all that by now. * If you *absolutely* want to multiplex both debug messages and console IO to both PL011 and the XenPV console, then a new DXE driver will be necessary that produces another EfiSerialIoProtocol instance, without going through the SerialPortLib class. This driver could be a clone of SerialDxe, but instead of consuming SerialPortLib (which we resolve to XenConsoleSerialPortLib in ArmVirtXen), it could be modified to: - grab the PL011 location in a Xen-specific way, - talk to PL011UartLib directly. Again, I do not recommend this; it would just duplicate the number of devices on which you'd get a mixture of DEBUGs and console IO. If Xen is gaining another serial port, use that opportunity to separate DEBUGs from console IO, like OVMF does. Which device is going to be used for which role is a matter of taste, or maybe it can be deduced from the relevant specs (ARM VM spec or maybe the SBBR). Thanks, Laszlo PS: pls feel free to fwd this to some public list so that others can comment should they want to.
On Wed, 5 Jul 2017, Julien Grall wrote: > On 05/07/2017 20:06, Stefano Stabellini wrote: > > On Wed, 5 Jul 2017, Julien Grall wrote: > > > On 07/04/2017 08:31 AM, Bhupinder Thakur wrote: > > > > Hi Julien, > > > > > > Hi Bhupinder, > > > > > > Thank you for the summary! > > > > > > [...] > > > > > > > > Currently, UEFI firmware uses hvc as the console for input/output. Now > > > > with the support > > > > of SBSA UART in Xen, it is preferrable that UEFI firmware should be > > > > able to the uart > > > > as well. > > > > > > > > One option which was discussed was to use pl011 purely as a debug > > > > port. Currently the debug > > > > prints are intermixed with the normal console output. Now with uart > > > > port becoming available > > > > the debug prints can be redirected to pl011 thus cleaning up the console > > > > output. > > > > > > > > Other option is to output everything on both HVC and pl011 both but it > > > > takes away the advantage > > > > of separating out the debug and normal console prints. However, pl011 > > > > can be used as debug > > > > port based on a compile time flag. If this compile-time is off, then > > > > the output can be sent to both > > > > HVC and pl011. > > > > > > > > Based on this discussion I feel that: > > > > - the default behaviour should be writing the output to both HVC and > > > > pl011. > > > > > > Hmmm. If I remember correctly this was suggested but ruled out. It was > > > considered that pl011 and PV console should not be treated equal. PL011 > > > would > > > be used for boot diagnostics (i.e imagine an Image with no Xen support). > > > > Actually I remember the opposite: > > afd2e931-706b-6e25-1f0e-feee16e83c88@redhat.com (this was a private > > reply though). > > This was an answer to my question whether a user could select the serial by > himself. To this reply, you asked whether it was feasible to output on all the > serials console, but I don't see any yes/no answer. > > On the rest of the thread, it has been mentioned it was difficult to multiplex > to serial console (I forwarded you the thread). Christoffer, Laszlo and Ard > agreed that PL011 should only be used as boot diagnostics and debug (if > selected at compile time). I see now that I missed an important part of the thread. Just follow Lazlo's suggestion.
On 05/07/2017 21:05, Stefano Stabellini wrote: > On Wed, 5 Jul 2017, Julien Grall wrote: >> On 05/07/2017 20:06, Stefano Stabellini wrote: >>> On Wed, 5 Jul 2017, Julien Grall wrote: >>>> On 07/04/2017 08:31 AM, Bhupinder Thakur wrote: >>>>> Hi Julien, >>>> >>>> Hi Bhupinder, >>>> >>>> Thank you for the summary! >>>> >>>> [...] >>>>> >>>>> Currently, UEFI firmware uses hvc as the console for input/output. Now >>>>> with the support >>>>> of SBSA UART in Xen, it is preferrable that UEFI firmware should be >>>>> able to the uart >>>>> as well. >>>>> >>>>> One option which was discussed was to use pl011 purely as a debug >>>>> port. Currently the debug >>>>> prints are intermixed with the normal console output. Now with uart >>>>> port becoming available >>>>> the debug prints can be redirected to pl011 thus cleaning up the console >>>>> output. >>>>> >>>>> Other option is to output everything on both HVC and pl011 both but it >>>>> takes away the advantage >>>>> of separating out the debug and normal console prints. However, pl011 >>>>> can be used as debug >>>>> port based on a compile time flag. If this compile-time is off, then >>>>> the output can be sent to both >>>>> HVC and pl011. >>>>> >>>>> Based on this discussion I feel that: >>>>> - the default behaviour should be writing the output to both HVC and >>>>> pl011. >>>> >>>> Hmmm. If I remember correctly this was suggested but ruled out. It was >>>> considered that pl011 and PV console should not be treated equal. PL011 >>>> would >>>> be used for boot diagnostics (i.e imagine an Image with no Xen support). >>> >>> Actually I remember the opposite: >>> afd2e931-706b-6e25-1f0e-feee16e83c88@redhat.com (this was a private >>> reply though). >> >> This was an answer to my question whether a user could select the serial by >> himself. To this reply, you asked whether it was feasible to output on all the >> serials console, but I don't see any yes/no answer. >> >> On the rest of the thread, it has been mentioned it was difficult to multiplex >> to serial console (I forwarded you the thread). Christoffer, Laszlo and Ard >> agreed that PL011 should only be used as boot diagnostics and debug (if >> selected at compile time). > > I see now that I missed an important part of the thread. Just follow > Lazlo's suggestion. Sorry for that. I forgot to include you from beginning. Cheers,
SBSA UART emulation for guests in Xen ====================================== Linaro has published VM System specification for ARM Processors, which provides a set of guidelines for both guest OS and hypervisor implementations, such that building OS images according to these guidelines guarantees that those images can also run on hypervisors compliant with this specification. One of the spec requirements is that the hypervisor must provide an emulated SBSA UART as a serial console which meets the minimum requirements in SBSA UART as defined in appendix B of the following ARM Server Base Architecture Document: https://static.docs.arm.com/den0029/a/Server_Base_System_Architecture_v3_1_ARM_DEN_0029A.pdf. This feature allows the Xen guests to use SBSA compliant pl011 UART as as a console. Note that SBSA pl011 UART is a subset of full featured ARM pl011 UART and supports only a subset of registers as mentioned below. It does not support rx/tx DMA. Currently, Xen supports paravirtualized (aka PV console) and an emulated serial consoles. This feature will expose an emulated SBSA pl011 UART console to the guest, which a user can access using xenconsole. The device tree passed to the guest VM will contain the pl011 MMIO address range and an irq for receiving rx/tx pl011 interrupts. The device tree format is specified in Documentation/devicetree/bindings/serial/arm_sbsa_uart.txt. The Xen hypervisor will expose two types of interfaces to the backend and domU. The interface exposed to domU will be an emulated pl011 UART by emulating the access to the following pl011 registers by the guest. - Data register (DR) - RW - Raw interrupt status register (RIS) - RO - Masked interrupt status register (MIS)- RO - Interrupt Mask (IMSC) - RW - Interrupt Clear (ICR) - WO It will also inject the pl011 interrupts to the guest in the following conditions: - incoming data in the rx buffer for the guest - there is space in the tx buffer for the guest to write more data The interface exposed to the backend will be the same PV console interface, which minimizes the changes required in xenconsole to support a new pl011 console. This interface has rx and tx ring buffers and an event channel for sending/receiving events from the backend. So essentially Xen handles the data on behalf of domU and the backend. Any data written by domU is captured by Xen and written to the TX (OUT) ring buffer and a pl011 event is raised to the backend to read the TX ring buffer. Similarly on reciving a pl011 event, Xen injects an interrupt to guest to indicate there is data available in the RX (IN) ring buffer. The pl011 UART state is completely captured in the set of registers mentioned above and this state is updated everytime there is an event from the backend or there is register read/write access from domU. For example, if domU has masked the rx interrupt in the IMSC register, then Xen will not inject an interrupt to guest and will just update the RIS register. Once the interrupt is unmasked by guest, the interrupt will be delivered to the guest. Changes summary: Xen Hypervisor =============== 1. Add emulation code to emulate read/write access to pl011 registers and pl011 interrupts: - It emulates DR read/write by reading and writing from/to the IN and OUT ring buffers and raising an event to dom0 when there is data in the OUT ring buffer and injecting an interrupt to the guest when there is data in the IN ring buffer. - Other registers are related to interrupt management and essentially control when interrupts are delivered to the guest. 2. Add a new domctl API to initialize vpl011 emulation in Xen. 3. Enable vpl011 emulation for a domain based on a libxl option passed during domain creation. Toolstack ========== 1. Add a new option "vuart" in the domU configuration file to enable/disable vuart. 2. Create a SBSA UART DT node in the guest device tree. It uses a fixed vpl011 SPI IRQ number and MMIO address. 3. Call vpl011 init DOMCTL API to enable vpl011 emulation. 5. Add a new vuart xenstore node, which contains: - ring-ref - event channel - buffer limit - type Xenconsoled ============ 1. Split the domain structure to support multiple consoles. 2. Modify different APIs such as buffer_append() etc. to operate on the console structure. 3. Add support for handling multiple consoles. 4. Add support for vuart console: The vpl011 changes available at the following repo: url: https://git@git.linaro.org:/people/bhupinder.thakur/xen.git branch: vpl011_v5 Kindly wait for one day to checkout the code from the above URL as I am not able to push my changes from my office network. I will push the changes once I am at home. There are some TBD items which need to be looked at in the future: 1. Currently UEFI firmware logs the output to hvc console only. How can UEFI firmware be made aware of pl011 console and how it can use it as a console instead of hvc. 2. Linux seems to have hvc console as the default console i.e. if no console is specified then it uses hvc as the console. How can an option be provided in Linux to select either hvc or pl011 as the default console. 3. ACPI support for pl011 device. CC: Ian Jackson <ian.jackson@eu.citrix.com> CC: Wei Liu <wei.liu2@citrix.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Julien Grall <julien.grall@arm.com> CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Bhupinder Thakur (17): xen/arm: vpl011: Move vgic register access functions to vreg.h xen/arm: vpl011: Rename vgic_reg* functions definitions and calls to vreg_reg* xen/arm: vpl011: Define common ring buffer helper functions in console.h xen/arm: vpl011: Add SBSA UART emulation in Xen xen/arm: vpl011: Allocate a new GFN in the toolstack for vuart xen/arm: vpl011: Add support for vuart in libxl xen/arm: vpl011: Rearrange xen header includes in alphabetical order in domctl.c xen/arm: vpl011: Add a new domctl API to initialize vpl011 xen/arm: vpl011: Add a new vuart node in the xenstore xen/arm: vpl011: Modify xenconsole to define and use a new console structure xen/arm: vpl011: Rename the console structure field conspath to xspath xen/arm: vpl011: Modify xenconsole functions to take console structure as input xen/arm: vpl011: Modify xenconsole to support multiple consoles xen/arm: vpl011: Add support for vuart console in xenconsole xen/arm: vpl011: Add a new vuart console type to xenconsole client xen/arm: vpl011: Add a pl011 uart DT node in the guest device tree xen/arm: vpl011: Update documentation for vuart console support config/arm32.mk | 1 + config/arm64.mk | 1 + docs/man/xl.cfg.pod.5.in | 9 + docs/misc/console.txt | 44 ++- tools/console/Makefile | 3 +- tools/console/client/main.c | 13 +- tools/console/daemon/io.c | 658 +++++++++++++++++++++++------------ tools/libxc/include/xc_dom.h | 2 + tools/libxc/include/xenctrl.h | 20 ++ tools/libxc/xc_dom_arm.c | 5 +- tools/libxc/xc_dom_boot.c | 2 + tools/libxc/xc_domain.c | 25 ++ tools/libxl/libxl.h | 6 + tools/libxl/libxl_arch.h | 6 + tools/libxl/libxl_arm.c | 74 +++- tools/libxl/libxl_console.c | 47 +++ tools/libxl/libxl_create.c | 10 +- tools/libxl/libxl_device.c | 9 +- tools/libxl/libxl_dom.c | 5 + tools/libxl/libxl_internal.h | 6 + tools/libxl/libxl_types.idl | 7 + tools/libxl/libxl_types_internal.idl | 1 + tools/libxl/libxl_x86.c | 8 + tools/xl/xl_cmdtable.c | 2 +- tools/xl/xl_console.c | 5 +- tools/xl/xl_parse.c | 8 + xen/arch/arm/Kconfig | 7 + xen/arch/arm/Makefile | 1 + xen/arch/arm/domain.c | 5 + xen/arch/arm/domctl.c | 41 ++- xen/arch/arm/vgic-v2.c | 28 +- xen/arch/arm/vgic-v3-its.c | 28 +- xen/arch/arm/vgic-v3.c | 50 +-- xen/arch/arm/vpl011.c | 449 ++++++++++++++++++++++++ xen/include/asm-arm/domain.h | 6 + xen/include/asm-arm/pl011-uart.h | 2 + xen/include/asm-arm/vgic.h | 111 +----- xen/include/asm-arm/vpl011.h | 73 ++++ xen/include/asm-arm/vreg.h | 110 ++++++ xen/include/public/arch-arm.h | 6 + xen/include/public/domctl.h | 12 + xen/include/public/io/console.h | 4 + 42 files changed, 1507 insertions(+), 403 deletions(-) create mode 100644 xen/arch/arm/vpl011.c create mode 100644 xen/include/asm-arm/vpl011.h