[v3] tests/functional/aarch64: add tests for FEAT_RME

This boot an OP-TEE environment, and launch a nested guest VM inside it
using the Realms feature. We do it for virt and sbsa-ref platforms.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>

-----

v2:
- move test to its own file
- add sbsa test
- check output of `cca-workload-attestation report`

v3:
- build and run test with cca-v4 images
- factorize nested guest test between both tests
- remove accel tcg option as it is the default when running tests
Note: It's a long test and there is a work in progress to understand why
debug build is so slow (x12 vs optimized).

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
---
 tests/functional/meson.build                 |   4 +
 tests/functional/test_aarch64_rme_sbsaref.py |  70 +++++++++++++
 tests/functional/test_aarch64_rme_virt.py    | 100 +++++++++++++++++++
 3 files changed, 174 insertions(+)
 create mode 100755 tests/functional/test_aarch64_rme_sbsaref.py
 create mode 100755 tests/functional/test_aarch64_rme_virt.py

On Mon, Dec 02, 2024 at 04:17:41PM -0800, Pierrick Bouvier wrote:
> This boot an OP-TEE environment, and launch a nested guest VM inside it
> using the Realms feature. We do it for virt and sbsa-ref platforms.
> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> 
> -----
> 
> v2:
> - move test to its own file
> - add sbsa test
> - check output of `cca-workload-attestation report`
> 
> v3:
> - build and run test with cca-v4 images
> - factorize nested guest test between both tests
> - remove accel tcg option as it is the default when running tests
> Note: It's a long test and there is a work in progress to understand why
> debug build is so slow (x12 vs optimized).

Can the functional tests take as long as we want, or are they regularly
run by a CI?

The `cca-workload-attestation report` command is great for making sure the
RME feature is working and the guest is running in a Realm, and it's very
light. In my opinion it's sufficient here.

We could also run kvm-unit-tests on the host, various VM payloads and
configs (in particular edk2 boot), and performing local attestation, but
those would all be testing the other software components, so don't seem
relevant to this functional test of the QEMU emulator. That said we do
need regression tests for the whole software stack (including QEMU VMM)
and this test automation is really convenient so I'd like to extend it
with extra tests but outside the QEMU codebase.

Thanks,
Jean

> 
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> ---
>  tests/functional/meson.build                 |   4 +
>  tests/functional/test_aarch64_rme_sbsaref.py |  70 +++++++++++++
>  tests/functional/test_aarch64_rme_virt.py    | 100 +++++++++++++++++++
>  3 files changed, 174 insertions(+)
>  create mode 100755 tests/functional/test_aarch64_rme_sbsaref.py
>  create mode 100755 tests/functional/test_aarch64_rme_virt.py
> 
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> index 5c048cfac6d..b975a1560df 100644
> --- a/tests/functional/meson.build
> +++ b/tests/functional/meson.build
> @@ -13,6 +13,8 @@ endif
>  test_timeouts = {
>    'aarch64_aspeed' : 600,
>    'aarch64_raspi4' : 480,
> +  'aarch64_rme_virt' : 720,
> +  'aarch64_rme_sbsaref' : 720,
>    'aarch64_sbsaref_alpine' : 720,
>    'aarch64_sbsaref_freebsd' : 720,
>    'aarch64_tuxrun' : 240,
> @@ -52,6 +54,8 @@ tests_aarch64_system_thorough = [
>    'aarch64_aspeed',
>    'aarch64_raspi3',
>    'aarch64_raspi4',
> +  'aarch64_rme_virt',
> +  'aarch64_rme_sbsaref',
>    'aarch64_sbsaref',
>    'aarch64_sbsaref_alpine',
>    'aarch64_sbsaref_freebsd',
> diff --git a/tests/functional/test_aarch64_rme_sbsaref.py b/tests/functional/test_aarch64_rme_sbsaref.py
> new file mode 100755
> index 00000000000..398562c166f
> --- /dev/null
> +++ b/tests/functional/test_aarch64_rme_sbsaref.py
> @@ -0,0 +1,70 @@
> +#!/usr/bin/env python3
> +#
> +# Functional test that boots a Realms environment on sbsa-ref machine and a
> +# nested guest VM using it.
> +#
> +# Copyright (c) 2024 Linaro Ltd.
> +#
> +# Author: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +import time
> +import os
> +import logging
> +
> +from qemu_test import QemuSystemTest, Asset
> +from qemu_test import exec_command, wait_for_console_pattern
> +from qemu_test import exec_command_and_wait_for_pattern
> +from qemu_test.utils import archive_extract
> +from test_aarch64_rme_virt import test_realms_guest
> +
> +class Aarch64RMESbsaRefMachine(QemuSystemTest):
> +
> +    # Stack is built with OP-TEE build environment from those instructions:
> +    # https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/
> +    # https://github.com/pbo-linaro/qemu-rme-stack
> +    ASSET_RME_STACK_SBSA = Asset(
> +        ('https://fileserver.linaro.org/s/KJyeBxL82mz2r7F/'
> +         'download/rme-stack-op-tee-4.2.0-cca-v4-sbsa.tar.gz'),
> +         'dd9ab28ec869bdf3b5376116cb3689103b43433fd5c4bca0f4a8d8b3c104999e')
> +
> +    # This tests the FEAT_RME cpu implementation, by booting a VM supporting it,
> +    # and launching a nested VM using it.
> +    def test_aarch64_rme_sbsaref(self):
> +        stack_path_tar_gz = self.ASSET_RME_STACK_SBSA.fetch()
> +        archive_extract(stack_path_tar_gz, self.workdir)
> +
> +        self.set_machine('sbsa-ref')
> +        self.vm.set_console()
> +        self.require_accelerator('tcg')
> +
> +        rme_stack = os.path.join(self.workdir,
> +                                 'rme-stack-op-tee-4.2.0-cca-v4-sbsa')
> +        pflash0 = os.path.join(rme_stack, 'images', 'SBSA_FLASH0.fd')
> +        pflash1 = os.path.join(rme_stack, 'images', 'SBSA_FLASH1.fd')
> +        virtual = os.path.join(rme_stack, 'images', 'disks', 'virtual')
> +        drive = os.path.join(rme_stack, 'out-br', 'images', 'rootfs.ext4')
> +
> +        self.vm.add_args('-cpu', 'max,x-rme=on')
> +        self.vm.add_args('-m', '2G')
> +        self.vm.add_args('-M', 'sbsa-ref')
> +        self.vm.add_args('-drive', f'file={pflash0},format=raw,if=pflash')
> +        self.vm.add_args('-drive', f'file={pflash1},format=raw,if=pflash')
> +        self.vm.add_args('-drive', f'file=fat:rw:{virtual},format=raw')
> +        self.vm.add_args('-drive', f'format=raw,if=none,file={drive},id=hd0')
> +        self.vm.add_args('-device', 'virtio-blk-pci,drive=hd0')
> +        self.vm.add_args('-device', 'virtio-9p-pci,fsdev=shr0,mount_tag=shr0')
> +        self.vm.add_args('-fsdev', f'local,security_model=none,path={rme_stack},id=shr0')
> +        self.vm.add_args('-device', 'virtio-net-pci,netdev=net0')
> +        self.vm.add_args('-netdev', 'user,id=net0')
> +
> +        self.vm.launch()
> +        # Wait for host VM boot to complete.
> +        wait_for_console_pattern(self, 'Welcome to Buildroot')
> +        exec_command_and_wait_for_pattern(self, 'root', '#')
> +
> +        test_realms_guest(self)
> +
> +if __name__ == '__main__':
> +    QemuSystemTest.main()
> diff --git a/tests/functional/test_aarch64_rme_virt.py b/tests/functional/test_aarch64_rme_virt.py
> new file mode 100755
> index 00000000000..be240c691c2
> --- /dev/null
> +++ b/tests/functional/test_aarch64_rme_virt.py
> @@ -0,0 +1,100 @@
> +#!/usr/bin/env python3
> +#
> +# Functional test that boots a Realms environment on virt machine and a nested
> +# guest VM using it.
> +#
> +# Copyright (c) 2024 Linaro Ltd.
> +#
> +# Author: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +import time
> +import os
> +import logging
> +
> +from qemu_test import QemuSystemTest, Asset
> +from qemu_test import exec_command, wait_for_console_pattern
> +from qemu_test import exec_command_and_wait_for_pattern
> +from qemu_test.utils import archive_extract
> +
> +def test_realms_guest(test_rme_instance):
> +
> +    # Boot the (nested) guest VM
> +    exec_command(test_rme_instance,
> +                 'qemu-system-aarch64 -M virt,gic-version=3 '
> +                 '-cpu host -enable-kvm -m 512M '
> +                 '-M confidential-guest-support=rme0 '
> +                 '-object rme-guest,id=rme0 '
> +                 '-device virtio-net-pci,netdev=net0,romfile= '
> +                 '-netdev user,id=net0 '
> +                 '-kernel /mnt/out/bin/Image '
> +                 '-initrd /mnt/out-br/images/rootfs.cpio '
> +                 '-serial stdio')
> +    # Detect Realm activation during (nested) guest boot.
> +    wait_for_console_pattern(test_rme_instance,
> +                             'SMC_RMI_REALM_ACTIVATE')
> +    # Wait for (nested) guest boot to complete.
> +    wait_for_console_pattern(test_rme_instance,
> +                             'Welcome to Buildroot')
> +    exec_command_and_wait_for_pattern(test_rme_instance, 'root', '#')
> +    # query (nested) guest cca report
> +    exec_command(test_rme_instance, 'cca-workload-attestation report')
> +    wait_for_console_pattern(test_rme_instance,
> +                             '"cca-platform-hash-algo-id": "sha-256"')
> +    wait_for_console_pattern(test_rme_instance,
> +                             '"cca-realm-hash-algo-id": "sha-512"')
> +    wait_for_console_pattern(test_rme_instance,
> +                             '"cca-realm-public-key-hash-algo-id": "sha-256"')
> +
> +class Aarch64RMEVirtMachine(QemuSystemTest):
> +
> +    # Stack is built with OP-TEE build environment from those instructions:
> +    # https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/
> +    # https://github.com/pbo-linaro/qemu-rme-stack
> +    ASSET_RME_STACK_VIRT = Asset(
> +        ('https://fileserver.linaro.org/s/iaRsNDJp2CXHMSJ/'
> +         'download/rme-stack-op-tee-4.2.0-cca-v4-qemu_v8.tar.gz'),
> +         '1851adc232b094384d8b879b9a2cfff07ef3d6205032b85e9b3a4a9ae6b0b7ad')
> +
> +    # This tests the FEAT_RME cpu implementation, by booting a VM supporting it,
> +    # and launching a nested VM using it.
> +    def test_aarch64_rme_virt(self):
> +        stack_path_tar_gz = self.ASSET_RME_STACK_VIRT.fetch()
> +        archive_extract(stack_path_tar_gz, self.workdir)
> +
> +        self.set_machine('virt')
> +        self.vm.set_console()
> +        self.require_accelerator('tcg')
> +
> +        rme_stack = os.path.join(self.workdir,
> +                                 'rme-stack-op-tee-4.2.0-cca-v4-qemu_v8')
> +        kernel = os.path.join(rme_stack, 'out', 'bin', 'Image')
> +        bios = os.path.join(rme_stack, 'out', 'bin', 'flash.bin')
> +        drive = os.path.join(rme_stack, 'out-br', 'images', 'rootfs.ext4')
> +
> +        self.vm.add_args('-cpu', 'max,x-rme=on')
> +        self.vm.add_args('-m', '2G')
> +        self.vm.add_args('-M', 'virt,acpi=off,'
> +                         'virtualization=on,'
> +                         'secure=on,'
> +                         'gic-version=3')
> +        self.vm.add_args('-bios', bios)
> +        self.vm.add_args('-kernel', kernel)
> +        self.vm.add_args('-drive', f'format=raw,if=none,file={drive},id=hd0')
> +        self.vm.add_args('-device', 'virtio-blk-pci,drive=hd0')
> +        self.vm.add_args('-device', 'virtio-9p-device,fsdev=shr0,mount_tag=shr0')
> +        self.vm.add_args('-fsdev', f'local,security_model=none,path={rme_stack},id=shr0')
> +        self.vm.add_args('-device', 'virtio-net-pci,netdev=net0')
> +        self.vm.add_args('-netdev', 'user,id=net0')
> +        self.vm.add_args('-append', 'root=/dev/vda')
> +
> +        self.vm.launch()
> +        # Wait for host VM boot to complete.
> +        wait_for_console_pattern(self, 'Welcome to Buildroot')
> +        exec_command_and_wait_for_pattern(self, 'root', '#')
> +
> +        test_realms_guest(self)
> +
> +if __name__ == '__main__':
> +    QemuSystemTest.main()
> -- 
> 2.39.5
>

Hi Jean-Philippe,
(adding Daniel and Thomas as well)

On 12/3/24 07:54, Jean-Philippe Brucker wrote:
> On Mon, Dec 02, 2024 at 04:17:41PM -0800, Pierrick Bouvier wrote:
>> This boot an OP-TEE environment, and launch a nested guest VM inside it
>> using the Realms feature. We do it for virt and sbsa-ref platforms.
>>
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>>
>> -----
>>
>> v2:
>> - move test to its own file
>> - add sbsa test
>> - check output of `cca-workload-attestation report`
>>
>> v3:
>> - build and run test with cca-v4 images
>> - factorize nested guest test between both tests
>> - remove accel tcg option as it is the default when running tests
>> Note: It's a long test and there is a work in progress to understand why
>> debug build is so slow (x12 vs optimized).
> 
> Can the functional tests take as long as we want, or are they regularly
> run by a CI?
> 

It is run by the CI and developers as part of functional tests (make 
check-functional). The shorter the better thus.
Adding more (shorter) tests would be acceptable though, as they can be 
parallelized.

> The `cca-workload-attestation report` command is great for making sure the
> RME feature is working and the guest is running in a Realm, and it's very
> light. In my opinion it's sufficient here.
> 
> We could also run kvm-unit-tests on the host, various VM payloads and
> configs (in particular edk2 boot), and performing local attestation, but
> those would all be testing the other software components, so don't seem
> relevant to this functional test of the QEMU emulator. That said we do
> need regression tests for the whole software stack (including QEMU VMM)
> and this test automation is really convenient so I'd like to extend it
> with extra tests but outside the QEMU codebase.
>

I saw that kvm-unit-tests are mentioned in our documentation, but I'm 
not sure if we have any functional test image running them.

I'm not familiar enough with this topic to know what would be the value 
to add such tests, or edk2 boot.
That said, if we can have extra (short) tests upstream, I don't think it 
would hurt anyone.

For your need when working on an RME stack, it may be easier to keep 
those specific tests on your side.

Someone on this thread will probably have a better answer than mine :).

> Thanks,
> Jean
>