| Message ID | 20241023120839.437771-1-mikko.rapeli@linaro.org |
|---|---|
| Headers | show |
| Series | systemd uki support | expand |
On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via lists.openembedded.org wrote: > These changes enable building systemd uki images which combine > kernel, kernel command line, initrd and possibly signatures to > a single UEFI binary. This binary can be booted with UEFI firmware > and systemd-boot. No grub is needed and UEFI firmware and/or > systemd-boot provide possibilities for boot menus. > The uki binary can also be signed for UEFI secure boot > so the secure boot extends from firmware to kernel and initrd. > Binding secure boot to full userspace is then easier since for example > kernel command line and initrd contain the support needed to mount > encrypted dm-verity etc partitions, and/or create partitions on demand > with systemd-repart using device specific TPM devices for encryption. > > Tested on qemuarm64-secureboot machine from meta-arm with changes to > support secure boot. Slightly different configuration tested on > multiple arm64 System Ready boards with UEFI firmware, real and firmware > based TPM devices. Tested with ovmf firmware on x86_64 with selftests but > without secure boot which seems to be harder to setup in ovmf. > > Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu and > wic.Wic2.test_expand_mbr_image, failing when executing all wic selftests > on a build machine with zfs filesystem. Will investigate this further. > The issue seems to be in mkfs.ext4 producing broken filesystem, and partially > in the tests which don't run the correct rootfs file (.ext4 vs .wic). > Will debug this further and it is IMO unrelated to these changes since > they reproduce on pure master branch without this series. > > v10: disabled kvm support in new tests since it breaks qemu boot on aarch64 > build machine, removed "testimage" from IMAGE_CLASS as well since > can end up testing qemu machine during build. I hate to say this but wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is still failing: wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 :( (I know there is another failure in there too). Cheers, Richard
On Wed, 2024-10-23 at 17:44 +0100, Richard Purdie via lists.openembedded.org wrote: > On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via > lists.openembedded.org wrote: > > These changes enable building systemd uki images which combine > > kernel, kernel command line, initrd and possibly signatures to > > a single UEFI binary. This binary can be booted with UEFI firmware > > and systemd-boot. No grub is needed and UEFI firmware and/or > > systemd-boot provide possibilities for boot menus. > > The uki binary can also be signed for UEFI secure boot > > so the secure boot extends from firmware to kernel and initrd. > > Binding secure boot to full userspace is then easier since for > > example > > kernel command line and initrd contain the support needed to mount > > encrypted dm-verity etc partitions, and/or create partitions on > > demand > > with systemd-repart using device specific TPM devices for > > encryption. > > > > Tested on qemuarm64-secureboot machine from meta-arm with changes > > to > > support secure boot. Slightly different configuration tested on > > multiple arm64 System Ready boards with UEFI firmware, real and > > firmware > > based TPM devices. Tested with ovmf firmware on x86_64 with > > selftests but > > without secure boot which seems to be harder to setup in ovmf. > > > > Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu > > and > > wic.Wic2.test_expand_mbr_image, failing when executing all wic > > selftests > > on a build machine with zfs filesystem. Will investigate this > > further. > > The issue seems to be in mkfs.ext4 producing broken filesystem, and > > partially > > in the tests which don't run the correct rootfs file (.ext4 vs > > .wic). > > Will debug this further and it is IMO unrelated to these changes > > since > > they reproduce on pure master branch without this series. > > > > v10: disabled kvm support in new tests since it breaks qemu boot on > > aarch64 > > build machine, removed "testimage" from IMAGE_CLASS as well > > since > > can end up testing qemu machine during build. > > I hate to say this but > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is still > failing: > https://valkyrie.yoctoproject.org/#/builders/23/builds/320/steps/14/logs/stdio > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 > > :( > > (I know there is another failure in there too). > > Cheers, > > Richard > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#206255): > https://lists.openembedded.org/g/openembedded-core/message/206255 > Mute This Topic: https://lists.openembedded.org/mt/109169005/1686473 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub [ > richard.purdie@linuxfoundation.org] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, 2024-10-23 at 17:58 +0100, Richard Purdie via lists.openembedded.org wrote: > On Wed, 2024-10-23 at 17:44 +0100, Richard Purdie via > lists.openembedded.org wrote: > > On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via > > lists.openembedded.org wrote: > > > These changes enable building systemd uki images which combine > > > kernel, kernel command line, initrd and possibly signatures to > > > a single UEFI binary. This binary can be booted with UEFI > > > firmware > > > and systemd-boot. No grub is needed and UEFI firmware and/or > > > systemd-boot provide possibilities for boot menus. > > > The uki binary can also be signed for UEFI secure boot > > > so the secure boot extends from firmware to kernel and initrd. > > > Binding secure boot to full userspace is then easier since for > > > example > > > kernel command line and initrd contain the support needed to > > > mount > > > encrypted dm-verity etc partitions, and/or create partitions on > > > demand > > > with systemd-repart using device specific TPM devices for > > > encryption. > > > > > > Tested on qemuarm64-secureboot machine from meta-arm with changes > > > to > > > support secure boot. Slightly different configuration tested on > > > multiple arm64 System Ready boards with UEFI firmware, real and > > > firmware > > > based TPM devices. Tested with ovmf firmware on x86_64 with > > > selftests but > > > without secure boot which seems to be harder to setup in ovmf. > > > > > > Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu > > > and > > > wic.Wic2.test_expand_mbr_image, failing when executing all wic > > > selftests > > > on a build machine with zfs filesystem. Will investigate this > > > further. > > > The issue seems to be in mkfs.ext4 producing broken filesystem, > > > and > > > partially > > > in the tests which don't run the correct rootfs file (.ext4 vs > > > .wic). > > > Will debug this further and it is IMO unrelated to these changes > > > since > > > they reproduce on pure master branch without this series. > > > > > > v10: disabled kvm support in new tests since it breaks qemu boot > > > on > > > aarch64 > > > build machine, removed "testimage" from IMAGE_CLASS as well > > > since > > > can end up testing qemu machine during build. > > > > I hate to say this but > > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is still > > failing: > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/320/steps/14/logs/stdio and: https://valkyrie.yoctoproject.org/#/builders/23/builds/323/steps/14/logs/stdio which is clearer without the other failure. Cheers, Richard
Hi, On Wed, Oct 23, 2024 at 07:39:51PM +0100, Richard Purdie wrote: > On Wed, 2024-10-23 at 17:58 +0100, Richard Purdie via > lists.openembedded.org wrote: > > On Wed, 2024-10-23 at 17:44 +0100, Richard Purdie via > > lists.openembedded.org wrote: > > > On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via > > > lists.openembedded.org wrote: > > > > These changes enable building systemd uki images which combine > > > > kernel, kernel command line, initrd and possibly signatures to > > > > a single UEFI binary. This binary can be booted with UEFI > > > > firmware > > > > and systemd-boot. No grub is needed and UEFI firmware and/or > > > > systemd-boot provide possibilities for boot menus. > > > > The uki binary can also be signed for UEFI secure boot > > > > so the secure boot extends from firmware to kernel and initrd. > > > > Binding secure boot to full userspace is then easier since for > > > > example > > > > kernel command line and initrd contain the support needed to > > > > mount > > > > encrypted dm-verity etc partitions, and/or create partitions on > > > > demand > > > > with systemd-repart using device specific TPM devices for > > > > encryption. > > > > > > > > Tested on qemuarm64-secureboot machine from meta-arm with changes > > > > to > > > > support secure boot. Slightly different configuration tested on > > > > multiple arm64 System Ready boards with UEFI firmware, real and > > > > firmware > > > > based TPM devices. Tested with ovmf firmware on x86_64 with > > > > selftests but > > > > without secure boot which seems to be harder to setup in ovmf. > > > > > > > > Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu > > > > and > > > > wic.Wic2.test_expand_mbr_image, failing when executing all wic > > > > selftests > > > > on a build machine with zfs filesystem. Will investigate this > > > > further. > > > > The issue seems to be in mkfs.ext4 producing broken filesystem, > > > > and > > > > partially > > > > in the tests which don't run the correct rootfs file (.ext4 vs > > > > .wic). > > > > Will debug this further and it is IMO unrelated to these changes > > > > since > > > > they reproduce on pure master branch without this series. > > > > > > > > v10: disabled kvm support in new tests since it breaks qemu boot > > > > on > > > > aarch64 > > > > build machine, removed "testimage" from IMAGE_CLASS as well > > > > since > > > > can end up testing qemu machine during build. > > > > > > I hate to say this but > > > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is still > > > failing: > > > > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/320/steps/14/logs/stdio > > and: > > https://valkyrie.yoctoproject.org/#/builders/23/builds/323/steps/14/logs/stdio > > which is clearer without the other failure. There are severe issues[1] running aarch64 target image and firmware via qemu on an aarch64 build machine. Specs of the machine would be good to know. The test works on x86_64 build machine for targets qemuarm64 and genericarm64, and qemux86_64 with a different test using edk2/ovmf. All wic, uki and efibootpartition tests pass on x86_64 build host. The failure is in u-boot and qemu on aarch64 and have nothing to do with high level systemd and Unified Kernel Image support which these patches setup. Ross, why can't I mark the test for x86_64 build hosts for now? v9 of the series has it and could be applied right now. I'm in the process of setting up an aarch64 machine for yocto builds so I can debug this issue and run tests. Right now I simply can't reproduce, debug or fix the issue. Cheers, -Mikko [1] https://valkyrie.yoctoproject.org/#/builders/23/builds/323/steps/14/logs/stdio 2024-10-23 20:28:56,148 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/utils/commands.py", line 375, in runqemu qemu.start(params=qemuparams, ssh=ssh, runqemuparams=runqemuparams, launch_cmd=launch_cmd, discard_writes=discard_writes) File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/targetcontrol.py", line 179, in start raise RuntimeError("%s - FAILED to start qemu - check the task log and the boot log" % self.pn) RuntimeError: core-image-base - FAILED to start qemu - check the task log and the boot log During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 1379, in test_efi_plugin_plain_systemd_boot_qemu_aarch64 with runqemu('core-image-base', ssh=False, File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/utils/commands.py", line 381, in runqemu raise Exception(msg) Exception: core-image-base - FAILED to start qemu - check the task log and the boot log Failed to start QEMU - see the logs in /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimageQemurunner log output from /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/qemurunner_log.20241023173919: INFO: rootfs file: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.wic INFO: Qemu log file: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/qemu_boot_log.20241023173919 INFO: SSH log file: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/ssh_target_log.20241023173919 DEBUG: Using kvm for runqemu INFO: QMP Available for connection at /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/.a8wewvdb DEBUG: Created listening socket for qemu serial console on: 127.0.0.1:60105 DEBUG: Created listening socket for qemu serial console on: 127.0.0.1:37965 DEBUG: launchcmd=runqemu snapshot kvm nographic qemuarm64 /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.wic tcpserial=60105:37965 bootparams=" printk.time=1" qemuparams="-pidfile /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/pidfile_1891679 -S -qmp unix:./.358r3f9e,server,wait -qmp unix:./.a8wewvdb,server,nowait" DEBUG: runqemu started, pid is 2886855 DEBUG: waiting at most 300 seconds for qemu pid (10/23/24 19:39:27) DEBUG: QMP Initializing to /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/.358r3f9e DEBUG: QMP Connecting to /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/.358r3f9e INFO: QMP connected to QEMU at 10/23/24 19:39:28 and took 0.56 seconds INFO: QMP released QEMU at 10/23/24 19:39:28 and took 0.09 seconds from connect DEBUG: qemu started in 0.65 seconds - qemu procces pid is 2887295 (10/23/24 19:39:28) DEBUG: Target IP: None DEBUG: Server IP: None DEBUG: Starting logging thread DEBUG: Starting thread event loop DEBUG: Connection request received DEBUG: Setting connection established event DEBUG: Output from runqemu: runqemu - INFO - Continuing with the following parameters: MACHINE: [qemuarm64] FSTYPE: [wic] ROOTFS: [/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.wic] CONFFILE: [/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.qemuboot.conf] runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock failed: [Errno 11] Resource temporarily unavailable runqemu - INFO - Using preconfigured tap device tap1 runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap1.skip to make runqemu skip tap1. runqemu - INFO - Network configuration: ip=192.168.7.4::192.168.7.3:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 runqemu - INFO - Copying rootfs to /home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896 runqemu - INFO - Copy done in 0.17237091064453125 seconds runqemu - INFO - Running /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/aarch64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-aarch64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:04 -netdev tap,id=net0,ifname=tap1,script=no,downscript=no -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -drive if=virtio,file=/home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896,format=raw -device qemu-xhci -device usb-tablet -device usb-kbd -machine virt -cpu host -machine gic-version=3 -smp 4 -enable-kvm -m 256 -device virtio-serial-pci -chardev socket,id=virtcon,port=60105,host=127.0.0.1,nodelay=on -device virtconsole,chardev=virtcon -serial tcp:127.0.0.1:37965,nodelay=on -pidfile /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/pidfile_1891679 -S -qmp unix:./.358r3f9e,server,wait -qmp unix:./.a8wewvdb,server,nowait -serial null -serial mon:stdio -nographic -device virtio-gpu-pci -bios /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/u-boot.bin runqemu - INFO - Host uptime: 2757248.55 QEMU 9.1.0 monitor - type 'help' for more information (qemu) DEBUG: Waiting at most 1000 seconds for login banner (10/23/24 19:39:28) DEBUG: Connection from 127.0.0.1:49726 WARNING: Target didn't reach login banner in 1000 seconds (10/23/24 19:56:08) WARNING: Last 25 lines of login console (2035481): Flash: 64 MiB Loading Environment from Flash... *** Warning - bad CRC, using default environment In: serial,usbkbd Out: serial,vidconsole Err: serial,vidconsole Bus xhci_pci: Register 8001040 NbrPorts 8 Starting the controller "Synchronous Abort" handler, esr 0x96000010, far 0x10088040 elr: 0000000000057c7c lr : 0000000000057c60 (reloc) elr: 000000004f71dc7c lr : 000000004f71dc60 x0 : 0000000010088040 x1 : 0000000000000001 x2 : 0000000000000000 x3 : 0000000000003e80 x4 : 0000000000000038 x5 : 000000004e585792 x6 : 0000000000000000 x7 : 0000000000000000 x8 : 000000004e585c20 x9 : 00000000ffffffd8 x10: 000000000000000d x11: 0000000000000006 x12: 000000004e585a78 x13: 000000004e585d90 x14: 0000000000000000 x15: 000000004e585792 x16: 000000004f710788 x17: 0000000000000000 x18: 000000004e685d90 x19: 000000004e68cc00 x20: 0000000000000000 x21: 0000000010088040 x22: 0000000010088000 x23: 000000004f7925a9 x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000 x27: 0000000000000000 WARNING: Last 25 lines of all logging (2035481): Flash: 64 MiB Loading Environment from Flash... *** Warning - bad CRC, using default environment In: serial,usbkbd Out: serial,vidconsole Err: serial,vidconsole Bus xhci_pci: Register 8001040 NbrPorts 8 Starting the controller "Synchronous Abort" handler, esr 0x96000010, far 0x10088040 elr: 0000000000057c7c lr : 0000000000057c60 (reloc) elr: 000000004f71dc7c lr : 000000004f71dc60 x0 : 0000000010088040 x1 : 0000000000000001 x2 : 0000000000000000 x3 : 0000000000003e80 x4 : 0000000000000038 x5 : 000000004e585792 x6 : 0000000000000000 x7 : 0000000000000000 x8 : 000000004e585c20 x9 : 00000000ffffffd8 x10: 000000000000000d x11: 0000000000000006 x12: 000000004e585a78 x13: 000000004e585d90 x14: 0000000000000000 x15: 000000004e585792 x16: 000000004f710788 x17: 0000000000000000 x18: 000000004e685d90 x19: 000000004e68cc00 x20: 0000000000000000 x21: 0000000010088040 x22: 0000000010088000 x23: 000000004f7925a9 x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000 x27: 0000000000000000 WARNING: Check full boot log: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/qemu_boot_log.20241023173919 DEBUG: Stopping logging thread DEBUG: Stop event received DEBUG: Tearing down logging thread DEBUG: Sending SIGTERM to runqemu INFO: Output from runqemu: runqemu - INFO - Received signal: 15 runqemu - INFO - Cleaning up runqemu - INFO - Host uptime: 2758254.35 runqemu - INFO - Removing /home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896 tput: No value for $TERM and no -T specified WARNING: Extra log data read: x28: 0000000000000000 x29: 000000004e585c20 Code: d5033fbf aa1503e0 5287d003 52800002 (b8004401) Resetting CPU ... resetting ... WARNING: Extra log data read: DEBUG: Sending SIGTERM to runqemu INFO: Output from runqemu: runqemu - INFO - Received signal: 15 runqemu - INFO - Cleaning up runqemu - INFO - Host uptime: 2758254.35 runqemu - INFO - Removing /home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896 tput: No value for $TERM and no -T specified Stderr: 2024-10-23 18:55:56,461 - oe-selftest - INFO - Adding: "include selftest.inc" in /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/conf/local.conf 2024-10-23 18:55:56,462 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", NOTE: Reconnecting to bitbake server... WARNING: /usr/lib/python3.12/unittest/case.py:580: RuntimeWarning: TestResult has no addDuration method warnings.warn("TestResult has no addDuration method", Loading cache...done. Removing 1 recipes from the allarch sysroot...done. Removing 2 recipes from the cortexa57 sysroot...done. Removing 5 recipes from the qemuarm64 sysroot...done. rootfs file: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.wic Qemu log file: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/qemu_boot_log.20241023173919 SSH log file: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/ssh_target_log.20241023173919 Using kvm for runqemu QMP Available for connection at /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/.a8wewvdb Created listening socket for qemu serial console on: 127.0.0.1:60105 Created listening socket for qemu serial console on: 127.0.0.1:37965 launchcmd=runqemu snapshot kvm nographic qemuarm64 /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.wic tcpserial=60105:37965 bootparams=" printk.time=1" qemuparams="-pidfile /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/pidfile_1891679 -S -qmp unix:./.358r3f9e,server,wait -qmp unix:./.a8wewvdb,server,nowait" runqemu started, pid is 2886855 waiting at most 300 seconds for qemu pid (10/23/24 19:39:27) QMP Initializing to /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/.358r3f9e QMP Connecting to /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/.358r3f9e QMP connected to QEMU at 10/23/24 19:39:28 and took 0.56 seconds QMP released QEMU at 10/23/24 19:39:28 and took 0.09 seconds from connect qemu started in 0.65 seconds - qemu procces pid is 2887295 (10/23/24 19:39:28) Target IP: None Server IP: None Starting logging thread Starting thread event loop Connection request received Setting connection established event Output from runqemu: runqemu - INFO - Continuing with the following parameters: MACHINE: [qemuarm64] FSTYPE: [wic] ROOTFS: [/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.wic] CONFFILE: [/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/core-image-base-qemuarm64.rootfs.qemuboot.conf] runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock failed: [Errno 11] Resource temporarily unavailable runqemu - INFO - Using preconfigured tap device tap1 runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap1.skip to make runqemu skip tap1. runqemu - INFO - Network configuration: ip=192.168.7.4::192.168.7.3:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 runqemu - INFO - Copying rootfs to /home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896 runqemu - INFO - Copy done in 0.17237091064453125 seconds runqemu - INFO - Running /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/aarch64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-aarch64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:04 -netdev tap,id=net0,ifname=tap1,script=no,downscript=no -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -drive if=virtio,file=/home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896,format=raw -device qemu-xhci -device usb-tablet -device usb-kbd -machine virt -cpu host -machine gic-version=3 -smp 4 -enable-kvm -m 256 -device virtio-serial-pci -chardev socket,id=virtcon,port=60105,host=127.0.0.1,nodelay=on -device virtconsole,chardev=virtcon -serial tcp:127.0.0.1:37965,nodelay=on -pidfile /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/pidfile_1891679 -S -qmp unix:./.358r3f9e,server,wait -qmp unix:./.a8wewvdb,server,nowait -serial null -serial mon:stdio -nographic -device virtio-gpu-pci -bios /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/deploy/images/qemuarm64/u-boot.bin runqemu - INFO - Host uptime: 2757248.55 QEMU 9.1.0 monitor - type 'help' for more information (qemu) Waiting at most 1000 seconds for login banner (10/23/24 19:39:28) Connection from 127.0.0.1:49726 Target didn't reach login banner in 1000 seconds (10/23/24 19:56:08) Last 25 lines of login console (2035481): Flash: 64 MiB Loading Environment from Flash... *** Warning - bad CRC, using default environment In: serial,usbkbd Out: serial,vidconsole Err: serial,vidconsole Bus xhci_pci: Register 8001040 NbrPorts 8 Starting the controller "Synchronous Abort" handler, esr 0x96000010, far 0x10088040 elr: 0000000000057c7c lr : 0000000000057c60 (reloc) elr: 000000004f71dc7c lr : 000000004f71dc60 x0 : 0000000010088040 x1 : 0000000000000001 x2 : 0000000000000000 x3 : 0000000000003e80 x4 : 0000000000000038 x5 : 000000004e585792 x6 : 0000000000000000 x7 : 0000000000000000 x8 : 000000004e585c20 x9 : 00000000ffffffd8 x10: 000000000000000d x11: 0000000000000006 x12: 000000004e585a78 x13: 000000004e585d90 x14: 0000000000000000 x15: 000000004e585792 x16: 000000004f710788 x17: 0000000000000000 x18: 000000004e685d90 x19: 000000004e68cc00 x20: 0000000000000000 x21: 0000000010088040 x22: 0000000010088000 x23: 000000004f7925a9 x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000 x27: 0000000000000000 Last 25 lines of all logging (2035481): Flash: 64 MiB Loading Environment from Flash... *** Warning - bad CRC, using default environment In: serial,usbkbd Out: serial,vidconsole Err: serial,vidconsole Bus xhci_pci: Register 8001040 NbrPorts 8 Starting the controller "Synchronous Abort" handler, esr 0x96000010, far 0x10088040 elr: 0000000000057c7c lr : 0000000000057c60 (reloc) elr: 000000004f71dc7c lr : 000000004f71dc60 x0 : 0000000010088040 x1 : 0000000000000001 x2 : 0000000000000000 x3 : 0000000000003e80 x4 : 0000000000000038 x5 : 000000004e585792 x6 : 0000000000000000 x7 : 0000000000000000 x8 : 000000004e585c20 x9 : 00000000ffffffd8 x10: 000000000000000d x11: 0000000000000006 x12: 000000004e585a78 x13: 000000004e585d90 x14: 0000000000000000 x15: 000000004e585792 x16: 000000004f710788 x17: 0000000000000000 x18: 000000004e685d90 x19: 000000004e68cc00 x20: 0000000000000000 x21: 0000000010088040 x22: 0000000010088000 x23: 000000004f7925a9 x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000 x27: 0000000000000000 Check full boot log: /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1891679/tmp/work/qemuarm64-poky-linux/core-image-base/1.0/testimage/qemu_boot_log.20241023173919 Stopping logging thread Stop event received Tearing down logging thread Sending SIGTERM to runqemu Output from runqemu: runqemu - INFO - Received signal: 15 runqemu - INFO - Cleaning up runqemu - INFO - Host uptime: 2758254.35 runqemu - INFO - Removing /home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896 tput: No value for $TERM and no -T specified Extra log data read: x28: 0000000000000000 x29: 000000004e585c20 Code: d5033fbf aa1503e0 5287d003 52800002 (b8004401) Resetting CPU ... resetting ... Extra log data read: Sending SIGTERM to runqemu Output from runqemu: runqemu - INFO - Received signal: 15 runqemu - INFO - Cleaning up runqemu - INFO - Host uptime: 2758254.35 runqemu - INFO - Removing /home/pokybuild/tmp/core-image-base-qemuarm64.rootfs.wic.2886896 tput: No value for $TERM and no -T specified ---------------------------------------------------------------------- 2024-10-23 20:28:56,148 - oe-selftest - INFO - Ran 609 tests in 5579.874s 2024-10-23 20:28:56,148 - oe-selftest - INFO - FAILED
On Wed, Oct 23, 2024 at 07:39:51PM +0100, Richard Purdie wrote: > On Wed, 2024-10-23 at 17:58 +0100, Richard Purdie via > lists.openembedded.org wrote: > > On Wed, 2024-10-23 at 17:44 +0100, Richard Purdie via > > lists.openembedded.org wrote: > > > On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via > > > lists.openembedded.org wrote: > > > > These changes enable building systemd uki images which combine > > > > kernel, kernel command line, initrd and possibly signatures to > > > > a single UEFI binary. This binary can be booted with UEFI > > > > firmware > > > > and systemd-boot. No grub is needed and UEFI firmware and/or > > > > systemd-boot provide possibilities for boot menus. > > > > The uki binary can also be signed for UEFI secure boot > > > > so the secure boot extends from firmware to kernel and initrd. > > > > Binding secure boot to full userspace is then easier since for > > > > example > > > > kernel command line and initrd contain the support needed to > > > > mount > > > > encrypted dm-verity etc partitions, and/or create partitions on > > > > demand > > > > with systemd-repart using device specific TPM devices for > > > > encryption. > > > > > > > > Tested on qemuarm64-secureboot machine from meta-arm with changes > > > > to > > > > support secure boot. Slightly different configuration tested on > > > > multiple arm64 System Ready boards with UEFI firmware, real and > > > > firmware > > > > based TPM devices. Tested with ovmf firmware on x86_64 with > > > > selftests but > > > > without secure boot which seems to be harder to setup in ovmf. > > > > > > > > Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu > > > > and > > > > wic.Wic2.test_expand_mbr_image, failing when executing all wic > > > > selftests > > > > on a build machine with zfs filesystem. Will investigate this > > > > further. > > > > The issue seems to be in mkfs.ext4 producing broken filesystem, > > > > and > > > > partially > > > > in the tests which don't run the correct rootfs file (.ext4 vs > > > > .wic). > > > > Will debug this further and it is IMO unrelated to these changes > > > > since > > > > they reproduce on pure master branch without this series. > > > > > > > > v10: disabled kvm support in new tests since it breaks qemu boot > > > > on > > > > aarch64 > > > > ���� build machine, removed "testimage" from IMAGE_CLASS as well > > > > since > > > > ���� can end up testing qemu machine during build. > > > > > > I hate to say this but > > > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is still > > > failing: > > > > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/320/steps/14/logs/stdio > > and: > > https://valkyrie.yoctoproject.org/#/builders/23/builds/323/steps/14/logs/stdio > > which is clearer without the other failure. Comparing x86_64 and aarch64 build host runqemu command lines from wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 selftest, two things pop up. aarch64 still enables KVM via "-enable-kvm". This is despite QEMU_USE_KVM = "" in the bitbake build config. Oh, this is only applied to the build configuration and bitbake build command but removed before runqemu is called. I can try to apply this config also for runqemu. This pattern is used in several tests. Build config is set temporarily and then removed before calling runqemu. I'll send a separate patch to master-next to reduce spam. aarch64 host uses "-cpu host" with qemu while x86_64 sets the CPU variant explicitly to "-cpu cortex-a76". I can't see from build logs which CPU variant the host really is. There are a lot of CPU variants in aarch64 world and I don't think they are all compatible, or detect CPU features at runtime which can impact things like firmware code badly. I don't know how to fix this. Cheers, -Mikko
On Thu, 2024-10-24 at 11:55 +0300, Mikko Rapeli wrote: > On Wed, Oct 23, 2024 at 07:39:51PM +0100, Richard Purdie wrote: > > On Wed, 2024-10-23 at 17:58 +0100, Richard Purdie via > > lists.openembedded.org wrote: > > > On Wed, 2024-10-23 at 17:44 +0100, Richard Purdie via > > > lists.openembedded.org wrote: > > > > On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via > > > > lists.openembedded.org wrote: > > > > > These changes enable building systemd uki images which combine > > > > > kernel, kernel command line, initrd and possibly signatures to > > > > > a single UEFI binary. This binary can be booted with UEFI > > > > > firmware > > > > > and systemd-boot. No grub is needed and UEFI firmware and/or > > > > > systemd-boot provide possibilities for boot menus. > > > > > The uki binary can also be signed for UEFI secure boot > > > > > so the secure boot extends from firmware to kernel and initrd. > > > > > Binding secure boot to full userspace is then easier since for > > > > > example > > > > > kernel command line and initrd contain the support needed to > > > > > mount > > > > > encrypted dm-verity etc partitions, and/or create partitions on > > > > > demand > > > > > with systemd-repart using device specific TPM devices for > > > > > encryption. > > > > > > > > > > Tested on qemuarm64-secureboot machine from meta-arm with changes > > > > > to > > > > > support secure boot. Slightly different configuration tested on > > > > > multiple arm64 System Ready boards with UEFI firmware, real and > > > > > firmware > > > > > based TPM devices. Tested with ovmf firmware on x86_64 with > > > > > selftests but > > > > > without secure boot which seems to be harder to setup in ovmf. > > > > > > > > > > Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu > > > > > and > > > > > wic.Wic2.test_expand_mbr_image, failing when executing all wic > > > > > selftests > > > > > on a build machine with zfs filesystem. Will investigate this > > > > > further. > > > > > The issue seems to be in mkfs.ext4 producing broken filesystem, > > > > > and > > > > > partially > > > > > in the tests which don't run the correct rootfs file (.ext4 vs > > > > > .wic). > > > > > Will debug this further and it is IMO unrelated to these changes > > > > > since > > > > > they reproduce on pure master branch without this series. > > > > > > > > > > v10: disabled kvm support in new tests since it breaks qemu boot > > > > > on > > > > > aarch64 > > > > > build machine, removed "testimage" from IMAGE_CLASS as well > > > > > since > > > > > can end up testing qemu machine during build. > > > > > > > > I hate to say this but > > > > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is still > > > > failing: > > > > > > > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/320/steps/14/logs/stdio > > > > and: > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/323/steps/14/logs/stdio > > > > which is clearer without the other failure. > > Comparing x86_64 and aarch64 build host runqemu command lines from > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 selftest, > two things pop up. > > aarch64 still enables KVM via "-enable-kvm". This is despite QEMU_USE_KVM = "" > in the bitbake build config. Oh, this is only applied to the build configuration > and bitbake build command but removed before runqemu is called. I can try to apply > this config also for runqemu. This pattern is used in several tests. Build config > is set temporarily and then removed before calling runqemu. I'll send a separate > patch to master-next to reduce spam. > > aarch64 host uses "-cpu host" with qemu while x86_64 sets the CPU variant explicitly > to "-cpu cortex-a76". I can't see from build logs which CPU variant the > host really is. There are a lot of CPU variants in aarch64 world and I don't think > they are all compatible, or detect CPU features at runtime which can impact > things like firmware code badly. I don't know how to fix this. Well spotted! I'm running a test build with your patch: https://valkyrie.yoctoproject.org/#/builders/23/builds/329 The CPU info is: $ cat /proc/cpuinfo processor : 0 BogoMIPS : 50.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x3 CPU part : 0xd0c CPU revision : 1 $ lscpu Architecture: aarch64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 80 On-line CPU(s) list: 0-79 Vendor ID: ARM Model name: Neoverse-N1 Model: 1 Thread(s) per core: 1 Core(s) per socket: 80 Socket(s): 1 Stepping: r3p1 Frequency boost: disabled CPU max MHz: 3000.0000 CPU min MHz: 1000.0000 BogoMIPS: 50.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs Caches (sum of all): L1d: 5 MiB (80 instances) L1i: 5 MiB (80 instances) L2: 80 MiB (80 instances) NUMA: NUMA node(s): 1 NUMA node0 CPU(s): 0-79 Vulnerabilities: Gather data sampling: Not affected Itlb multihit: Not affected L1tf: Not affected Mds: Not affected Meltdown: Not affected Mmio stale data: Not affected Reg file data sampling: Not affected Retbleed: Not affected Spec rstack overflow: Not affected Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Spectre v1: Mitigation; __user pointer sanitization Spectre v2: Mitigation; CSV2, BHB Srbds: Not affected Tsx async abort: Not affected
On Thu, 2024-10-24 at 10:22 +0100, Richard Purdie via lists.openembedded.org wrote: > On Thu, 2024-10-24 at 11:55 +0300, Mikko Rapeli wrote: > > On Wed, Oct 23, 2024 at 07:39:51PM +0100, Richard Purdie wrote: > > > On Wed, 2024-10-23 at 17:58 +0100, Richard Purdie via > > > lists.openembedded.org wrote: > > > > On Wed, 2024-10-23 at 17:44 +0100, Richard Purdie via > > > > lists.openembedded.org wrote: > > > > > On Wed, 2024-10-23 at 15:08 +0300, Mikko Rapeli via > > > > > lists.openembedded.org wrote: > > > > > > These changes enable building systemd uki images which > > > > > > combine > > > > > > kernel, kernel command line, initrd and possibly signatures > > > > > > to > > > > > > a single UEFI binary. This binary can be booted with UEFI > > > > > > firmware > > > > > > and systemd-boot. No grub is needed and UEFI firmware > > > > > > and/or > > > > > > systemd-boot provide possibilities for boot menus. > > > > > > The uki binary can also be signed for UEFI secure boot > > > > > > so the secure boot extends from firmware to kernel and > > > > > > initrd. > > > > > > Binding secure boot to full userspace is then easier since > > > > > > for > > > > > > example > > > > > > kernel command line and initrd contain the support needed > > > > > > to > > > > > > mount > > > > > > encrypted dm-verity etc partitions, and/or create > > > > > > partitions on > > > > > > demand > > > > > > with systemd-repart using device specific TPM devices for > > > > > > encryption. > > > > > > > > > > > > Tested on qemuarm64-secureboot machine from meta-arm with > > > > > > changes > > > > > > to > > > > > > support secure boot. Slightly different configuration > > > > > > tested on > > > > > > multiple arm64 System Ready boards with UEFI firmware, real > > > > > > and > > > > > > firmware > > > > > > based TPM devices. Tested with ovmf firmware on x86_64 with > > > > > > selftests but > > > > > > without secure boot which seems to be harder to setup in > > > > > > ovmf. > > > > > > > > > > > > Sadly I see two wic selftests, > > > > > > wic.Wic2.test_rawcopy_plugin_qemu > > > > > > and > > > > > > wic.Wic2.test_expand_mbr_image, failing when executing all > > > > > > wic > > > > > > selftests > > > > > > on a build machine with zfs filesystem. Will investigate > > > > > > this > > > > > > further. > > > > > > The issue seems to be in mkfs.ext4 producing broken > > > > > > filesystem, > > > > > > and > > > > > > partially > > > > > > in the tests which don't run the correct rootfs file (.ext4 > > > > > > vs > > > > > > .wic). > > > > > > Will debug this further and it is IMO unrelated to these > > > > > > changes > > > > > > since > > > > > > they reproduce on pure master branch without this series. > > > > > > > > > > > > v10: disabled kvm support in new tests since it breaks qemu > > > > > > boot > > > > > > on > > > > > > aarch64 > > > > > > build machine, removed "testimage" from IMAGE_CLASS as > > > > > > well > > > > > > since > > > > > > can end up testing qemu machine during build. > > > > > > > > > > I hate to say this but > > > > > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 is > > > > > still > > > > > failing: > > > > > > > > > > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/320/steps/14/logs/stdio > > > > > > and: > > > > > > https://valkyrie.yoctoproject.org/#/builders/23/builds/323/steps/14/logs/stdio > > > > > > which is clearer without the other failure. > > > > Comparing x86_64 and aarch64 build host runqemu command lines from > > wic.Wic2.test_efi_plugin_plain_systemd_boot_qemu_aarch64 selftest, > > two things pop up. > > > > aarch64 still enables KVM via "-enable-kvm". This is despite > > QEMU_USE_KVM = "" > > in the bitbake build config. Oh, this is only applied to the build > > configuration > > and bitbake build command but removed before runqemu is called. I > > can try to apply > > this config also for runqemu. This pattern is used in several > > tests. Build config > > is set temporarily and then removed before calling runqemu. I'll > > send a separate > > patch to master-next to reduce spam. > > > > aarch64 host uses "-cpu host" with qemu while x86_64 sets the CPU > > variant explicitly > > to "-cpu cortex-a76". I can't see from build logs which CPU variant > > the > > host really is. There are a lot of CPU variants in aarch64 world > > and I don't think > > they are all compatible, or detect CPU features at runtime which > > can impact > > things like firmware code badly. I don't know how to fix this. > > Well spotted! I'm running a test build with your patch: > > https://valkyrie.yoctoproject.org/#/builders/23/builds/329 Good news is that one passed, thanks for working through this! Cheers, Richard
Hi Mikko, On Wed Oct 23, 2024 at 2:08 PM CEST, Mikko Rapeli wrote: > These changes enable building systemd uki images which combine > kernel, kernel command line, initrd and possibly signatures to > a single UEFI binary. This binary can be booted with UEFI firmware > and systemd-boot. No grub is needed and UEFI firmware and/or > systemd-boot provide possibilities for boot menus. > The uki binary can also be signed for UEFI secure boot > so the secure boot extends from firmware to kernel and initrd. > Binding secure boot to full userspace is then easier since for example > kernel command line and initrd contain the support needed to mount > encrypted dm-verity etc partitions, and/or create partitions on demand > with systemd-repart using device specific TPM devices for encryption. Now that this class has made it into master, we need to document it in https://git.yoctoproject.org/yocto-docs. Would you be able to help writing some documentation about this class and the related variables it defines? It would need to be part of documentation/ref-manual/classes.rst, and the variables would need to be documented in documentation/ref-manual/variables.rst. Regards, Antonin -- Antonin Godard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hi, On Mon, Nov 04, 2024 at 03:42:10PM +0100, Antonin Godard wrote: > Hi Mikko, > > On Wed Oct 23, 2024 at 2:08 PM CEST, Mikko Rapeli wrote: > > These changes enable building systemd uki images which combine > > kernel, kernel command line, initrd and possibly signatures to > > a single UEFI binary. This binary can be booted with UEFI firmware > > and systemd-boot. No grub is needed and UEFI firmware and/or > > systemd-boot provide possibilities for boot menus. > > The uki binary can also be signed for UEFI secure boot > > so the secure boot extends from firmware to kernel and initrd. > > Binding secure boot to full userspace is then easier since for example > > kernel command line and initrd contain the support needed to mount > > encrypted dm-verity etc partitions, and/or create partitions on demand > > with systemd-repart using device specific TPM devices for encryption. > > Now that this class has made it into master, we need to document it in > https://git.yoctoproject.org/yocto-docs. Would you be able to help writing some > documentation about this class and the related variables it defines? It would > need to be part of documentation/ref-manual/classes.rst, and the variables would > need to be documented in documentation/ref-manual/variables.rst. Sure, on my todo. Feel free to send something if I don't get into it in time. Struggling with some trivialities currently: rm_work wiping rootfs and wic creating and empty rootfs instead, breaking builds and tests and hindering debugging... Cheers, -Mikko
Hi Mikko, On Mon Nov 4, 2024 at 3:46 PM CET, Mikko Rapeli wrote: [...] >> Now that this class has made it into master, we need to document it in >> https://git.yoctoproject.org/yocto-docs. Would you be able to help writing some >> documentation about this class and the related variables it defines? It would >> need to be part of documentation/ref-manual/classes.rst, and the variables would >> need to be documented in documentation/ref-manual/variables.rst. > > Sure, on my todo. Feel free to send something if I don't get into it in > time. Struggling with some trivialities currently: rm_work wiping rootfs > and wic creating and empty rootfs instead, breaking builds and tests > and hindering debugging... I've created a bug to track the missing doc bits: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15650 Regards, Antonin
These changes enable building systemd uki images which combine kernel, kernel command line, initrd and possibly signatures to a single UEFI binary. This binary can be booted with UEFI firmware and systemd-boot. No grub is needed and UEFI firmware and/or systemd-boot provide possibilities for boot menus. The uki binary can also be signed for UEFI secure boot so the secure boot extends from firmware to kernel and initrd. Binding secure boot to full userspace is then easier since for example kernel command line and initrd contain the support needed to mount encrypted dm-verity etc partitions, and/or create partitions on demand with systemd-repart using device specific TPM devices for encryption. Tested on qemuarm64-secureboot machine from meta-arm with changes to support secure boot. Slightly different configuration tested on multiple arm64 System Ready boards with UEFI firmware, real and firmware based TPM devices. Tested with ovmf firmware on x86_64 with selftests but without secure boot which seems to be harder to setup in ovmf. Sadly I see two wic selftests, wic.Wic2.test_rawcopy_plugin_qemu and wic.Wic2.test_expand_mbr_image, failing when executing all wic selftests on a build machine with zfs filesystem. Will investigate this further. The issue seems to be in mkfs.ext4 producing broken filesystem, and partially in the tests which don't run the correct rootfs file (.ext4 vs .wic). Will debug this further and it is IMO unrelated to these changes since they reproduce on pure master branch without this series. v10: disabled kvm support in new tests since it breaks qemu boot on aarch64 build machine, removed "testimage" from IMAGE_CLASS as well since can end up testing qemu machine during build. v9: Fixed wic test on genericarm64. Disabled new aarch64 wic test on aarch64 build host since boot with qemu is currently failing. Tested wic, uki and efibootpartition selftests with target machines qemux86_64, qemuarm64 and genericarm64 on x86_64 build host. v8: fixed comments from Ross Burton: debug print from warning to debug, dropped duplicate DISTRO_FEATURE setting for systemd in tests, removed aarch64 comment from tests which are currently x86 only. Fixed the new aarch64 wic selftest to run on both genericarm64 and qemuarm64 by adding bios, virtio disk driver etc settings for runqemu (already set in genericarm64 but missing from qemuarm64). v7: add missing "ovmf" to runqemu argument to test_efi_plugin_plain_systemd_boot_qemu_x86 to fix boot hang v6: fixes wic refactoring botch which broken non-uki systemd-boot usage on genericarm64 reported by Ross Burton <Ross.Burton@arm.com>, added selftest to cover this wks usage on x86 and aarch64 v5: drop patch "image_types_wic.bbclass: set systemd-boot and os-release dependency for all archs" since systemd-boot does not support all architectures v4: handle missing runqemu variable from build config, add python3-pefile to fast ptest list v3: rebased, fixed and added more sefltests, removed wic plugin side uki support v2: https://lists.openembedded.org/g/openembedded-core/message/204090 *** BLURB HERE *** Michelle Lin (1): uki.bbclass: add class for building Unified Kernel Images (UKI) Mikko Rapeli (8): wic bootimg-efi.py: keep timestamps and add debug prints wic bootimg-efi.py: change UKI support from wic plugin to uki.bbclass oeqa selftest uki.py: add tests for uki.bbclass oeqa selftest efibootpartition.py: add TEST_RUNQEMUPARAMS to runqemu oeqa selftest efibootpartition.py: remove systemd-boot from grub-efi test oeqa selftest wic.py: add TEST_RUNQEMUPARAMS to runqemu oeqa selftest wic.py: support UKIs via uki.bbclass oeqa selftest wic.py: fix missing ext4 image meta-selftest/wic/test_efi_plugin.wks | 5 +- .../test_efi_plugin_plain_systemd-boot.wks | 5 + meta/classes-recipe/image_types_wic.bbclass | 3 +- meta/classes-recipe/uki.bbclass | 195 ++++++++++++++++++ .../oeqa/selftest/cases/efibootpartition.py | 11 +- meta/lib/oeqa/selftest/cases/uki.py | 142 +++++++++++++ meta/lib/oeqa/selftest/cases/wic.py | 140 +++++++++++-- .../wic/canned-wks/efi-uki-bootdisk.wks.in | 3 + scripts/lib/wic/plugins/source/bootimg-efi.py | 183 +++++----------- 9 files changed, 529 insertions(+), 158 deletions(-) create mode 100644 meta-selftest/wic/test_efi_plugin_plain_systemd-boot.wks create mode 100644 meta/classes-recipe/uki.bbclass create mode 100644 meta/lib/oeqa/selftest/cases/uki.py create mode 100644 scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in