diff mbox series

[3/3] oeqa selftest uboot.py: add qemu KVM test case

Message ID 20250522134116.254270-3-mikko.rapeli@linaro.org
State New
Headers show
Series [v3,1/3] u-boot: disable CONFIG_BLOBLIST on genericarm64 and qemuarm64 | expand

Commit Message

Mikko Rapeli May 22, 2025, 1:41 p.m. UTC
Add a test case to boot target system via u-boot
using qemu with KVM. This was broken recently
and workaround proposed to u-boot. Test case
works with genericarm64 and qemuarm64 target machines
compiled and tested on aarch64 build host with KVM
support.

Test execution time with full sstate cache is
around 170 seconds. qemu boot itself takes just
a few seconds to full userspace.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/selftest/cases/uboot.py | 55 ++++++++++++++++++++++++++-
 1 file changed, 53 insertions(+), 2 deletions(-)

Comments

Mathieu Dubois-Briand May 23, 2025, 1:15 p.m. UTC | #1
On Thu May 22, 2025 at 3:41 PM CEST, Mikko Rapeli via lists.openembedded.org wrote:
> Add a test case to boot target system via u-boot
> using qemu with KVM. This was broken recently
> and workaround proposed to u-boot. Test case
> works with genericarm64 and qemuarm64 target machines
> compiled and tested on aarch64 build host with KVM
> support.
>
> Test execution time with full sstate cache is
> around 170 seconds. qemu boot itself takes just
> a few seconds to full userspace.
>
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---

Hi Mikko,

I saw your other mail saying you will make an update, but as I had
already picked these patches, here is the build result: the selftest is
failing on armhost:

---
2025-05-23 08:50:18,542 - oe-selftest - INFO - uboot.UBootTest.test_boot_uboot_kvm_to_full_target (subunit.RemotedTestCase)
2025-05-23 08:50:18,543 - oe-selftest - INFO -  ... ERROR
...
QEMU 10.0.0 monitor - type 'help' for more information
(qemu)
Waiting at most 1000 seconds for login banner (05/23/25 08:33:29)
Connection from 127.0.0.1:43786
Target didn't reach login banner in 1000 seconds (05/23/25 08:50:12)
Last 25 lines of login console (1721):
Scanning bootdev 'virtio-blk#36.bootdev':
Scanning bootdev 'virtio-net#32.bootdev':
BOOTP broadcast 1
...
BOOTP broadcast 17

Retry time exceeded; starting again
No more bootdevs
---  -----------  ------  --------  ----  ------------------------  ----------------
(1 bootflow, 1 valid)
=>
...
RuntimeError: core-image-minimal - FAILED to start qemu - check the task log and the boot log
---

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1749
Mikko Rapeli May 23, 2025, 2:16 p.m. UTC | #2
Hi,

On Fri, May 23, 2025 at 03:15:51PM +0200, Mathieu Dubois-Briand wrote:
> On Thu May 22, 2025 at 3:41 PM CEST, Mikko Rapeli via lists.openembedded.org wrote:
> > Add a test case to boot target system via u-boot
> > using qemu with KVM. This was broken recently
> > and workaround proposed to u-boot. Test case
> > works with genericarm64 and qemuarm64 target machines
> > compiled and tested on aarch64 build host with KVM
> > support.
> >
> > Test execution time with full sstate cache is
> > around 170 seconds. qemu boot itself takes just
> > a few seconds to full userspace.
> >
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> > ---
> 
> Hi Mikko,
> 
> I saw your other mail saying you will make an update, but as I had
> already picked these patches, here is the build result: the selftest is
> failing on armhost:
> 
> ---
> 2025-05-23 08:50:18,542 - oe-selftest - INFO - uboot.UBootTest.test_boot_uboot_kvm_to_full_target (subunit.RemotedTestCase)
> 2025-05-23 08:50:18,543 - oe-selftest - INFO -  ... ERROR
> ...
> QEMU 10.0.0 monitor - type 'help' for more information
> (qemu)
> Waiting at most 1000 seconds for login banner (05/23/25 08:33:29)
> Connection from 127.0.0.1:43786
> Target didn't reach login banner in 1000 seconds (05/23/25 08:50:12)
> Last 25 lines of login console (1721):
> Scanning bootdev 'virtio-blk#36.bootdev':
> Scanning bootdev 'virtio-net#32.bootdev':
> BOOTP broadcast 1
> ...
> BOOTP broadcast 17
> 
> Retry time exceeded; starting again
> No more bootdevs
> ---  -----------  ------  --------  ----  ------------------------  ----------------
> (1 bootflow, 1 valid)
> =>
> ...
> RuntimeError: core-image-minimal - FAILED to start qemu - check the task log and the boot log
> ---
> 
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1749

Thanks, I will take a look. Boot to u-boot seems to ok so KVM is working on
the aarch64 build machine but something in wic image ESP partition detection is not
working in the same way as on my aarch64 build machine and qemuarm64 and
genericarm64 configs.

I will compare the runqemu and qemu-system-aarch64 arguments to figure out what is
different.

One problem I thought I solved was to change from virtio scsi to virtio blk emulation
but that seems to be effecttive albeit with a warning:

https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2468510/raw_inline

runqemu - WARNING - Unknown QB_DRIVE_TYPE: vd
runqemu - WARNING - Failed to figure out drive type, consider define or fix QB_DRIVE_TYPE
runqemu - WARNING - Trying to use virtio block drive

u-boot defconfig does not have scsi support but virtio blk works for disk with ESP
partition in fat format.

Cheers,

-Mikko
Mikko Rapeli May 26, 2025, 8:45 a.m. UTC | #3
Hi,

On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
> On Fri, May 23, 2025 at 03:15:51PM +0200, Mathieu Dubois-Briand wrote:
> > On Thu May 22, 2025 at 3:41 PM CEST, Mikko Rapeli via lists.openembedded.org wrote:
> > > Add a test case to boot target system via u-boot
> > > using qemu with KVM. This was broken recently
> > > and workaround proposed to u-boot. Test case
> > > works with genericarm64 and qemuarm64 target machines
> > > compiled and tested on aarch64 build host with KVM
> > > support.
> > >
> > > Test execution time with full sstate cache is
> > > around 170 seconds. qemu boot itself takes just
> > > a few seconds to full userspace.
> > >
> > > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> > > ---
> > 
> > Hi Mikko,
> > 
> > I saw your other mail saying you will make an update, but as I had
> > already picked these patches, here is the build result: the selftest is
> > failing on armhost:
> > 
> > ---
> > 2025-05-23 08:50:18,542 - oe-selftest - INFO - uboot.UBootTest.test_boot_uboot_kvm_to_full_target (subunit.RemotedTestCase)
> > 2025-05-23 08:50:18,543 - oe-selftest - INFO -  ... ERROR
> > ...
> > QEMU 10.0.0 monitor - type 'help' for more information
> > (qemu)
> > Waiting at most 1000 seconds for login banner (05/23/25 08:33:29)
> > Connection from 127.0.0.1:43786
> > Target didn't reach login banner in 1000 seconds (05/23/25 08:50:12)
> > Last 25 lines of login console (1721):
> > Scanning bootdev 'virtio-blk#36.bootdev':
> > Scanning bootdev 'virtio-net#32.bootdev':
> > BOOTP broadcast 1
> > ...
> > BOOTP broadcast 17
> > 
> > Retry time exceeded; starting again
> > No more bootdevs
> > ---  -----------  ------  --------  ----  ------------------------  ----------------
> > (1 bootflow, 1 valid)
> > =>
> > ...
> > RuntimeError: core-image-minimal - FAILED to start qemu - check the task log and the boot log
> > ---
> > 
> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1749
> 
> Thanks, I will take a look. Boot to u-boot seems to ok so KVM is working on
> the aarch64 build machine but something in wic image ESP partition detection is not
> working in the same way as on my aarch64 build machine and qemuarm64 and
> genericarm64 configs.
> 
> I will compare the runqemu and qemu-system-aarch64 arguments to figure out what is
> different.
> 
> One problem I thought I solved was to change from virtio scsi to virtio blk emulation
> but that seems to be effecttive albeit with a warning:
> 
> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2468510/raw_inline
> 
> runqemu - WARNING - Unknown QB_DRIVE_TYPE: vd
> runqemu - WARNING - Failed to figure out drive type, consider define or fix QB_DRIVE_TYPE
> runqemu - WARNING - Trying to use virtio block drive
> 
> u-boot defconfig does not have scsi support but virtio blk works for disk with ESP
> partition in fat format.

I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
One difference is that autobuilder runs with tap and I with slirp networking, but that
should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
If this still fails on autobuilder, then I'd need know what config fragments
are applied. I've configured qemuarm64 and genericarm64 with
". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
scripts add something more.

Cheers,

-Mikko
Mathieu Dubois-Briand May 27, 2025, 6:34 a.m. UTC | #4
On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
> Hi,
>
> On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
>> On Fri, May 23, 2025 at 03:15:51PM +0200, Mathieu Dubois-Briand wrote:
>> > On Thu May 22, 2025 at 3:41 PM CEST, Mikko Rapeli via lists.openembedded.org wrote:
>> > > Add a test case to boot target system via u-boot
>> > > using qemu with KVM. This was broken recently
>> > > and workaround proposed to u-boot. Test case
>> > > works with genericarm64 and qemuarm64 target machines
>> > > compiled and tested on aarch64 build host with KVM
>> > > support.
>> > >
>> > > Test execution time with full sstate cache is
>> > > around 170 seconds. qemu boot itself takes just
>> > > a few seconds to full userspace.
>> > >
>> > > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
>> > > ---
>> > 
>> > Hi Mikko,
>> > 
>> > I saw your other mail saying you will make an update, but as I had
>> > already picked these patches, here is the build result: the selftest is
>> > failing on armhost:
>> > 
>> > ---
>> > 2025-05-23 08:50:18,542 - oe-selftest - INFO - uboot.UBootTest.test_boot_uboot_kvm_to_full_target (subunit.RemotedTestCase)
>> > 2025-05-23 08:50:18,543 - oe-selftest - INFO -  ... ERROR
>> > ...
>> > QEMU 10.0.0 monitor - type 'help' for more information
>> > (qemu)
>> > Waiting at most 1000 seconds for login banner (05/23/25 08:33:29)
>> > Connection from 127.0.0.1:43786
>> > Target didn't reach login banner in 1000 seconds (05/23/25 08:50:12)
>> > Last 25 lines of login console (1721):
>> > Scanning bootdev 'virtio-blk#36.bootdev':
>> > Scanning bootdev 'virtio-net#32.bootdev':
>> > BOOTP broadcast 1
>> > ...
>> > BOOTP broadcast 17
>> > 
>> > Retry time exceeded; starting again
>> > No more bootdevs
>> > ---  -----------  ------  --------  ----  ------------------------  ----------------
>> > (1 bootflow, 1 valid)
>> > =>
>> > ...
>> > RuntimeError: core-image-minimal - FAILED to start qemu - check the task log and the boot log
>> > ---
>> > 
>> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1749
>> 
>> Thanks, I will take a look. Boot to u-boot seems to ok so KVM is working on
>> the aarch64 build machine but something in wic image ESP partition detection is not
>> working in the same way as on my aarch64 build machine and qemuarm64 and
>> genericarm64 configs.
>> 
>> I will compare the runqemu and qemu-system-aarch64 arguments to figure out what is
>> different.
>> 
>> One problem I thought I solved was to change from virtio scsi to virtio blk emulation
>> but that seems to be effecttive albeit with a warning:
>> 
>> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2468510/raw_inline
>> 
>> runqemu - WARNING - Unknown QB_DRIVE_TYPE: vd
>> runqemu - WARNING - Failed to figure out drive type, consider define or fix QB_DRIVE_TYPE
>> runqemu - WARNING - Trying to use virtio block drive
>> 
>> u-boot defconfig does not have scsi support but virtio blk works for disk with ESP
>> partition in fat format.
>
> I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
> One difference is that autobuilder runs with tap and I with slirp networking, but that
> should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
> If this still fails on autobuilder, then I'd need know what config fragments
> are applied. I've configured qemuarm64 and genericarm64 with
> ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
> and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
> scripts add something more.
>
> Cheers,
>
> -Mikko

Hi Mikko,

Thanks for the new version, but I confirm it still fails.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772

About the configuration, I believe everything is there:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772/steps/14/logs/stdio

But anyway, here is the configuration on the worker:

$ cat conf/auto.conf
MACHINE = "qemuarm64"
DISTRO = "poky"
SDKMACHINE = "aarch64"
PACKAGE_CLASSES = "package_rpm"
DL_DIR = '/srv/autobuilder/valkyrie.yocto.io/current_sources'
SSTATE_DIR ?= '/srv/autobuilder/valkyrie.yocto.io/pub/sstate'
RPM_GPG_SIGN_CHUNK = '1'
IMAGE_INSTALL:append = ' ssh-pregen-hostkeys'
SANITY_TESTED_DISTROS = ''
BB_HASHSERVE = 'wss://hashserv-eu.yoctoproject.org/ws'
OE_FRAGMENTS += 'core/yocto-autobuilder/autobuilder core/yocto-autobuilder/autobuilder-resource-constraints'

local.conf is unmodified (local.conf.sample).

Best regards,
Mathieu
Mikko Rapeli May 27, 2025, 7:17 a.m. UTC | #5
Hi,

On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
> >> On Fri, May 23, 2025 at 03:15:51PM +0200, Mathieu Dubois-Briand wrote:
> >> > On Thu May 22, 2025 at 3:41 PM CEST, Mikko Rapeli via lists.openembedded.org wrote:
> >> > > Add a test case to boot target system via u-boot
> >> > > using qemu with KVM. This was broken recently
> >> > > and workaround proposed to u-boot. Test case
> >> > > works with genericarm64 and qemuarm64 target machines
> >> > > compiled and tested on aarch64 build host with KVM
> >> > > support.
> >> > >
> >> > > Test execution time with full sstate cache is
> >> > > around 170 seconds. qemu boot itself takes just
> >> > > a few seconds to full userspace.
> >> > >
> >> > > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> >> > > ---
> >> > 
> >> > Hi Mikko,
> >> > 
> >> > I saw your other mail saying you will make an update, but as I had
> >> > already picked these patches, here is the build result: the selftest is
> >> > failing on armhost:
> >> > 
> >> > ---
> >> > 2025-05-23 08:50:18,542 - oe-selftest - INFO - uboot.UBootTest.test_boot_uboot_kvm_to_full_target (subunit.RemotedTestCase)
> >> > 2025-05-23 08:50:18,543 - oe-selftest - INFO -  ... ERROR
> >> > ...
> >> > QEMU 10.0.0 monitor - type 'help' for more information
> >> > (qemu)
> >> > Waiting at most 1000 seconds for login banner (05/23/25 08:33:29)
> >> > Connection from 127.0.0.1:43786
> >> > Target didn't reach login banner in 1000 seconds (05/23/25 08:50:12)
> >> > Last 25 lines of login console (1721):
> >> > Scanning bootdev 'virtio-blk#36.bootdev':
> >> > Scanning bootdev 'virtio-net#32.bootdev':
> >> > BOOTP broadcast 1
> >> > ...
> >> > BOOTP broadcast 17
> >> > 
> >> > Retry time exceeded; starting again
> >> > No more bootdevs
> >> > ---  -----------  ------  --------  ----  ------------------------  ----------------
> >> > (1 bootflow, 1 valid)
> >> > =>
> >> > ...
> >> > RuntimeError: core-image-minimal - FAILED to start qemu - check the task log and the boot log
> >> > ---
> >> > 
> >> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1749
> >> 
> >> Thanks, I will take a look. Boot to u-boot seems to ok so KVM is working on
> >> the aarch64 build machine but something in wic image ESP partition detection is not
> >> working in the same way as on my aarch64 build machine and qemuarm64 and
> >> genericarm64 configs.
> >> 
> >> I will compare the runqemu and qemu-system-aarch64 arguments to figure out what is
> >> different.
> >> 
> >> One problem I thought I solved was to change from virtio scsi to virtio blk emulation
> >> but that seems to be effecttive albeit with a warning:
> >> 
> >> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2468510/raw_inline
> >> 
> >> runqemu - WARNING - Unknown QB_DRIVE_TYPE: vd
> >> runqemu - WARNING - Failed to figure out drive type, consider define or fix QB_DRIVE_TYPE
> >> runqemu - WARNING - Trying to use virtio block drive
> >> 
> >> u-boot defconfig does not have scsi support but virtio blk works for disk with ESP
> >> partition in fat format.
> >
> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
> > One difference is that autobuilder runs with tap and I with slirp networking, but that
> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
> > If this still fails on autobuilder, then I'd need know what config fragments
> > are applied. I've configured qemuarm64 and genericarm64 with
> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
> > scripts add something more.
> >
> > Cheers,
> >
> > -Mikko
> 
> Hi Mikko,
> 
> Thanks for the new version, but I confirm it still fails.
> 
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772

Is it possible to grab the full boot log
/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?

It was referred in
https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline

That shows multiple tries to boot qemu into the rootfs.
One of them try the ext4 rootfs image which should not be possible
since the test configures wic with an ESP partition and
u-boot should be able to detect that if the block devices
was correct setup in qemu.

Maybe these logs are from previous test and not the one which is failing
which could be the last boot try in the log.

> About the configuration, I believe everything is there:
> 
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772/steps/14/logs/stdio
> 
> But anyway, here is the configuration on the worker:
> 
> $ cat conf/auto.conf
> MACHINE = "qemuarm64"
> DISTRO = "poky"
> SDKMACHINE = "aarch64"
> PACKAGE_CLASSES = "package_rpm"
> DL_DIR = '/srv/autobuilder/valkyrie.yocto.io/current_sources'
> SSTATE_DIR ?= '/srv/autobuilder/valkyrie.yocto.io/pub/sstate'
> RPM_GPG_SIGN_CHUNK = '1'
> IMAGE_INSTALL:append = ' ssh-pregen-hostkeys'
> SANITY_TESTED_DISTROS = ''
> BB_HASHSERVE = 'wss://hashserv-eu.yoctoproject.org/ws'
> OE_FRAGMENTS += 'core/yocto-autobuilder/autobuilder core/yocto-autobuilder/autobuilder-resource-constraints'
> 
> local.conf is unmodified (local.conf.sample).

Thanks, I will try again to reproduce. Something is different
but what...

Cheers,

-Mikko
 
> Best regards,
> Mathieu
> 
> -- 
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
Mathieu Dubois-Briand May 27, 2025, 7:43 a.m. UTC | #6
On Tue May 27, 2025 at 9:17 AM CEST, Mikko Rapeli wrote:
> Hi,
>
> On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
>> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
>> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
>> >
>> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
>> > One difference is that autobuilder runs with tap and I with slirp networking, but that
>> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
>> > If this still fails on autobuilder, then I'd need know what config fragments
>> > are applied. I've configured qemuarm64 and genericarm64 with
>> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
>> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
>> > scripts add something more.
>> >
>> > Cheers,
>> >
>> > -Mikko
>> 
>> Hi Mikko,
>> 
>> Thanks for the new version, but I confirm it still fails.
>> 
>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772
>
> Is it possible to grab the full boot log
> /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?
>
> It was referred in
> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline
>

The file is gone :(

I will launch a new build so I can capture the file.
Mikko Rapeli May 27, 2025, 8:54 a.m. UTC | #7
Hi,

On Tue, May 27, 2025 at 09:43:02AM +0200, Mathieu Dubois-Briand wrote:
> On Tue May 27, 2025 at 9:17 AM CEST, Mikko Rapeli wrote:
> > On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
> >> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
> >> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
> >> >
> >> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
> >> > One difference is that autobuilder runs with tap and I with slirp networking, but that
> >> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
> >> > If this still fails on autobuilder, then I'd need know what config fragments
> >> > are applied. I've configured qemuarm64 and genericarm64 with
> >> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
> >> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
> >> > scripts add something more.
> >> >
> >> > Cheers,
> >> >
> >> > -Mikko
> >> 
> >> Hi Mikko,
> >> 
> >> Thanks for the new version, but I confirm it still fails.
> >> 
> >> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772
> >
> > Is it possible to grab the full boot log
> > /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?
> >
> > It was referred in
> > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline
> >
> 
> The file is gone :(
> 
> I will launch a new build so I can capture the file.

I finally reproduced this. The ESP partition is missing grub-efi
binaries and only config file is there. I have fixes pending for this
missing dependency on aarch64 and of course was building with them
in my tree. I will add a workaround to this test case and
try get to rebasing the fixes to wic...

Cheers,

-Mikko
Mathieu Dubois-Briand May 27, 2025, 9:36 a.m. UTC | #8
On Tue May 27, 2025 at 9:43 AM CEST, Mathieu Dubois-Briand wrote:
> On Tue May 27, 2025 at 9:17 AM CEST, Mikko Rapeli wrote:
>> Hi,
>>
>> On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
>>> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
>>> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
>>> >
>>> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
>>> > One difference is that autobuilder runs with tap and I with slirp networking, but that
>>> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
>>> > If this still fails on autobuilder, then I'd need know what config fragments
>>> > are applied. I've configured qemuarm64 and genericarm64 with
>>> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
>>> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
>>> > scripts add something more.
>>> >
>>> > Cheers,
>>> >
>>> > -Mikko
>>> 
>>> Hi Mikko,
>>> 
>>> Thanks for the new version, but I confirm it still fails.
>>> 
>>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772
>>
>> Is it possible to grab the full boot log
>> /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?
>>
>> It was referred in
>> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline
>>
>
> The file is gone :(
>
> I will launch a new build so I can capture the file.

New build. There is also a failure on bitbake selftest here, but it
seems to be an unrelated intermittent issue.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1778

I believe the file you are looking for is
/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-932426/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250527082811.2,
I'm attaching it to my mail. But I'm not sure there is more data than
what you already had.

I will try to backup the whole build directory once the build is done,
in the case you need some other file.
Mathieu Dubois-Briand May 27, 2025, 9:37 a.m. UTC | #9
On Tue May 27, 2025 at 10:54 AM CEST, Mikko Rapeli wrote:
> Hi,
>
> On Tue, May 27, 2025 at 09:43:02AM +0200, Mathieu Dubois-Briand wrote:
>> On Tue May 27, 2025 at 9:17 AM CEST, Mikko Rapeli wrote:
>> > On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
>> >> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
>> >> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
>> >> >
>> >> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
>> >> > One difference is that autobuilder runs with tap and I with slirp networking, but that
>> >> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
>> >> > If this still fails on autobuilder, then I'd need know what config fragments
>> >> > are applied. I've configured qemuarm64 and genericarm64 with
>> >> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
>> >> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
>> >> > scripts add something more.
>> >> >
>> >> > Cheers,
>> >> >
>> >> > -Mikko
>> >> 
>> >> Hi Mikko,
>> >> 
>> >> Thanks for the new version, but I confirm it still fails.
>> >> 
>> >> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772
>> >
>> > Is it possible to grab the full boot log
>> > /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?
>> >
>> > It was referred in
>> > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline
>> >
>> 
>> The file is gone :(
>> 
>> I will launch a new build so I can capture the file.
>
> I finally reproduced this. The ESP partition is missing grub-efi
> binaries and only config file is there. I have fixes pending for this
> missing dependency on aarch64 and of course was building with them
> in my tree. I will add a workaround to this test case and
> try get to rebasing the fixes to wic...
>
> Cheers,
>
> -Mikko

Oops, I replied before I saw your own reply. Thanks for the update!
Mikko Rapeli May 27, 2025, 9:52 a.m. UTC | #10
Hi,

On Tue, May 27, 2025 at 11:36:10AM +0200, Mathieu Dubois-Briand wrote:
> On Tue May 27, 2025 at 9:43 AM CEST, Mathieu Dubois-Briand wrote:
> > On Tue May 27, 2025 at 9:17 AM CEST, Mikko Rapeli wrote:
> >> On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
> >>> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
> >>> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
> >>> >
> >>> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
> >>> > One difference is that autobuilder runs with tap and I with slirp networking, but that
> >>> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
> >>> > If this still fails on autobuilder, then I'd need know what config fragments
> >>> > are applied. I've configured qemuarm64 and genericarm64 with
> >>> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
> >>> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
> >>> > scripts add something more.
> >>> >
> >>> > Cheers,
> >>> >
> >>> > -Mikko
> >>> 
> >>> Hi Mikko,
> >>> 
> >>> Thanks for the new version, but I confirm it still fails.
> >>> 
> >>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772
> >>
> >> Is it possible to grab the full boot log
> >> /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?
> >>
> >> It was referred in
> >> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline
> >>
> >
> > The file is gone :(
> >
> > I will launch a new build so I can capture the file.
> 
> New build. There is also a failure on bitbake selftest here, but it
> seems to be an unrelated intermittent issue.
> 
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1778
> 
> I believe the file you are looking for is
> /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-932426/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250527082811.2,
> I'm attaching it to my mail. But I'm not sure there is more data than
> what you already had.
> 
> I will try to backup the whole build directory once the build is done,
> in the case you need some other file.

Contents of the file is short and show that u-boot failed to find the binaries
needed from ESP or other locations. I had the exact same failure, finally:

U-Boot 2025.04 (Apr 07 2025 - 21:57:55 +0000)

DRAM:  256 MiB
Core:  52 devices, 14 uclasses, devicetree: board
Flash: 64 MiB
Loading Environment from Flash... *** Warning - bad CRC, using default environment

In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
No USB controllers found
Net:   eth0: virtio-net#32
[?25h
starting USB...
No USB controllers found
Hit any key to stop autoboot:  2  1  0 
Scanning for bootflows in all bootdevs
Seq  Method       State   Uclass    Part  Name                      Filename
---  -----------  ------  --------  ----  ------------------------  ----------------
Scanning global bootmeth 'efi_mgr':
78Cannot persist EFI variables without system partition
Missing TPMv2 device for EFI_TCG_PROTOCOL
  0  efi_mgr      ready   (none)       0  <NULL>                    
** Booting bootflow '<NULL>' with efi_mgr
Loading Boot0000 'virtio 0' failed
EFI boot manager: Cannot load any image
Boot failed (err=-14)
Scanning bootdev 'fw-cfg@9020000.bootdev':
fatal: no kernel available
No USB controllers found
scanning bus for devices...
Scanning bootdev 'virtio-blk#36.bootdev':
Scanning bootdev 'virtio-net#32.bootdev':
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
BOOTP broadcast 4
BOOTP broadcast 5
BOOTP broadcast 6
BOOTP broadcast 7
BOOTP broadcast 8
BOOTP broadcast 9
BOOTP broadcast 10
BOOTP broadcast 11
BOOTP broadcast 12
BOOTP broadcast 13
BOOTP broadcast 14
BOOTP broadcast 15
BOOTP broadcast 16
BOOTP broadcast 17

Retry time exceeded; starting again
No more bootdevs
---  -----------  ------  --------  ----  ------------------------  ----------------
(1 bootflow, 1 valid)
=> 

So this is the race condition and problem in wic with aarch64 targets where
systemd-boot or grub-efi are not compiled before wic creates
images and missing EFI loader files on the ESP partition
generated by wic don't fail the build. In my case and I suspect
on autobuilder as well, the core-image-minimal-qemuarm64.rootfs.wic is missing
/efi/boot/bootaa64.efi file which is the expected default on aarch64.
This can be checked in build directory with:

$ wic ls tmp/deploy/images/qemuarm64/core-image-minimal-qemuarm64.rootfs.wic:1/efi/boot/bootaa64.efi

These are fixed in my tree with patches
"image_types_wic.bbclass: depend on grub-efi and systemd-boot on aarch64, systemd-boot on arm"
https://lists.openembedded.org/g/openembedded-core/topic/112395863#msg215349

and

"wic bootimg-efi.py: fail build if no binaries installed"
https://lists.openembedded.org/g/openembedded-core/topic/112395862#msg215222

The latter fails the build if EFI loader binary is not installed.
The patches have some open review comments on mailing list and I'm
trying to get to fixing them.

Cheers,

-Mikko
Mikko Rapeli May 27, 2025, 9:55 a.m. UTC | #11
Hi,

On Tue, May 27, 2025 at 11:37:33AM +0200, Mathieu Dubois-Briand wrote:
> On Tue May 27, 2025 at 10:54 AM CEST, Mikko Rapeli wrote:
> > On Tue, May 27, 2025 at 09:43:02AM +0200, Mathieu Dubois-Briand wrote:
> >> On Tue May 27, 2025 at 9:17 AM CEST, Mikko Rapeli wrote:
> >> > On Tue, May 27, 2025 at 08:34:52AM +0200, Mathieu Dubois-Briand wrote:
> >> >> On Mon May 26, 2025 at 10:45 AM CEST, Mikko Rapeli wrote:
> >> >> > On Fri, May 23, 2025 at 05:16:58PM +0300, Mikko Rapeli via lists.openembedded.org wrote:
> >> >> >
> >> >> > I reviewed the runqemu and qemu-system-aarch64 arguments but could not see major differences.
> >> >> > One difference is that autobuilder runs with tap and I with slirp networking, but that
> >> >> > should not affect rootfs/ESP partition detection in u-boot. I've sent out v4 now.
> >> >> > If this still fails on autobuilder, then I'd need know what config fragments
> >> >> > are applied. I've configured qemuarm64 and genericarm64 with
> >> >> > ". oe-init-build-env" and then added TEST_RUNQEMUPARAMS += "slirp"
> >> >> > and SANITY_TESTED_DISTROS = "" to conf/local.conf. Maybe autobuilder
> >> >> > scripts add something more.
> >> >> >
> >> >> > Cheers,
> >> >> >
> >> >> > -Mikko
> >> >> 
> >> >> Hi Mikko,
> >> >> 
> >> >> Thanks for the new version, but I confirm it still fails.
> >> >> 
> >> >> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1772
> >> >
> >> > Is it possible to grab the full boot log
> >> > /srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1032659/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/testimage/qemu_boot_log.20250526163143 ?
> >> >
> >> > It was referred in
> >> > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/2499407/raw_inline
> >> >
> >> 
> >> The file is gone :(
> >> 
> >> I will launch a new build so I can capture the file.
> >
> > I finally reproduced this. The ESP partition is missing grub-efi
> > binaries and only config file is there. I have fixes pending for this
> > missing dependency on aarch64 and of course was building with them
> > in my tree. I will add a workaround to this test case and
> > try get to rebasing the fixes to wic...
> >
> > Cheers,
> >
> > -Mikko
> 
> Oops, I replied before I saw your own reply. Thanks for the update!

No worries, it's good to see the confirmation that issues are the same.

Maybe these files could be captured fully from autobuilder in case there
are failures?

Like in this case, it's a race condition which is not so easy to reproduce.

Cheers,

-Mikko
diff mbox series

Patch

diff --git a/meta/lib/oeqa/selftest/cases/uboot.py b/meta/lib/oeqa/selftest/cases/uboot.py
index 39e48f8bdb..982822b389 100644
--- a/meta/lib/oeqa/selftest/cases/uboot.py
+++ b/meta/lib/oeqa/selftest/cases/uboot.py
@@ -6,8 +6,8 @@ 
 #
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, runqemu, get_bb_var
-from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.utils.commands import bitbake, runqemu, get_bb_var, runCmd
+from oeqa.core.decorator.data import skipIfNotArch, skipIfNotMachine
 from oeqa.core.decorator import OETestTag
 
 uboot_boot_patterns = {
@@ -42,3 +42,54 @@  QEMU_USE_KVM = "False"
             status, output = qemu.run_serial(cmd)
             self.assertEqual(status, 1, msg=output)
             self.assertTrue("U-Boot" in output, msg=output)
+
+    @skipIfNotArch(['aarch64'])
+    # Also works on genericarm64 but no way to encode that currently
+    @skipIfNotMachine('qemuarm64', 'KVM depends that host and target architectures match, e.g. aarch64 and qemuarm64')
+    @OETestTag("runqemu")
+    def test_boot_uboot_kvm_to_full_target(self):
+        """
+        Tests building u-boot and booting it with QEMU and KVM.
+        Requires working KVM on build host. See "kvm-ok" output.
+        """
+
+        runCmd("kvm-ok")
+
+        self.write_config("""
+QEMU_USE_KVM = "1"
+
+# Using u-boot in EFI mode, need ESP partition for grub/systemd-boot/kernel etc
+IMAGE_FSTYPES:pn-core-image-minimal:append = " wic"
+
+# easiest to follow genericarm64 setup with wks file, initrd and EFI loader
+INITRAMFS_IMAGE="core-image-initramfs-boot"
+EFI_PROVIDER = "${@bb.utils.contains("DISTRO_FEATURES", "systemd", "systemd-boot", "grub-efi", d)}"
+WKS_FILE = "genericarm64.wks.in"
+
+# use wic image with ESP for u-boot, not ext4
+QB_DEFAULT_FSTYPE = "wic"
+
+PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
+QB_DEFAULT_BIOS = "u-boot.bin"
+
+# let u-boot or EFI loader load kernel from ESP
+QB_DEFAULT_KERNEL = "none"
+
+# virt pci, not scsi because support not in u-boot to find ESP
+QB_DRIVE_TYPE = "vd"
+# qemu usb causes u-boot reset atm
+QB_OPT_APPEND = ""
+""")
+        bitbake("virtual/bootloader core-image-minimal")
+
+        runqemu_params = get_bb_var('TEST_RUNQEMUPARAMS', "core-image-minimal") or ""
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic kvm %s' % runqemu_params) as qemu:
+
+            # boot to target and login worked, should have been fast with kvm
+            cmd = "dmesg"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            # Machine is qemu
+            self.assertTrue("Machine model: linux,dummy-virt" in output, msg=output)
+            # with KVM enabled
+            self.assertTrue("KVM: hypervisor services detected" in output, msg=output)