[3/3] qemurunner: add boot logs to do_testimage task logs in real time

Complex boot sequences take a lot of time to reach serial console login.
In our case boot involves multiple bootloaders like u-boot and grub,
then hypervisors like xen, then initramfs images which do things like
validating secure boot and encrypting the main rootfs. All these happen
before main rootfs serial console login prompt is reached and thus many
things can go wrong, including race conditions, hangs and deadlocks, and
thus logging the details is important for development.

This change adds the qemu serial console logs to do_testimage task logs while
they are being read out. The logs are not added to bitbake output. This
enables debugging and tracing the do_testimage via task log file while the boot
and tests are running.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/utils/qemurunner.py | 1 +
 1 file changed, 1 insertion(+)

On 10/5/23 14:59, Mikko Rapeli wrote:
> Complex boot sequences take a lot of time to reach serial console login.
> In our case boot involves multiple bootloaders like u-boot and grub,
> then hypervisors like xen, then initramfs images which do things like
> validating secure boot and encrypting the main rootfs. All these happen
> before main rootfs serial console login prompt is reached and thus many
> things can go wrong, including race conditions, hangs and deadlocks, and
> thus logging the details is important for development.
> 
> This change adds the qemu serial console logs to do_testimage task logs while
> they are being read out. The logs are not added to bitbake output. This
> enables debugging and tracing the do_testimage via task log file while the boot
> and tests are running.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>   meta/lib/oeqa/utils/qemurunner.py | 1 +
>   1 file changed, 1 insertion(+)

Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Hi Richard,

Could you accept this with some conditional flag in testimage.bbclass maybe?

I really feel the need for this when debugging issues with boot sequence.

Cheers,

-Mikko

On Wed, May 10, 2023 at 03:59:35PM +0300, Mikko Rapeli wrote:
> Complex boot sequences take a lot of time to reach serial console login.
> In our case boot involves multiple bootloaders like u-boot and grub,
> then hypervisors like xen, then initramfs images which do things like
> validating secure boot and encrypting the main rootfs. All these happen
> before main rootfs serial console login prompt is reached and thus many
> things can go wrong, including race conditions, hangs and deadlocks, and
> thus logging the details is important for development.
> 
> This change adds the qemu serial console logs to do_testimage task logs while
> they are being read out. The logs are not added to bitbake output. This
> enables debugging and tracing the do_testimage via task log file while the boot
> and tests are running.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>  meta/lib/oeqa/utils/qemurunner.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
> index c06268f5bc..7f520d47c3 100644
> --- a/meta/lib/oeqa/utils/qemurunner.py
> +++ b/meta/lib/oeqa/utils/qemurunner.py
> @@ -480,6 +480,7 @@ class QemuRunner:
>                          self.logger.error('Invalid file type: %s\n%s' % (file))
>                          read = b''
>  
> +                    self.logger.debug2('Partial boot log:\n%s' % (read.decode('utf-8', errors='ignore')))
>                      data = data + read
>                      if data:
>                          bootlog += data
> -- 
> 2.34.1
>