diff mbox series

oeqa: do not fail when sdk package is not available

Message ID 20250723062616.101590-1-peter.marko@siemens.com
State New
Headers show
Series oeqa: do not fail when sdk package is not available | expand

Commit Message

Marko, Peter July 23, 2025, 6:26 a.m. UTC 
From: Peter Marko <peter.marko@siemens.com>

Prior to commits:
* 11277efd057685558a744e98082b5709e849dd2a
* d0e8b83d05957b1f22d08582e364afa4b522801e
the tests were skipped if package was not available.
Now the code calls function ensure_host_package which says
"try to sdk-install missing dependencies", however in fact for sdkext it
causes a failure if the installation is not available.

Since maturin is not installed in any image, it cannot be installed
unless it's downloaded from sstate-cache mirror populated by a world
build. These builds are however now not done for powerpc and mips.
It also does not work in local builds without sstate-cache mirror.

Fix this by skipping the test if the package cannot be installed to
match the original behavior before those commits.

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Cc: Ross Burton <ross.burton@arm.com>
Cc: Khem Raj <raj.khem@gmail.com>
---
 meta/lib/oeqa/sdk/case.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Khem Raj July 23, 2025, 6:36 a.m. UTC | #1 
On Tue, Jul 22, 2025 at 11:26 PM Peter Marko <peter.marko@siemens.com> wrote:
>
> From: Peter Marko <peter.marko@siemens.com>
>
> Prior to commits:
> * 11277efd057685558a744e98082b5709e849dd2a
> * d0e8b83d05957b1f22d08582e364afa4b522801e
> the tests were skipped if package was not available.
> Now the code calls function ensure_host_package which says
> "try to sdk-install missing dependencies", however in fact for sdkext it
> causes a failure if the installation is not available.
>
> Since maturin is not installed in any image, it cannot be installed
> unless it's downloaded from sstate-cache mirror populated by a world
> build. These builds are however now not done for powerpc and mips.

Yeah I was doing riscv32 build which is also not part of world targets

> It also does not work in local builds without sstate-cache mirror.
>
> Fix this by skipping the test if the package cannot be installed to
> match the original behavior before those commits.
>
> Signed-off-by: Peter Marko <peter.marko@siemens.com>
> Cc: Ross Burton <ross.burton@arm.com>
> Cc: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/lib/oeqa/sdk/case.py | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
> index 1fd3b3b569..002eed271d 100644
> --- a/meta/lib/oeqa/sdk/case.py
> +++ b/meta/lib/oeqa/sdk/case.py
> @@ -44,7 +44,10 @@ class OESDKTestCase(OETestCase):
>              if isinstance(self.tc, OESDKExtTestContext):
>                  recipe = (recipe or packages[0]) + "-native"
>                  print("Trying to install %s..." % recipe)
> -                self._run('devtool sdk-install %s' % recipe)
> +                try:
> +                    self._run('devtool sdk-install %s' % recipe)
> +                except:
> +                    raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))
>              else:
>                  raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))
>
Ross Burton July 29, 2025, 10:22 a.m. UTC | #2 
On 23 Jul 2025, at 07:26, Peter Marko <peter.marko@siemens.com> wrote:
> 
> From: Peter Marko <peter.marko@siemens.com>
> 
> Prior to commits:
> * 11277efd057685558a744e98082b5709e849dd2a
> * d0e8b83d05957b1f22d08582e364afa4b522801e
> the tests were skipped if package was not available.
> Now the code calls function ensure_host_package which says
> "try to sdk-install missing dependencies", however in fact for sdkext it
> causes a failure if the installation is not available.
> 
> Since maturin is not installed in any image, it cannot be installed
> unless it's downloaded from sstate-cache mirror populated by a world
> build. These builds are however now not done for powerpc and mips.
> It also does not work in local builds without sstate-cache mirror.
> 
> Fix this by skipping the test if the package cannot be installed to
> match the original behavior before those commits.
> 
> Signed-off-by: Peter Marko <peter.marko@siemens.com>
> Cc: Ross Burton <ross.burton@arm.com>
> Cc: Khem Raj <raj.khem@gmail.com>
> ---
> meta/lib/oeqa/sdk/case.py | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
> index 1fd3b3b569..002eed271d 100644
> --- a/meta/lib/oeqa/sdk/case.py
> +++ b/meta/lib/oeqa/sdk/case.py
> @@ -44,7 +44,10 @@ class OESDKTestCase(OETestCase):
>             if isinstance(self.tc, OESDKExtTestContext):
>                 recipe = (recipe or packages[0]) + "-native"
>                 print("Trying to install %s..." % recipe)
> -                self._run('devtool sdk-install %s' % recipe)
> +                try:
> +                    self._run('devtool sdk-install %s' % recipe)
> +                except:
> +                    raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))
>             else:
>                 raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))

Yeah, I managed to break the logic slightly there.

I think it does, but can you confirm that the sdk-install output is visible in the test output when it is skipped?  Also, the except should catch just subprocessed.CalledProcessException.

Thanks,
Ross
Marko, Peter July 31, 2025, 7:14 p.m. UTC | #3 
> -----Original Message-----
> From: Ross Burton <Ross.Burton@arm.com>
> Sent: Tuesday, July 29, 2025 12:22
> To: Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com>
> Cc: openembedded-core@lists.openembedded.org; Khem Raj
> <raj.khem@gmail.com>
> Subject: Re: [OE-core][PATCH] oeqa: do not fail when sdk package is not
> available
> 
> On 23 Jul 2025, at 07:26, Peter Marko <peter.marko@siemens.com> wrote:
> >
> > From: Peter Marko <peter.marko@siemens.com>
> >
> > Prior to commits:
> > * 11277efd057685558a744e98082b5709e849dd2a
> > * d0e8b83d05957b1f22d08582e364afa4b522801e
> > the tests were skipped if package was not available.
> > Now the code calls function ensure_host_package which says
> > "try to sdk-install missing dependencies", however in fact for sdkext it
> > causes a failure if the installation is not available.
> >
> > Since maturin is not installed in any image, it cannot be installed
> > unless it's downloaded from sstate-cache mirror populated by a world
> > build. These builds are however now not done for powerpc and mips.
> > It also does not work in local builds without sstate-cache mirror.
> >
> > Fix this by skipping the test if the package cannot be installed to
> > match the original behavior before those commits.
> >
> > Signed-off-by: Peter Marko <peter.marko@siemens.com>
> > Cc: Ross Burton <ross.burton@arm.com>
> > Cc: Khem Raj <raj.khem@gmail.com>
> > ---
> > meta/lib/oeqa/sdk/case.py | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
> > index 1fd3b3b569..002eed271d 100644
> > --- a/meta/lib/oeqa/sdk/case.py
> > +++ b/meta/lib/oeqa/sdk/case.py
> > @@ -44,7 +44,10 @@ class OESDKTestCase(OETestCase):
> >             if isinstance(self.tc, OESDKExtTestContext):
> >                 recipe = (recipe or packages[0]) + "-native"
> >                 print("Trying to install %s..." % recipe)
> > -                self._run('devtool sdk-install %s' % recipe)
> > +                try:
> > +                    self._run('devtool sdk-install %s' % recipe)
> > +                except:
> > +                    raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ",
> ".join(packages)))
> >             else:
> >                 raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ",
> ".join(packages)))
> 
> Yeah, I managed to break the logic slightly there.
> 
> I think it does, but can you confirm that the sdk-install output is visible in the test
> output when it is skipped?  Also, the except should catch just
> subprocessed.CalledProcessException.

I will send a v2 with limiting the caught exception.

Regarding the output - it is not visible in logs.
When I added logging, it was a 105 lines of bitbake logs full of setscene tasks.
(see attached relevant log part from log.do_testsdkext)
So I don't think it's worth to add as it distorts the log readability.
But feel free to object and I can add that log anyway.

But something else good came out of it, too.
In the testing process I also had to fix OETestCalledProcessError, patch will come.

> 
> Thanks,
> Ross
NOTE: test_manifests (manifest.ManifestTest.test_manifests)
NOTE:  ... ok
NOTE: test_maturin_develop (maturin.MaturinDevelopTest.test_maturin_develop)
NOTE: devtool sdk-install threw: Command '. /home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/environment-setup-ppc7400-poky-linux > /dev/null; devtool sdk-install python3-maturin-native;' returned non-zero exit status 1.
Standard Output: NOTE: Starting bitbake server...
WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.
Loading cache...done.
Loaded 2015 entries from dependency cache.

Summary: There was 1 WARNING message.
INFO: Installing python3-maturin-native...
NOTE: Reconnecting to bitbake server...
WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.
Loading cache...done.
Loaded 2015 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks...Sstate summary: Wanted 85 Local 16 Mirrors 66 Missed 3 Current 68 (96% match, 98% complete)
done.
NOTE: Executing Tasks
NOTE: Running setscene task 107 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/python/python3-setuptools-rust_1.11.1.bb:do_populate_sysroot_setscene)
NOTE: recipe python3-setuptools-rust-native-1.11.1-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe python3-setuptools-rust-native-1.11.1-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 113 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/rust/cargo_1.87.0.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 114 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/python/python3-semantic-version_2.10.0.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 116 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/python/python3-setuptools-rust_1.11.1.bb:do_recipe_qa_setscene)
NOTE: recipe python3-semantic-version-native-2.10.0-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe cargo-native-1.87.0-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe python3-setuptools-rust-native-1.11.1-r0: task do_recipe_qa_setscene: Started
NOTE: recipe python3-semantic-version-native-2.10.0-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: recipe python3-setuptools-rust-native-1.11.1-r0: task do_recipe_qa_setscene: Succeeded
NOTE: Running setscene task 118 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/python/python3-semantic-version_2.10.0.bb:do_recipe_qa_setscene)
NOTE: recipe cargo-native-1.87.0-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 119 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/rust/rust_1.87.0.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 120 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-support/ca-certificates/ca-certificates_20250419.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 122 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-support/libssh2/libssh2_1.11.1.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 123 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/rust/cargo_1.87.0.bb:do_recipe_qa_setscene)
NOTE: recipe python3-semantic-version-native-2.10.0-r0: task do_recipe_qa_setscene: Started
NOTE: recipe ca-certificates-native-20250419-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe python3-semantic-version-native-2.10.0-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe libssh2-native-1.11.1-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe rust-native-1.87.0-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe cargo-native-1.87.0-r0: task do_recipe_qa_setscene: Started
NOTE: recipe cargo-native-1.87.0-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe libssh2-native-1.11.1-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 126 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-support/libssh2/libssh2_1.11.1.bb:do_recipe_qa_setscene)
NOTE: recipe ca-certificates-native-20250419-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 130 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-support/ca-certificates/ca-certificates_20250419.bb:do_recipe_qa_setscene)
NOTE: recipe ca-certificates-native-20250419-r0: task do_recipe_qa_setscene: Started
NOTE: recipe libssh2-native-1.11.1-r0: task do_recipe_qa_setscene: Started
NOTE: recipe ca-certificates-native-20250419-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe libssh2-native-1.11.1-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe rust-native-1.87.0-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 131 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/patchelf/patchelf_0.18.0.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 133 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/rust/rust-llvm_1.87.0.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 134 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/rust/rust_1.87.0.bb:do_recipe_qa_setscene)
NOTE: recipe patchelf-native-0.18.0-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe rust-llvm-native-1.87.0-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe rust-native-1.87.0-r0: task do_recipe_qa_setscene: Started
NOTE: recipe rust-native-1.87.0-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe patchelf-native-0.18.0-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 136 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/patchelf/patchelf_0.18.0.bb:do_recipe_qa_setscene)
NOTE: recipe patchelf-native-0.18.0-r0: task do_recipe_qa_setscene: Started
NOTE: recipe patchelf-native-0.18.0-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe rust-llvm-native-1.87.0-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 140 of 153 (virtual:native:/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/rust/rust-llvm_1.87.0.bb:do_recipe_qa_setscene)
NOTE: Running setscene task 151 of 153 (/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.69.bb:do_populate_sysroot_setscene)
NOTE: recipe quilt-native-0.69-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe rust-llvm-native-1.87.0-r0: task do_recipe_qa_setscene: Started
NOTE: recipe quilt-native-0.69-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 153 of 153 (/home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.69.bb:do_recipe_qa_setscene)
NOTE: recipe rust-llvm-native-1.87.0-r0: task do_recipe_qa_setscene: Succeeded
NOTE: recipe quilt-native-0.69-r0: task do_recipe_qa_setscene: Started
NOTE: recipe quilt-native-0.69-r0: task do_recipe_qa_setscene: Succeeded
NOTE: Tasks Summary: Attempted 0 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message.
ERROR: Failed to install python3-maturin-native - unavailable

NOTE:  ... skipped 'Test maturin.MaturinDevelopTest.test_maturin_develop needs one of python3-maturin'
Test maturin.MaturinDevelopTest.test_maturin_develop needs one of python3-maturin
NOTE: test_maturin_list_python (maturin.MaturinTest.test_maturin_list_python)
NOTE: devtool sdk-install threw: Command '. /home/projects/builds/poky-build/poky/build/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0/testsdkext/environment-setup-ppc7400-poky-linux > /dev/null; devtool sdk-install python3-maturin-native;' returned non-zero exit status 1.
Standard Output: NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)... (18:53:41.681808)
NOTE: Reconnecting to bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)... (18:53:41.681808)
NOTE: Retrying server connection (#1)... (18:53:41.681808)
NOTE: Starting bitbake server...
WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.
Loading cache...done.
Loaded 2015 entries from dependency cache.

Summary: There was 1 WARNING message.
INFO: Installing python3-maturin-native...
NOTE: Reconnecting to bitbake server...
WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.
Loading cache...done.
Loaded 2015 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks...Sstate summary: Wanted 67 Local 14 Mirrors 50 Missed 3 Current 86 (95% match, 98% complete)
done.
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 0 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message.
ERROR: Failed to install python3-maturin-native - unavailable

NOTE:  ... skipped 'Test maturin.MaturinTest.test_maturin_list_python needs one of python3-maturin'
Test maturin.MaturinTest.test_maturin_list_python needs one of python3-maturin
NOTE: test_epoxy (meson.MesonTest.test_epoxy)
NOTE:  ... skipped 'meson.MesonTest.test_epoxy does not support eSDK (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15854)'
meson.MesonTest.test_epoxy does not support eSDK (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15854)
diff mbox series

Patch

diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
index 1fd3b3b569..002eed271d 100644
--- a/meta/lib/oeqa/sdk/case.py
+++ b/meta/lib/oeqa/sdk/case.py
@@ -44,7 +44,10 @@  class OESDKTestCase(OETestCase):
             if isinstance(self.tc, OESDKExtTestContext):
                 recipe = (recipe or packages[0]) + "-native"
                 print("Trying to install %s..." % recipe)
-                self._run('devtool sdk-install %s' % recipe)
+                try:
+                    self._run('devtool sdk-install %s' % recipe)
+                except:
+                    raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))
             else:
                 raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))