| Message ID | 20250510084400.269726-3-ross.burton@arm.com |
|---|---|
| State | Accepted, archived |
| Commit | b293c44f87b6a52e4239ce14066514e87d9b08d0 |
| Headers | show |
| Series | [01/23] buildtools-tarball: fix default_cases assignment | expand |
Mercy me. How do we still have sort-of-functional SDKs in Yocto? There's one more place where SDK tests are run: test_meta_ide_can_run_sdk_tests() selftest (in meta/lib/oeqa/selftest/cases/meta_ide.py) Can you confirm that it still passes and runs a reasonable set of SDK tests with your patchset? Alex On Sat, 10 May 2025 at 10:44, Ross Burton via lists.openembedded.org <ross.burton=arm.com@lists.openembedded.org> wrote: > > The SDK manifests are generated by listing the sstate was that used, but > it hardcodes that the sstate data filenames end in .tgz. > > This has not been the case since sstate switched to Zstd[1] in 2021, > which meant that all of the tests which checked for packages existing > were being skipped as the manifests were empty. For example, see a > representative core-image-sato eSDK test run[2]: > > RESULTS - cmake.CMakeTest.test_assimp: SKIPPED (0.00s) > RESULTS - gtk3.GTK3Test.test_galculator: SKIPPED (0.00s) > RESULTS - kmod.KernelModuleTest.test_cryptodev: SKIPPED (0.00s) > RESULTS - maturin.MaturinDevelopTest.test_maturin_develop: SKIPPED (0.00s) > RESULTS - maturin.MaturinTest.test_maturin_list_python: SKIPPED (0.00s) > RESULTS - meson.MesonTest.test_epoxy: SKIPPED (0.00s) > RESULTS - perl.PerlTest.test_perl: SKIPPED (0.00s) > RESULTS - python.Python3Test.test_python3: SKIPPED (0.00s) > > All of those tests should have been ran. > > Solve this by generalising the filename check so that it doesn't care > what specfic compression algorithm is used. > > [1] oe-core 0710e98f40e ("sstate: Switch to ZStandard compressor support") > [2] https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/1517/steps/15/logs/stdio > > Signed-off-by: Ross Burton <ross.burton@arm.com> > --- > meta/lib/oe/sdk.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py > index 11759aba489..9fe0fbb752d 100644 > --- a/meta/lib/oe/sdk.py > +++ b/meta/lib/oe/sdk.py > @@ -148,7 +148,8 @@ def get_extra_sdkinfo(sstate_dir): > extra_info['filesizes'] = {} > for root, _, files in os.walk(sstate_dir): > for fn in files: > - if fn.endswith('.tgz'): > + # Note that this makes an assumption about the sstate filenames > + if '.tar.' in fn and not fn.endswith('.siginfo'): > fsize = int(math.ceil(float(os.path.getsize(os.path.join(root, fn))) / 1024)) > task = fn.rsplit(':',1)[1].split('_',1)[1].split(',')[0] > origtotal = extra_info['tasksizes'].get(task, 0) > -- > 2.43.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#216251): https://lists.openembedded.org/g/openembedded-core/message/216251 > Mute This Topic: https://lists.openembedded.org/mt/113037458/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
> On 12 May 2025, at 10:25, Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > Mercy me. How do we still have sort-of-functional SDKs in Yocto? > > There's one more place where SDK tests are run: > test_meta_ide_can_run_sdk_tests() selftest > (in meta/lib/oeqa/selftest/cases/meta_ide.py) > > Can you confirm that it still passes and runs a reasonable set of SDK > tests with your patchset? The AB didn’t fail, but for reference: 2025-05-16 12:04:32,319 - oe-selftest - INFO - RESULTS: 2025-05-16 12:04:32,319 - oe-selftest - INFO - RESULTS - meta_ide.MetaIDE.test_meta_ide_can_build_cpio_project: PASSED (53.39s) 2025-05-16 12:04:32,320 - oe-selftest - INFO - RESULTS - meta_ide.MetaIDE.test_meta_ide_can_compile_c_program: PASSED (0.25s) 2025-05-16 12:04:32,320 - oe-selftest - INFO - RESULTS - meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests: PASSED (539.39s) 2025-05-16 12:04:32,320 - oe-selftest - INFO - RESULTS - meta_ide.MetaIDE.test_meta_ide_had_installed_meta_ide_support: PASSED (0.01s) 2025-05-16 12:04:32,321 - oe-selftest - INFO - SUMMARY: 2025-05-16 12:04:32,321 - oe-selftest - INFO - oe-selftest () - Ran 4 tests in 1272.775s 2025-05-16 12:04:32,321 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=4, skipped=0, failures=0, errors=0) Ross
diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py index 11759aba489..9fe0fbb752d 100644 --- a/meta/lib/oe/sdk.py +++ b/meta/lib/oe/sdk.py @@ -148,7 +148,8 @@ def get_extra_sdkinfo(sstate_dir): extra_info['filesizes'] = {} for root, _, files in os.walk(sstate_dir): for fn in files: - if fn.endswith('.tgz'): + # Note that this makes an assumption about the sstate filenames + if '.tar.' in fn and not fn.endswith('.siginfo'): fsize = int(math.ceil(float(os.path.getsize(os.path.join(root, fn))) / 1024)) task = fn.rsplit(':',1)[1].split('_',1)[1].split(',')[0] origtotal = extra_info['tasksizes'].get(task, 0)
The SDK manifests are generated by listing the sstate was that used, but it hardcodes that the sstate data filenames end in .tgz. This has not been the case since sstate switched to Zstd[1] in 2021, which meant that all of the tests which checked for packages existing were being skipped as the manifests were empty. For example, see a representative core-image-sato eSDK test run[2]: RESULTS - cmake.CMakeTest.test_assimp: SKIPPED (0.00s) RESULTS - gtk3.GTK3Test.test_galculator: SKIPPED (0.00s) RESULTS - kmod.KernelModuleTest.test_cryptodev: SKIPPED (0.00s) RESULTS - maturin.MaturinDevelopTest.test_maturin_develop: SKIPPED (0.00s) RESULTS - maturin.MaturinTest.test_maturin_list_python: SKIPPED (0.00s) RESULTS - meson.MesonTest.test_epoxy: SKIPPED (0.00s) RESULTS - perl.PerlTest.test_perl: SKIPPED (0.00s) RESULTS - python.Python3Test.test_python3: SKIPPED (0.00s) All of those tests should have been ran. Solve this by generalising the filename check so that it doesn't care what specfic compression algorithm is used. [1] oe-core 0710e98f40e ("sstate: Switch to ZStandard compressor support") [2] https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/1517/steps/15/logs/stdio Signed-off-by: Ross Burton <ross.burton@arm.com> --- meta/lib/oe/sdk.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)