mbox series

[v3,0/8] Add SPDX 3 Recipe Information

Message ID 20260226173930.2847872-1-JPEWhacker@gmail.com
Headers show
Series Add SPDX 3 Recipe Information | expand

Message

Joshua Watt Feb. 26, 2026, 5:33 p.m. UTC
Changes the SPDX 3 output to include a "recipe" package that describe
static information available at parse time (without building). This is
primarily useful for gathering SPDX 3 VEX information about some or all
recipes, enabling SPDX 3 to be used in place of cve_check.bbclass and
vex.bbclass.

Special thanks to Benjamin Robin <benjamin.robin@bootlin.com> for
helping work through this.

V2: Fixes a bug where do_populate_sysroot was running when it should not
be. Drops the patch to ignore ASSUME_PROVIDES recipes, since this is
incorrect (this is already handled by bitbake in the taskgraph, and
doesn't need to be manually removed).

V3: Fixes a bug where meta-world-recipe-sbom was reporting a circular
dependency. meta-world-recipe-sbom also no longer runs in world builds,
as there's no reason to this. Finally, fixes a bug where
NO_GENERIC_LICENSE files would fail to be found in do_create_spdx
(because do_unpack was not run).

Joshua Watt (8):
  llvm-project-source: Use allarch.bbclass
  gcc-source: Use allarch.bbclass
  spdx3: Add recipe SPDX data
  spdx3: Add recipe SBoM task
  spdx3: Add is-native property
  spdx30: Include patch file information in VEX
  spdx: De-duplicate CreationInfo
  spdx_common: Check for dependent task in task flags

 meta/classes-global/sstate.bbclass            |   4 +-
 .../create-spdx-image-3.0.bbclass             |   4 +-
 .../create-spdx-sdk-3.0.bbclass               |   4 +-
 meta/classes-recipe/kernel.bbclass            |   2 +-
 meta/classes-recipe/nospdx.bbclass            |   1 +
 meta/classes/create-spdx-2.2.bbclass          |  12 +-
 meta/classes/create-spdx-3.0.bbclass          |  76 ++-
 meta/classes/spdx-common.bbclass              |  16 +-
 meta/lib/oe/sbom30.py                         | 192 ++++---
 meta/lib/oe/spdx30.py                         |   2 +-
 meta/lib/oe/spdx30_tasks.py                   | 487 +++++++++++++-----
 meta/lib/oe/spdx_common.py                    |  11 +
 .../meta/meta-world-recipe-sbom.bb            |  28 +
 .../clang/llvm-project-source.inc             |   8 +-
 meta/recipes-devtools/gcc/gcc-source.inc      |  16 +-
 15 files changed, 618 insertions(+), 245 deletions(-)
 create mode 100644 meta/recipes-core/meta/meta-world-recipe-sbom.bb

Comments

Mathieu Dubois-Briand Feb. 27, 2026, 7:32 a.m. UTC | #1
On Thu Feb 26, 2026 at 6:33 PM CET, Joshua Watt via lists.openembedded.org wrote:
> Changes the SPDX 3 output to include a "recipe" package that describe
> static information available at parse time (without building). This is
> primarily useful for gathering SPDX 3 VEX information about some or all
> recipes, enabling SPDX 3 to be used in place of cve_check.bbclass and
> vex.bbclass.
>
> Special thanks to Benjamin Robin <benjamin.robin@bootlin.com> for
> helping work through this.
>
> V2: Fixes a bug where do_populate_sysroot was running when it should not
> be. Drops the patch to ignore ASSUME_PROVIDES recipes, since this is
> incorrect (this is already handled by bitbake in the taskgraph, and
> doesn't need to be manually removed).
>
> V3: Fixes a bug where meta-world-recipe-sbom was reporting a circular
> dependency. meta-world-recipe-sbom also no longer runs in world builds,
> as there's no reason to this. Finally, fixes a bug where
> NO_GENERIC_LICENSE files would fail to be found in do_create_spdx
> (because do_unpack was not run).
>

Hi Joshua,

Thanks for new version, results looks way better overall, but we still
have a few errors.

I now have this issue on some builds:
ERROR: nativesdk-sdk-provides-dummy-1.0-r0 do_create_spdx: Could not find a static SPDX document named static-nativesdk-sdk-provides-dummy

https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/3335
https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/1105
https://autobuilder.yoctoproject.org/valkyrie/#/builders/40/builds/3238
https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/3233

And some errors in oe-selftests:
2026-02-26 20:46:06,214 - oe-selftest - INFO - newlib.NewlibTest.test_newlib (subunit.RemotedTestCase)
2026-02-26 20:46:06,215 - oe-selftest - INFO -  ... FAIL
...
ERROR: gcc-cross-x86_64-15.2.0-r0 do_create_spdx: Could not find a builds SPDX document named build-linux-libc-headers
...
ERROR: sysroot-test-1.0-r0 do_create_spdx: Could not find a builds SPDX document named build-sysroot-test-arch1
...
2026-02-26 21:03:31,870 - oe-selftest - INFO - sysroot.SysrootTests.test_sysroot_cleanup (subunit.RemotedTestCase)
2026-02-26 21:03:31,870 - oe-selftest - INFO -  ... FAIL
...
2026-02-26 21:20:09,006 - oe-selftest - INFO - spdx.SPDX30Check.test_custom_annotation_vars (subunit.RemotedTestCase)
2026-02-26 21:20:09,006 - oe-selftest - INFO -  ... FAIL
...
2026-02-26 21:20:09,006 - oe-selftest - INFO - 4: 37/50 612/670 (18.90s) (0 failed) (spdx.SPDX30Check.test_custom_annotation_vars)
2026-02-26 21:20:09,006 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/spdx.py", line 306, in test_custom_annotation_vars
    objset = self.check_recipe_spdx(
        "base-files",
    ...<7 lines>...
        ),
    )
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/spdx.py", line 123, in check_recipe_spdx
    return self.check_spdx_file(filename)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/spdx.py", line 81, in check_spdx_file
    self.assertExists(filename)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/case.py", line 249, in assertExists
    raise self.failureException(msg)
AssertionError: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2700993/tmp/deploy/spdx/3.0.1/qemux86_64/recipes/recipe-base-files.spdx.json' does not exist
...
ERROR: gawk-native-5.3.2-r0 do_create_spdx: Applying patch '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/recipes-extended/gawk/gawk/0001-configure.ac-re-enable-disabled-printf-features.patch' on target directory '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2700993/tmp/work/x86_64-linux/gawk-native/5.3.2/spdx/3.0.1/work/sources/gawk-5.3.2'
CmdError('quilt --quiltrc /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2700993/tmp/work/x86_64-linux/gawk-native/5.3.2/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout:
stderr: /bin/sh: 1: quilt: not found
')
ERROR: bzip2-native-1.0.8-r0 do_create_spdx: Applying patch '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/recipes-extended/bzip2/bzip2/0001-fix-bzip2-version-tmp-aaa-will-hang.patch' on target directory '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2700993/tmp/work/x86_64-linux/bzip2-native/1.0.8/spdx/3.0.1/work/sources/bzip2-1.0.8'
CmdError('quilt --quiltrc /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-2700993/tmp/work/x86_64-linux/bzip2-native/1.0.8/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout:
stderr: /bin/sh: 1: quilt: not found
')
...
2026-02-26 21:27:01,979 - oe-selftest - INFO - spdx.SPDX30Check.test_gcc_include_source (subunit.RemotedTestCase)
2026-02-26 21:27:01,980 - oe-selftest - INFO -  ... FAIL
...
2026-02-26 21:27:26,620 - oe-selftest - INFO - spdx.SPDX30Check.test_kernel_config_spdx (subunit.RemotedTestCase)
2026-02-26 21:27:26,621 - oe-selftest - INFO -  ... FAIL
...

2026-02-26 21:27:50,868 - oe-selftest - INFO - spdx.SPDX30Check.test_packageconfig_spdx (subunit.RemotedTestCase)
2026-02-26 21:27:50,868 - oe-selftest - INFO -  ... FAIL
...

Also, it looks like you did not add yourself as a maintainer of
meta-world-recipe-sbom.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3288
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3405
https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3463

Thanks,
Mathieu