| Message ID | 20260107232019.2649430-1-JPEWhacker@gmail.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | spdx: Add yocto PURLs | expand |
On Thu Jan 8, 2026 at 12:20 AM CET, Joshua Watt via lists.openembedded.org wrote: > Adds code to add PURLs to packages based on the PURL specification for > Yocto packages [1]. > > The SPDX_PACKAGE_URL variable is renamed SPDX_PACKAGE_URLS to make it > clear that it can now be a list of PURLs. SPDX_PACKAGE_URL is retained, > but marked as deprecated. > > [1]: https://github.com/package-url/purl-spec/blob/main/types-doc/yocto-definition.md > > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> > --- Hi Joshua, Thanks for your patch. > --- a/meta/lib/oe/spdx_common.py > +++ b/meta/lib/oe/spdx_common.py > @@ -10,6 +10,7 @@ import json > import oe.packagedata > import re > import shutil > +import urllib.parse > > from pathlib import Path > from dataclasses import dataclass > @@ -288,3 +289,14 @@ def get_compiled_sources(d): > types.add(ext) > bb.debug(1, f"Num of sources: {len(sources)} and types: {len(types)} {str(types)}") > return sources, types > + > + > +def purl_quote(s): > + return urllib.parse.quote(s, safe="") > + > + > +def get_base_purl(d): > + layername = d.getVar("FILE_LAYERNAME").lower() > + bpn = d.getVar("BPN").lower() > + pv = d.getVar("PV") > + return f"pkg:yocto/{purl_quote(layername)}/{purl_quote(bpn)}@{purl_quote(pv)}" It looks like sometimes, FILE_LAYERNAME is not defined during selftests: 2026-01-08 12:00:20,364 - oe-selftest - INFO - buildoptions.ImageOptionsTests.test_ccache_tool (subunit.RemotedTestCase) 2026-01-08 12:00:20,365 - oe-selftest - INFO - ... FAIL ... ERROR: Unable to read virtual:native:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/recipes-devtools/ccache/ccache_4.12.2.bb Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/data_smart.py", line 465, in expandWithRefs s = __expand_python_regexp__.sub(varparse.python_sub, s) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/data_smart.py", line 154, in python_sub value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/utils.py", line 489, in better_eval return eval(source, ctx, locals) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "Var <SPDX_PACKAGE_URLS>", line 1, in <module> File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oe/spdx_common.py", line 299, in get_base_purl layername = d.getVar("FILE_LAYERNAME").lower() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'lower' ... 2026-01-08 12:04:59,178 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_sign_cascaded_uboot_fit_image (subunit.RemotedTestCase) 2026-01-08 12:04:59,179 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:06:05,604 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_sign_standalone_uboot_atf_tee_fit_image (subunit.RemotedTestCase) 2026-01-08 12:06:05,604 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:06:58,169 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_sign_standalone_uboot_fit_image (subunit.RemotedTestCase) 2026-01-08 12:06:58,169 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:07:40,880 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_sign_uboot_fit_image_without_spl (subunit.RemotedTestCase) 2026-01-08 12:07:40,880 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:08:11,262 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_sign_uboot_kernel_individual (subunit.RemotedTestCase) 2026-01-08 12:08:11,262 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:08:50,675 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_uboot_atf_tee_fit_image (subunit.RemotedTestCase) 2026-01-08 12:08:50,675 - oe-selftest - INFO - ... FAIL ... 026-01-08 12:08:50,857 - oe-selftest - INFO - liboe.CopyTreeTests.test_copy_tree_xattr (subunit.RemotedTestCase) 2026-01-08 12:08:50,857 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:09:29,398 - oe-selftest - INFO - fitimage.UBootFitImageTests.test_uboot_fit_image (subunit.RemotedTestCase) 2026-01-08 12:09:29,399 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:20:27,984 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_fit_image (subunit.RemotedTestCase) 2026-01-08 12:20:27,984 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:21:02,937 - oe-selftest - INFO - recipetool.RecipetoolCreateTests.test_recipetool_create_go (subunit.RemotedTestCase) 2026-01-08 12:21:02,937 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:21:03,484 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_fit_image_ext_dtb_dtbo (subunit.RemotedTestCase) 2026-01-08 12:21:03,484 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:21:34,129 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_fit_image_sign_initramfs (subunit.RemotedTestCase) 2026-01-08 12:21:34,129 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:22:00,488 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_fit_image_sign_initramfs_bundle (subunit.RemotedTestCase) 2026-01-08 12:22:00,489 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:22:23,908 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_get_compatible_from_dtb (subunit.RemotedTestCase) 2026-01-08 12:22:23,909 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:22:50,980 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_sign_fit_image_configurations (subunit.RemotedTestCase) 2026-01-08 12:22:50,980 - oe-selftest - INFO - ... FAIL ... 2026-01-08 12:23:38,183 - oe-selftest - INFO - fitimage.KernelFitImageRecipeTests.test_sign_fit_image_individual (subunit.RemotedTestCase) 2026-01-08 12:23:38,184 - oe-selftest - INFO - ... FAIL I did not copy the exact error for each test fail, but is is basically the same every times. https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2986 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2877 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3124 Can you have a look at these? Thanks, Mathieu
diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index 96c0b9722b..3adda24003 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -131,7 +131,16 @@ SPDX_PACKAGE_VERSION[doc] = "The version of a package, software_packageVersion \ SPDX_PACKAGE_URL ??= "" SPDX_PACKAGE_URL[doc] = "Provides a place for the SPDX data creator to record \ the package URL string (in accordance with the Package URL specification) for \ -a software Package." +a software Package. DEPRECATED - use SPDX_PACKAGE_URLS instead" + +SPDX_PACKAGE_URLS ?= "${SPDX_PACKAGE_URL} ${@oe.spdx_common.get_base_purl(d)}" +SPDX_PACKAGE_URLS[doc] = "A space separated list of Package URLs (purls) for \ + the software Package. The first item in this list will be listed as the \ + packageUrl property of the packages, and all purls (including the first \ + one) will be listed as external references. The default value is an auto \ + generated pkg:yocto purl based on the recipe name, version, and layer name. \ + Override this variable to replace the default, otherwise append or prepend \ + to add additional purls." IMAGE_CLASSES:append = " create-spdx-image-3.0" SDK_CLASSES += "create-spdx-sdk-3.0" diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index f731a709e3..01e7dcbbc6 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -639,12 +639,21 @@ def create_spdx(d): set_var_field("SUMMARY", spdx_package, "summary", package=package) set_var_field("DESCRIPTION", spdx_package, "description", package=package) - if d.getVar("SPDX_PACKAGE_URL:%s" % package) or d.getVar("SPDX_PACKAGE_URL"): - set_var_field( - "SPDX_PACKAGE_URL", - spdx_package, - "software_packageUrl", - package=package + purls = ( + d.getVar("SPDX_PACKAGE_URLS:%s" % package) + or d.getVar("SPDX_PACKAGE_URLS") + or "" + ).split() + + if purls: + spdx_package.software_packageUrl = purls[0] + + for p in sorted(set(purls)): + spdx_package.externalIdentifier.append( + oe.spdx30.ExternalIdentifier( + externalIdentifierType=oe.spdx30.ExternalIdentifierType.packageUrl, + identifier=p, + ) ) pkg_objset.new_scoped_relationship( diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 72c24180d5..3f0b25ddef 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -10,6 +10,7 @@ import json import oe.packagedata import re import shutil +import urllib.parse from pathlib import Path from dataclasses import dataclass @@ -288,3 +289,14 @@ def get_compiled_sources(d): types.add(ext) bb.debug(1, f"Num of sources: {len(sources)} and types: {len(types)} {str(types)}") return sources, types + + +def purl_quote(s): + return urllib.parse.quote(s, safe="") + + +def get_base_purl(d): + layername = d.getVar("FILE_LAYERNAME").lower() + bpn = d.getVar("BPN").lower() + pv = d.getVar("PV") + return f"pkg:yocto/{purl_quote(layername)}/{purl_quote(bpn)}@{purl_quote(pv)}"
Adds code to add PURLs to packages based on the PURL specification for Yocto packages [1]. The SPDX_PACKAGE_URL variable is renamed SPDX_PACKAGE_URLS to make it clear that it can now be a list of PURLs. SPDX_PACKAGE_URL is retained, but marked as deprecated. [1]: https://github.com/package-url/purl-spec/blob/main/types-doc/yocto-definition.md Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> --- meta/classes/create-spdx-3.0.bbclass | 11 ++++++++++- meta/lib/oe/spdx30_tasks.py | 21 +++++++++++++++------ meta/lib/oe/spdx_common.py | 12 ++++++++++++ 3 files changed, 37 insertions(+), 7 deletions(-)