From patchwork Wed Mar 18 13:44:29 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7AB7103E17F for ; Wed, 18 Mar 2026 13:47:09 +0000 (UTC) Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14405.1773841620596471280 for ; Wed, 18 Mar 2026 06:47:00 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=EQ+nhRq9; spf=pass (domain: gmail.com, ip: 209.85.210.42, mailfrom: jpewhacker@gmail.com) Received: by mail-ot1-f42.google.com with SMTP id 46e09a7af769-7d738fe814cso5345532a34.3 for ; Wed, 18 Mar 2026 06:47:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841619; x=1774446419; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GXI7idm+D1nl9zMby/r1oELBEq5rGXbSp2Z1dA7pCn0=; b=EQ+nhRq9C1XxXUTjYY9s39cZnSh4Zbsj9MIDYeEkbs48maKMLURMNoDdZR7sXQy9q9 byYXrRv2I+5p1R/qQn0sujq/u/VVsnPqAmJyK02ygoS5MGy9ZT8NN9ddVWO73rjyuDTj vD5OX8qO+Ab4G8XhKV+kSBPF2zd9TkRatsml/TW3toxe+hW2Wb6KEctG1lXbJiv4/IEs K47lRhayEtiImWHtAKnF+o544XE8JqgClMvQZPPF2pfrDFd7XFHGON5kJXSml91HI9Zn ePN5LI8A7PDWgLsdSyVvQjwwwyIaZFijLrnmQOvyfDsD79unkWNcFcVHzU1BIDqSM+P/ jZKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841619; x=1774446419; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=GXI7idm+D1nl9zMby/r1oELBEq5rGXbSp2Z1dA7pCn0=; b=cgLPlRWoNa/K0N8IUEIfkyb4LtkwaLGg6QXYnaIPp7t0giPpTtkhlkHs1DHyyC83o/ pUdHhN9GsqZ2WpGQ1pPMxGg2M4TtWVG2w7GlQO+OBL6WIH5fr788CrUrpIl3kIPxoVQY okX9Q98syuuj3g8z/YPljfgYM+6VULgOiML39II8mUX8KBwL51ovGua39dU02hpp0jLL fasADjAynmtGQ83eqS6ub8KhuyVBRMgq/c3y03N5pEGz/0da/2+FNIDurNlF6DB1eqvs 2V1PuudCCqDSfVTXo6IWG7W28vmn1gLj1fk+UvWXmOKLUPL+Ata+6IZff/mga2TfFfPu k19g== X-Gm-Message-State: AOJu0YyEjkoP9AIu5IBK0B8IOw0OHngjEkbqK+nkfLcyJZF/fIOnAwLv w+ZrcVb0yopkonwu3O3DHGO/17mFa6C6WVNqHNCYwLnVdHpXw/4wC0uiuGi6aA== X-Gm-Gg: ATEYQzwutt3L2oBELgns2g0MiZk7q5KKo1ZN7P+Q/VCHcaBVtlQmvQ21FpHKwM5FE3x 45iDSZUPUAzdDnjYTvLUOi1F1Wu/KJYI/HMBPULK37TdsJNgjpHkv2rfByQgpxL9esK01aUPbz/ ZMHN/0bvCA5cugrioCcd9V5WkB1hn/zIY/iCP7ASSMYLRe82fAxB4LmIOqPS9JEszzsp/Wq9DGZ 57YoHg5Dghu6+5YBRRb45ovVAkfJJT537MMZ5Xng9VBd0R190gXwn1Mqm1FSP0MTDM2D1Mn4Cyl Yj93UJI1f1H+ViMTJU/FC6zetbDzjIB2jlfQhH/gwdgmnzYagtQqjdPyKBnxl/isb2G0QWh0bju H8VHfiIyYE5EI1iikO0TfkJFiXPEi7UCAoNKppkDh2MscTngQ60ZkVbqiaNH3kYPdlgDOPQpkIo 4g3an2X2va5G9IDdaAcDOJ X-Received: by 2002:a05:6820:1b18:b0:67b:ab43:778d with SMTP id 006d021491bc7-67c0daa3720mr2372244eaf.28.1773841619245; Wed, 18 Mar 2026 06:46:59 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.46.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:46:58 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 01/12] spdx3: Add recipe SPDX data Date: Wed, 18 Mar 2026 07:44:29 -0600 Message-ID: <20260318134655.953233-2-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233391 Adds a new package to the SPDX output that represents the recipe data for a given recipe. Importantly, this data contains only things that can be determined statically from only the recipe, so it doesn't require fetching or building anything. This means that build time dependencies and CVE information for recipes can be analyzed without needing to actually do any builds. Sadly, license data cannot be included because NO_GENERIC_LICENSE means that actual license text might only be available after do_fetch Signed-off-by: Joshua Watt --- meta/classes-global/sstate.bbclass | 4 +- .../create-spdx-image-3.0.bbclass | 4 +- .../create-spdx-sdk-3.0.bbclass | 4 +- meta/classes-recipe/kernel.bbclass | 2 +- meta/classes-recipe/nospdx.bbclass | 1 + meta/classes/create-spdx-2.2.bbclass | 12 +- meta/classes/create-spdx-3.0.bbclass | 51 ++- meta/classes/spdx-common.bbclass | 21 +- meta/lib/oe/spdx30_tasks.py | 402 ++++++++++++------ meta/lib/oeqa/selftest/cases/spdx.py | 19 +- 10 files changed, 354 insertions(+), 166 deletions(-) diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass index 1c3df0f544..88449d19c7 100644 --- a/meta/classes-global/sstate.bbclass +++ b/meta/classes-global/sstate.bbclass @@ -945,7 +945,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True, extrapath = d.getVar("NATIVELSBSTRING") + "/" else: extrapath = "" - + tname = bb.runqueue.taskname_from_tid(task)[3:] if tname in ["fetch", "unpack", "patch", "populate_lic", "preconfigure"] and splithashfn[2]: @@ -1107,7 +1107,7 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None): logit("Considering setscene task: %s" % (str(taskdependees[task])), log) - directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_spdx", "do_deploy_archives"] + directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_spdx", "do_create_recipe_spdx", "do_deploy_archives"] def isNativeCross(x): return x.endswith("-native") or "-cross-" in x or "-crosssdk" in x or x.endswith("-cross") diff --git a/meta/classes-recipe/create-spdx-image-3.0.bbclass b/meta/classes-recipe/create-spdx-image-3.0.bbclass index 636ab14eb0..15a91e90e2 100644 --- a/meta/classes-recipe/create-spdx-image-3.0.bbclass +++ b/meta/classes-recipe/create-spdx-image-3.0.bbclass @@ -34,7 +34,7 @@ addtask do_create_rootfs_spdx after do_rootfs before do_image SSTATETASKS += "do_create_rootfs_spdx" do_create_rootfs_spdx[sstate-inputdirs] = "${SPDXROOTFSDEPLOY}" do_create_rootfs_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" -do_create_rootfs_spdx[recrdeptask] += "do_create_spdx do_create_package_spdx" +do_create_rootfs_spdx[recrdeptask] += "do_create_recipe_spdx do_create_spdx do_create_package_spdx" do_create_rootfs_spdx[cleandirs] += "${SPDXROOTFSDEPLOY}" do_create_rootfs_spdx[file-checksums] += "${SPDX3_DEP_FILES}" @@ -76,7 +76,7 @@ do_create_image_sbom_spdx[sstate-inputdirs] = "${SPDXIMAGEDEPLOYDIR}" do_create_image_sbom_spdx[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" do_create_image_sbom_spdx[stamp-extra-info] = "${MACHINE_ARCH}" do_create_image_sbom_spdx[cleandirs] = "${SPDXIMAGEDEPLOYDIR}" -do_create_image_sbom_spdx[recrdeptask] += "do_create_spdx do_create_package_spdx" +do_create_image_sbom_spdx[recrdeptask] += "do_create_recipe_spdx do_create_spdx do_create_package_spdx" do_create_image_sbom_spdx[file-checksums] += "${SPDX3_DEP_FILES}" python do_create_image_sbom_spdx_setscene() { diff --git a/meta/classes-recipe/create-spdx-sdk-3.0.bbclass b/meta/classes-recipe/create-spdx-sdk-3.0.bbclass index e5f220cdfa..a4b8ed3bf9 100644 --- a/meta/classes-recipe/create-spdx-sdk-3.0.bbclass +++ b/meta/classes-recipe/create-spdx-sdk-3.0.bbclass @@ -5,14 +5,14 @@ # # SPDX SDK tasks -do_populate_sdk[recrdeptask] += "do_create_spdx do_create_package_spdx" +do_populate_sdk[recrdeptask] += "do_create_recipe_spdx do_create_spdx do_create_package_spdx" do_populate_sdk[cleandirs] += "${SPDXSDKWORK}" do_populate_sdk[postfuncs] += "sdk_create_sbom" do_populate_sdk[file-checksums] += "${SPDX3_DEP_FILES}" POPULATE_SDK_POST_HOST_COMMAND:append:task-populate-sdk = " sdk_host_create_spdx" POPULATE_SDK_POST_TARGET_COMMAND:append:task-populate-sdk = " sdk_target_create_spdx" -do_populate_sdk_ext[recrdeptask] += "do_create_spdx do_create_package_spdx" +do_populate_sdk_ext[recrdeptask] += "do_create_recipe_spdx do_create_spdx do_create_package_spdx" do_populate_sdk_ext[cleandirs] += "${SPDXSDKEXTWORK}" do_populate_sdk_ext[postfuncs] += "sdk_ext_create_sbom" do_populate_sdk_ext[file-checksums] += "${SPDX3_DEP_FILES}" diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index 094c1148b6..2d8565bd55 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -883,7 +883,7 @@ do_create_spdx:append() { except Exception as e: bb.error(f"Failed to parse kernel config file: {e}") - path = oe.sbom30.jsonld_arch_path(d, pkg_arch, "recipes", f"recipe-{pn}", deploydir=deploydir) + path = oe.sbom30.jsonld_arch_path(d, pkg_arch, "builds", f"build-{pn}", deploydir=deploydir) build_objset = oe.sbom30.load_jsonld(d, path, required=True) build = build_objset.find_root(oe.spdx30.build_Build) if not build: diff --git a/meta/classes-recipe/nospdx.bbclass b/meta/classes-recipe/nospdx.bbclass index b20e28218b..90e14442ba 100644 --- a/meta/classes-recipe/nospdx.bbclass +++ b/meta/classes-recipe/nospdx.bbclass @@ -5,6 +5,7 @@ # deltask do_collect_spdx_deps +deltask do_create_recipe_spdx deltask do_create_spdx deltask do_create_spdx_runtime deltask do_create_package_spdx diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 65d10d86db..3288cdf75a 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -399,6 +399,15 @@ def get_license_list_version(license_data, d): return ".".join(license_data["licenseListVersion"].split(".")[:2]) +# This task is added for compatibility with tasks shared with SPDX 3, but +# doesn't do anything +do_create_recipe_spdx() { + : +} +do_create_recipe_spdx[noexec] = "1" +addtask do_create_recipe_spdx after do_collect_spdx_deps + + python do_create_spdx() { from datetime import datetime, timezone import oe.sbom @@ -594,7 +603,7 @@ python do_create_spdx() { } do_create_spdx[vardepsexclude] += "BB_NUMBER_THREADS" # NOTE: depending on do_unpack is a hack that is necessary to get it's dependencies for archive the source -addtask do_create_spdx after do_package do_packagedata do_unpack do_collect_spdx_deps before do_populate_sdk do_build do_rm_work +addtask do_create_spdx after do_create_recipe_spdx do_package do_packagedata do_unpack do_patch do_collect_spdx_deps before do_populate_sdk do_build do_rm_work SSTATETASKS += "do_create_spdx" do_create_spdx[sstate-inputdirs] = "${SPDXDEPLOY}" @@ -605,6 +614,7 @@ python do_create_spdx_setscene () { } addtask do_create_spdx_setscene +do_create_spdx[deptask] += "do_create_spdx" do_create_spdx[dirs] = "${SPDXWORK}" do_create_spdx[cleandirs] = "${SPDXDEPLOY} ${SPDXWORK}" do_create_spdx[depends] += " \ diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index d4575d61c4..672ca27cd0 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -159,11 +159,18 @@ SPDX3_DEP_FILES = "\ ${SPDX_LICENSES}:True \ " -python do_create_spdx() { +python do_create_recipe_spdx() { import oe.spdx30_tasks - oe.spdx30_tasks.create_spdx(d) + oe.spdx30_tasks.create_recipe_spdx(d) } -do_create_spdx[vardeps] += "\ +addtask do_create_recipe_spdx after do_collect_spdx_deps + +SSTATETASKS += "do_create_recipe_spdx" +do_create_recipe_spdx[sstate-inputdirs] = "${SPDXRECIPEDEPLOY}" +do_create_recipe_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" +do_create_recipe_spdx[file-checksums] += "${SPDX3_DEP_FILES}" +do_create_recipe_spdx[cleandirs] = "${SPDXRECIPEDEPLOY}" +do_create_recipe_spdx[vardeps] += "\ SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ SPDX_PACKAGE_ADDITIONAL_PURPOSE \ SPDX_PROFILES \ @@ -171,7 +178,19 @@ do_create_spdx[vardeps] += "\ SPDX_UUID_NAMESPACE \ " +python do_create_recipe_spdx_setscene () { + sstate_setscene(d) +} +addtask do_create_recipe_spdx_setscene + +python do_create_spdx() { + import oe.spdx30_tasks + oe.spdx30_tasks.create_spdx(d) +} addtask do_create_spdx after \ + do_unpack \ + do_patch \ + do_create_recipe_spdx \ do_collect_spdx_deps \ do_deploy_source_date_epoch \ do_populate_sysroot do_package do_packagedata \ @@ -181,18 +200,25 @@ SSTATETASKS += "do_create_spdx" do_create_spdx[sstate-inputdirs] = "${SPDXDEPLOY}" do_create_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" do_create_spdx[file-checksums] += "${SPDX3_DEP_FILES}" - -python do_create_spdx_setscene () { - sstate_setscene(d) -} -addtask do_create_spdx_setscene - +do_create_spdx[deptask] += "do_create_spdx" do_create_spdx[dirs] = "${SPDXWORK}" do_create_spdx[cleandirs] = "${SPDXDEPLOY} ${SPDXWORK}" do_create_spdx[depends] += " \ ${PATCHDEPENDENCY} \ ${@create_spdx_source_deps(d)} \ " +do_create_spdx[vardeps] += "\ + SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ + SPDX_PACKAGE_ADDITIONAL_PURPOSE \ + SPDX_PROFILES \ + SPDX_NAMESPACE_PREFIX \ + SPDX_UUID_NAMESPACE \ + " + +python do_create_spdx_setscene () { + sstate_setscene(d) +} +addtask do_create_spdx_setscene python do_create_package_spdx() { import oe.spdx30_tasks @@ -205,16 +231,15 @@ SSTATETASKS += "do_create_package_spdx" do_create_package_spdx[sstate-inputdirs] = "${SPDXRUNTIMEDEPLOY}" do_create_package_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" do_create_package_spdx[file-checksums] += "${SPDX3_DEP_FILES}" +do_create_package_spdx[dirs] = "${SPDXRUNTIMEDEPLOY}" +do_create_package_spdx[cleandirs] = "${SPDXRUNTIMEDEPLOY}" +do_create_package_spdx[rdeptask] = "do_create_spdx" python do_create_package_spdx_setscene () { sstate_setscene(d) } addtask do_create_package_spdx_setscene -do_create_package_spdx[dirs] = "${SPDXRUNTIMEDEPLOY}" -do_create_package_spdx[cleandirs] = "${SPDXRUNTIMEDEPLOY}" -do_create_package_spdx[rdeptask] = "do_create_spdx" - python spdx30_build_started_handler () { import oe.spdx30_tasks d = e.data.createCopy() diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index 3110230c9e..3c239a718b 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -23,6 +23,7 @@ SPDXDEPS = "${SPDXDIR}/deps.json" SPDX_TOOL_NAME ??= "oe-spdx-creator" SPDX_TOOL_VERSION ??= "1.0" +SPDXRECIPEDEPLOY = "${SPDXDIR}/recipe-deploy" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" SPDX_INCLUDE_SOURCES ??= "0" @@ -67,12 +68,6 @@ def create_spdx_source_deps(d): deps = [] if d.getVar("SPDX_INCLUDE_SOURCES") == "1": pn = d.getVar('PN') - # do_unpack is a hack for now; we only need it to get the - # dependencies do_unpack already has so we can extract the source - # ourselves - if oe.spdx_common.has_task(d, "do_unpack"): - deps.append("%s:do_unpack" % pn) - if oe.spdx_common.is_work_shared_spdx(d) and \ oe.spdx_common.process_sources(d): # For kernel source code @@ -84,8 +79,6 @@ def create_spdx_source_deps(d): # For gcc-source-${PV} source code if oe.spdx_common.has_task(d, "do_preconfigure"): deps.append("%s:do_preconfigure" % pn) - elif oe.spdx_common.has_task(d, "do_patch"): - deps.append("%s:do_patch" % pn) # For gcc-cross-x86_64 source code elif oe.spdx_common.has_task(d, "do_configure"): deps.append("%s:do_configure" % pn) @@ -97,8 +90,8 @@ python do_collect_spdx_deps() { # This task calculates the build time dependencies of the recipe, and is # required because while a task can deptask on itself, those dependencies # do not show up in BB_TASKDEPDATA. To work around that, this task does the - # deptask on do_create_spdx and writes out the dependencies it finds, then - # do_create_spdx reads in the found dependencies when writing the actual + # deptask on do_create_recipe_spdx and writes out the dependencies it finds, then + # downstream tasks read in the found dependencies when writing the actual # SPDX document import json import oe.spdx_common @@ -106,15 +99,13 @@ python do_collect_spdx_deps() { spdx_deps_file = Path(d.getVar("SPDXDEPS")) - deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + deps = oe.spdx_common.collect_direct_deps(d, "do_create_recipe_spdx") with spdx_deps_file.open("w") as f: json.dump(deps, f) } -# NOTE: depending on do_unpack is a hack that is necessary to get it's dependencies for archive the source -addtask do_collect_spdx_deps after do_unpack -do_collect_spdx_deps[depends] += "${PATCHDEPENDENCY}" -do_collect_spdx_deps[deptask] = "do_create_spdx" +addtask do_collect_spdx_deps +do_collect_spdx_deps[deptask] = "do_create_recipe_spdx" do_collect_spdx_deps[dirs] = "${SPDXDIR}" oe.spdx_common.collect_direct_deps[vardepsexclude] += "BB_TASKDEPDATA" diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 99f2892dfb..a8b4525e3d 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -32,7 +32,9 @@ def set_timestamp_now(d, o, prop): delattr(o, prop) -def add_license_expression(d, objset, license_expression, license_data): +def add_license_expression( + d, objset, license_expression, license_data, search_objsets=[] +): simple_license_text = {} license_text_map = {} license_ref_idx = 0 @@ -44,14 +46,15 @@ def add_license_expression(d, objset, license_expression, license_data): if name in simple_license_text: return simple_license_text[name] - lic = objset.find_filter( - oe.spdx30.simplelicensing_SimpleLicensingText, - name=name, - ) + for o in [objset] + search_objsets: + lic = o.find_filter( + oe.spdx30.simplelicensing_SimpleLicensingText, + name=name, + ) - if lic is not None: - simple_license_text[name] = lic - return lic + if lic is not None: + simple_license_text[name] = lic + return lic lic = objset.add( oe.spdx30.simplelicensing_SimpleLicensingText( @@ -178,7 +181,9 @@ def add_package_files( # Check if file is compiled if check_compiled_sources: - if not oe.spdx_common.is_compiled_source(filename, compiled_sources, types): + if not oe.spdx_common.is_compiled_source( + filename, compiled_sources, types + ): continue spdx_file = objset.new_file( @@ -293,17 +298,16 @@ def get_package_sources_from_debug( return dep_source_files -def collect_dep_objsets(d, build): +def collect_dep_objsets(d, subdir, fn_prefix, obj_type, **attr_filter): deps = oe.spdx_common.get_spdx_deps(d) dep_objsets = [] - dep_builds = set() + dep_objs = set() - dep_build_spdxids = set() for dep in deps: bb.debug(1, "Fetching SPDX for dependency %s" % (dep.pn)) - dep_build, dep_objset = oe.sbom30.find_root_obj_in_jsonld( - d, "recipes", "recipe-" + dep.pn, oe.spdx30.build_Build + dep_obj, dep_objset = oe.sbom30.find_root_obj_in_jsonld( + d, subdir, fn_prefix + dep.pn, obj_type, **attr_filter ) # If the dependency is part of the taskhash, return it to be linked # against. Otherwise, it cannot be linked against because this recipe @@ -311,10 +315,10 @@ def collect_dep_objsets(d, build): if dep.in_taskhash: dep_objsets.append(dep_objset) - # The build _can_ be linked against (by alias) - dep_builds.add(dep_build) + # The object _can_ be linked against (by alias) + dep_objs.add(dep_obj) - return dep_objsets, dep_builds + return dep_objsets, dep_objs def index_sources_by_hash(sources, dest): @@ -423,9 +427,7 @@ def add_download_files(d, objset): if fd.method.supports_checksum(fd): # TODO Need something better than hard coding this for checksum_id in ["sha256", "sha1"]: - expected_checksum = getattr( - fd, "%s_expected" % checksum_id, None - ) + expected_checksum = getattr(fd, "%s_expected" % checksum_id, None) if expected_checksum is None: continue @@ -462,50 +464,96 @@ def set_purposes(d, element, *var_names, force_purposes=[]): ] -def create_spdx(d): - def set_var_field(var, obj, name, package=None): - val = None - if package: - val = d.getVar("%s:%s" % (var, package)) +def set_purls(spdx_package, purls): + if purls: + spdx_package.software_packageUrl = purls[0] - if not val: - val = d.getVar(var) + for p in sorted(set(purls)): + spdx_package.externalIdentifier.append( + oe.spdx30.ExternalIdentifier( + externalIdentifierType=oe.spdx30.ExternalIdentifierType.packageUrl, + identifier=p, + ) + ) - if val: - setattr(obj, name, val) + +def create_recipe_spdx(d): + deploydir = Path(d.getVar("SPDXRECIPEDEPLOY")) + deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) + pn = d.getVar("PN") license_data = oe.spdx_common.load_spdx_license_data(d) - deploydir = Path(d.getVar("SPDXDEPLOY")) - deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) - spdx_workdir = Path(d.getVar("SPDXWORK")) - include_sources = d.getVar("SPDX_INCLUDE_SOURCES") == "1" - pkg_arch = d.getVar("SSTATE_PKGARCH") - is_native = bb.data.inherits_class("native", d) or bb.data.inherits_class( - "cross", d - ) include_vex = d.getVar("SPDX_INCLUDE_VEX") if not include_vex in ("none", "current", "all"): bb.fatal("SPDX_INCLUDE_VEX must be one of 'none', 'current', 'all'") - build_objset = oe.sbom30.ObjectSet.new_objset(d, "recipe-" + d.getVar("PN")) + recipe_objset = oe.sbom30.ObjectSet.new_objset(d, "static-" + pn) - build = build_objset.new_task_build("recipe", "recipe") - build_objset.set_element_alias(build) + recipe = recipe_objset.add_root( + oe.spdx30.software_Package( + _id=recipe_objset.new_spdxid("recipe", pn), + creationInfo=recipe_objset.doc.creationInfo, + name=d.getVar("PN"), + software_packageVersion=d.getVar("PV"), + software_primaryPurpose=oe.spdx30.software_SoftwarePurpose.specification, + software_sourceInfo=json.dumps( + { + "FILENAME": os.path.basename(d.getVar("FILE")), + "FILE_LAYERNAME": d.getVar("FILE_LAYERNAME"), + }, + separators=(",", ":"), + ), + ) + ) - build_objset.doc.rootElement.append(build) + set_purls(recipe, (d.getVar("SPDX_PACKAGE_URLS") or "").split()) + + # TODO: This doesn't work before do_unpack because the license text has to + # be available for recipes with NO_GENERIC_LICENSE + # recipe_spdx_license = add_license_expression( + # d, + # recipe_objset, + # d.getVar("LICENSE"), + # license_data, + # ) + # recipe_objset.new_relationship( + # [recipe], + # oe.spdx30.RelationshipType.hasDeclaredLicense, + # [oe.sbom30.get_element_link_id(recipe_spdx_license)], + # ) + + if val := d.getVar("HOMEPAGE"): + recipe.software_homePage = val + + if val := d.getVar("SUMMARY"): + recipe.summary = val + + if val := d.getVar("DESCRIPTION"): + recipe.description = val + + for cpe_id in oe.cve_check.get_cpe_ids( + d.getVar("CVE_PRODUCT"), d.getVar("CVE_VERSION") + ): + recipe.externalIdentifier.append( + oe.spdx30.ExternalIdentifier( + externalIdentifierType=oe.spdx30.ExternalIdentifierType.cpe23, + identifier=cpe_id, + ) + ) - build_objset.set_is_native(is_native) + dep_objsets, dep_recipes = collect_dep_objsets( + d, "static", "static-", oe.spdx30.software_Package + ) - for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): - build_objset.new_annotation( - build, - "%s=%s" % (var, d.getVar(var)), - oe.spdx30.AnnotationType.other, + if dep_recipes: + recipe_objset.new_scoped_relationship( + [recipe], + oe.spdx30.RelationshipType.dependsOn, + oe.spdx30.LifecycleScopeType.build, + sorted(oe.sbom30.get_element_link_id(dep) for dep in dep_recipes), ) - build_inputs = set() - # Add CVEs cve_by_status = {} if include_vex != "none": @@ -514,7 +562,7 @@ def create_spdx(d): decoded_status = { "mapping": patched_cve["abbrev-status"], "detail": patched_cve["status"], - "description": patched_cve.get("justification", None) + "description": patched_cve.get("justification", None), } # If this CVE is fixed upstream, skip it unless all CVEs are @@ -531,8 +579,7 @@ def create_spdx(d): bb.debug(1, "Skipping %s since it is already fixed upstream" % cve) continue - spdx_cve = build_objset.new_cve_vuln(cve) - build_objset.set_element_alias(spdx_cve) + spdx_cve = recipe_objset.new_cve_vuln(cve) cve_by_status.setdefault(decoded_status["mapping"], {})[cve] = ( spdx_cve, @@ -540,13 +587,118 @@ def create_spdx(d): decoded_status["description"], ) + all_cves = set() + for status, cves in cve_by_status.items(): + for cve, items in cves.items(): + spdx_cve, detail, description = items + spdx_cve_id = oe.sbom30.get_element_link_id(spdx_cve) + + all_cves.add(spdx_cve) + + if status == "Patched": + recipe_objset.new_vex_patched_relationship([spdx_cve_id], [recipe]) + elif status == "Unpatched": + recipe_objset.new_vex_unpatched_relationship([spdx_cve_id], [recipe]) + elif status == "Ignored": + spdx_vex = recipe_objset.new_vex_ignored_relationship( + [spdx_cve_id], + [recipe], + impact_statement=description, + ) + + vex_just_type = d.getVarFlag("CVE_CHECK_VEX_JUSTIFICATION", detail) + if vex_just_type: + if ( + vex_just_type + not in oe.spdx30.security_VexJustificationType.NAMED_INDIVIDUALS + ): + bb.fatal( + f"Unknown vex justification '{vex_just_type}', detail '{detail}', for ignored {cve}" + ) + + for v in spdx_vex: + v.security_justificationType = ( + oe.spdx30.security_VexJustificationType.NAMED_INDIVIDUALS[ + vex_just_type + ] + ) + + elif status == "Unknown": + bb.note(f"Skipping {cve} with status 'Unknown'") + else: + bb.fatal(f"Unknown {cve} status '{status}'") + + if all_cves: + recipe_objset.new_relationship( + [recipe], + oe.spdx30.RelationshipType.hasAssociatedVulnerability, + sorted(list(all_cves)), + ) + + oe.sbom30.write_recipe_jsonld_doc(d, recipe_objset, "static", deploydir) + + +def load_recipe_spdx(d): + + return oe.sbom30.find_root_obj_in_jsonld( + d, + "static", + "static-" + d.getVar("PN"), + oe.spdx30.software_Package, + ) + + +def create_spdx(d): + def set_var_field(var, obj, name, package=None): + val = None + if package: + val = d.getVar("%s:%s" % (var, package)) + + if not val: + val = d.getVar(var) + + if val: + setattr(obj, name, val) + + license_data = oe.spdx_common.load_spdx_license_data(d) + + pn = d.getVar("PN") + deploydir = Path(d.getVar("SPDXDEPLOY")) + deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) + spdx_workdir = Path(d.getVar("SPDXWORK")) + include_sources = d.getVar("SPDX_INCLUDE_SOURCES") == "1" + pkg_arch = d.getVar("SSTATE_PKGARCH") + is_native = bb.data.inherits_class("native", d) or bb.data.inherits_class( + "cross", d + ) + + recipe, recipe_objset = load_recipe_spdx(d) + + build_objset = oe.sbom30.ObjectSet.new_objset(d, "build-" + pn) + + build = build_objset.new_task_build("recipe", "recipe") + build_objset.set_element_alias(build) + + build_objset.doc.rootElement.append(build) + + build_objset.set_is_native(is_native) + + for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): + build_objset.new_annotation( + build, + "%s=%s" % (var, d.getVar(var)), + oe.spdx30.AnnotationType.other, + ) + + build_inputs = set() + cpe_ids = oe.cve_check.get_cpe_ids(d.getVar("CVE_PRODUCT"), d.getVar("CVE_VERSION")) source_files = add_download_files(d, build_objset) build_inputs |= source_files recipe_spdx_license = add_license_expression( - d, build_objset, d.getVar("LICENSE"), license_data + d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset] ) build_objset.new_relationship( source_files, @@ -575,7 +727,10 @@ def create_spdx(d): build_inputs |= files index_sources_by_hash(files, dep_sources) - dep_objsets, dep_builds = collect_dep_objsets(d, build) + dep_objsets, dep_builds = collect_dep_objsets( + d, "builds", "build-", oe.spdx30.build_Build + ) + if dep_builds: build_objset.new_scoped_relationship( [build], @@ -587,6 +742,22 @@ def create_spdx(d): debug_source_ids = set() source_hash_cache = {} + # Collect all VEX statements from the recipe + vex_statements = {} + for rel in recipe_objset.foreach_filter( + oe.spdx30.Relationship, + relationshipType=oe.spdx30.RelationshipType.hasAssociatedVulnerability, + ): + for cve in rel.to: + vex_statements[cve] = [] + + for cve in vex_statements.keys(): + for rel in recipe_objset.foreach_filter( + oe.spdx30.security_VexVulnAssessmentRelationship, + from_=cve, + ): + vex_statements[cve].append(rel) + # Write out the package SPDX data now. It is not complete as we cannot # write the runtime data, so write it to a staging area and a later task # will write out the final collection @@ -645,16 +816,7 @@ def create_spdx(d): or "" ).split() - if purls: - spdx_package.software_packageUrl = purls[0] - - for p in sorted(set(purls)): - spdx_package.externalIdentifier.append( - oe.spdx30.ExternalIdentifier( - externalIdentifierType=oe.spdx30.ExternalIdentifierType.packageUrl, - identifier=p, - ) - ) + set_purls(spdx_package, purls) pkg_objset.new_scoped_relationship( [oe.sbom30.get_element_link_id(build)], @@ -663,6 +825,13 @@ def create_spdx(d): [spdx_package], ) + pkg_objset.new_scoped_relationship( + [oe.sbom30.get_element_link_id(recipe)], + oe.spdx30.RelationshipType.generates, + oe.spdx30.LifecycleScopeType.build, + [spdx_package], + ) + for cpe_id in cpe_ids: spdx_package.externalIdentifier.append( oe.spdx30.ExternalIdentifier( @@ -696,7 +865,11 @@ def create_spdx(d): package_license = d.getVar("LICENSE:%s" % package) if package_license and package_license != d.getVar("LICENSE"): package_spdx_license = add_license_expression( - d, build_objset, package_license, license_data + d, + build_objset, + package_license, + license_data, + [recipe_objset], ) else: package_spdx_license = recipe_spdx_license @@ -721,58 +894,41 @@ def create_spdx(d): [oe.sbom30.get_element_link_id(concluded_spdx_license)], ) - # NOTE: CVE Elements live in the recipe collection - all_cves = set() - for status, cves in cve_by_status.items(): - for cve, items in cves.items(): - spdx_cve, detail, description = items - spdx_cve_id = oe.sbom30.get_element_link_id(spdx_cve) - - all_cves.add(spdx_cve_id) + # Copy CVEs from recipe + if vex_statements: + pkg_objset.new_relationship( + [spdx_package], + oe.spdx30.RelationshipType.hasAssociatedVulnerability, + sorted( + oe.sbom30.get_element_link_id(cve) + for cve in vex_statements.keys() + ), + ) - if status == "Patched": + for cve, vexes in vex_statements.items(): + for vex in vexes: + if vex.relationshipType == oe.spdx30.RelationshipType.fixedIn: pkg_objset.new_vex_patched_relationship( - [spdx_cve_id], [spdx_package] + [oe.sbom30.get_element_link_id(cve)], [spdx_package] ) - elif status == "Unpatched": + elif vex.relationshipType == oe.spdx30.RelationshipType.affects: pkg_objset.new_vex_unpatched_relationship( - [spdx_cve_id], [spdx_package] + [oe.sbom30.get_element_link_id(cve)], [spdx_package] ) - elif status == "Ignored": + elif ( + vex.relationshipType == oe.spdx30.RelationshipType.doesNotAffect + ): spdx_vex = pkg_objset.new_vex_ignored_relationship( - [spdx_cve_id], + [oe.sbom30.get_element_link_id(cve)], [spdx_package], - impact_statement=description, + impact_statement=vex.security_impactStatement, ) - vex_just_type = d.getVarFlag( - "CVE_CHECK_VEX_JUSTIFICATION", detail - ) - if vex_just_type: - if ( - vex_just_type - not in oe.spdx30.security_VexJustificationType.NAMED_INDIVIDUALS - ): - bb.fatal( - f"Unknown vex justification '{vex_just_type}', detail '{detail}', for ignored {cve}" - ) - + if vex.security_justificationType: for v in spdx_vex: - v.security_justificationType = oe.spdx30.security_VexJustificationType.NAMED_INDIVIDUALS[ - vex_just_type - ] - - elif status == "Unknown": - bb.note(f"Skipping {cve} with status 'Unknown'") - else: - bb.fatal(f"Unknown {cve} status '{status}'") - - if all_cves: - pkg_objset.new_relationship( - [spdx_package], - oe.spdx30.RelationshipType.hasAssociatedVulnerability, - sorted(list(all_cves)), - ) + v.security_justificationType = ( + vex.security_justificationType + ) bb.debug(1, "Adding package files to SPDX for package %s" % pkg_name) package_files = add_package_files( @@ -851,14 +1007,15 @@ def create_spdx(d): status = "enabled" if feature in enabled else "disabled" build.build_parameter.append( oe.spdx30.DictionaryEntry( - key=f"PACKAGECONFIG:{feature}", - value=status + key=f"PACKAGECONFIG:{feature}", value=status ) ) - bb.note(f"Added PACKAGECONFIG entries: {len(enabled)} enabled, {len(disabled)} disabled") + bb.note( + f"Added PACKAGECONFIG entries: {len(enabled)} enabled, {len(disabled)} disabled" + ) - oe.sbom30.write_recipe_jsonld_doc(d, build_objset, "recipes", deploydir) + oe.sbom30.write_recipe_jsonld_doc(d, build_objset, "builds", deploydir) def create_package_spdx(d): @@ -1197,17 +1354,17 @@ def create_image_spdx(d): image_path = image_deploy_dir / image_filename if os.path.isdir(image_path): a = add_package_files( - d, - objset, - image_path, - lambda file_counter: objset.new_spdxid( - "imagefile", str(file_counter) - ), - lambda filepath: [], - license_data=None, - ignore_dirs=[], - ignore_top_level_dirs=[], - archive=None, + d, + objset, + image_path, + lambda file_counter: objset.new_spdxid( + "imagefile", str(file_counter) + ), + lambda filepath: [], + license_data=None, + ignore_dirs=[], + ignore_top_level_dirs=[], + archive=None, ) artifacts.extend(a) else: @@ -1234,7 +1391,6 @@ def create_image_spdx(d): set_timestamp_now(d, a, "builtTime") - if artifacts: objset.new_scoped_relationship( [image_build], diff --git a/meta/lib/oeqa/selftest/cases/spdx.py b/meta/lib/oeqa/selftest/cases/spdx.py index 5830d7c087..759ca86b73 100644 --- a/meta/lib/oeqa/selftest/cases/spdx.py +++ b/meta/lib/oeqa/selftest/cases/spdx.py @@ -141,6 +141,11 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): SPDX_CLASS = "create-spdx-3.0" def test_base_files(self): + self.check_recipe_spdx( + "base-files", + "{DEPLOY_DIR_SPDX}/{MACHINE_ARCH}/static/static-base-files.spdx.json", + task="create_recipe_spdx", + ) self.check_recipe_spdx( "base-files", "{DEPLOY_DIR_SPDX}/{MACHINE_ARCH}/packages/package-base-files.spdx.json", @@ -149,7 +154,7 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): def test_gcc_include_source(self): objset = self.check_recipe_spdx( "gcc", - "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/recipes/recipe-gcc.spdx.json", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-gcc.spdx.json", extraconf="""\ SPDX_INCLUDE_SOURCES = "1" """, @@ -162,12 +167,12 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): if software_file.name == filename: found = True self.logger.info( - f"The spdxId of {filename} in recipe-gcc.spdx.json is {software_file.spdxId}" + f"The spdxId of {filename} in build-gcc.spdx.json is {software_file.spdxId}" ) break self.assertTrue( - found, f"Not found source file {filename} in recipe-gcc.spdx.json\n" + found, f"Not found source file {filename} in build-gcc.spdx.json\n" ) def test_core_image_minimal(self): @@ -305,7 +310,7 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): # This will fail with NameError if new_annotation() is called incorrectly objset = self.check_recipe_spdx( "base-files", - "{DEPLOY_DIR_SPDX}/{MACHINE_ARCH}/recipes/recipe-base-files.spdx.json", + "{DEPLOY_DIR_SPDX}/{MACHINE_ARCH}/builds/build-base-files.spdx.json", extraconf=textwrap.dedent( f"""\ ANNOTATION1 = "{ANNOTATION_VAR1}" @@ -360,8 +365,8 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): def test_kernel_config_spdx(self): kernel_recipe = get_bb_var("PREFERRED_PROVIDER_virtual/kernel") - spdx_file = f"recipe-{kernel_recipe}.spdx.json" - spdx_path = f"{{DEPLOY_DIR_SPDX}}/{{SSTATE_PKGARCH}}/recipes/{spdx_file}" + spdx_file = f"build-{kernel_recipe}.spdx.json" + spdx_path = f"{{DEPLOY_DIR_SPDX}}/{{SSTATE_PKGARCH}}/builds/{spdx_file}" # Make sure kernel is configured first bitbake(f"-c configure {kernel_recipe}") @@ -392,7 +397,7 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): def test_packageconfig_spdx(self): objset = self.check_recipe_spdx( "tar", - "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/recipes/recipe-tar.spdx.json", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-tar.spdx.json", extraconf="""\ SPDX_INCLUDE_PACKAGECONFIG = "1" """, From patchwork Wed Mar 18 13:44:30 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83736 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0287A103E181 for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oo1-f52.google.com (mail-oo1-f52.google.com [209.85.161.52]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14406.1773841620949327854 for ; Wed, 18 Mar 2026 06:47:01 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=aOyJ4ZaS; spf=pass (domain: gmail.com, ip: 209.85.161.52, mailfrom: jpewhacker@gmail.com) Received: by mail-oo1-f52.google.com with SMTP id 006d021491bc7-67bb3a6c6f3so564688eaf.1 for ; Wed, 18 Mar 2026 06:47:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841620; x=1774446420; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G2XuK1/Em0BrU2YK1t33ZIx4D+QmhWiHU8dV8JDK01I=; b=aOyJ4ZaSCQzvv2twndUrM4BTvXeJ43kTlX+77xH88tF2CBtLWNTEi272IS6kIibpWp P2P91AnVEyinu0W5OW5fWqVG8zhrgWlQTGwDCEt2gFiAxjBTtwsN70YXgNK0J2LUPdDr +sn5h+aGfDUsm2gi9iB8Eumhw0ppi/1wmhu+CUoeOku8pLaORyK2OAKNFgY+P35lOxtK yjFlYspwRzsMwvj/obxHa4vxFWtzA//48fNOe+JX8KUtpid7ImGsj3sNok5pZgQ3oCl3 Je7ldGHBKAwS1C9CRnM0TiEtLbcS/pBu0l2IqH+EAuOn3CvRj1dNxUx4jCIpkkmdUzZo SbnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841620; x=1774446420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=G2XuK1/Em0BrU2YK1t33ZIx4D+QmhWiHU8dV8JDK01I=; b=FMuzOi/XjuVe2UzkPa5Z2zPYqdz7AkrE11An+41FhVfXds3m40Z/MYrk7JYKzXSnX0 6nd2GCcrNi8OAGGIyUUhW9k8XavaZin8JpYrp+VMWlLc0OaMjqWIXaLd+Ch7aQKAhVlT c19T2LjhLSJJ5O07Co0JkKUNhlg+00LmP7CXxO37rUPpalRCGnWfqVkja0bApT5/rKkn T1NHInwceSs7b+zniortiQoXTP5PEptqBju+HqahWY2FxVQ9vuPlmk06Epv4H9HXRBJW nCTp4USRqFqRAy9lot5J8+lLUWuvUVF8tkd7gpGYwzaoZMKb/9zJSO2UIgRFS6ckpEu9 sJBg== X-Gm-Message-State: AOJu0YyfcrXVeCVxhEpxXbo3sxMJ1NC7tE6LJ6EA+QtLq+krcGpQwpnr WSkGQGgpnRpIo1pv8ocezEm5hsEab6Zx9dL1AFCFYbXdN/pN/TAmxDhtfoBmjg== X-Gm-Gg: ATEYQzzkxHMtYvRIdqzpfCD53hNj++fQcCz+qvLNy0+Z9RjC2wTm/BMIf2YI/Jxx/Pi pubo122qzdMShFQGWRCW7BSiB9paHTsioqg1IJsSrLBW97aBbqTTpCK22wtoeZL0hSqaWj2Qetz XmChYo9aI20fuAnPivmPEVPkN42DdVs3lBImDqx86miA20PQv2dZ3YBmghUHBimRlj7/27yjzw4 P+2pxh1ntZWaJS379xip0E+b87tA+vNVAnJYexQOukcj+TzET1979w90Hq1I/1tzZWa8bEsod2a 2MCSll0zaw5InYBkYuovcAJHQ2UOavwz9rrfygrlPGuWLHzh/LGl7GzKf+3tPcCZr1mfs/56+WD g6XLWx5hMkTXSeSEyU/fiszl6/A2SWDdnfx35YXWN8/4QqREsOYunVHI0lVX1cAPutIdSkXotLy h/MK1viDguHMPWtKtIs39/ X-Received: by 2002:a05:6820:20a:b0:67b:f12a:e020 with SMTP id 006d021491bc7-67c0ceb19f6mr2304356eaf.0.1773841619971; Wed, 18 Mar 2026 06:46:59 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.46.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:46:59 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 02/12] spdx3: Add recipe SBoM task Date: Wed, 18 Mar 2026 07:44:30 -0600 Message-ID: <20260318134655.953233-3-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233392 Adds a task that will create the complete recipe-level SBoM for a given target recipe, following all dependencies. For example: ``` bitbake -c create_recipe_sbom zstd ``` Would produce the complete recipe SBoM for the zstd recipe, include all build time dependencies (recursively). The complete SBoM for all (target) recipes can be built with: ``` bitbake meta-world-recipe-sbom ``` Signed-off-by: Joshua Watt --- meta/classes/create-spdx-3.0.bbclass | 32 ++++++++++++++++ meta/classes/spdx-common.bbclass | 1 + meta/conf/distro/include/maintainers.inc | 1 + meta/lib/oe/spdx30_tasks.py | 10 +++++ meta/lib/oeqa/selftest/cases/spdx.py | 9 +++++ .../meta/meta-world-recipe-sbom.bb | 38 +++++++++++++++++++ 6 files changed, 91 insertions(+) create mode 100644 meta/recipes-core/meta/meta-world-recipe-sbom.bb diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index 672ca27cd0..c3ea95b8bc 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -142,6 +142,10 @@ SPDX_PACKAGE_URLS[doc] = "A space separated list of Package URLs (purls) for \ Override this variable to replace the default, otherwise append or prepend \ to add additional purls." +SPDX_RECIPE_SBOM_NAME ?= "${PN}-recipe-sbom" +SPDX_RECIPE_SBOM_NAME[doc] = "The name of output recipe SBoM when using \ + create_recipe_sbom" + IMAGE_CLASSES:append = " create-spdx-image-3.0" SDK_CLASSES += "create-spdx-sdk-3.0" @@ -240,6 +244,34 @@ python do_create_package_spdx_setscene () { } addtask do_create_package_spdx_setscene +addtask do_create_recipe_sbom after create_recipe_spdx +python do_create_recipe_sbom() { + import oe.spdx30_tasks + from pathlib import Path + deploydir = Path(d.getVar("SPDXRECIPESBOMDEPLOY")) + oe.spdx30_tasks.create_recipe_sbom(d, deploydir) +} + +SSTATETASKS += "do_create_recipe_sbom" +do_create_recipe_sbom[recrdeptask] = "do_create_recipe_spdx" +do_create_recipe_sbom[nostamp] = "1" +do_create_recipe_sbom[sstate-inputdirs] = "${SPDXRECIPESBOMDEPLOY}" +do_create_recipe_sbom[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" +do_create_recipe_sbom[file-checksums] += "${SPDX3_DEP_FILES}" +do_create_recipe_sbom[cleandirs] = "${SPDXRECIPESBOMDEPLOY}" +do_create_recipe_sbom[vardeps] += "\ + SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ + SPDX_PACKAGE_ADDITIONAL_PURPOSE \ + SPDX_PROFILES \ + SPDX_NAMESPACE_PREFIX \ + SPDX_UUID_NAMESPACE \ + " + +python do_create_recipe_sbom_setscene () { + sstate_setscene(d) +} +addtask do_create_recipe_sbom_setscene + python spdx30_build_started_handler () { import oe.spdx30_tasks d = e.data.createCopy() diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index 3c239a718b..abf2332bee 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -25,6 +25,7 @@ SPDX_TOOL_VERSION ??= "1.0" SPDXRECIPEDEPLOY = "${SPDXDIR}/recipe-deploy" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" +SPDXRECIPESBOMDEPLOY = "${SPDXDIR}/recipes-bom-deploy" SPDX_INCLUDE_SOURCES ??= "0" SPDX_INCLUDE_COMPILED_SOURCES ??= "0" diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc index c7a646a643..86a11867a1 100644 --- a/meta/conf/distro/include/maintainers.inc +++ b/meta/conf/distro/include/maintainers.inc @@ -534,6 +534,7 @@ RECIPE_MAINTAINER:pn-meta-go-toolchain = "Richard Purdie X-Patchwork-Id: 83735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F560103E183 for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oa1-f54.google.com (mail-oa1-f54.google.com [209.85.160.54]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14407.1773841621896179765 for ; Wed, 18 Mar 2026 06:47:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=eb/DR3iE; spf=pass (domain: gmail.com, ip: 209.85.160.54, mailfrom: jpewhacker@gmail.com) Received: by mail-oa1-f54.google.com with SMTP id 586e51a60fabf-4042905015cso4354524fac.0 for ; Wed, 18 Mar 2026 06:47:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841621; x=1774446421; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+wYSZS61IRGHmLD9OmiNc9hgO2b0xZj5mQwHOFCwfH8=; b=eb/DR3iENbNVwlUcPmJS/sTyr7cAhpFV5h3V5QaSJTyk+cRg6OPtTnZYQ54CTb8rZW e2JSVAupujiWHUTHEZ/iofwfTPpUNniwqnioRyhjHJvQrIyBOXBVOPLzF4CGakUErkEL yg1jsoLDnJoe9R5wx4saV+19tjpVJEzFZP36fCH2sQn0Be+kRNMf+nJMpcR5Gz2BMBz0 vsf5zh6OwAI2rPp0IXvFYfp1a+jG2/16q9SszaZpCmhu0z5tX1fOJa9m5TYkYrc1mgDm I3k+hGUV/SWPLOOfTKdEl6SAj1YqA6KSauG57pzenEmfRLhg/IjUfeOIxCd/wtfyoUUk /LfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841621; x=1774446421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+wYSZS61IRGHmLD9OmiNc9hgO2b0xZj5mQwHOFCwfH8=; b=gm0cYPncreYucVc11Jdzro9nLNPpcNWbyGIqsEG+qxFgRixHComBOgB2Ch8Zt24OeY 88a9fb2nPvz2OZH0WA01X71al/uWWQmWWCyFT/SGXZMSBlR8bfxtT5k87SUEIBwJOzBX u6kytSgLNBLH1mazZmL94dAxspRPi8qOEOjSVOudYm9hYZEfaBjsiSTlkIarVPxAvMMI kD7D6sCnKnWaWXw0MfHtjfy1hI+2gXVIxTMb4Pp5//65rjN9pBtTWSD/WyLaXunG0yGH as9YYPYcaIdn+IC3fY8tLpd13zBZTPYmmpXzg2ey4dryhr20zs3XV1ccMh0Cpo8Aemx6 L4lw== X-Gm-Message-State: AOJu0Yys+OEuUOXqB/EyQIK0e5oRuhy9bh8Ch4Zaer8apyfkp9ZIMjnh yeNLT+5OYFJ7KzIdXzDA6C/9D2hFuq3m+bIjX+oyIK0tWvBUpoExPHTyPTEJZw== X-Gm-Gg: ATEYQzwcu1QRLxD9Hm1eMINSTDvlRWHqflCiNxrj8+pbga8QKAfhArRynCN6XQvRon7 KITVHeoDQbATan+5YrQpZ3X/WfeGsJwX3Y2fnbNUbrmd/OwItGni4V1wk3bhS7DGE2XaSrOXsCc Yi24fF1oZFlHaq7feUyfIbkmz1Sj2jDj6REEUT/hn/li/n/62AftDp9GjH5q2ouXQgL6tSaEWUD emOfDRbis3qwuh+grgkUTv39iLJWzlpdrLpJ+gvMFy9fP+shH4406WwBC0HoIfUK0PB815+kzfm Ilk4xQpuhN8AGeFboNCCs0A8vquT3VE+jRA5jMUpSvR8rIdCtiKbCUODauLmQSGo+sg9wEDDHM1 dWEKJ5bnL3modW0WnCkM6JknHe5TrwKYq1Oez/yLPOQmyoYJj4lSi4uPHdCsIo0Glyo76Wv6TRj EPP01aEuidElYsIq0A4ORl X-Received: by 2002:a05:6820:2014:b0:67b:a77a:e678 with SMTP id 006d021491bc7-67c0db1c763mr2499051eaf.54.1773841620991; Wed, 18 Mar 2026 06:47:00 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:00 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 03/12] spdx3: Add is-native property Date: Wed, 18 Mar 2026 07:44:31 -0600 Message-ID: <20260318134655.953233-4-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233393 Adds a custom is-native property to the recipe package to indicate if it is a native recipe Signed-off-by: Joshua Watt --- meta/lib/oe/sbom30.py | 20 ++++++++++++++++++++ meta/lib/oe/spdx30_tasks.py | 18 +++++++++++------- 2 files changed, 31 insertions(+), 7 deletions(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 227ac51877..50a72fce39 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -118,6 +118,26 @@ class OEDocumentExtension(oe.spdx30.extension_Extension): ) +@oe.spdx30.register(OE_SPDX_BASE + "recipe-extension") +class OERecipeExtension(oe.spdx30.extension_Extension): + """ + This extension is added to recipe software_Packages to indicate various + useful bits of information about the recipe + """ + + CLOSED = True + + @classmethod + def _register_props(cls): + super()._register_props() + cls._add_property( + "is_native", + oe.spdx30.BooleanProp(), + OE_SPDX_BASE + "is-native", + max_count=1, + ) + + def spdxid_hash(*items): h = hashlib.md5() for i in items: diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index b6c917045e..a8fffbb085 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -477,6 +477,10 @@ def set_purls(spdx_package, purls): ) +def get_is_native(d): + return bb.data.inherits_class("native", d) or bb.data.inherits_class("cross", d) + + def create_recipe_spdx(d): deploydir = Path(d.getVar("SPDXRECIPEDEPLOY")) deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) @@ -507,6 +511,11 @@ def create_recipe_spdx(d): ) ) + if get_is_native(d): + ext = oe.sbom30.OERecipeExtension() + ext.is_native = True + recipe.extension.append(ext) + set_purls(recipe, (d.getVar("SPDX_PACKAGE_URLS") or "").split()) # TODO: This doesn't work before do_unpack because the license text has to @@ -668,9 +677,7 @@ def create_spdx(d): spdx_workdir = Path(d.getVar("SPDXWORK")) include_sources = d.getVar("SPDX_INCLUDE_SOURCES") == "1" pkg_arch = d.getVar("SSTATE_PKGARCH") - is_native = bb.data.inherits_class("native", d) or bb.data.inherits_class( - "cross", d - ) + is_native = get_is_native(d) recipe, recipe_objset = load_recipe_spdx(d) @@ -1021,14 +1028,11 @@ def create_spdx(d): def create_package_spdx(d): deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) deploydir = Path(d.getVar("SPDXRUNTIMEDEPLOY")) - is_native = bb.data.inherits_class("native", d) or bb.data.inherits_class( - "cross", d - ) providers = oe.spdx_common.collect_package_providers(d) pkg_arch = d.getVar("SSTATE_PKGARCH") - if is_native: + if get_is_native(d): return bb.build.exec_func("read_subpackage_metadata", d) From patchwork Wed Mar 18 13:44:32 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3662D103E185 for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14408.1773841623144559638 for ; Wed, 18 Mar 2026 06:47:03 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=BEQfEMw4; spf=pass (domain: gmail.com, ip: 209.85.167.176, mailfrom: jpewhacker@gmail.com) Received: by mail-oi1-f176.google.com with SMTP id 5614622812f47-46701f2077cso705571b6e.0 for ; Wed, 18 Mar 2026 06:47:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841622; x=1774446422; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GygUv99XXYXKewxHRq+dVr3zF/qZkskkMg0YO5XWbQU=; b=BEQfEMw4A/w6r5B+zcpq1qQ1NRh4jc6Jw0F1PzglslkMQSzd8sOsRDcyn7h10XbArv orv2vRroIzano3tY7H+VtylYIKfvMHg6+40xcwgAtDi0eiBAIaafimVal/hOoNaLeyY2 8qD/rHLgFGgPGAPx+0RlvVpX9AtVYhqpf0nyECAsQ0Yw363fIBOEtByzCOjgpPFyk+Gz b/i8c3PmMXfERpcKQhacVn+uuCAFCkWveMCMDie3qi3yCAVJePGXcGZlaoyNB+53YjPL B0zqFlcDIV3oZQlDp+T7A/tk0lKRGSbhnN/vZcUFNl1V1wc4cXA+YBMSvKBu91UQLtZ5 nkpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841622; x=1774446422; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=GygUv99XXYXKewxHRq+dVr3zF/qZkskkMg0YO5XWbQU=; b=EZOZf+tlUTxVkCDnrrSOzrGYY6TYDFcJKnRvEJRWP7LRsbJVMj1S/hV6WtimOWwDQX fWPfB5KS5TN7fL1Js18jTFt/tKhmKITBFVYBCnKiSS1SggmThAA5Gj2wmgNTu9OLlx7q XLnUs6v8v5sx7jZD5HRri5SF7i5SJHkRdBW6JYwuh9T/hZCXASKTcWqfvbhKcEuYVBku yOnzEtMbFVPuF5ogUqAnhE67Z4SrdSmKjNrBkmp43UTA38vFvwKG9iiHUuCzEox5X0RO SJ4G89NojcDMCCtZm87XxRyItmPqQy9x7+/HVnQWncwIbN6DPKXNpqeCtzzKCgB5SF1K 0YFg== X-Gm-Message-State: AOJu0YxGG3RGutZk+Yrce+ghScnCCTiFQz30unM6kksBTbdcW0nVly1z cQWRXilsycUZdjvd05XdGic0R2pnS7RJdVT9JyXGmFXcBLSapx9UAiFK8rXZOA== X-Gm-Gg: ATEYQzwLUBIUWHuEGVsyQ4VmgyaBYijC8bJnGM2nbNT+YQMNKFSCitqcWDLN4U67DEh 87dKPtL33+Pny55IufRslMHXHnezEzCTNK4ukjkaskWJGHZHRxiwZaSMaY5Joh3S3+XNlSDK1ZL AW5B13Kz6O6h6HrWTScu6kck2MTFuPdHFDrthuPHTw/GHTIaKtMNw5JR5KSVMVWsmoGiYXzXy2D YZVjJR2BBCNwBdK0+btKX/NL2MTm2l0eaTwrvBQDD6FZMjMiU/x6EKwDm8+wbh7tuvj3BKsa3Sw ZLIqeM0a9sPeWOAEOw+xqUAysOQ/d/D45BfLosOeBSXUHO5kNJCjbyeaYhfpBk+GuER2cGLY5tv TB2RAv5a9DIOVRxyGkck2TfmehjogP3B1EIQTtG3oGFrSa72gOWuR2Ltxv8eeoReQvh+D7KbFRB 5erWyqOZ0HlrkpqN5fNcNavbFpbk6ASNA= X-Received: by 2002:a05:6820:f02e:b0:67b:f13d:afe8 with SMTP id 006d021491bc7-67c0d04d9e0mr2090156eaf.22.1773841622028; Wed, 18 Mar 2026 06:47:02 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:01 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 04/12] spdx30: Include patch file information in VEX Date: Wed, 18 Mar 2026 07:44:32 -0600 Message-ID: <20260318134655.953233-5-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233394 Modifies the SPDX VEX output to include the patches that fix a particular vulnerability. This is done by adding a `patchedBy` relationship from the `VexFixedVulnAssessmentRelationship` to the `File` that provides the fix. If the file can be located without fetching (e.g. is a file:// in SRC_URI), the checksum will be included. Signed-off-by: Joshua Watt --- meta/lib/oe/sbom30.py | 60 ++++++++++++++------------- meta/lib/oe/spdx30_tasks.py | 81 ++++++++++++++++++++++++++++--------- 2 files changed, 92 insertions(+), 49 deletions(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 50a72fce39..21f084dc16 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -620,37 +620,38 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): ) spdx_file.extension.append(OELicenseScannedExtension()) - def new_file(self, _id, name, path, *, purposes=[]): - sha256_hash = bb.utils.sha256_file(path) + def new_file(self, _id, name, path, *, purposes=[], hashfile=True): + if hashfile: + sha256_hash = bb.utils.sha256_file(path) - for f in self.by_sha256_hash.get(sha256_hash, []): - if not isinstance(f, oe.spdx30.software_File): - continue + for f in self.by_sha256_hash.get(sha256_hash, []): + if not isinstance(f, oe.spdx30.software_File): + continue - if purposes: - new_primary = purposes[0] - new_additional = [] + if purposes: + new_primary = purposes[0] + new_additional = [] - if f.software_primaryPurpose: - new_additional.append(f.software_primaryPurpose) - new_additional.extend(f.software_additionalPurpose) + if f.software_primaryPurpose: + new_additional.append(f.software_primaryPurpose) + new_additional.extend(f.software_additionalPurpose) - new_additional = sorted( - list(set(p for p in new_additional if p != new_primary)) - ) + new_additional = sorted( + list(set(p for p in new_additional if p != new_primary)) + ) - f.software_primaryPurpose = new_primary - f.software_additionalPurpose = new_additional + f.software_primaryPurpose = new_primary + f.software_additionalPurpose = new_additional - if f.name != name: - for e in f.extension: - if isinstance(e, OEFileNameAliasExtension): - e.aliases.append(name) - break - else: - f.extension.append(OEFileNameAliasExtension(aliases=[name])) + if f.name != name: + for e in f.extension: + if isinstance(e, OEFileNameAliasExtension): + e.aliases.append(name) + break + else: + f.extension.append(OEFileNameAliasExtension(aliases=[name])) - return f + return f spdx_file = oe.spdx30.software_File( _id=_id, @@ -661,12 +662,13 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): spdx_file.software_primaryPurpose = purposes[0] spdx_file.software_additionalPurpose = purposes[1:] - spdx_file.verifiedUsing.append( - oe.spdx30.Hash( - algorithm=oe.spdx30.HashAlgorithm.sha256, - hashValue=sha256_hash, + if hashfile: + spdx_file.verifiedUsing.append( + oe.spdx30.Hash( + algorithm=oe.spdx30.HashAlgorithm.sha256, + hashValue=sha256_hash, + ) ) - ) return self.add(spdx_file) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index a8fffbb085..aec47d4f81 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -568,44 +568,63 @@ def create_recipe_spdx(d): if include_vex != "none": patched_cves = oe.cve_check.get_patched_cves(d) for cve, patched_cve in patched_cves.items(): - decoded_status = { - "mapping": patched_cve["abbrev-status"], - "detail": patched_cve["status"], - "description": patched_cve.get("justification", None), - } + mapping = patched_cve["abbrev-status"] + detail = patched_cve["status"] + description = patched_cve.get("justification", None) + resources = patched_cve.get("resource", []) # If this CVE is fixed upstream, skip it unless all CVEs are # specified. - if ( - include_vex != "all" - and "detail" in decoded_status - and decoded_status["detail"] - in ( - "fixed-version", - "cpe-stable-backport", - ) + if include_vex != "all" and detail in ( + "fixed-version", + "cpe-stable-backport", ): bb.debug(1, "Skipping %s since it is already fixed upstream" % cve) continue spdx_cve = recipe_objset.new_cve_vuln(cve) - cve_by_status.setdefault(decoded_status["mapping"], {})[cve] = ( + cve_by_status.setdefault(mapping, {})[cve] = ( spdx_cve, - decoded_status["detail"], - decoded_status["description"], + detail, + description, + resources, ) all_cves = set() for status, cves in cve_by_status.items(): for cve, items in cves.items(): - spdx_cve, detail, description = items + spdx_cve, detail, description, resources = items spdx_cve_id = oe.sbom30.get_element_link_id(spdx_cve) all_cves.add(spdx_cve) if status == "Patched": - recipe_objset.new_vex_patched_relationship([spdx_cve_id], [recipe]) + spdx_vex = recipe_objset.new_vex_patched_relationship( + [spdx_cve_id], [recipe] + ) + patches = [] + for idx, filepath in enumerate(resources): + patches.append( + recipe_objset.new_file( + recipe_objset.new_spdxid( + "patch", str(idx), os.path.basename(filepath) + ), + os.path.basename(filepath), + filepath, + purposes=[oe.spdx30.software_SoftwarePurpose.patch], + hashfile=os.path.isfile(filepath), + ) + ) + + if patches: + recipe_objset.new_scoped_relationship( + spdx_vex, + oe.spdx30.RelationshipType.patchedBy, + oe.spdx30.LifecycleScopeType.build, + patches, + ) + elif status == "Unpatched": recipe_objset.new_vex_unpatched_relationship([spdx_cve_id], [recipe]) elif status == "Ignored": @@ -751,12 +770,14 @@ def create_spdx(d): # Collect all VEX statements from the recipe vex_statements = {} + vex_patches = {} for rel in recipe_objset.foreach_filter( oe.spdx30.Relationship, relationshipType=oe.spdx30.RelationshipType.hasAssociatedVulnerability, ): for cve in rel.to: vex_statements[cve] = [] + vex_patches[cve] = [] for cve in vex_statements.keys(): for rel in recipe_objset.foreach_filter( @@ -764,6 +785,13 @@ def create_spdx(d): from_=cve, ): vex_statements[cve].append(rel) + if rel.relationshipType == oe.spdx30.RelationshipType.fixedIn: + for patch_rel in recipe_objset.foreach_filter( + oe.spdx30.Relationship, + relationshipType=oe.spdx30.RelationshipType.patchedBy, + from_=rel, + ): + vex_patches[cve].extend(patch_rel.to) # Write out the package SPDX data now. It is not complete as we cannot # write the runtime data, so write it to a staging area and a later task @@ -889,7 +917,9 @@ def create_spdx(d): # Add concluded license relationship if manually set # Only add when license analysis has been explicitly performed - concluded_license_str = d.getVar("SPDX_CONCLUDED_LICENSE:%s" % package) or d.getVar("SPDX_CONCLUDED_LICENSE") + concluded_license_str = d.getVar( + "SPDX_CONCLUDED_LICENSE:%s" % package + ) or d.getVar("SPDX_CONCLUDED_LICENSE") if concluded_license_str: concluded_spdx_license = add_license_expression( d, build_objset, concluded_license_str, license_data @@ -915,9 +945,20 @@ def create_spdx(d): for cve, vexes in vex_statements.items(): for vex in vexes: if vex.relationshipType == oe.spdx30.RelationshipType.fixedIn: - pkg_objset.new_vex_patched_relationship( + spdx_vex = pkg_objset.new_vex_patched_relationship( [oe.sbom30.get_element_link_id(cve)], [spdx_package] ) + if vex_patches[cve]: + pkg_objset.new_scoped_relationship( + spdx_vex, + oe.spdx30.RelationshipType.patchedBy, + oe.spdx30.LifecycleScopeType.build, + [ + oe.sbom30.get_element_link_id(p) + for p in vex_patches[cve] + ], + ) + elif vex.relationshipType == oe.spdx30.RelationshipType.affects: pkg_objset.new_vex_unpatched_relationship( [oe.sbom30.get_element_link_id(cve)], [spdx_package] From patchwork Wed Mar 18 13:44:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B78A103E187 for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oa1-f43.google.com (mail-oa1-f43.google.com [209.85.160.43]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14409.1773841624080905191 for ; Wed, 18 Mar 2026 06:47:04 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gjXgshl2; spf=pass (domain: gmail.com, ip: 209.85.160.43, mailfrom: jpewhacker@gmail.com) Received: by mail-oa1-f43.google.com with SMTP id 586e51a60fabf-40974bf7781so645680fac.0 for ; Wed, 18 Mar 2026 06:47:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841623; x=1774446423; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RgZWWYUETcp+KAK54Y1gHYnk1xGLeXOCGllecsNhDhw=; b=gjXgshl2ex3TjCtBIVAiMvdUYQMnoU/NxMtDSsz+PIClwxrg9eGfy0D8cG6GoH9wZC RH01ogaXctku0a+wI+ytsmChmbwZnwtmIO/k9SYl8WdJYYYyGOtud9PO9dyxQ3E9MEiI 5bwgA+1vNKx5Pp7B7vbDOvIhfI3fyi4433BBq0hcXF7rDaL2sZNWBdxQ3eeOgAvDifX2 IbaeCBJX2e3A+guCH9mcws7uffNguK5C+h8ps0EuLSp6UeKwTLCvRKRoWx7ov206SSCx loRmn5kajDnfG1G2tbHnDl/QWoI0SWfGI3zczm1mrC70/RXVsevDadcx2IY4zH+5UJ9G yJyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841623; x=1774446423; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=RgZWWYUETcp+KAK54Y1gHYnk1xGLeXOCGllecsNhDhw=; b=mwNVOfk0oemtgNcFdF59Tb/EeZ3/05sBtHC9KNUwEnp1xr0rGK8C6rc7I3SGeg64Ps EzG/T+oQRyRgFYdBUmbOXUF+2SuPfWQPR4Bi//E0bas0au7RUKglfPki1rbZOlwiF8ws l4bI4BDprX2LrHmb3oWMK219j3zRls3AjpZ9dfRLETsYCGKQ0AIL9tq1GAy07HJsS3A/ QYGEXfymFAyLXY+bIcF9PDMgDmxg3M+EFuxG6hfzbRK4Q2qa6PfEZngCmZ12044bcJ4O 7DBqEmgp5TbIciJrZlKBWqudNXE28XuAIj7AhPNiBy1jU9GNwkppGoD+SvBPrsK0IbB5 CtgA== X-Gm-Message-State: AOJu0Yzs1M1IknuioPD14WIrHvyuuhxOcne7MIxLaLxkqgOOcXZ/vqX1 OB382P3aW8I62eohkOHRjPkNezBscpaDt0BzzB6pIW4boG7Qn3rIdFzZhiI8qQ== X-Gm-Gg: ATEYQzzavRWwbT7idm5+uFap3h4/cG/Xa2h4UJJD0kAugeHk29Q9FlBJOgpA7fqYpHp poRcX+st0mcwympK4bxDzLsELrkL+WFC8WwGFsysTaiMi1h4i8kYlNV60wTwFZbbLdh1GAgkFTn wZshn5hROuuRERyUA4XfX1mMPB4IoPrJBzFjlXJdAK3nc+yRbMFr2U7C80j96Dl7IQemdYIZTxg bIsoGjbQUZV6QfIvDbpx82BdP65sVSHFPsDu31FrtTPP83yE2jQpbk4h/TdIDOsxjvkhQIUFxc0 13iwkqeaBIOk4/G0fq6qmRPwPl3Mcw1P7OfRQGsX6zi+ASFyp6EesBLNKw7BRwbAaRfXms9z0St K5GeMdUuZfnuwpDg4DBj/v7YN8llEMSRnaEn7DXvw8Qa3O7doZJ2PwFNlXGNluxg4Cu+3a/yz1r /ViraRek+Kb/UmJg7vc9UI4PWl4hV//HU= X-Received: by 2002:a05:6820:f04:b0:67b:a85e:6ca5 with SMTP id 006d021491bc7-67c0d04d23cmr2212913eaf.18.1773841623057; Wed, 18 Mar 2026 06:47:03 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:02 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 05/12] spdx: De-duplicate CreationInfo Date: Wed, 18 Mar 2026 07:44:33 -0600 Message-ID: <20260318134655.953233-6-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233395 De-duplicates CreationInfo objects that are identical (except for ID) when writing out an SBoM. This significantly reduces the number of CreationInfo objects that end up in the final document. Signed-off-by: Joshua Watt --- meta/lib/oe/sbom30.py | 112 ++++++++++++++++++++++++++++++------------ meta/lib/oe/spdx30.py | 2 +- 2 files changed, 81 insertions(+), 33 deletions(-) diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py index 21f084dc16..55a2863d2d 100644 --- a/meta/lib/oe/sbom30.py +++ b/meta/lib/oe/sbom30.py @@ -14,6 +14,7 @@ import uuid import os import oe.spdx_common from datetime import datetime, timezone +from contextlib import contextmanager OE_SPDX_BASE = "https://rdf.openembedded.org/spdx/3.0/" @@ -191,6 +192,25 @@ def to_list(l): return l +class Dedup(object): + def __init__(self, objset): + self.unique = set() + self.dedup = {} + self.objset = objset + + def find_duplicates(self, cmp, typ, **kwargs): + for o in self.objset.foreach_filter(typ, **kwargs): + for u in self.unique: + if cmp(u, o): + self.dedup[o] = u + break + else: + self.unique.add(o) + + def get(self, o): + return self.dedup.get(o, o) + + class ObjectSet(oe.spdx30.SHACLObjectSet): def __init__(self, d): super().__init__() @@ -895,6 +915,45 @@ class ObjectSet(oe.spdx30.SHACLObjectSet): self.missing_ids -= set(imports.keys()) return self.missing_ids + @contextmanager + def deduplicate(self): + d = Dedup(self) + + yield d + + visited = set() + + def visit(o, path): + if isinstance(o, oe.spdx30.SHACLObject): + if o in visited: + return False + visited.add(o) + + for k in o: + v = o[k] + if isinstance(v, oe.spdx30.SHACLObject): + o[k] = d.get(v) + + elif isinstance(o, oe.spdx30.ListProxy): + for idx, v in enumerate(o): + if isinstance(v, oe.spdx30.SHACLObject): + o[idx] = d.get(v) + + return True + + if d.dedup: + for o in self.objects: + o.walk(visit) + + for k, v in d.dedup.items(): + bb.debug( + 1, + f"Removing duplicate {k.__class__.__name__} {k._id or id(k)} -> {v._id or id(v)}", + ) + self.objects.discard(k) + + self.create_index() + def load_jsonld(d, path, required=False): deserializer = oe.spdx30.JSONLDDeserializer() @@ -1080,39 +1139,28 @@ def create_sbom(d, name, root_elements, add_objectsets=[]): # SBoM should be the only root element of the document objset.doc.rootElement = [sbom] - # De-duplicate licenses - unique = set() - dedup = {} - for lic in objset.foreach_type(oe.spdx30.simplelicensing_LicenseExpression): - for u in unique: - if ( - u.simplelicensing_licenseExpression - == lic.simplelicensing_licenseExpression - and u.simplelicensing_licenseListVersion - == lic.simplelicensing_licenseListVersion - ): - dedup[lic] = u - break - else: - unique.add(lic) - - if dedup: - for rel in objset.foreach_filter( - oe.spdx30.Relationship, - relationshipType=oe.spdx30.RelationshipType.hasDeclaredLicense, - ): - rel.to = [dedup.get(to, to) for to in rel.to] - - for rel in objset.foreach_filter( - oe.spdx30.Relationship, - relationshipType=oe.spdx30.RelationshipType.hasConcludedLicense, - ): - rel.to = [dedup.get(to, to) for to in rel.to] + def cmp_license_expression(a, b): + return ( + a.simplelicensing_licenseExpression == b.simplelicensing_licenseExpression + and a.simplelicensing_licenseListVersion + == b.simplelicensing_licenseListVersion + ) - for k, v in dedup.items(): - bb.debug(1, f"Removing duplicate License {k._id} -> {v._id}") - objset.objects.remove(k) + def cmp_creation_info(a, b): + data_a = {k: a[k] for k in a} + data_b = {k: b[k] for k in b} + data_a["@id"] = "" + data_b["@id"] = "" + return data_a == data_b + + with objset.deduplicate() as dedup: + # De-duplicate licenses + dedup.find_duplicates( + cmp_license_expression, + oe.spdx30.simplelicensing_LicenseExpression, + ) - objset.create_index() + # Deduplicate creation info + dedup.find_duplicates(cmp_creation_info, oe.spdx30.CreationInfo) return objset, sbom diff --git a/meta/lib/oe/spdx30.py b/meta/lib/oe/spdx30.py index cd97eebd18..1f58402ffc 100644 --- a/meta/lib/oe/spdx30.py +++ b/meta/lib/oe/spdx30.py @@ -701,7 +701,7 @@ class SHACLObject(object): self.__dict__["_obj_data"][iri] = prop.init() def __iter__(self): - return self._OBJ_PROPERTIES.keys() + return iter(self._OBJ_PROPERTIES.keys()) def walk(self, callback, path=None): """ From patchwork Wed Mar 18 13:44:34 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C404103E189 for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14411.1773841625043908134 for ; Wed, 18 Mar 2026 06:47:05 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=L5lBhIby; spf=pass (domain: gmail.com, ip: 209.85.161.45, mailfrom: jpewhacker@gmail.com) Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-67bb17938d5so481644eaf.1 for ; Wed, 18 Mar 2026 06:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841624; x=1774446424; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Tj4cGGvEEbE1QT7Q+4TdJLcoNUT6Vuz9fhKtg+CMko=; b=L5lBhIbyw+9aJChVQKxpObOXPtuFufsCM2xf8w/ut0e69DbuS9KKpicga0oaIsJoFJ gMJbFZmwva3Fn7mdjsCiCMtvs9kuQ/1Q8sAof/eKPKM+VziLvtV34OWVR/DmuwI8eJln dBvNJ3sF2b2OEZC9z+imREVsf0sDK5BDKtr5WtKEeZQSbCj3eDAY8/r5GDFDLPOlOo0l OHiLrtKpyC5TTEjDJOgxwc6VSKzpu0tjXF2CbIZARVFRxxom0TaRnj+2KSutSERcUVcN OSFObJCKqwMw6PTj2016fOZbLGI1z6LLOCPdzwtk4hQdeuV7f6tVNOsVqY0JsTqSNxFi +g6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841624; x=1774446424; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6Tj4cGGvEEbE1QT7Q+4TdJLcoNUT6Vuz9fhKtg+CMko=; b=o8FXYQiwaAAK7WLVFV23MlVHzHRx3RuC+YgrChvE0Ek9bskwXm7Zlc4PiP9Z6rPNrT +rL+MXmmOLbKMXcW8NCuZNEpWogO53awyK1a0bCe7rKkFpdbExkJ7OPFmxmZIPHOf3Ev LGTvGWyaLmKVlhZrMtpFnp3NF7q3YRybF16N5KTYFnBr8WPFMd1lZXc1xXEm51LRnOoO EqpVQtHaWQ5UJowDr22memoyBTiGvJ81bkP7gvMQC5yCGxLvisYNfKSxRacxp+zgxfD7 se+8XHUXTF4OUv4+Lo8d5AHk0Fror8rcY6R6YaGCzZjLkS1AEYjYZpTTnuiszsvZgMY9 7DRw== X-Gm-Message-State: AOJu0YzFqkpOfzul44kNJ/EM0Ff13nOB1mLbxGijhITfX7pbIuu0dgFL B3UX6yhzeoQpHApa1+wdZ6CX/MvbcAweYf+6Pv9i838ZBNaoko2gqk015KP9Zg== X-Gm-Gg: ATEYQzwqogGYQaqceQ/37Np5KC3Ku2KmUuqsAyBcDsWicWH4pETO8yvjZ7wauLGR30l b9DOJNDgcbWdNkH1/80MUr7PHR7bi30ThfRtm+1qnjgqaH8SnwzqVKQjXUSALDNXhefJx5Hurmp aylIMzs+5Rh8zJUAXCYtPmWVM1wHb34hn9o4dgQx6BH6P+LAuK6dXWkdnnhGW3QxSW4jf40VP4I yfe+kB6fKuLZDyZ4TIo5LSAGoohU4P/If9nxxJZCiT1Oqc90R5XwNkyUOTN9LUvzGkN/1SvAvJ2 lsnRkHvYoad+mK6sBv9Gx7Hu4O+Fj9mP25yC7JGnKY+eZMZjXMYIUGWU68QR7KV9rjM4QGZcdi6 nR15iyPyiR3/M5gj6afCYP6Wyg126yCR+JPSzr1V15xjsctfazNH7ESK6qVf/ahP1nkCkNWIGc1 ePLj/s/B/ajZbnrA2Jl89k X-Received: by 2002:a4a:e901:0:b0:67b:af79:4c28 with SMTP id 006d021491bc7-67c0da70e69mr2398651eaf.13.1773841624001; Wed, 18 Mar 2026 06:47:04 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:03 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 06/12] spdx_common: Check for dependent task in task flags Date: Wed, 18 Mar 2026 07:44:34 -0600 Message-ID: <20260318134655.953233-7-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233396 Checks that the task being used to detect dependencies is present in at least one dependency task flag of the current task. This helps prevent errors where the wrong task is specified and never found. Signed-off-by: Joshua Watt --- meta/lib/oe/spdx_common.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 72c24180d5..3aaf2a9c8b 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -96,6 +96,17 @@ def collect_direct_deps(d, dep_task): taskdepdata = d.getVar("BB_TASKDEPDATA", False) + # Check that the task is listed one of the task dependency flags of the + # current task + depflags = ( + set((d.getVarFlag(current_task, "deptask") or "").split()) + | set((d.getVarFlag(current_task, "rdeptask") or "").split()) + | set((d.getVarFlag(current_task, "recrdeptask") or "").split()) + ) + + if not dep_task in depflags: + bb.fatal(f"Task {dep_task} was not found in any dependency flag of {pn}:{current_task}") + for this_dep in taskdepdata.values(): if this_dep[0] == pn and this_dep[1] == current_task: break From patchwork Wed Mar 18 13:44:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ED2E103E18B for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oa1-f51.google.com (mail-oa1-f51.google.com [209.85.160.51]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14412.1773841626115694985 for ; Wed, 18 Mar 2026 06:47:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Mg2U304R; spf=pass (domain: gmail.com, ip: 209.85.160.51, mailfrom: jpewhacker@gmail.com) Received: by mail-oa1-f51.google.com with SMTP id 586e51a60fabf-4152698e745so2997107fac.1 for ; Wed, 18 Mar 2026 06:47:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841625; x=1774446425; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eVHszXY8NilPQUg/rlhRHWFKY4L68xQCK+mRhzQQ1rQ=; b=Mg2U304RqRLwO0YTaX9EEMkH7CCRn+Ers6BMTCzh+ztWmNrB3x9vBgpOLuR4PxFveB 61PgnHLEh2nF3zHfnIfJufNjiQxrWiuMlBaSkAeG9zb+i5hIl9XXtQQP+CObQhi6cYpW 807mrSdwBYe39ckcm1aCdampb6Z6c1yfM6JkSDkF9nyA5uu01Yy/V3pkmAPFJfSct8/P a/mCA/dybdaYo60gHhxxpd5SDqWlKv1tXDR+fTs134kBhlOm0VXsaLJ9wVBYzNvqRs4g 1IUuwR/gRyR/udrrSVjI7TZmvMgLrlr+D2KAT1Jq7/DNWKPvm6gMFYBD+SbNKOk3e42n CDJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841625; x=1774446425; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=eVHszXY8NilPQUg/rlhRHWFKY4L68xQCK+mRhzQQ1rQ=; b=BylPcgsSNWOGpS7kETz8IOa9nOrhOEAbIn6wsvYc6oxFhAnEIedJfocH7YCD91LdQ/ j0PL0ADpgpmKUYGV7x25GPiZBUClXyjJv+jQLOtWOU/sfb+uQovvkEMK2H2dKE6xqbTR 923UmpU6E5nVOb+x2DUG1cT+mgfkVeB5ckpzYjwFWxpUVUiRpbzyQOlNdlPYcXR/xDjX KJ4BIKss7vZQhJyN87ywiaI3iqoHH4XaHXh5WS1AFOdEqnUNwaxsvXKIZ11KK6ohuejb 9hoj0Df5pWioUTVdcZRyfOltMLAvwbr2SqNMsm+AdhaRjlOGxo2pMv1ddWkfPXyZr10f 261A== X-Gm-Message-State: AOJu0YxpwrIpVxAMZNE8ZgG6CfGFFzrmHEYIRE3gH/w2jBAFlrl0RTSY dAsiSp/kbJydf0okVtM1bcAyTPibZAHzbDrpL22fqEPCwoHNmT/4clAZ2ypqQg== X-Gm-Gg: ATEYQzxuWshhTkvC7COWTdIkGFg0NXDsnqDrKIhQpqdhp3SWVxUcfg8Nbm/OQ1XRKTl PXI4bYqBaTPUW6NOYadO9WdhAq4/7qIx9v3wi8zUsJQORTxPKvTxeqfxinCdJScuqb4tDR6KSKP Q6YOh3vYPoivzYsCCwzM554RRfBpS1TbTdv5nvIOrxEl1QZrZI+bfGaJfBuiQBv2HEbBHvnTIX4 hy+nkzVHUiMwpeIv8rEF2wQdGBBwZS5YvqU5oN21I8KtQrbfW2RrrEnTPSYgvO0Dimx3RSvnHVu lRC1JU9A8dHqucBcXP/cdgaxhBEP/cu+1TIiea2EZAaWF2szc90/JbV+e+60/VExFgJYmKU1nU3 Dj2gZLAwwxFTCJRi8evn9AAgjUDmirXBcaMDciecpfdV4diw4uBUyJhvjTT9YxrMmBtZ3LGK70N VDxrrG9zhZ5lHX7a1P82GC X-Received: by 2002:a05:6820:4b06:b0:67b:f1f1:1abd with SMTP id 006d021491bc7-67c0db3cb74mr2426813eaf.54.1773841624941; Wed, 18 Mar 2026 06:47:04 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:04 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 07/12] spdx30: Remove package VEX Date: Wed, 18 Mar 2026 07:44:35 -0600 Message-ID: <20260318134655.953233-8-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233397 Removes VEX statements from packages. These are no longer necessary since the VEX data is now attached to the recipes, which significantly reduces the duplication of the data, and thus the size of the SPDX output files. Signed-off-by: Joshua Watt --- meta/lib/oe/spdx30_tasks.py | 72 ------------------------------------- 1 file changed, 72 deletions(-) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index aec47d4f81..5b651900c4 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -768,31 +768,6 @@ def create_spdx(d): debug_source_ids = set() source_hash_cache = {} - # Collect all VEX statements from the recipe - vex_statements = {} - vex_patches = {} - for rel in recipe_objset.foreach_filter( - oe.spdx30.Relationship, - relationshipType=oe.spdx30.RelationshipType.hasAssociatedVulnerability, - ): - for cve in rel.to: - vex_statements[cve] = [] - vex_patches[cve] = [] - - for cve in vex_statements.keys(): - for rel in recipe_objset.foreach_filter( - oe.spdx30.security_VexVulnAssessmentRelationship, - from_=cve, - ): - vex_statements[cve].append(rel) - if rel.relationshipType == oe.spdx30.RelationshipType.fixedIn: - for patch_rel in recipe_objset.foreach_filter( - oe.spdx30.Relationship, - relationshipType=oe.spdx30.RelationshipType.patchedBy, - from_=rel, - ): - vex_patches[cve].extend(patch_rel.to) - # Write out the package SPDX data now. It is not complete as we cannot # write the runtime data, so write it to a staging area and a later task # will write out the final collection @@ -931,53 +906,6 @@ def create_spdx(d): [oe.sbom30.get_element_link_id(concluded_spdx_license)], ) - # Copy CVEs from recipe - if vex_statements: - pkg_objset.new_relationship( - [spdx_package], - oe.spdx30.RelationshipType.hasAssociatedVulnerability, - sorted( - oe.sbom30.get_element_link_id(cve) - for cve in vex_statements.keys() - ), - ) - - for cve, vexes in vex_statements.items(): - for vex in vexes: - if vex.relationshipType == oe.spdx30.RelationshipType.fixedIn: - spdx_vex = pkg_objset.new_vex_patched_relationship( - [oe.sbom30.get_element_link_id(cve)], [spdx_package] - ) - if vex_patches[cve]: - pkg_objset.new_scoped_relationship( - spdx_vex, - oe.spdx30.RelationshipType.patchedBy, - oe.spdx30.LifecycleScopeType.build, - [ - oe.sbom30.get_element_link_id(p) - for p in vex_patches[cve] - ], - ) - - elif vex.relationshipType == oe.spdx30.RelationshipType.affects: - pkg_objset.new_vex_unpatched_relationship( - [oe.sbom30.get_element_link_id(cve)], [spdx_package] - ) - elif ( - vex.relationshipType == oe.spdx30.RelationshipType.doesNotAffect - ): - spdx_vex = pkg_objset.new_vex_ignored_relationship( - [oe.sbom30.get_element_link_id(cve)], - [spdx_package], - impact_statement=vex.security_impactStatement, - ) - - if vex.security_justificationType: - for v in spdx_vex: - v.security_justificationType = ( - vex.security_justificationType - ) - bb.debug(1, "Adding package files to SPDX for package %s" % pkg_name) package_files = add_package_files( d, From patchwork Wed Mar 18 13:44:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E98F103E18D for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oo1-f52.google.com (mail-oo1-f52.google.com [209.85.161.52]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.14441.1773841627118326339 for ; Wed, 18 Mar 2026 06:47:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=H+1Y8T4r; spf=pass (domain: gmail.com, ip: 209.85.161.52, mailfrom: jpewhacker@gmail.com) Received: by mail-oo1-f52.google.com with SMTP id 006d021491bc7-67bbea1e090so397359eaf.3 for ; Wed, 18 Mar 2026 06:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841626; x=1774446426; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FCaaEZW0U/AEs0relez15VB/j0mdCHwpFh7bL0M3bMQ=; b=H+1Y8T4r/D6uLM6MG7moPZkdJY4VZ8k6tU77mqiT0aKzdO/9WY2guBOppiXGBJjWs8 yER38T9ZX+S7yvd1JgFD39xcfG+p3OYTERuEi70gq2sv8EkcUAhhwADc6Dp/qC6dnHyK iN65MEQF3A8QXufTf1Juj3rUKUXB75er30mpuFZstJKA5HwmpJCgIn7S8iQTSsUwzehS 4HMvHKXGHdZnnsENUPJynN42XNYwwQuvETnkLbBLj89n9LlQxJ7XM/KtL614Xvq8Xc+g jyLTCOysPMARZ9v3AhjCJ7uHC4fevrp+9TFSzahfhpgYM26pJsYVi626gJdHJ9VkXtJr zPgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841626; x=1774446426; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=FCaaEZW0U/AEs0relez15VB/j0mdCHwpFh7bL0M3bMQ=; b=SZ9uNAEtjviEkINDYV1CYjTqBXp569oearTSFwgAK/xmLPfJt4ntgWs7+rsVVvYBUp rC3xhRFunciMNuVrEAgpxqIsPd+GI9a114g1MnQdGehKwMpaiLj/S/ai0W+x3WeDAQ91 NhUpQxqDtBIB2HIvjBLmtScL17p7szRcGmmcccvAE6TWNWYLEA4ZSO11KszbahiSStbT gy7NfvMH7OCNqIVSkray1S1FYz6LoRkWfu1weno8mjWIgb1Q3DPHkywZx4CrEBQM6Pdq KgHdWLZ/dknJ28KPR6byjUU1VwAh8wp2MwFOcSE1CJsxXQ/uclEiFUFtI61bkpmH3aXC 7u9A== X-Gm-Message-State: AOJu0YygaKNDe7bwci9jide9ShnTp9SXtHk/Zr6haHX4jL9j+v2ojiTO hVwTL1beVPkI0OgYSrk4LkS2Ijs4SAmNPrbJvYeFEK4+yhMbYUfdpt3hYWoTgA== X-Gm-Gg: ATEYQzzOfXL6PWo6XGDnjS8K/Jc8pvegxura+1+gtkXcp2PDYN40qVIR/nsJpe1wtZ9 h+XDAuhFph3ipdKKBERyGpGBlG6GvkkKJBaBC5HkW2cEOZPxWFlTGE9EFEtfteFLvehpQ5TSBpN MVtGg3R2v9i59SlEmPZnShrdCpww83d2U7sn+rTxFOsDN0GUeFP1e3BV/MABMaGqTpZC056Ac7O NfVxnFrFIzKsOufF4wAimJ7+r+s/UZFTr4fFQ2EWCFazXpY0DqNOgaJ77ZZ19bydZOT7JIbCmk5 U3IlfZNtUEGd9tz89UqgV0m59nF+8cQXrxnFV5FRDK6uneTTBqaablguPTgS9xqWktMsNzGYu8Y r86hYFaaEudbLy86u5EgmxIZW9zdIuYIXvqOGJoE+WESGqaNorOCc8OntBIj9fzHi0BftzP1W6U 1OVxvIfEi5t/zsmyFYP0xy X-Received: by 2002:a4a:e84b:0:b0:67b:d213:e64e with SMTP id 006d021491bc7-67c0daf32bamr2145569eaf.34.1773841625934; Wed, 18 Mar 2026 06:47:05 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:05 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 08/12] spdx: Remove fatal errors for missing providers Date: Wed, 18 Mar 2026 07:44:36 -0600 Message-ID: <20260318134655.953233-9-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233398 When creating images and SDKs, do not error on missing providers. This allows recipes to use the `nospdx` inherit to prevent SPDX from being generated, but not result in an error when assembling the final image. Note that runtime packages generation already ignored missing providers, so this is changing image and SDK generation to match Signed-off-by: Joshua Watt --- meta/classes/create-spdx-2.2.bbclass | 3 ++- meta/lib/oe/spdx30_tasks.py | 8 +------- 2 files changed, 3 insertions(+), 8 deletions(-) diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 3288cdf75a..aa39208eae 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -858,7 +858,8 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx if packages: for name in sorted(packages.keys()): if name not in providers: - bb.fatal("Unable to find SPDX provider for '%s'" % name) + bb.note("Unable to find SPDX provider for '%s'" % name) + continue pkg_name, pkg_hashfn = providers[name] diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 5b651900c4..c4af191974 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -1178,11 +1178,10 @@ def collect_build_package_inputs(d, objset, build, packages, files_by_hash=None) providers = oe.spdx_common.collect_package_providers(d) build_deps = set() - missing_providers = set() for name in sorted(packages.keys()): if name not in providers: - missing_providers.add(name) + bb.note(f"Unable to find SPDX provider for '{name}'") continue pkg_name, pkg_hashfn = providers[name] @@ -1201,11 +1200,6 @@ def collect_build_package_inputs(d, objset, build, packages, files_by_hash=None) for h, f in pkg_objset.by_sha256_hash.items(): files_by_hash.setdefault(h, set()).update(f) - if missing_providers: - bb.fatal( - f"Unable to find SPDX provider(s) for: {', '.join(sorted(missing_providers))}" - ) - if build_deps: objset.new_scoped_relationship( [build], From patchwork Wed Mar 18 13:44:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84907103E18F for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.14442.1773841628123874849 for ; Wed, 18 Mar 2026 06:47:08 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=NFfVuXar; spf=pass (domain: gmail.com, ip: 209.85.167.169, mailfrom: jpewhacker@gmail.com) Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-4670676ba03so2437625b6e.1 for ; Wed, 18 Mar 2026 06:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841627; x=1774446427; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4GQr03avjj4R9pZ+lu2O4W8U9AvEqC/e3JtfjqlshRU=; b=NFfVuXarA1jFTY7izgzHmM3ysry32D8dZ2QzMJM28P9BDDftPaLFEVy8bTCtuWyUOJ dA5C9fAAy1RLDqc8V8fUUyrF2CirVbBd+zJdq3JEBtKFRpoRuRTROKvXP+mRZLjPinTG cZpL1i00p3nD0/HwX4DfQJwXvzvW6Zf+L4HRfQLwleodUwMQc/KgpECkZ9925Y5hgPbN +hY0w8neRKmRysLeeLv8JT/eVNOarSYWz2emqhRsvB7X2K+KolmGX4zPGu5DdoC/rMK3 B8Nr5g03PaSIkTqsLnwY8QcMV3p7sBa7BUze7HPwkiaFIu7YiV2rjQh83r8+PyDIrR45 Gptw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841627; x=1774446427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4GQr03avjj4R9pZ+lu2O4W8U9AvEqC/e3JtfjqlshRU=; b=cOqdNMNq8ESBC9LjXXyXw2X4CAIhnb1QtDM0oZTPVaZ/hBNzbsGi5VZj0IdvVdQ0ku 34pd1jq81xUViH8j9y2JEtT3Dh2PLapTMWHpaoi15GxwoMIKSgbVJMlRLDcAvNxbs7N8 A2rQwWOP1SDF5/kPBMOxZETJpKxvzcPEBFrFNGOqTeOeiQKSULaxxyUIfEyDIha6nlbb iJRQzMl9oA2GBxyDJarZMdYPEuNZBAVjileDy43iIt5QsLWnaAU8BwPyAamvN6LFoXs/ sldC8u80GDKDJ5lMEV+arSrQWNGWBY7NhLkWyY5Mjr7f4/7tCIvHt1QwyTf3vY3Bme7l 12ww== X-Gm-Message-State: AOJu0YyFrEbt49mIVbDL9nB1073nWFv5ibAMIx1oNdsdFn5xydXwMcm9 0J72NFoJ/HrRVlcWaYPFhT+298ielPR0PXFWz6nr4EQaACh3aLMY9fnOfSC4Gg== X-Gm-Gg: ATEYQzy63hc0q8IDe1Jw4vvwf8xOk3szxNxmZ12YJsbkycjG/4Iot7PVr6SbJFysmSJ KDFe/vMW9BEtbJfserfKAUjbjkhy6Tw/1TtsYfkQrXvThI3TktDMUmJif9ItsXYgJ+5uDvK3dVe ovIQ/5M6gy029LRmiJHDVVg8irRQMwxmufNrh0GBAbHws5GxHfj7Y3DNbbAvWWoO6SmRcGrwU00 LjqOE5CHc85GlmizMa6XCdiMOC/X01MW0WRnjJQBuWo7uRl9QM3XxhI3b3COki9Xaw5jeDDtogY ZubmgBz8BlWFmhSrBcld66aLozjNEEK3NrtNAQ+59KDPYOHCDJiLKoCT+h5qZ4iiuGg65Fgepxr rTnLXo4bGy+/El7gOzxtBocrz1gcHgLvtsHjL0Ai2xT9eaUZ/562jYJdNZZBPRqWeGRQe9dlaAc zmjihnOzGTx79hx0hbix0N6wz3Li+lkHQ= X-Received: by 2002:a05:6820:f029:b0:67b:b01c:5887 with SMTP id 006d021491bc7-67c0db41fecmr2309940eaf.56.1773841627100; Wed, 18 Mar 2026 06:47:07 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:06 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 09/12] spdx3: Use common variable for vardeps Date: Wed, 18 Mar 2026 07:44:37 -0600 Message-ID: <20260318134655.953233-10-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233399 Instead of repeating the vardeps for each SPDX task with the necessary variables, use a common variable to make it easier to manage Signed-off-by: Joshua Watt --- meta/classes/create-spdx-3.0.bbclass | 33 ++++++++++------------------ 1 file changed, 12 insertions(+), 21 deletions(-) diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index c3ea95b8bc..869bbd472f 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -163,6 +163,14 @@ SPDX3_DEP_FILES = "\ ${SPDX_LICENSES}:True \ " +SPDX3_VAR_DEPS = "\ + SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ + SPDX_PACKAGE_ADDITIONAL_PURPOSE \ + SPDX_PROFILES \ + SPDX_NAMESPACE_PREFIX \ + SPDX_UUID_NAMESPACE \ + " + python do_create_recipe_spdx() { import oe.spdx30_tasks oe.spdx30_tasks.create_recipe_spdx(d) @@ -174,13 +182,7 @@ do_create_recipe_spdx[sstate-inputdirs] = "${SPDXRECIPEDEPLOY}" do_create_recipe_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" do_create_recipe_spdx[file-checksums] += "${SPDX3_DEP_FILES}" do_create_recipe_spdx[cleandirs] = "${SPDXRECIPEDEPLOY}" -do_create_recipe_spdx[vardeps] += "\ - SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ - SPDX_PACKAGE_ADDITIONAL_PURPOSE \ - SPDX_PROFILES \ - SPDX_NAMESPACE_PREFIX \ - SPDX_UUID_NAMESPACE \ - " +do_create_recipe_spdx[vardeps] += "${SPDX3_VAR_DEPS}" python do_create_recipe_spdx_setscene () { sstate_setscene(d) @@ -211,13 +213,7 @@ do_create_spdx[depends] += " \ ${PATCHDEPENDENCY} \ ${@create_spdx_source_deps(d)} \ " -do_create_spdx[vardeps] += "\ - SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ - SPDX_PACKAGE_ADDITIONAL_PURPOSE \ - SPDX_PROFILES \ - SPDX_NAMESPACE_PREFIX \ - SPDX_UUID_NAMESPACE \ - " +do_create_spdx[vardeps] += "${SPDX3_VAR_DEPS}" python do_create_spdx_setscene () { sstate_setscene(d) @@ -238,6 +234,7 @@ do_create_package_spdx[file-checksums] += "${SPDX3_DEP_FILES}" do_create_package_spdx[dirs] = "${SPDXRUNTIMEDEPLOY}" do_create_package_spdx[cleandirs] = "${SPDXRUNTIMEDEPLOY}" do_create_package_spdx[rdeptask] = "do_create_spdx" +do_create_package_spdx[vardeps] += "${SPDX3_VAR_DEPS}" python do_create_package_spdx_setscene () { sstate_setscene(d) @@ -259,13 +256,7 @@ do_create_recipe_sbom[sstate-inputdirs] = "${SPDXRECIPESBOMDEPLOY}" do_create_recipe_sbom[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" do_create_recipe_sbom[file-checksums] += "${SPDX3_DEP_FILES}" do_create_recipe_sbom[cleandirs] = "${SPDXRECIPESBOMDEPLOY}" -do_create_recipe_sbom[vardeps] += "\ - SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ - SPDX_PACKAGE_ADDITIONAL_PURPOSE \ - SPDX_PROFILES \ - SPDX_NAMESPACE_PREFIX \ - SPDX_UUID_NAMESPACE \ - " +do_create_recipe_sbom[vardeps] += "${SPDX3_VAR_DEPS}" python do_create_recipe_sbom_setscene () { sstate_setscene(d) From patchwork Wed Mar 18 13:44:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B1D7103E17D for ; Wed, 18 Mar 2026 13:47:09 +0000 (UTC) Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.14414.1773841628994759940 for ; Wed, 18 Mar 2026 06:47:09 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=fgpr/tW5; spf=pass (domain: gmail.com, ip: 209.85.210.44, mailfrom: jpewhacker@gmail.com) Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-7d77b179b52so5887732a34.2 for ; Wed, 18 Mar 2026 06:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841628; x=1774446428; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wxzrZp1LjKbEzkovZzvYqu+W5DOt8PCBhIjLmqSvOVA=; b=fgpr/tW55J05Sy27AT4DSNTc57YILJzmOsLVuGsYGWsOtZCaJnQkdLQNBS3aChnJIz otiZ6dofRhzb50wY4IdrfEEZFc52rH4zWR+D0JLzWTfHcANZzfE5E/D4m+z9/AEx/P3r sLv+BHyoHOLmIs5SL3y7RiZGLrQbMrHLuqxzc87wh200iLq50uhQZlfpswAMmnpxKc5M QgppnzjIGUukXmRIX0gZ00/oWhE8qCzwZY/mF2ngnL0cr/7M3sG4F0sboJeOvRsNki8t LqB7E+tUzg+719dgRycAf2dMYkrxw8OfE2s39clBrd8Dj+s9YvR0v7KHQ78EJMsVG72E Fd5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841628; x=1774446428; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=wxzrZp1LjKbEzkovZzvYqu+W5DOt8PCBhIjLmqSvOVA=; b=OlZ/kzhcqrAYNcVW+LyNmZ1xj8Z4plLvNM3K3QCX8H8IUGRt9XOj9TaMKvUgqGi4Ee 5IXCPthv9pu7mCUI+Vj4cq3mgjlkMc4nYdTFiV5exVe+aN9xXhrFp4BZgz/8Tf8YFx/w /ZqyYvbQF7GTSb+XcHINdi8kVgStgmB8tElPPVF1plKE5rcWGvRxJw3VhCnn6zzlXZEA BT40SISJHNxxHciudNzzg0+Uj+pW/BzejCcDF1OLyfEirUDU7O6yOeIp+INHxNN6khOC TWDaNqdT+0JckROrMazbua7FkyKMcNguGmforwJWyuzUcDHuNtQBD70XdyRGwEvXLv5D zsKg== X-Gm-Message-State: AOJu0YzYZlulimhYA/Y9Zgtgw+dEn7pFH7zSJ2JiXfRM3jcipxRMBx14 I4N/tjK3bh52mPZBhZ46phspzatkbQEaWy1rGlRBeRrx/HSbfPh+JaIZZDA36Q== X-Gm-Gg: ATEYQzwhdWeykFjdz1V2Snd72yGSamOtI9WRR1kkSLhvhLyNfZ5eFF0L0+rOkzo0ZR7 eWBYAV/VLPctOYv5d4ZIqO5HqHmsI3laVkp810bsfpWwDmoJV1/PCAv6UoBQFvXjt/QxOH1wgXy SKPqzXgEmaKpdHeeXHYAkXy+iCAJ/hgFpsltzTNHNdLxmESuOhODvVPQ+fya0NmnyUfHYHKqzHG mHe549lwP/569X6Jltn2RlogGhgDOCc9Z6c+15OzuPJKf60FSQjUMcp4PYnm6rxanjbfWHvjBJ1 CIKYWbXJlPrcjy0sbjLYeFKoNHcygYfOLyKAnEVVmNZg42YNN4k8rd3J59+PScj41OxcBeWrxp2 q5dE29plxsdIWv/PgPUhp0l6AqPlNhXJ32u+XX2AA/nX2xMmmV81PcUTQnByEPCnT+4dvvpoRra pv273ebi+eg0C5ZVpK1kGB X-Received: by 2002:a05:6820:2981:b0:677:4fc0:a852 with SMTP id 006d021491bc7-67c0dae9889mr2325661eaf.34.1773841628068; Wed, 18 Mar 2026 06:47:08 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:07 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 10/12] glibc-testsuite: Do not generate SPDX Date: Wed, 18 Mar 2026 07:44:38 -0600 Message-ID: <20260318134655.953233-11-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233400 glibc-testsuite does not run on target or factor into the build supply chain, since its purpose is run tests in Qemu at build time Signed-off-by: Joshua Watt --- meta/recipes-core/glibc/glibc-testsuite_2.43.bb | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.43.bb b/meta/recipes-core/glibc/glibc-testsuite_2.43.bb index 28af6961c3..899955adfb 100644 --- a/meta/recipes-core/glibc/glibc-testsuite_2.43.bb +++ b/meta/recipes-core/glibc/glibc-testsuite_2.43.bb @@ -64,6 +64,7 @@ do_check:append () { } inherit nopackages +inherit nospdx deltask do_stash_locale deltask do_install deltask do_populate_sysroot From patchwork Wed Mar 18 13:44:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Watt X-Patchwork-Id: 83744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 966A5103E191 for ; Wed, 18 Mar 2026 13:47:10 +0000 (UTC) Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.14443.1773841630199579783 for ; Wed, 18 Mar 2026 06:47:10 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=eBLTYbnJ; spf=pass (domain: gmail.com, ip: 209.85.210.46, mailfrom: jpewhacker@gmail.com) Received: by mail-ot1-f46.google.com with SMTP id 46e09a7af769-7d748ca8708so3956255a34.0 for ; Wed, 18 Mar 2026 06:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773841629; x=1774446429; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MncNZoRlG6mpRhzsgf6Te3StVhbrPIWxLWm4MTT+B5g=; b=eBLTYbnJbswUAAMqKp6sac0+Wi90+To6JXymRhZ3wmaXJO7pufQMZCF275EIPyOl9c ox+BMo7nK26Ck4m+EBkL39KqfAhzs6sDuh9mK4Zi3lru2Y2Nvr62nmAZlvG/o/HDAnu0 +ubjuIG5m2BcHmxjMrfJYVluJjE4SFVtnQ/zAHrt1NtLuFGWrQm9mqcKPwdNpcMFZqOc 4bQYl5ZEz2Y4OMBkblaaCdB+MSRT0bq8HdromZglVwxsEzow8W3GHi7DfxnUK3v8SCm6 ipvDjZVsvwB0XbF86AOXMptlb83beSOr0dXnjGOFhM7p6u0PcpFE9YPZ6FzPRBywQ4AC nS6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773841629; x=1774446429; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MncNZoRlG6mpRhzsgf6Te3StVhbrPIWxLWm4MTT+B5g=; b=O1x/fjWSfkoIa7bUZMDhgosJWFN4Dy2OJNpBMwp4UuqOB5hu2KpJB2DZfigWejzPSS mj8L5wZZDB5MRoVYDspuMz5/FBQCZWUorSIoZFuF5gW1KOeDEDqygar6c02KwlG+N2CP DKwuLAgkZ4qwInllSIk9d02uFivT+gbpaj4p7zW2jeksPvU+zxe4ybhfW7DJwD+rccr3 1zD+bsg87pyDywiMNdVJJvSPMtOL6jpW56nGxzA1AZ5VrTEHciqozqC2zLOwkV1gTPDx cMbPtuEaJoK+9ddPgZBG7PRvzCAdA9L2ew4g+tUXdd7qdp1xjMnxg9xz9WJpD9NGG2h4 A1Cw== X-Gm-Message-State: AOJu0YxIhyvdtaZGusne5WiCRav2r/RVDQbPbA+y9olX7QCxsT+VPcUh WsQ51sFcpVB6ZTOqZ/c9UANH3vvLUfLzee/lFCkudosed5ieDfMP4t13toBPWA== X-Gm-Gg: ATEYQzzLULGXCqv7eIDGlnYoDulHvzUJAHDO2pSYqpeoBLVpS3yHgXznylSFw/hwge8 hUGzPkxr9AHfRT8nGAct5GiNftWV3VCVb+qD5EiAcvBEMXiq1RzfprZw7m7HqUP/G7JKnSVQzoj xsptkwbrztupeajm/lLn2NT44RslsI6ERcPlExJ6vuXQPn/UT7LV89NXTtk7SVAQcX+k1zFDlOg BtYrKvFOVoNMojZ8EyLpkwSETco2YxZlJrjidTMTRHkVspb3V7YDDEXrj7+F4QLZv0MADwKrrQ2 NEDQ8nyI87ZtHX6qTeNTB2wfzIZR0qKvHiQbfUiOd7uja5EOwYluAUnr4MBJgtvHdrvs6j330/d dRzknJLq1THNa2xJfOjlaLdcFTl7TLbmJbTYXUkQlnsZGAuWeEaqf/MYYEX4h4wL/y17g437VKH 0LdcNkMq2KX0g1FA3o1AV3 X-Received: by 2002:a05:6820:6aef:b0:679:976a:98bc with SMTP id 006d021491bc7-67c0daf571fmr2429345eaf.36.1773841629074; Wed, 18 Mar 2026 06:47:09 -0700 (PDT) Received: from localhost.localdomain ([2601:282:4200:11c0::8279]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-67c0d89c739sm1763625eaf.13.2026.03.18.06.47.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 06:47:08 -0700 (PDT) From: Joshua Watt X-Google-Original-From: Joshua Watt To: openembedded-core@lists.openembedded.org Cc: Joshua Watt Subject: [OE-core][PATCH v7 11/12] spdx: Remove do_collect_spdx_deps task Date: Wed, 18 Mar 2026 07:44:39 -0600 Message-ID: <20260318134655.953233-12-JPEWhacker@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260318134655.953233-1-JPEWhacker@gmail.com> References: <20260310184058.533343-1-JPEWhacker@gmail.com> <20260318134655.953233-1-JPEWhacker@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 18 Mar 2026 13:47:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/233401 Removes the do_collect_spdx_deps task. This task was added a long time ago, and appears to have been added due to a misunderstanding about how the task graph works. It is not necessary since tasks can directly call collect_direct_deps() with the appropriate task that they depend on to get their dependencies. This should fix several classes of SPDX bug where documents could not be found because the wrong deps were being looked for due to which tasks were re-run Signed-off-by: Joshua Watt --- meta/classes-recipe/nospdx.bbclass | 1 - meta/classes/create-spdx-2.2.bbclass | 22 +++++---- meta/classes/create-spdx-3.0.bbclass | 5 +- meta/classes/spdx-common.bbclass | 22 --------- meta/lib/oe/spdx30_tasks.py | 22 +++++---- meta/lib/oe/spdx_common.py | 69 +++++++++++++--------------- 6 files changed, 62 insertions(+), 79 deletions(-) diff --git a/meta/classes-recipe/nospdx.bbclass b/meta/classes-recipe/nospdx.bbclass index 90e14442ba..7c99fcd1ec 100644 --- a/meta/classes-recipe/nospdx.bbclass +++ b/meta/classes-recipe/nospdx.bbclass @@ -4,7 +4,6 @@ # SPDX-License-Identifier: MIT # -deltask do_collect_spdx_deps deltask do_create_recipe_spdx deltask do_create_spdx deltask do_create_spdx_runtime diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index aa39208eae..1c43156559 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -277,7 +277,7 @@ def add_package_sources_from_debug(d, package_doc, spdx_package, package, packag add_package_sources_from_debug[vardepsexclude] += "STAGING_KERNEL_DIR" -def collect_dep_recipes(d, doc, spdx_recipe): +def collect_dep_recipes(d, doc, spdx_recipe, direct_deps): import json from pathlib import Path import oe.sbom @@ -290,9 +290,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): dep_recipes = [] - deps = oe.spdx_common.get_spdx_deps(d) - - for dep in deps: + for dep in direct_deps: # If this dependency is not calculated in the taskhash skip it. # Otherwise, it can result in broken links since this task won't # rebuild and see the new SPDX ID if the dependency changes @@ -405,7 +403,7 @@ do_create_recipe_spdx() { : } do_create_recipe_spdx[noexec] = "1" -addtask do_create_recipe_spdx after do_collect_spdx_deps +addtask do_create_recipe_spdx python do_create_spdx() { @@ -532,7 +530,8 @@ python do_create_spdx() { if archive is not None: recipe.packageFileName = str(recipe_archive.name) - dep_recipes = collect_dep_recipes(d, doc, recipe) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + dep_recipes = collect_dep_recipes(d, doc, recipe, direct_deps) doc_sha1 = oe.sbom.write_doc(d, doc, pkg_arch, "recipes", indent=get_json_indent(d)) dep_recipes.append(oe.sbom.DepRecipe(doc, doc_sha1, recipe)) @@ -603,7 +602,7 @@ python do_create_spdx() { } do_create_spdx[vardepsexclude] += "BB_NUMBER_THREADS" # NOTE: depending on do_unpack is a hack that is necessary to get it's dependencies for archive the source -addtask do_create_spdx after do_create_recipe_spdx do_package do_packagedata do_unpack do_patch do_collect_spdx_deps before do_populate_sdk do_build do_rm_work +addtask do_create_spdx after do_create_recipe_spdx do_package do_packagedata do_unpack do_patch before do_populate_sdk do_build do_rm_work SSTATETASKS += "do_create_spdx" do_create_spdx[sstate-inputdirs] = "${SPDXDEPLOY}" @@ -638,7 +637,9 @@ python do_create_runtime_spdx() { license_data = oe.spdx_common.load_spdx_license_data(d) - providers = oe.spdx_common.collect_package_providers(d) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + + providers = oe.spdx_common.collect_package_providers(d, direct_deps) pkg_arch = d.getVar("SSTATE_PKGARCH") package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() @@ -760,6 +761,7 @@ addtask do_create_runtime_spdx_setscene do_create_runtime_spdx[dirs] = "${SPDXRUNTIMEDEPLOY}" do_create_runtime_spdx[cleandirs] = "${SPDXRUNTIMEDEPLOY}" +do_create_runtime_spdx[deptask] = "do_create_spdx" do_create_runtime_spdx[rdeptask] = "do_create_spdx" do_rootfs[recrdeptask] += "do_create_spdx do_create_runtime_spdx" @@ -829,7 +831,9 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx license_data = oe.spdx_common.load_spdx_license_data(d) - providers = oe.spdx_common.collect_package_providers(d) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + + providers = oe.spdx_common.collect_package_providers(d, direct_deps) package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index 869bbd472f..c3deb22598 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -175,13 +175,14 @@ python do_create_recipe_spdx() { import oe.spdx30_tasks oe.spdx30_tasks.create_recipe_spdx(d) } -addtask do_create_recipe_spdx after do_collect_spdx_deps +addtask do_create_recipe_spdx SSTATETASKS += "do_create_recipe_spdx" do_create_recipe_spdx[sstate-inputdirs] = "${SPDXRECIPEDEPLOY}" do_create_recipe_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" do_create_recipe_spdx[file-checksums] += "${SPDX3_DEP_FILES}" do_create_recipe_spdx[cleandirs] = "${SPDXRECIPEDEPLOY}" +do_create_recipe_spdx[deptask] += "do_create_recipe_spdx" do_create_recipe_spdx[vardeps] += "${SPDX3_VAR_DEPS}" python do_create_recipe_spdx_setscene () { @@ -197,7 +198,6 @@ addtask do_create_spdx after \ do_unpack \ do_patch \ do_create_recipe_spdx \ - do_collect_spdx_deps \ do_deploy_source_date_epoch \ do_populate_sysroot do_package do_packagedata \ before do_populate_sdk do_populate_sdk_ext do_build do_rm_work @@ -233,6 +233,7 @@ do_create_package_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}" do_create_package_spdx[file-checksums] += "${SPDX3_DEP_FILES}" do_create_package_spdx[dirs] = "${SPDXRUNTIMEDEPLOY}" do_create_package_spdx[cleandirs] = "${SPDXRUNTIMEDEPLOY}" +do_create_package_spdx[deptask] = "do_create_spdx" do_create_package_spdx[rdeptask] = "do_create_spdx" do_create_package_spdx[vardeps] += "${SPDX3_VAR_DEPS}" diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index abf2332bee..4b40cbf75c 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -87,28 +87,6 @@ def create_spdx_source_deps(d): return " ".join(deps) -python do_collect_spdx_deps() { - # This task calculates the build time dependencies of the recipe, and is - # required because while a task can deptask on itself, those dependencies - # do not show up in BB_TASKDEPDATA. To work around that, this task does the - # deptask on do_create_recipe_spdx and writes out the dependencies it finds, then - # downstream tasks read in the found dependencies when writing the actual - # SPDX document - import json - import oe.spdx_common - from pathlib import Path - - spdx_deps_file = Path(d.getVar("SPDXDEPS")) - - deps = oe.spdx_common.collect_direct_deps(d, "do_create_recipe_spdx") - - with spdx_deps_file.open("w") as f: - json.dump(deps, f) -} -addtask do_collect_spdx_deps -do_collect_spdx_deps[deptask] = "do_create_recipe_spdx" -do_collect_spdx_deps[dirs] = "${SPDXDIR}" - oe.spdx_common.collect_direct_deps[vardepsexclude] += "BB_TASKDEPDATA" oe.spdx_common.collect_direct_deps[vardeps] += "DEPENDS" oe.spdx_common.collect_package_providers[vardepsexclude] += "BB_TASKDEPDATA" diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index c4af191974..353d783fa2 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -298,13 +298,11 @@ def get_package_sources_from_debug( return dep_source_files -def collect_dep_objsets(d, subdir, fn_prefix, obj_type, **attr_filter): - deps = oe.spdx_common.get_spdx_deps(d) - +def collect_dep_objsets(d, direct_deps, subdir, fn_prefix, obj_type, **attr_filter): dep_objsets = [] dep_objs = set() - for dep in deps: + for dep in direct_deps: bb.debug(1, "Fetching SPDX for dependency %s" % (dep.pn)) dep_obj, dep_objset = oe.sbom30.find_root_obj_in_jsonld( d, subdir, fn_prefix + dep.pn, obj_type, **attr_filter @@ -551,8 +549,10 @@ def create_recipe_spdx(d): ) ) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_recipe_spdx") + dep_objsets, dep_recipes = collect_dep_objsets( - d, "static", "static-", oe.spdx30.software_Package + d, direct_deps, "static", "static-", oe.spdx30.software_Package ) if dep_recipes: @@ -753,8 +753,10 @@ def create_spdx(d): build_inputs |= files index_sources_by_hash(files, dep_sources) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + dep_objsets, dep_builds = collect_dep_objsets( - d, "builds", "build-", oe.spdx30.build_Build + d, direct_deps, "builds", "build-", oe.spdx30.build_Build ) if dep_builds: @@ -998,7 +1000,9 @@ def create_package_spdx(d): deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) deploydir = Path(d.getVar("SPDXRUNTIMEDEPLOY")) - providers = oe.spdx_common.collect_package_providers(d) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + + providers = oe.spdx_common.collect_package_providers(d, direct_deps) pkg_arch = d.getVar("SSTATE_PKGARCH") if get_is_native(d): @@ -1175,7 +1179,9 @@ def write_bitbake_spdx(d): def collect_build_package_inputs(d, objset, build, packages, files_by_hash=None): import oe.sbom30 - providers = oe.spdx_common.collect_package_providers(d) + direct_deps = oe.spdx_common.collect_direct_deps(d, "do_create_spdx") + + providers = oe.spdx_common.collect_package_providers(d, direct_deps) build_deps = set() diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 3aaf2a9c8b..c0ef11f199 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -38,7 +38,7 @@ def extract_licenses(filename): def is_work_shared_spdx(d): - return '/work-shared/' in d.getVar('S') + return "/work-shared/" in d.getVar("S") def load_spdx_license_data(d): @@ -77,12 +77,15 @@ def process_sources(d): return True -@dataclass(frozen=True) +@dataclass(frozen=True, eq=True, order=True) class Dep(object): pn: str hashfn: str in_taskhash: bool + def to_tuple(self): + return (self.pn, self.hashfn, self.in_taskhash) + def collect_direct_deps(d, dep_task): """ @@ -105,7 +108,9 @@ def collect_direct_deps(d, dep_task): ) if not dep_task in depflags: - bb.fatal(f"Task {dep_task} was not found in any dependency flag of {pn}:{current_task}") + bb.fatal( + f"Task {dep_task} was not found in any dependency flag of {pn}:{current_task}" + ) for this_dep in taskdepdata.values(): if this_dep[0] == pn and this_dep[1] == current_task: @@ -118,25 +123,14 @@ def collect_direct_deps(d, dep_task): for dep_name in this_dep.deps: dep_data = taskdepdata[dep_name] if dep_data.taskname == dep_task and dep_data.pn != pn: - deps.add((dep_data.pn, dep_data.hashfn, dep_name in this_dep.taskhash_deps)) + deps.add( + Dep(dep_data.pn, dep_data.hashfn, dep_name in this_dep.taskhash_deps) + ) return sorted(deps) -def get_spdx_deps(d): - """ - Reads the SPDX dependencies JSON file and returns the data - """ - spdx_deps_file = Path(d.getVar("SPDXDEPS")) - - deps = [] - with spdx_deps_file.open("r") as f: - for d in json.load(f): - deps.append(Dep(*d)) - return deps - - -def collect_package_providers(d): +def collect_package_providers(d, direct_deps): """ Returns a dictionary where each RPROVIDES is mapped to the package that provides it @@ -145,16 +139,15 @@ def collect_package_providers(d): providers = {} - deps = collect_direct_deps(d, "do_create_spdx") - deps.append((d.getVar("PN"), d.getVar("BB_HASHFILENAME"), True)) + all_deps = direct_deps + [Dep(d.getVar("PN"), d.getVar("BB_HASHFILENAME"), True)] - for dep_pn, dep_hashfn, _ in deps: + for dep in all_deps: localdata = d - recipe_data = oe.packagedata.read_pkgdata(dep_pn, localdata) + recipe_data = oe.packagedata.read_pkgdata(dep.pn, localdata) if not recipe_data: localdata = bb.data.createCopy(d) localdata.setVar("PKGDATA_DIR", "${PKGDATA_DIR_SDK}") - recipe_data = oe.packagedata.read_pkgdata(dep_pn, localdata) + recipe_data = oe.packagedata.read_pkgdata(dep.pn, localdata) for pkg in recipe_data.get("PACKAGES", "").split(): pkg_data = oe.packagedata.read_subpkgdata_dict(pkg, localdata) @@ -171,7 +164,7 @@ def collect_package_providers(d): rprovides.add(pkg) for r in rprovides: - providers[r] = (pkg, dep_hashfn) + providers[r] = (pkg, dep.hashfn) return providers @@ -202,25 +195,21 @@ def get_patched_src(d): bb.build.exec_func("do_unpack", d) if d.getVar("SRC_URI") != "": - if bb.data.inherits_class('dos2unix', d): - bb.build.exec_func('do_convert_crlf_to_lf', d) + if bb.data.inherits_class("dos2unix", d): + bb.build.exec_func("do_convert_crlf_to_lf", d) bb.build.exec_func("do_patch", d) # Copy source from work-share to spdx_workdir if is_work_shared_spdx(d): - share_src = d.getVar('S') + share_src = d.getVar("S") d.setVar("WORKDIR", spdx_workdir) d.setVar("STAGING_DIR_NATIVE", spdx_sysroot_native) # Copy source to ${SPDXWORK}, same basename dir of ${S}; - src_dir = ( - spdx_workdir - + "/" - + os.path.basename(share_src) - ) + src_dir = spdx_workdir + "/" + os.path.basename(share_src) # For kernel souce, rename suffix dir 'kernel-source' # to ${BP} (${BPN}-${PV}) if bb.data.inherits_class("kernel", d): - src_dir = spdx_workdir + "/" + d.getVar('BP') + src_dir = spdx_workdir + "/" + d.getVar("BP") bb.note(f"copyhardlinktree {share_src} to {src_dir}") oe.path.copyhardlinktree(share_src, src_dir) @@ -233,7 +222,9 @@ def get_patched_src(d): def has_task(d, task): - return bool(d.getVarFlag(task, "task", False)) and not bool(d.getVarFlag(task, "noexec", False)) + return bool(d.getVarFlag(task, "task", False)) and not bool( + d.getVarFlag(task, "noexec", False) + ) def fetch_data_to_uri(fd, name): @@ -243,8 +234,8 @@ def fetch_data_to_uri(fd, name): uri = fd.type # crate: is not a valid URL. Use url field instead if exist - if uri == "crate" and hasattr(fd,"url"): - return fd.url + if uri == "crate" and hasattr(fd, "url"): + return fd.url # Map gitsm to git, since gitsm:// is not a valid URI protocol if uri == "gitsm": @@ -259,11 +250,13 @@ def fetch_data_to_uri(fd, name): return uri -def is_compiled_source (filename, compiled_sources, types): + +def is_compiled_source(filename, compiled_sources, types): """ Check if the file is a compiled file """ import os + # If we don't have compiled source, we assume all are compiled. if not compiled_sources: return True @@ -278,11 +271,13 @@ def is_compiled_source (filename, compiled_sources, types): # Check that the file is in the list return filename in compiled_sources + def get_compiled_sources(d): """ Get list of compiled sources from debug information and normalize the paths """ import itertools + source_info = oe.package.read_debugsources_info(d) if not source_info: bb.debug(1, "Do not have debugsources.list. Skipping")