From patchwork Thu Nov 19 20:49:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105000 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.4746.1795121354145221436 for ; Thu, 19 Nov 2026 12:49:14 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=PXNo03z9; spf=pass (domain: gmail.com, ip: 209.85.221.45, mailfrom: stondo@gmail.com) Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-492fcbfaa88so82511f8f.3 for ; Thu, 19 Nov 2026 12:49:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795121352; x=1795726152; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=HDrfGaEx8zpGcWQX4X/LNZ+KM0hAVNCKWLtbOnESC60=; b=PXNo03z9hKNlmr1tgax7HKi8gqgdypDVBn3vFz+Ml2aF86/ztSsBDb7iewcBH7SDcG EG2gHUdPMuAsfOty1goKT+DOfkYA+e5GBB1iq+xgywbMn/iqy1WqExFDgzfA9R1HSvVz JvobeHnuQye3vhjg/KgCE080pWbKzTxH5LUUlV8gSdN6mBI4WNgNM3PHhxpx9Dcbhb2G fVYTvLDgVlygErJf4llUt8AjLbeHRKFyGX2EzRqwZx0JYJdiFTt5MOyyyTzGnlb8xkJ5 eyjeZ9vIy6vgeMh6cUGxL2csYp/ZT7h7vawcD/AMOYAQ9LYJ04h3PX8hbjpOo1K+IRwM CKbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795121352; x=1795726152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=HDrfGaEx8zpGcWQX4X/LNZ+KM0hAVNCKWLtbOnESC60=; b=WHuTbNUeIBI3MajBQ/DCrQ3Hwa479/+vuEP3i3wsEXqgP+7t5Lrgh5PiQU2DSUbLob REXljzQbD2JWLd/dDnZsN2LYHhMFduOPPaAXwo3uXOn3MxdvNfZjmH6ShUyy/7Xmm77d /kWixFwG2ge3fwT8O/Y3ZDnm38X4f5tXtQ+S8gJ5wOZFJQRS14nwHgKyWut668GvxP2u EeoBgPddAwYc+J/kBgDE+lyhEMoEyKQt+jiSPeKCfXOE238UNKqfl0dytZYYTAoqspF0 56fUahWAY9sr4KXZfbbbUMr0A6S9Q8tCvT6IwFodbT7OWttQJ7+3DKKlJWCi85K7c+hF pUxA== X-Gm-Message-State: APuU4Z9qOT6hbqAo7GsbGcr55uxahdv+sosUrBNZ9yU5k0GCOOn5jC2v TKeylkyMlAozksLdl9spR+84vsIJ2pYSU//Bge1cslQkQlQ3c1YOEEbDxbi6WHJq X-Gm-Gg: AZFdTZcaZ1Zw96eWIDiova3eLjndJQmzECCBRSBRqC849+nJcD56992V+LuNMLHKuDo hrhIyefBF72HM87ULCcZtZsEj+9g+wcz1YXwG4B3UcZRXVvLt7g9acGtA2LSFJ6nTZg801EtavS JnsL7pppQOLDXJGJWMYMIc/hj5KyyXa0o2sArtITYP5xy0oD/v0BUZfluA/QzhE+CQRJbn1KFoB kjxqdBAwwAKQ4l+QcQ5sCvH3vwg0Lx3jRl1IMEkR2+ZOVhGGcqgK/YQJgXY9xTPX9cSFr+EcXrY Ydx90qSO2R0s5bb5dcaqxHQGNLCUWY3zTFBkaH/hyhWPg9VaUS391zXtl191MoWr8e3UOLzVTNQ oD9anm1FHLrscfJRgabeA7vyR4ayDuB5DryvuF42sVcszghptJCQwEzrKt+OXoJ72weji13ef3E TASNb7ZW1D2avrluMPfu8yfplLWYiTlrNoDHvOJDzUArV86uvGe9FiIHrHsB2iJjH8MuCi6SBSz C3lgYUOCnqoF1i0kIW81q0zl2OirgrTsrJrbw/+lXo6gokiXsLNVHmNmB0+KuEX7t9Y1XVre31l nb/gWE/F2ik= X-Received: by 2002:a05:6000:4381:b0:4a6:5a2a:f5a3 with SMTP id ffacd0b85a97d-4abaa9773abmr1306358f8f.3.1795121352298; Thu, 19 Nov 2026 12:49:12 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4ab6c43563esm1606171f8f.57.2026.11.19.12.49.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 12:49:11 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Paul Barker , Mathieu Dubois-Briand , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH v3 1/4] spdx30: Take name, version and PURL of downloads from the fetcher Date: Thu, 19 Nov 2026 21:49:06 +0100 Message-ID: <20261119204909.2931233-2-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261119204909.2931233-1-stondo@gmail.com> References: <20261119204909.2931233-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Thu, 19 Nov 2026 20:49:19 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250718 From: Stefano Tondo The fetchers of package registries (crate, gomod, npm) know the name and the version of each package they download. Today the SPDX of such a download only has the file name, so an SBOM scanner cannot match the dependency. Ask the fetcher with package_identity() and set the name, the version and the PURL of the download. This needs package_identity() in bitbake. Signed-off-by: Stefano Tondo --- meta/lib/oe/spdx30_tasks.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 6eaa74ce3d..eded315819 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -463,6 +463,19 @@ def enrich_source_package(d, dl, fd, file_name, primary_purpose): ) +def set_package_identity(d, dl, fd): + """Set name, version and PURL of a download that names a package. + + Fetchers of package registries (crate, gomod, npm) know which package + a URL downloads, see FetchMethod.package_identity() in bitbake. + """ + identity = fd.method.package_identity(fd, d) + if identity: + dl.name = identity.name + dl.software_packageVersion = identity.version + dl.software_packageUrl = identity.purl + + def add_download_files(d, objset): inputs = set() release_date = get_release_date(d) @@ -528,6 +541,7 @@ def add_download_files(d, objset): ) enrich_source_package(d, dl, fd, file_name, primary_purpose) + set_package_identity(d, dl, fd) if release_date is not None: dl.releaseTime = release_date From patchwork Thu Nov 19 20:49:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105001 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.4747.1795121355438119450 for ; Thu, 19 Nov 2026 12:49:15 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=sLPMWUyC; spf=pass (domain: gmail.com, ip: 209.85.221.53, mailfrom: stondo@gmail.com) Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-490d59c3f0aso73212f8f.1 for ; Thu, 19 Nov 2026 12:49:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795121354; x=1795726154; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=iyEY1MwSnkpJ/cglrwrz43/rdUxqcF5jiCN7lY+MY00=; b=sLPMWUyCw5UpNXGdG9UvoW3auCbsujfbZZQqVKfofY2THQk+WRbXnd4v5i6Axi9aWz 8BrIFswj2kK8m0LyBJIzEOrHqJHJGnFGi9OMyGOlhQDAVKln0g7ceOfBao1aqDx1ejqV OvXdCK/Cea7nfbZJFsb/fDCkLH47xIUdeb6PmzeWcQJIR5sTTO0t3ldGJqHIqCLJYeVV 0WbQ4Q+vsDo3WhsmeaWQFemdDUUE05qRnIayaLWZk7wKEjefgdRBgcmC4u2JM/t6rHH4 3oufsSsR6knEjEJB3w5RLnPCOZsVj4J1Gw1OoeHdUZzgtfVtoW6iuGY51pmZ02M+LpPs ft+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795121354; x=1795726154; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=iyEY1MwSnkpJ/cglrwrz43/rdUxqcF5jiCN7lY+MY00=; b=MZhpY//uL7N3kDqN/RDCAqPAG8CIZgQqaJRoGgAbO2qEfepIP62yQvfQ3/v/g517yb zRr8sLIv2q+eNOIjF65Vuz+s+kGgECit1b0oOvNuI8deG2WYauCAdoUMzcSFRTyOXFf+ HxpoKgVR3Q8XtfOoQO+9Bt9on6NMnIuSFEEhv7iSUGKp2LfPOuoTxENbn3kBjhV6NICY odsMdJPBjrspc3otD0nE8oSFLKEanNaACkllQHuECTZ+So96F5+nR9dfFo5sZJDlgYW6 QkMDSC6uSCO/0qjFs8rwhCxKo/svj0QDRrxEJG5LmyV538pn/e7LtCHMGGHSpML3MNk7 7kgw== X-Gm-Message-State: APuU4Z8PYcFDidPdLdKZ2TtPsq6N2yxRTCSf3ZFwfwySe+/TaakLSWHj AUHo3qio4qs3X8wHLNezq9YUyAwK4jgVlGpSD440vOIzFhlcE2AVYSS4lYccjH67 X-Gm-Gg: AZFdTZc2jWLQf23qDdJqXIetsDZCL0FTsUSo5vt3sbWCkrc5AscQmVHosUamD48Sdvx kRKJIVBhDZSFKw0XdSg/SzsLM4mcxadxwrtPgWpNowE+AGKXgCYPG7c4qaMMw3JRINT1ic3K+S0 jH6aRvM0oWG+ahfWEoohpsc5hfxeassMUYVb8cksv3ARyo54K5c/liMmt9x3Um+matcYqN3P8SX CBtV6ia/SY419DC/6OhmGjfZdEkOpZr+HzvEkpUb26X8OOr/ij96T564WL7ym9bFsSJ9S9+8u9i 1hWs8px07Tp5JNXrJgRHQuCFXjdpMOyU5iursy1BT8APn/WOdZ3MyxB/f+i0mkdabyBqLFqB5gU x/o3B0KfZXlxuTwFH4zEsER7bTq6Btttg+3g4DXleBLz8fgX7vKAmNtKz4RixKWGU/YMB+d0We5 CFtXcdTtLuAE+Qd798/MaYSPd8DjvCV2GYowQSnsjsFh7iPVHyf7aglJDILUOMYWiYoLUmRUtEB gNy2CG+49v6fh8x0cPF5raaJ9zNUtEoXn7XO+mQb4pZV7ZiHNRf6H05LgAnFScLxas2BBDT9bVk 84NAuv64W4g= X-Received: by 2002:a05:6000:4904:b0:4a6:60b2:afed with SMTP id ffacd0b85a97d-4abb3c37a2dmr1047312f8f.11.1795121353562; Thu, 19 Nov 2026 12:49:13 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4ab6c43563esm1606171f8f.57.2026.11.19.12.49.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 12:49:12 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Paul Barker , Mathieu Dubois-Briand , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH v3 2/4] spdx30: Record the lifecycle scope of crate dependencies Date: Thu, 19 Nov 2026 21:49:07 +0100 Message-ID: <20261119204909.2931233-3-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261119204909.2931233-1-stondo@gmail.com> References: <20261119204909.2931233-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Thu, 19 Nov 2026 20:49:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250719 From: Stefano Tondo A Cargo.lock file lists the crates for all targets, and some crates only run at build time (build scripts, proc-macros and the crates they use). The fetcher downloads all of them, but only some end up in the binary. Add a do_cargo_metadata task that writes the dependency graph for the target with cargo metadata --filter-platform. The create_spdx task reads this graph. It links the build to each crate download with a scoped dependsOn relationship: runtime, build or test. A crate that is not used on the target gets an annotation instead. The walk expands each workspace member only once. Members can depend on each other in a cycle, and a member can be a dev dependency of itself, as uv-preview is in python3-uv-build. Add unit tests for the scopes, with member cycles. Signed-off-by: Stefano Tondo --- meta/classes-recipe/cargo.bbclass | 17 ++++ meta/lib/oe/spdx30_tasks.py | 90 +++++++++++++++++++ .../oeqa/selftest/cases/oelib/cargo_scopes.py | 73 +++++++++++++++ 3 files changed, 180 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py diff --git a/meta/classes-recipe/cargo.bbclass b/meta/classes-recipe/cargo.bbclass index 586071988a..b3e82f9d5b 100644 --- a/meta/classes-recipe/cargo.bbclass +++ b/meta/classes-recipe/cargo.bbclass @@ -26,6 +26,23 @@ cargo_do_compile () { "${CARGO}" build ${CARGO_BUILD_FLAGS} ${PACKAGECONFIG_CONFARGS} "$@" } +# The crate dependency graph for the target, with the crates that are not +# used on it removed. create-spdx uses it to give each crate download a +# lifecycle scope. +CARGO_METADATA_FILE = "${WORKDIR}/cargo-metadata.json" + +do_cargo_metadata () { + rm -f "${CARGO_METADATA_FILE}" + # Some recipes inherit cargo without being a cargo project + if [ ! -f "${CARGO_MANIFEST_PATH}" ]; then + return + fi + "${CARGO}" metadata --format-version 1 --offline --locked \ + --manifest-path "${CARGO_MANIFEST_PATH}" \ + --filter-platform "${RUST_HOST_SYS}" > "${CARGO_METADATA_FILE}" +} +addtask cargo_metadata after do_configure before do_compile do_create_spdx + cargo_do_install () { local have_installed=false for tgt in "${B}/target/${CARGO_TARGET_SUBDIR}/"*; do diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index eded315819..01e4eefa2d 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -476,6 +476,94 @@ def set_package_identity(d, dl, fd): dl.software_packageUrl = identity.purl +def get_cargo_scopes(d): + """Return {(crate, version): scope} from CARGO_METADATA_FILE. + + The file is the output of cargo metadata --filter-platform for the + target, so crates used only on other targets are absent. Walks the + dependency graph from the workspace members: normal dependencies are + runtime, build dependencies and proc-macro crates (and everything they + pull in) are build, dev dependencies of the members are test. + """ + path = d.getVar("CARGO_METADATA_FILE") + if not path or not os.path.exists(path): + return None + + with open(path) as f: + metadata = json.load(f) + + packages = {p["id"]: p for p in metadata["packages"]} + nodes = {n["id"]: n for n in metadata["resolve"]["nodes"]} + members = set(metadata["workspace_members"]) + rank = {"test": 1, "build": 2, "runtime": 3} + + def is_proc_macro(pkg_id): + return any("proc-macro" in t["kind"] for t in packages[pkg_id]["targets"]) + + scopes = {} + expanded_members = set() + stack = [(m, "runtime") for m in members] + while stack: + pkg_id, scope = stack.pop() + if pkg_id in members: + # A member is a root with the runtime scope. Expand it only + # once: members can depend on each other in a cycle, and a + # member can even be a dev dependency of itself. + if pkg_id in expanded_members: + continue + expanded_members.add(pkg_id) + scope = "runtime" + else: + if scope == "runtime" and is_proc_macro(pkg_id): + scope = "build" + if rank.get(scopes.get(pkg_id), 0) >= rank[scope]: + continue + scopes[pkg_id] = scope + for dep in nodes[pkg_id]["deps"]: + for kind in (k["kind"] for k in dep["dep_kinds"]): + if kind == "dev": + if pkg_id in members: + stack.append((dep["pkg"], "test")) + elif kind == "build": + stack.append((dep["pkg"], "test" if scope == "test" else "build")) + else: + stack.append((dep["pkg"], scope)) + + return { + (packages[i]["name"], packages[i]["version"]): s for i, s in scopes.items() + } + + +def add_dependency_scopes(d, objset, build, source_files): + """Give crate downloads a lifecycle scope, see get_cargo_scopes().""" + scopes = get_cargo_scopes(d) + if scopes is None: + return + + by_scope = {} + for dl in source_files: + if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"): + continue + scope = scopes.get((dl.name, dl.software_packageVersion)) + if scope: + by_scope.setdefault(scope, []).append(dl) + else: + objset.new_annotation( + dl, + "%s %s is fetched but not built: not used on target %s" + % (dl.name, dl.software_packageVersion, d.getVar("RUST_HOST_SYS")), + oe.spdx30.AnnotationType.other, + ) + + for scope, dls in sorted(by_scope.items()): + objset.new_scoped_relationship( + [build], + oe.spdx30.RelationshipType.dependsOn, + getattr(oe.spdx30.LifecycleScopeType, scope), + sorted(oe.sbom30.get_element_link_id(dl) for dl in dls), + ) + + def add_download_files(d, objset): inputs = set() release_date = get_release_date(d) @@ -978,6 +1066,8 @@ def create_spdx(d): source_files = add_download_files(d, build_objset) build_inputs |= source_files + if bb.data.inherits_class("cargo", d): + add_dependency_scopes(d, build_objset, build, source_files) recipe_spdx_license = add_license_expression( d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset] diff --git a/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py new file mode 100644 index 0000000000..6273346668 --- /dev/null +++ b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py @@ -0,0 +1,73 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +import json +import os +import tempfile +from unittest.case import TestCase + +import bb.data +import oe.spdx30_tasks + + +def package(name, proc_macro=False): + kind = ["proc-macro"] if proc_macro else ["lib"] + return {"id": name, "name": name, "version": "1.0.0", "targets": [{"kind": kind}]} + + +def dep(name, *kinds): + return {"pkg": name, "dep_kinds": [{"kind": kind} for kind in kinds or (None,)]} + + +class TestCargoScopes(TestCase): + def get_scopes(self, members, packages, deps): + metadata = { + "packages": packages, + "workspace_members": members, + "resolve": {"nodes": [{"id": p["id"], "deps": deps.get(p["id"], [])} for p in packages]}, + } + with tempfile.TemporaryDirectory() as tmpdir: + path = os.path.join(tmpdir, "cargo-metadata.json") + with open(path, "w") as f: + json.dump(metadata, f) + d = bb.data.init() + d.setVar("CARGO_METADATA_FILE", path) + scopes = oe.spdx30_tasks.get_cargo_scopes(d) + return {name: scope for (name, version), scope in scopes.items()} + + def test_scopes(self): + packages = [package(n) for n in ("app", "runtime-dep", "build-dep", "build-dep-dep", + "dev-dep", "shared")] + packages.append(package("derive", proc_macro=True)) + deps = { + "app": [dep("runtime-dep"), dep("build-dep", "build"), dep("dev-dep", "dev"), + dep("derive"), dep("shared", "dev")], + "build-dep": [dep("build-dep-dep")], + "runtime-dep": [dep("shared")], + } + self.assertEqual(self.get_scopes(["app"], packages, deps), { + "runtime-dep": "runtime", + "shared": "runtime", + "build-dep": "build", + "build-dep-dep": "build", + "derive": "build", + "dev-dep": "test", + }) + + def test_member_cycles(self): + # Workspace members that depend on each other in a cycle, and a member + # that is a dev dependency of itself, as in uv. Each member is a root + # and is expanded once. + packages = [package(n) for n in ("member-a", "member-b", "member-c", "crate-a", "crate-c")] + deps = { + "member-a": [dep("member-b"), dep("member-a", "dev"), dep("crate-a")], + "member-b": [dep("member-c"), dep("member-a", "dev")], + "member-c": [dep("member-a"), dep("member-c", None, "dev"), dep("crate-c", "dev")], + } + self.assertEqual(self.get_scopes(["member-a", "member-b", "member-c"], packages, deps), { + "crate-a": "runtime", + "crate-c": "test", + }) From patchwork Thu Nov 19 20:49:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 104999 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.4792.1795121357828128177 for ; Thu, 19 Nov 2026 12:49:18 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=JeqJG+60; spf=pass (domain: gmail.com, ip: 209.85.221.45, mailfrom: stondo@gmail.com) Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-490c3ae81d3so82726f8f.2 for ; Thu, 19 Nov 2026 12:49:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795121356; x=1795726156; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=PvCH4R90rfne2Q7UD9n5NqK14z1DQrcr6x0XCs0igVY=; b=JeqJG+60F9k2l+zL8q/N0hxAHJLbgQMZKEalvBXQFowYElcc19pqGAArc4/jwivk// 8kJnH4yBXC4hek9dbT/aqkB9mz3tBuuChqC5owAF7LfPyXFaK2E4U/B+ViYZVQjOdW7M v4/ylduM0XqpaDwyTSwwSvIUdy01Fph/lZ1p2ZEmjtzSk9Xo+XY4KH3p5sVQJULgBuXx o9ooi8Qrf5TVnlQdbxjZBh+rcNtYwO3uITVqvs8Z6Y/CAJw6tLHVqR9i1S3/7/kqzYmf C52eJHfow+WFQgO0LfoFExlUUGBdFv3qQmvoWCldnyiLon7LIcM/zo7ijV8A/085rlfg e/Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795121356; x=1795726156; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=PvCH4R90rfne2Q7UD9n5NqK14z1DQrcr6x0XCs0igVY=; b=Pvabnnqt7RUhRoROG5MHuo9hgx9TXIWLF8FcqYrjifIWfcEHXYSRpFlxixM6Rp7yHT yevoTDSiWxyLtrPLFTAv7tPONa+H3yy3XKEoFicsTr+PRRO2DD5TDINItFsJ60IjIOXC AfhrweNQEfsw3U7d4WmMn9NpKwIDOPVlicDBDoRBV30Yqmsyv8ei7cyV59mV5J9OJ6sN RH+/eGDMeujgcFjqeEugYlJ3ghBuLiFXFfHiekuficK0TZ0jo3qJOhZanpHrfDVZru69 Vj/GVFXnaBgajGjLR5x8ZJMGmOP/4oRihpMlji2jTifP7dkwslF0aTOQ0q2LX04LRx+D MxDA== X-Gm-Message-State: APuU4Z9tKXkyvQoxRzAKzueIO6mnMNZZWySbtZTIJwhlg/Lxh3Cj74OW aYbLimOea/C74zA819exBMP3jID717FYTc+4yfDvfl0UUwTTbdHHfLXapkT0E32z X-Gm-Gg: AZFdTZcHmL/BJQLK506rCUFmXqxqio1ByUFH5ReLQRzA/2bZt+BSy8esfD93VMel6aI qNdwsdp9av+70VbFUEG0TsTtPXBP04JUSaf+f4CYnc//0NAAztRvnafpTpSAL80CbOwjVWSEgYr H53UNkiNFuxRi5+1ZZSR1c9Z71KigNExUiPPEeA77+aglbTbcjZQoW4ZZq3nK2VWGRBsnfF6BmW zYmBJpPLEQmOAtTU9f/eAZfvQimCKuRlRfv68hguDxKvDi2UKWxUh59PtHQJ+nnzKMX+Ea4ESNI LubTm7vxv3dSCCFlESBSwZvYn+yNDreBrqVj8s/qMOwyJqaJ6IZJ9a4OEF5O70iZHFiJKLlqeS4 cTg9AsstblGLPdQd1iWJzu/0XwVWOl2DtbEVlJVvBbAeNbGXzS8QOkVjEyoW+5A99TXp7w8E01+ x4ikyXhdRobdakRRqrGtwyyR89mgDc0JfgrUjg5I/e8xn/yykOfeZSCphDn6zdMq2NAmmfOirdY SEruhGh3DGeIHH1Gv58AKY4nrh97y+gxz4qCuE0RD4ik6vjUSRxPzdERyVQhlviQfuKiUFnREBO mRZZBihSdlc= X-Received: by 2002:a05:6000:47d4:b0:493:8c32:94 with SMTP id ffacd0b85a97d-4abb400ae31mr774991f8f.38.1795121355964; Thu, 19 Nov 2026 12:49:15 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4ab6c43563esm1606171f8f.57.2026.11.19.12.49.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 12:49:14 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Paul Barker , Mathieu Dubois-Briand , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH v3 3/4] spdx30: Add the PURLs of the source downloads to the output packages Date: Thu, 19 Nov 2026 21:49:08 +0100 Message-ID: <20261119204909.2931233-4-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261119204909.2931233-1-stondo@gmail.com> References: <20261119204909.2931233-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Thu, 19 Nov 2026 20:49:19 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250720 From: Stefano Tondo Statically linked code, for example a Rust crate, is part of the binary, but the SPDX of the output package only names the recipe. A vulnerability scanner that checks the packages of an image does not find the dependency. Add the PURLs of the source downloads to the external identifiers of each output package. Leave out the crates that the cargo dependency graph proves are not built into the target. The software_packageUrl of the package stays the first PURL in SPDX_PACKAGE_URLS. Signed-off-by: Stefano Tondo --- meta/lib/oe/spdx30_tasks.py | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 01e4eefa2d..6887f408ea 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -535,16 +535,24 @@ def get_cargo_scopes(d): def add_dependency_scopes(d, objset, build, source_files): - """Give crate downloads a lifecycle scope, see get_cargo_scopes().""" + """Give crate downloads a lifecycle scope, see get_cargo_scopes(). + + Returns the crate downloads that are not built into the target: crates + that are not used on it, build scripts, proc-macros and the crates + they use, and test-only crates. + """ scopes = get_cargo_scopes(d) if scopes is None: - return + return set() by_scope = {} + not_in_target = set() for dl in source_files: if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"): continue scope = scopes.get((dl.name, dl.software_packageVersion)) + if scope != "runtime": + not_in_target.add(dl) if scope: by_scope.setdefault(scope, []).append(dl) else: @@ -563,6 +571,8 @@ def add_dependency_scopes(d, objset, build, source_files): sorted(oe.sbom30.get_element_link_id(dl) for dl in dls), ) + return not_in_target + def add_download_files(d, objset): inputs = set() @@ -1066,8 +1076,20 @@ def create_spdx(d): source_files = add_download_files(d, build_objset) build_inputs |= source_files + + # The output packages inherit the PURLs of the downloads they are built + # from, as the code of statically linked dependencies is in them. Only + # the downloads known not to be built into the target are left out. + not_in_target = set() if bb.data.inherits_class("cargo", d): - add_dependency_scopes(d, build_objset, build, source_files) + not_in_target = add_dependency_scopes(d, build_objset, build, source_files) + dependency_purls = sorted( + { + dl.software_packageUrl + for dl in source_files + if getattr(dl, "software_packageUrl", None) and dl not in not_in_target + } + ) recipe_spdx_license = add_license_expression( d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset] @@ -1180,7 +1202,7 @@ def create_spdx(d): d.getVar("SPDX_PACKAGE_URLS:%s" % package) or d.getVar("SPDX_PACKAGE_URLS") or "" - ).split() + ).split() + dependency_purls set_purls(spdx_package, purls) From patchwork Thu Nov 19 20:49:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105002 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.4750.1795121360260638830 for ; Thu, 19 Nov 2026 12:49:20 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=xCOYIuJf; spf=pass (domain: gmail.com, ip: 209.85.221.45, mailfrom: stondo@gmail.com) Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-49465892f7bso151263f8f.1 for ; Thu, 19 Nov 2026 12:49:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795121358; x=1795726158; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=0ddKFm4edcu46UL0qqikK24536Nx+0m4JDW2BPdxXAg=; b=xCOYIuJfPYp1+xYUZ8puI8oXsaBSmcZC4LMwz6uTLyLb70bg7HlYQxdSA5pqQhXeGo nBfVYjxPJ+wHNsA96Dg5doGm3aVhPkNm8nOFulT9cx7EgAxQ8zlztMQfHhehqMMlZNJs XtqalRDs2VgpM8oA0PsTmOmpQT1ImxSo2+c/l59LA0VCzOLe6clE5C6k/ZPieharPpl0 dg4UT/f3Xry/Yv5CQyTJoZBlHDomwwkytzBBnUkn+X/rIySN+XArPqfwaA4vJ0f1/9a7 DnaaqZQR/D5hL4/p8CZ5BpXgLntxEmKaWZ5FK9GtsKsr7wKPe3RzQKjEFxKsLTKqiA4f OvZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795121358; x=1795726158; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=0ddKFm4edcu46UL0qqikK24536Nx+0m4JDW2BPdxXAg=; b=hj5aNQMW5yaNmgwP93eYfsFJeMs+euXm/xWhyu6gb9qdTpPPISvO8yqa0RSbs+wa3N tG7RZuajsNgEs5/Dp8+C3u9VIda4eY5NsPrxOWrCsse72Gd5aYDa4X0hxu59tIG8uj1x itYDPHEHLCZuk71z4F/6DjWypsbKEQ5RpqpUTMfHVMB8Vh9RhRb86aL3HUFD1UZ2FBdQ SIs308TDRMu3sWP+RnotdspsiXbm3m/cAB1atAJD/sz2IRgyOZaRRT7jt1u3dTc93gTm 8Zg8QBoRqT5NBQK82RtUrdyBEDBBmCcR8R+RpgxcXxJ1k89lO1b+k4VAzQrfEFxvtsZF +9Dg== X-Gm-Message-State: APuU4Z9VtYgPf1RHFRtnHhfQzQdFEbImt/BBarzakX7iFkM/zj9yLaXy 6nFyOG1kgwXabkDu5wBtB7lipoNjo0E0F+5yMRCZ6qHej/OEbodLMMh0xyVbaZPD X-Gm-Gg: AZFdTZd+GluNfH2XQaFa2zA9q00ykQ5OQ2KObcg9wmUt2Be5FuUXquj3aqNlz1Dhgg1 PuY0GwpERdalOAKzBoDk1YmTp3fn/bCux2OReyfmB/aZw8K2OWyhXH20QsIGidWyrKLjW6HyTNn FX3FI8u6aIdhPZAyG+Fo0A4rPTQQmxGk3rvo9ErYJQruoG0Vqtju6cO6OGA3XG6JuNpYX1cIO4R 22g17z3/xh8cdAf4hj5DruB0RnCx9zVQMKj79/K5aN9MscasKhcstOZZBaybU38tPw5dJEGo2wO 2t+mDOggt+VdPeDCOyKH5CcLoanizDxkGUem8tB2iEmAT/Dci8ckY2HxlR5WTVnflRFGk122bqD GX/DwLORymA01cAkD8i69e1LlMakX97HQrRTBur4rLzpXjRyRLcNJ1zoJCAG9HfbILh8M0TQV6R ITnreHFckBEfgSd9PHbhPlNKzPCOrqh8pb8nY9gmeENzvhzbP6oPydHL7AJpz/6Mt54bJ77A01+ TRBw0sYNA3j3Cswk8odamzIjoLNkIG+zZtvru3wQqlcXWJyCf5+xSGDYahMvNsFV4n47x8VidXz zgvg9KgM1Qs= X-Received: by 2002:a5d:5f85:0:b0:4a7:1613:7206 with SMTP id ffacd0b85a97d-4abb57f7b6fmr1197202f8f.47.1795121358353; Thu, 19 Nov 2026 12:49:18 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4ab6c43563esm1606171f8f.57.2026.11.19.12.49.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 12:49:17 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Paul Barker , Mathieu Dubois-Briand , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH v3 4/4] oeqa/selftest: spdx: Test crate identity, scope and output PURLs Date: Thu, 19 Nov 2026 21:49:09 +0100 Message-ID: <20261119204909.2931233-5-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261119204909.2931233-1-stondo@gmail.com> References: <20261119204909.2931233-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Thu, 19 Nov 2026 20:49:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250721 From: Stefano Tondo python3-bcrypt fetches 42 crates. The new tests check three things: - Each crate download has its name, version and PURL. - Each crate download has a lifecycle scope, or an annotation that it is not built. - The output package gets the PURLs of the runtime crates only. The tests name only crates that do not change on recipe upgrades. blowfish is the cipher of bcrypt. Only the pyo3 proc-macros use syn, at build time. The other checks do not use crate names. Signed-off-by: Stefano Tondo --- meta/lib/oeqa/selftest/cases/spdx.py | 122 +++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/spdx.py b/meta/lib/oeqa/selftest/cases/spdx.py index ce9a8af4f6..d75c6146c7 100644 --- a/meta/lib/oeqa/selftest/cases/spdx.py +++ b/meta/lib/oeqa/selftest/cases/spdx.py @@ -9,6 +9,7 @@ import hashlib from datetime import datetime, timezone from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars +import oe.purl import oe.spdx30 @@ -543,3 +544,124 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): # Document should be fully linked self.check_objset_missing_ids(objset) + + def test_crate_package_identity(self): + """Test that crate downloads carry name, version and a cargo PURL.""" + objset = self.check_recipe_spdx( + "python3-bcrypt", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-python3-bcrypt.spdx.json", + ) + + prefix = "https://static.crates.io/crates/" + crates = 0 + for pkg in objset.foreach_type(oe.spdx30.software_Package): + location = pkg.software_downloadLocation or "" + if not location.startswith(prefix): + continue + name, version, _ = location[len(prefix):].split("/") + self.assertEqual(pkg.name, name) + self.assertEqual(pkg.software_packageVersion, version) + self.assertEqual( + pkg.software_packageUrl, + f"pkg:cargo/{oe.purl.quote(name)}@{oe.purl.quote(version)}", + ) + crates += 1 + + self.assertGreater(crates, 0, "no crate downloads found") + + def test_crate_dependency_scope(self): + """Test that crate downloads get a lifecycle scope for the target. + + Every crate download is either the target of one scoped dependsOn + relationship or annotated as not built. python3-bcrypt fetches crates + for other platforms (WebAssembly, UEFI), which are never built for a + Linux target. blowfish is the cipher of bcrypt, and syn only runs at + build time in proc-macros, so these names do not change on upgrades. + """ + objset = self.check_recipe_spdx( + "python3-bcrypt", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-python3-bcrypt.spdx.json", + ) + + crates = { + pkg._id: pkg.name + for pkg in objset.foreach_type(oe.spdx30.software_Package) + if (pkg.software_packageUrl or "").startswith("pkg:cargo/") + and (pkg.software_downloadLocation or "").startswith("https://static.crates.io/") + } + + scope_of = {} + for rel in objset.foreach_type(oe.spdx30.LifecycleScopedRelationship): + if rel.relationshipType != oe.spdx30.RelationshipType.dependsOn: + continue + for to in rel.to: + to_id = to if isinstance(to, str) else to._id + if to_id in crates: + self.assertNotIn(crates[to_id], scope_of) + scope_of[crates[to_id]] = rel.scope + + not_built = set() + for annotation in objset.foreach_type(oe.spdx30.Annotation): + subject = annotation.subject + subject_id = subject if isinstance(subject, str) else subject._id + if subject_id in crates and "not built" in annotation.statement: + not_built.add(crates[subject_id]) + + self.assertEqual(set(scope_of) | not_built, set(crates.values())) + self.assertFalse(set(scope_of) & not_built) + self.assertTrue(not_built, "no crate for another platform found") + self.assertEqual(scope_of["syn"], oe.spdx30.LifecycleScopeType.build) + self.assertEqual(scope_of["blowfish"], oe.spdx30.LifecycleScopeType.runtime) + + def test_crate_purls_in_output_package(self): + """Test that the output package inherits the PURLs of its crates. + + The package gets the PURLs of the crates with the runtime scope in + the build SPDX, and no other crate PURLs. syn only runs at build + time in proc-macros, so it is not in the binary. + """ + objset = self.check_recipe_spdx( + "python3-bcrypt", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/packages/package-python3-bcrypt.spdx.json", + ) + + package = next( + pkg + for pkg in objset.foreach_type(oe.spdx30.software_Package) + if pkg.name == "python3-bcrypt" + ) + purls = { + e.identifier + for e in package.externalIdentifier + if e.externalIdentifierType == oe.spdx30.ExternalIdentifierType.packageUrl + } + crates = {p[len("pkg:cargo/"):].split("@")[0] for p in purls if p.startswith("pkg:cargo/")} + + # The package's own PURL stays first + own_purls = get_bb_var("SPDX_PACKAGE_URLS", "python3-bcrypt").split() + self.assertEqual(package.software_packageUrl, own_purls[0]) + self.assertTrue(set(own_purls) <= purls) + self.assertIn("blowfish", crates) + self.assertNotIn("syn", crates) + + spdx_vars = get_bb_vars(["DEPLOY_DIR_SPDX", "SSTATE_PKGARCH"], "python3-bcrypt") + build_objset = self.check_spdx_file( + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-python3-bcrypt.spdx.json".format(**spdx_vars) + ) + downloads = { + pkg._id: pkg.software_packageUrl + for pkg in build_objset.foreach_type(oe.spdx30.software_Package) + if (pkg.software_packageUrl or "").startswith("pkg:cargo/") + } + runtime = set() + for rel in build_objset.foreach_type(oe.spdx30.LifecycleScopedRelationship): + if rel.scope != oe.spdx30.LifecycleScopeType.runtime: + continue + for to in rel.to: + to_id = to if isinstance(to, str) else to._id + if to_id in downloads: + runtime.add(downloads[to_id]) + + self.assertTrue(runtime, "no crate with the runtime scope found") + cargo_purls = {p for p in purls if p.startswith("pkg:cargo/")} + self.assertEqual(cargo_purls - set(own_purls), runtime)