From patchwork Wed Nov 18 23:44:22 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 104811 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.4152.1795045472952607606 for ; Wed, 18 Nov 2026 15:44:33 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=Ch4Uvi9Z; spf=pass (domain: gmail.com, ip: 209.85.221.43, mailfrom: stondo@gmail.com) Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-4a7b8abe209so830057f8f.0 for ; Wed, 18 Nov 2026 15:44:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795045471; x=1795650271; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xO0OOZkadpAthd17lMA31hFxO3/QewgAd0/TDcrDg1w=; b=Ch4Uvi9Z02Xe1X6b2o0ncJ3okChz1V4UO4l5sOHoBlL5qsU72/56z2hmW9+JI2R5IF jL9f+z5KUfjkqrALIZFMjDenOPhZe0w3qeDB0sqxFCnwOoYtVjG5Apwit/81QpjXKs/Y vfdMEly8BJJbpVhNC0GU34wkNOJoDMp6p+HmSddtQJJ+jSbtIjqrzKFnKq439nWlNVF6 abnHxumk+mwczn6YpVqOk3gOOKVo2AI94k+wyE2X/tE45W7FrhNiJS8xiEXd8jmaLMaI fphfYRRolEX7M/W8PE2ciJgbWMqUGQVs2tiDeQybbCpZ5XbyMHQkVr26HgdKb5DI6sKG 6LVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795045471; x=1795650271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xO0OOZkadpAthd17lMA31hFxO3/QewgAd0/TDcrDg1w=; b=bEJVnMqgwQAjqTXOF1NKyDxi550/TzjhC0kKWGrMyPSgXr7V4LYK7yPx0iT2kMDZ0C 7DhGH8Tkp3kcQGdlXk0ozxl+0nDEcnkkkig4x1MoYzWtIyI5U46F+TbvDasKjMkUPSj6 XYTtoEjitp1Th3vswoBfNDUngUQVSelNhm/hslWc7ftj26al8n2qIXMOg32BKN51bRFH vLMTweAzgVUqd0lWlWKYl1/lNqXsckZFDWuJEvq9JlyqPNPTiFhCTBve3nxs+A1DPXv2 nrs3SkAzuJH2UnyvnZgXslPr/95NjiPtnvxFHiFeBYMEOyOPGwHzwdFbNXYUlTGBRQUD iI2A== X-Gm-Message-State: APuU4Z+RPy0cK3djWx9fdtFw3tXCeo2cL3qdaNw2nv9QiJvzL5sQIZk4 +L0pvf8oA9iYnv+y739Phf8SHcAP/evN0zV0wvDgEUGh6Z7jId/WC0bz722buQ== X-Gm-Gg: AZFdTZcAY4j85BwHHxa8Un+qzODKoi365ajlNsJNoKme8KEYULocJyzTeqbZGorl6y4 IkiT2vzv38UVMgkUukKab7qhYT6NE9hVqLBjS2+DKW8NPrxrPORYoV5yrDkRGFKtbXC8rGobeq+ 87Wp2/BN6gaapSHb3jcMQU1nEMn8rY3uqMQtJrbjSEaUf/3miYAkSQKbgq0NhQ/foIgKBre6wQm pON3exiM3t+ODlwAEMUI548YcDEQSu6rbu7N0qbnWNJ3WTrbf2hVMaL4WB4GJXFxBAF4O5gkyNl aPoeC3tNXMZki5p53a+YMHLdEpgFyNVAdZ17hDKxGJRszsaf1lFbiutCdT/psB3aE/9rBIbDN5k PiNPZnxkaM0KlaVKc5QW9gnyf/MhIGARMLAVBdXkaAVK3A/cEZlNtWUwoNLQ7a2sOTTUHpR1L1Y hQRgpj1D+Ytov3SvbZeC+FjTGk6X5tdj/38OWp9dj/kbNUmoKpSYxcM5gmqzuOr354gW1Lk13SX HyDtvNXA1Fo3QQQXSGN0IgBz2SATiOrdfC7nasq/NaOLwkquJsRDpJ0YprlJcmpHIB7P8vtv1fK iTerMuPwcGFx X-Received: by 2002:a05:6000:4a07:b0:48e:3767:d885 with SMTP id ffacd0b85a97d-4a63f64e505mr8642851f8f.53.1795045471028; Wed, 18 Nov 2026 15:44:31 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4a654a8c3c6sm6730718f8f.19.2026.11.18.15.44.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Nov 2026 15:44:29 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Mathieu Dubois-Briand , stefano.tondo.ext@siemens.com Subject: [PATCH v2 1/4] spdx30: Take name, version and PURL of downloads from the fetcher Date: Thu, 19 Nov 2026 00:44:22 +0100 Message-ID: <20261118234425.3681880-2-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261118234425.3681880-1-stondo@gmail.com> References: <20261118234425.3681880-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Wed, 18 Nov 2026 23:44:37 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250601 From: Stefano Tondo The fetchers of package registries (crate, gomod, npm) know the name and the version of each package they download. Today the SPDX of such a download only has the file name, so an SBOM scanner cannot match the dependency. Ask the fetcher with package_identity() and set the name, the version and the PURL of the download. This needs package_identity() in bitbake. Signed-off-by: Stefano Tondo --- meta/lib/oe/spdx30_tasks.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 6eaa74ce3d..58fe45ec7e 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -463,6 +463,19 @@ def enrich_source_package(d, dl, fd, file_name, primary_purpose): ) +def set_package_identity(d, dl, fd): + """Set name, version and PURL of a download that names a package. + + Fetchers of package registries (crate, gomod, npm) know which package + a URL downloads, see FetchMethod.package_identity() in bitbake. + """ + identity = fd.method.package_identity(fd, d) + if identity: + dl.name = identity["name"] + dl.software_packageVersion = identity["version"] + dl.software_packageUrl = identity["purl"] + + def add_download_files(d, objset): inputs = set() release_date = get_release_date(d) @@ -528,6 +541,7 @@ def add_download_files(d, objset): ) enrich_source_package(d, dl, fd, file_name, primary_purpose) + set_package_identity(d, dl, fd) if release_date is not None: dl.releaseTime = release_date From patchwork Wed Nov 18 23:44:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 104812 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.3928.1795045474576126922 for ; Wed, 18 Nov 2026 15:44:34 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=wqYqCAN6; spf=pass (domain: gmail.com, ip: 209.85.221.46, mailfrom: stondo@gmail.com) Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-49d64a88222so208126f8f.1 for ; Wed, 18 Nov 2026 15:44:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795045473; x=1795650273; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=slJC0v5xsKxvzx+hs6xcHaoETbX+btqhLuTIoYCNnzA=; b=wqYqCAN6LPUwhYy4pPc5lJX/ZbwsFvD/zLxfpri3XS0cbT+66p6UMS3NTftWsvKB8L YhoXB1f219RszcJQav9JEU95xBmmbm7UIU3OV2me9r21bekHNVjG7AHVRk7er7L9VNKy 6vmQronazyuhOXICliGM3wDNq3ylq3TS0nl+NenXmZP44tJrJJeRYF1ukJJJ53j93vUK FZBnBwX/I7VGcsbZ48Di6/zK7R9ztM1ubFZfeKOHLBU2i372LmH+DJAnNA35v4rWtWXX IrsiaFEYHu5qfB0/e8AhJR0876d7AwCBYR/X/V7tCx+b35SzFuPdTxm68DRFDxUzkdY8 aFOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795045473; x=1795650273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=slJC0v5xsKxvzx+hs6xcHaoETbX+btqhLuTIoYCNnzA=; b=PcyhwcSg9q2q6GGTTVcmVnIKyNXboUPa4Ljn1pTocEWn5oL27GTWZmwXknUrflGeoU hAwK+Fo+RQ6pUJSbfRpAKTgZQxK33kf8jo9ejdExkuWPEUz+WK25XUuHOnlf5qvq4wFa 5Czvp7FOJKftE9r+TasZi2c+Cr/QEx0r1Att0+CyKZu23c6nEdLiE5/P74wrZOo0A1Bx +Djxh7hTQ48Ya4qLFSkUkQIOk955O0RIGvkptCtdd6O3T7LphTwi10Xz2X+P6fipgIqG THxiyjwVfiOPe3ByliTt3NWWHc2TS+nM3CPi907xuVbzO3sSIIKzpJ0JjDK0a68SShTQ lFTQ== X-Gm-Message-State: APuU4Z9DnAWn+BJL9i1H5oWwHOjvpAOc+uikyz3MRm5BAgssKMnpvQFG BAVmKmg8oraQxFnWZXbnsEbJzyVcQYGYeyz7BNrvpt9vR0Xg4aa//q5QxnOXqg== X-Gm-Gg: AZFdTZc7BjTPxGlGsjI9DmXh2v26xzSbchD/L/GtFBMS054kUSRNwmQAb4TwjDvU4QU BHg8XtO6D/XhGJ1hjEGLJpGXCURaIPIdRQ4I2kqePNy7Wm0w8TMmk/oa6po5UL1B+B6ZBLRNa53 KgcAj+GhBMBmLmLCrd/BUidw1/BMw67Cmr0OFHhwgNPE7AdX7oMT8CHGZJIEChOfW/5ZgPPn6Mu Dvk8/ZgXNLoueMKgqyXl67WDioMvHVfWnoNFLCjJD9EHniHPwxdKhaqY49kzOUb2nLD2SYnMFB8 bhzhTAOgb5Ym0fIkJ8HKGZ9OaU37AblG0za4OsASgHdgIyAPi8JF062AfW8wZDpoGSwR1wDn5Fk E/fzzoUd7tmKaSp+MV6z995oSAlt9OQ5yoADSib9tEVgAeSaLgjlY4JW0DQd091i302anjHxo4C yXpI4Yph5FM0e277Vz+dY07uLNydmiljY+DmLdOHSkySqVUwnMjJOXSmb1HLMe7jD3C6/xwXDjJ ZTYFSZKVFKG7pd4+3gVL7Hgr2oKMR5XqVthP9a68xU/l+Dit2cwyle2Z/xg2k0iDrjI7gbGjyS6 dyWQ63IUexk= X-Received: by 2002:a05:6000:420d:b0:491:cd7:e96b with SMTP id ffacd0b85a97d-4a9a5cf6a6bmr203932f8f.15.1795045472701; Wed, 18 Nov 2026 15:44:32 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4a654a8c3c6sm6730718f8f.19.2026.11.18.15.44.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Nov 2026 15:44:32 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Mathieu Dubois-Briand , stefano.tondo.ext@siemens.com Subject: [PATCH v2 2/4] spdx30: Record the lifecycle scope of crate dependencies Date: Thu, 19 Nov 2026 00:44:23 +0100 Message-ID: <20261118234425.3681880-3-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261118234425.3681880-1-stondo@gmail.com> References: <20261118234425.3681880-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Wed, 18 Nov 2026 23:44:37 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250602 From: Stefano Tondo A Cargo.lock file lists the crates for all targets, and some crates only run at build time (build scripts, proc-macros and the crates they use). The fetcher downloads all of them, but only some end up in the binary. Add a do_cargo_metadata task that writes the dependency graph for the target with cargo metadata --filter-platform. The create_spdx task reads this graph. It links the build to each crate download with a scoped dependsOn relationship: runtime, build or test. A crate that is not used on the target gets an annotation instead. The walk expands each workspace member only once. Members can depend on each other in a cycle, and a member can be a dev dependency of itself, as uv-preview is in python3-uv-build. Add unit tests for the scopes, with member cycles. Signed-off-by: Stefano Tondo --- meta/classes-recipe/cargo.bbclass | 17 ++++ meta/lib/oe/spdx30_tasks.py | 90 +++++++++++++++++++ .../oeqa/selftest/cases/oelib/cargo_scopes.py | 73 +++++++++++++++ 3 files changed, 180 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py diff --git a/meta/classes-recipe/cargo.bbclass b/meta/classes-recipe/cargo.bbclass index 586071988a..b3e82f9d5b 100644 --- a/meta/classes-recipe/cargo.bbclass +++ b/meta/classes-recipe/cargo.bbclass @@ -26,6 +26,23 @@ cargo_do_compile () { "${CARGO}" build ${CARGO_BUILD_FLAGS} ${PACKAGECONFIG_CONFARGS} "$@" } +# The crate dependency graph for the target, with the crates that are not +# used on it removed. create-spdx uses it to give each crate download a +# lifecycle scope. +CARGO_METADATA_FILE = "${WORKDIR}/cargo-metadata.json" + +do_cargo_metadata () { + rm -f "${CARGO_METADATA_FILE}" + # Some recipes inherit cargo without being a cargo project + if [ ! -f "${CARGO_MANIFEST_PATH}" ]; then + return + fi + "${CARGO}" metadata --format-version 1 --offline --locked \ + --manifest-path "${CARGO_MANIFEST_PATH}" \ + --filter-platform "${RUST_HOST_SYS}" > "${CARGO_METADATA_FILE}" +} +addtask cargo_metadata after do_configure before do_compile do_create_spdx + cargo_do_install () { local have_installed=false for tgt in "${B}/target/${CARGO_TARGET_SUBDIR}/"*; do diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 58fe45ec7e..7aae73ed0d 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -476,6 +476,94 @@ def set_package_identity(d, dl, fd): dl.software_packageUrl = identity["purl"] +def get_cargo_scopes(d): + """Return {(crate, version): scope} from CARGO_METADATA_FILE. + + The file is the output of cargo metadata --filter-platform for the + target, so crates used only on other targets are absent. Walks the + dependency graph from the workspace members: normal dependencies are + runtime, build dependencies and proc-macro crates (and everything they + pull in) are build, dev dependencies of the members are test. + """ + path = d.getVar("CARGO_METADATA_FILE") + if not path or not os.path.exists(path): + return None + + with open(path) as f: + metadata = json.load(f) + + packages = {p["id"]: p for p in metadata["packages"]} + nodes = {n["id"]: n for n in metadata["resolve"]["nodes"]} + members = set(metadata["workspace_members"]) + rank = {"test": 1, "build": 2, "runtime": 3} + + def is_proc_macro(pkg_id): + return any("proc-macro" in t["kind"] for t in packages[pkg_id]["targets"]) + + scopes = {} + expanded_members = set() + stack = [(m, "runtime") for m in members] + while stack: + pkg_id, scope = stack.pop() + if pkg_id in members: + # A member is a root with the runtime scope. Expand it only + # once: members can depend on each other in a cycle, and a + # member can even be a dev dependency of itself. + if pkg_id in expanded_members: + continue + expanded_members.add(pkg_id) + scope = "runtime" + else: + if scope == "runtime" and is_proc_macro(pkg_id): + scope = "build" + if rank.get(scopes.get(pkg_id), 0) >= rank[scope]: + continue + scopes[pkg_id] = scope + for dep in nodes[pkg_id]["deps"]: + for kind in (k["kind"] for k in dep["dep_kinds"]): + if kind == "dev": + if pkg_id in members: + stack.append((dep["pkg"], "test")) + elif kind == "build": + stack.append((dep["pkg"], "test" if scope == "test" else "build")) + else: + stack.append((dep["pkg"], scope)) + + return { + (packages[i]["name"], packages[i]["version"]): s for i, s in scopes.items() + } + + +def add_dependency_scopes(d, objset, build, source_files): + """Give crate downloads a lifecycle scope, see get_cargo_scopes().""" + scopes = get_cargo_scopes(d) + if scopes is None: + return + + by_scope = {} + for dl in source_files: + if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"): + continue + scope = scopes.get((dl.name, dl.software_packageVersion)) + if scope: + by_scope.setdefault(scope, []).append(dl) + else: + objset.new_annotation( + dl, + "%s %s is fetched but not built: not used on target %s" + % (dl.name, dl.software_packageVersion, d.getVar("RUST_HOST_SYS")), + oe.spdx30.AnnotationType.other, + ) + + for scope, dls in sorted(by_scope.items()): + objset.new_scoped_relationship( + [build], + oe.spdx30.RelationshipType.dependsOn, + getattr(oe.spdx30.LifecycleScopeType, scope), + sorted(oe.sbom30.get_element_link_id(dl) for dl in dls), + ) + + def add_download_files(d, objset): inputs = set() release_date = get_release_date(d) @@ -978,6 +1066,8 @@ def create_spdx(d): source_files = add_download_files(d, build_objset) build_inputs |= source_files + if bb.data.inherits_class("cargo", d): + add_dependency_scopes(d, build_objset, build, source_files) recipe_spdx_license = add_license_expression( d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset] diff --git a/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py new file mode 100644 index 0000000000..6273346668 --- /dev/null +++ b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py @@ -0,0 +1,73 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +import json +import os +import tempfile +from unittest.case import TestCase + +import bb.data +import oe.spdx30_tasks + + +def package(name, proc_macro=False): + kind = ["proc-macro"] if proc_macro else ["lib"] + return {"id": name, "name": name, "version": "1.0.0", "targets": [{"kind": kind}]} + + +def dep(name, *kinds): + return {"pkg": name, "dep_kinds": [{"kind": kind} for kind in kinds or (None,)]} + + +class TestCargoScopes(TestCase): + def get_scopes(self, members, packages, deps): + metadata = { + "packages": packages, + "workspace_members": members, + "resolve": {"nodes": [{"id": p["id"], "deps": deps.get(p["id"], [])} for p in packages]}, + } + with tempfile.TemporaryDirectory() as tmpdir: + path = os.path.join(tmpdir, "cargo-metadata.json") + with open(path, "w") as f: + json.dump(metadata, f) + d = bb.data.init() + d.setVar("CARGO_METADATA_FILE", path) + scopes = oe.spdx30_tasks.get_cargo_scopes(d) + return {name: scope for (name, version), scope in scopes.items()} + + def test_scopes(self): + packages = [package(n) for n in ("app", "runtime-dep", "build-dep", "build-dep-dep", + "dev-dep", "shared")] + packages.append(package("derive", proc_macro=True)) + deps = { + "app": [dep("runtime-dep"), dep("build-dep", "build"), dep("dev-dep", "dev"), + dep("derive"), dep("shared", "dev")], + "build-dep": [dep("build-dep-dep")], + "runtime-dep": [dep("shared")], + } + self.assertEqual(self.get_scopes(["app"], packages, deps), { + "runtime-dep": "runtime", + "shared": "runtime", + "build-dep": "build", + "build-dep-dep": "build", + "derive": "build", + "dev-dep": "test", + }) + + def test_member_cycles(self): + # Workspace members that depend on each other in a cycle, and a member + # that is a dev dependency of itself, as in uv. Each member is a root + # and is expanded once. + packages = [package(n) for n in ("member-a", "member-b", "member-c", "crate-a", "crate-c")] + deps = { + "member-a": [dep("member-b"), dep("member-a", "dev"), dep("crate-a")], + "member-b": [dep("member-c"), dep("member-a", "dev")], + "member-c": [dep("member-a"), dep("member-c", None, "dev"), dep("crate-c", "dev")], + } + self.assertEqual(self.get_scopes(["member-a", "member-b", "member-c"], packages, deps), { + "crate-a": "runtime", + "crate-c": "test", + }) From patchwork Wed Nov 18 23:44:24 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 104813 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.4153.1795045477079480627 for ; Wed, 18 Nov 2026 15:44:37 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=uuKZpdeB; spf=pass (domain: gmail.com, ip: 209.85.128.52, mailfrom: stondo@gmail.com) Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-4ac95761c85so12810025e9.2 for ; Wed, 18 Nov 2026 15:44:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795045475; x=1795650275; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=4h39kvK8SGfHkypJnM+Og15Dbn7IP3iwkJ7gsJCdJqE=; b=uuKZpdeBfXf2DcevDMaf2KN28k3/ydrY9UZoPhhOmAdqnKk1YKVNNTMXwj3OVUcOHr yOspr2jsJb7BOeJNKb9aHdq13qldaFkCwpABJ6udhaX7ndUD70tRVgsrt5TM+S2cas60 zRIw6Dym4j/aRf03jc2k1mqCnBZRwKgIjWxF6yocJFF5HrARnyPXMoM8g7Pqv2gD64Ti Ef7tf6ToIehJ8suiHnUmc26C4wfDeyS1PxP/kLzljf3KuXbrVfdXfutT0ICnMIJzsKbl 7KyARcPqdEi94D7dEf+ZBXxAaFZ0WQX1R42LHe+IS5PEwaU8Q0eYwPouq1i2j6sX6dmR CDBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795045475; x=1795650275; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=4h39kvK8SGfHkypJnM+Og15Dbn7IP3iwkJ7gsJCdJqE=; b=1lW+s1T99YQP3yTHrwjvAeuyWXEPtiblL2Iplu8sbGH6StJij2B36C+LWvkDjIyV3B QwCMVJOMAB/SLiQEegpXmhRYXbn3/81INpg36mue5y7XleSH84OzMK+vzIBIhPYpSs4p OBJTte5P8eAOfn9bFf2klSJL9SbjQSgE1BmT3Z5vsKk9XwoAqfCtI7xd4i5kzAjW4IkR MagoXN+GMx1Spn44RwpJtqN7Dyofe6Ox0uGaG9/MISFJXsKllHbUFb8ybzgodHhd5AZD Q87Vg/7Kml651K8I3KHf9MLoelP9BeVM4FPAMKLjJRTczN0Eos0tPmNNT3Y57HjUO+mN UTJQ== X-Gm-Message-State: APuU4Z9o+3o6wjz6y59E/tOV1FR2vrQyz9N0JfiCoWT0ZlkoduBk49mX cxbQSdV0K7XDaDX/m8/W7uHcxXB5cJLgV6dKentg5PO3dR/VLtuSeZvZjPnElQ== X-Gm-Gg: AYBFou0qcRVN82uNTmqqDnH10igW+OUrOUIGOLAfB/TjiL5IZYV1RIrSRuhT4UcTqBf gjPrtKaogq7CLJR9xP6UyXBoAdd0nn7ue6AkWJiD7mS6akUWdMSznCdhwGlP5SxlEOfWMKCae7X oLYXvOR6HvAmP8Ow3qjF3kAr/JQayObvBdX3csAZY0Xv6p+NBI1DmqLDGA+8q+ZV55irQxvOHoA AaVGtaxJdrJfpmgZmUqG62eRGTKuZLjsfZ0JrYL/jqGr/qzmMz4DkE8Yg6IE/7XKMSJr5NBkq8F PK0kHFw+Dg7Hw8SbHWfmFriNPHgiirTXxJLtvxxQo98fMSIUc/LvXWONFndg2+9zJyO1oixYQ1G 9ZKWzRsdt8wYAwlSF6YwgSN2FFgM+A9Wg0dm4vn2APnDXFGvdxmLUzyt2W8QVErsqRvKyMKnVmf WlVM9ok3c6I+OGjq8j380OX1gEnbieMsaHUsxau0tzeNzDhqrifNW+ouAg5fZZpljhAHjZaClx8 WNDTCXb3fWZXiLiCze2I0KS0H0OZ4AeSuWIIpX8EMX12GispBeaJ67nJcaH2jqIdr9/WSaa6dcM 0gmn5M5WTv8= X-Received: by 2002:a05:600c:c174:b0:4a8:f2ad:a9e5 with SMTP id 5b1f17b1804b1-4bb7d5d4108mr77253235e9.3.1795045475276; Wed, 18 Nov 2026 15:44:35 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4a654a8c3c6sm6730718f8f.19.2026.11.18.15.44.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Nov 2026 15:44:33 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Mathieu Dubois-Briand , stefano.tondo.ext@siemens.com Subject: [PATCH v2 3/4] spdx30: Add the PURLs of the source downloads to the output packages Date: Thu, 19 Nov 2026 00:44:24 +0100 Message-ID: <20261118234425.3681880-4-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261118234425.3681880-1-stondo@gmail.com> References: <20261118234425.3681880-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Wed, 18 Nov 2026 23:44:37 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250603 From: Stefano Tondo Statically linked code, for example a Rust crate, is part of the binary, but the SPDX of the output package only names the recipe. A vulnerability scanner that checks the packages of an image does not find the dependency. Add the PURLs of the source downloads to the external identifiers of each output package. Leave out the crates that the cargo dependency graph proves are not built into the target. The software_packageUrl of the package stays the first PURL in SPDX_PACKAGE_URLS. Signed-off-by: Stefano Tondo --- meta/lib/oe/spdx30_tasks.py | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 7aae73ed0d..49494a5007 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -535,16 +535,24 @@ def get_cargo_scopes(d): def add_dependency_scopes(d, objset, build, source_files): - """Give crate downloads a lifecycle scope, see get_cargo_scopes().""" + """Give crate downloads a lifecycle scope, see get_cargo_scopes(). + + Returns the crate downloads that are not built into the target: crates + that are not used on it, build scripts, proc-macros and the crates + they use, and test-only crates. + """ scopes = get_cargo_scopes(d) if scopes is None: - return + return set() by_scope = {} + not_in_target = set() for dl in source_files: if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"): continue scope = scopes.get((dl.name, dl.software_packageVersion)) + if scope != "runtime": + not_in_target.add(dl) if scope: by_scope.setdefault(scope, []).append(dl) else: @@ -563,6 +571,8 @@ def add_dependency_scopes(d, objset, build, source_files): sorted(oe.sbom30.get_element_link_id(dl) for dl in dls), ) + return not_in_target + def add_download_files(d, objset): inputs = set() @@ -1066,8 +1076,20 @@ def create_spdx(d): source_files = add_download_files(d, build_objset) build_inputs |= source_files + + # The output packages inherit the PURLs of the downloads they are built + # from, as the code of statically linked dependencies is in them. Only + # the downloads known not to be built into the target are left out. + not_in_target = set() if bb.data.inherits_class("cargo", d): - add_dependency_scopes(d, build_objset, build, source_files) + not_in_target = add_dependency_scopes(d, build_objset, build, source_files) + dependency_purls = sorted( + { + dl.software_packageUrl + for dl in source_files + if getattr(dl, "software_packageUrl", None) and dl not in not_in_target + } + ) recipe_spdx_license = add_license_expression( d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset] @@ -1180,7 +1202,7 @@ def create_spdx(d): d.getVar("SPDX_PACKAGE_URLS:%s" % package) or d.getVar("SPDX_PACKAGE_URLS") or "" - ).split() + ).split() + dependency_purls set_purls(spdx_package, purls) From patchwork Wed Nov 18 23:44:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 104814 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.3930.1795045479524341824 for ; Wed, 18 Nov 2026 15:44:39 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=A9qV24FD; spf=pass (domain: gmail.com, ip: 209.85.221.51, mailfrom: stondo@gmail.com) Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-4a34ffd3e39so900052f8f.0 for ; Wed, 18 Nov 2026 15:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795045478; x=1795650278; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=0ddKFm4edcu46UL0qqikK24536Nx+0m4JDW2BPdxXAg=; b=A9qV24FD+yP2veXjlQBcAft5tAmZiTIukQb4I0XRCRreIhsXyD52vOsLr9t6PPr96w QtpdRN4tQTpVwiEvDtf6oYf9Q16/Npe16g/ushzFjOU/5HcehJv5k2GQxKnd1sBU8diw v6PWDhI38my0mqt8rH8ObF2jsLK7mYNenhAQAnvIBvckw/CX60dyaobKr6c1vyLV5xH6 mUBAiYZ/X06byHoSM6KKvLTFtePOJQc8IIM3rQfE1YOMoLdIhLk6zYZAw7oLSnWIR70y Cp4NO6+vrJFvfZgBBF87Iz6+h7j4GzOfVcmKYQ5gwPRnKA9IIrXYH9ZPIqk/JlJqDX52 qjJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795045478; x=1795650278; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=0ddKFm4edcu46UL0qqikK24536Nx+0m4JDW2BPdxXAg=; b=RCH98iI1epSAQxdg/05KkCAP0PMY/QaDDn8FdtxhHSxVAdGCocPatLxMFSa8tXEmcl K89OdHhkO5keHLNyH0/FMMgXxJbAOttJA9hd2LRZCI919RTsSTNSRmijYVD7RQd4ygsm IRi+z7kjrlldnC0vHulyYN5qV3U/jJFLbmf9koqkW7KBemSoBudIoEUml9gH+kq63xvU iaI7ixYBKO/RmCt9ql61yrxNyZH8wZnORr8yiLCFJChVpHXrxLFRQxVCJKHoh1go+l/a xistLlrcc7ld6h6CjuU2apSoifnqrDgF0LuscgO4LCx+PIMyeSNFwSPUDcb57T8KJicd ZpJg== X-Gm-Message-State: APuU4Z/sDtDKmYyvXb4Y3lYC5QIqNwJbuY+snEo/YJSBBDClD0/V6p0C rWAg/DsZ4jL1Y/0kWqH5jwD0BJaJ4vzg2+h1l8+SPhOiG7dM4wRxrUhsGJsE8g== X-Gm-Gg: AZFdTZcfGC4VhA2F5C9mF1xdOc97Hxqi8d4xmPhlCJbk+MhOmVlo7hg1Ysv7+unVqm5 ISnx7FYXtwa70ohVbLXg96OgcW0cL4MzWjmUagLZM3CnNJFxQ6baK/1DZEeS4rMCwIR5T9s+yoD kKppfO3M8tucR32oIosLRPifaDatFs9y90eWxh+zIrNpl7WeoxUsPH1oOsvPqFuWzRxNNR6xHzB 1gwg+IR5yjWFLOZVcvvHnXLC1Ihg0/a7feiPKK+ya+uHXDXuRUunzg/m/yAkfy/qOWVKNepvBDp YmsBD8drF8gFDC7TkE03iU7gPRB7H5Uf5DoWTCDGYXQublNpAlJSdAFyJMZkj+tsN3OUtEgWYYD weDxBjur9OqdTrvJe6q+Vow0c5gH7UOi/32vya5ZIpnEr0qWv291CKJScL+9Uc2orHTbvc0dtrD avYwVzpIGAZgq0bmnUvWQFxwx/p0so/Nc1QbY4UW7MC/rGYsmB8gv0xH7FHh3oGYyqIvvd0F1y7 hAMhpRc8pjTVtEScf7rYW2xCUt/wP68ANe0B9dfO1fhw03fRYDTOfKS4C4nf6xEOXUblTH2NDNr Y2MiRxHkugk= X-Received: by 2002:a05:6000:2512:b0:48d:f383:2356 with SMTP id ffacd0b85a97d-4a63c374724mr9828606f8f.53.1795045477714; Wed, 18 Nov 2026 15:44:37 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4a654a8c3c6sm6730718f8f.19.2026.11.18.15.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Nov 2026 15:44:36 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Joshua Watt , Alexander Kanavin , Stefan Herbrechtsmeier , Mathieu Dubois-Briand , stefano.tondo.ext@siemens.com Subject: [PATCH v2 4/4] oeqa/selftest: spdx: Test crate identity, scope and output PURLs Date: Thu, 19 Nov 2026 00:44:25 +0100 Message-ID: <20261118234425.3681880-5-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261118234425.3681880-1-stondo@gmail.com> References: <20261118234425.3681880-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Wed, 18 Nov 2026 23:44:47 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250604 From: Stefano Tondo python3-bcrypt fetches 42 crates. The new tests check three things: - Each crate download has its name, version and PURL. - Each crate download has a lifecycle scope, or an annotation that it is not built. - The output package gets the PURLs of the runtime crates only. The tests name only crates that do not change on recipe upgrades. blowfish is the cipher of bcrypt. Only the pyo3 proc-macros use syn, at build time. The other checks do not use crate names. Signed-off-by: Stefano Tondo --- meta/lib/oeqa/selftest/cases/spdx.py | 122 +++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/spdx.py b/meta/lib/oeqa/selftest/cases/spdx.py index ce9a8af4f6..d75c6146c7 100644 --- a/meta/lib/oeqa/selftest/cases/spdx.py +++ b/meta/lib/oeqa/selftest/cases/spdx.py @@ -9,6 +9,7 @@ import hashlib from datetime import datetime, timezone from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars +import oe.purl import oe.spdx30 @@ -543,3 +544,124 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): # Document should be fully linked self.check_objset_missing_ids(objset) + + def test_crate_package_identity(self): + """Test that crate downloads carry name, version and a cargo PURL.""" + objset = self.check_recipe_spdx( + "python3-bcrypt", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-python3-bcrypt.spdx.json", + ) + + prefix = "https://static.crates.io/crates/" + crates = 0 + for pkg in objset.foreach_type(oe.spdx30.software_Package): + location = pkg.software_downloadLocation or "" + if not location.startswith(prefix): + continue + name, version, _ = location[len(prefix):].split("/") + self.assertEqual(pkg.name, name) + self.assertEqual(pkg.software_packageVersion, version) + self.assertEqual( + pkg.software_packageUrl, + f"pkg:cargo/{oe.purl.quote(name)}@{oe.purl.quote(version)}", + ) + crates += 1 + + self.assertGreater(crates, 0, "no crate downloads found") + + def test_crate_dependency_scope(self): + """Test that crate downloads get a lifecycle scope for the target. + + Every crate download is either the target of one scoped dependsOn + relationship or annotated as not built. python3-bcrypt fetches crates + for other platforms (WebAssembly, UEFI), which are never built for a + Linux target. blowfish is the cipher of bcrypt, and syn only runs at + build time in proc-macros, so these names do not change on upgrades. + """ + objset = self.check_recipe_spdx( + "python3-bcrypt", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-python3-bcrypt.spdx.json", + ) + + crates = { + pkg._id: pkg.name + for pkg in objset.foreach_type(oe.spdx30.software_Package) + if (pkg.software_packageUrl or "").startswith("pkg:cargo/") + and (pkg.software_downloadLocation or "").startswith("https://static.crates.io/") + } + + scope_of = {} + for rel in objset.foreach_type(oe.spdx30.LifecycleScopedRelationship): + if rel.relationshipType != oe.spdx30.RelationshipType.dependsOn: + continue + for to in rel.to: + to_id = to if isinstance(to, str) else to._id + if to_id in crates: + self.assertNotIn(crates[to_id], scope_of) + scope_of[crates[to_id]] = rel.scope + + not_built = set() + for annotation in objset.foreach_type(oe.spdx30.Annotation): + subject = annotation.subject + subject_id = subject if isinstance(subject, str) else subject._id + if subject_id in crates and "not built" in annotation.statement: + not_built.add(crates[subject_id]) + + self.assertEqual(set(scope_of) | not_built, set(crates.values())) + self.assertFalse(set(scope_of) & not_built) + self.assertTrue(not_built, "no crate for another platform found") + self.assertEqual(scope_of["syn"], oe.spdx30.LifecycleScopeType.build) + self.assertEqual(scope_of["blowfish"], oe.spdx30.LifecycleScopeType.runtime) + + def test_crate_purls_in_output_package(self): + """Test that the output package inherits the PURLs of its crates. + + The package gets the PURLs of the crates with the runtime scope in + the build SPDX, and no other crate PURLs. syn only runs at build + time in proc-macros, so it is not in the binary. + """ + objset = self.check_recipe_spdx( + "python3-bcrypt", + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/packages/package-python3-bcrypt.spdx.json", + ) + + package = next( + pkg + for pkg in objset.foreach_type(oe.spdx30.software_Package) + if pkg.name == "python3-bcrypt" + ) + purls = { + e.identifier + for e in package.externalIdentifier + if e.externalIdentifierType == oe.spdx30.ExternalIdentifierType.packageUrl + } + crates = {p[len("pkg:cargo/"):].split("@")[0] for p in purls if p.startswith("pkg:cargo/")} + + # The package's own PURL stays first + own_purls = get_bb_var("SPDX_PACKAGE_URLS", "python3-bcrypt").split() + self.assertEqual(package.software_packageUrl, own_purls[0]) + self.assertTrue(set(own_purls) <= purls) + self.assertIn("blowfish", crates) + self.assertNotIn("syn", crates) + + spdx_vars = get_bb_vars(["DEPLOY_DIR_SPDX", "SSTATE_PKGARCH"], "python3-bcrypt") + build_objset = self.check_spdx_file( + "{DEPLOY_DIR_SPDX}/{SSTATE_PKGARCH}/builds/build-python3-bcrypt.spdx.json".format(**spdx_vars) + ) + downloads = { + pkg._id: pkg.software_packageUrl + for pkg in build_objset.foreach_type(oe.spdx30.software_Package) + if (pkg.software_packageUrl or "").startswith("pkg:cargo/") + } + runtime = set() + for rel in build_objset.foreach_type(oe.spdx30.LifecycleScopedRelationship): + if rel.scope != oe.spdx30.LifecycleScopeType.runtime: + continue + for to in rel.to: + to_id = to if isinstance(to, str) else to._id + if to_id in downloads: + runtime.add(downloads[to_id]) + + self.assertTrue(runtime, "no crate with the runtime scope found") + cargo_purls = {p for p in purls if p.startswith("pkg:cargo/")} + self.assertEqual(cargo_purls - set(own_purls), runtime)