diff mbox series

[3/4] spdx30: Add the PURLs of the source downloads to the output packages

Message ID 20261118092927.204959-4-stondo@gmail.com
State Changes Requested
Headers show
Series spdx30: identity and purls of dependency downloads | expand

Commit Message

Stefano Tondo Nov. 18, 2026, 9:29 a.m. UTC
From: Stefano Tondo <stefano.tondo.ext@siemens.com>

Statically linked code, for example a Rust crate, is part of the
binary, but the SPDX of the output package only names the recipe.
A vulnerability scanner that checks the packages of an image does
not find the dependency.

Add the PURLs of the source downloads to the external identifiers of
each output package. Leave out the crates that the cargo dependency
graph proves are not built into the target. The software_packageUrl
of the package stays the first PURL in SPDX_PACKAGE_URLS.

Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
 meta/lib/oe/spdx30_tasks.py | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

Comments

Mathieu Dubois-Briand Nov. 18, 2026, 3:22 p.m. UTC | #1
On Wed Nov 18, 2026 at 10:29 AM CET, Stefano Tondo via openembedded-core wrote:
> From: Stefano Tondo <stefano.tondo.ext@siemens.com>
>
> Statically linked code, for example a Rust crate, is part of the
> binary, but the SPDX of the output package only names the recipe.
> A vulnerability scanner that checks the packages of an image does
> not find the dependency.
>
> Add the PURLs of the source downloads to the external identifiers of
> each output package. Leave out the crates that the cargo dependency
> graph proves are not built into the target. The software_packageUrl
> of the package stays the first PURL in SPDX_PACKAGE_URLS.
>
> Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
> ---

Hi Stefano,

Thanks for your patches.

It looks like this is using memory in a way it got killed several times
on the autobuilder. Locally, I managed to have a do_create_spdx task
using more than 10GB of memory, with a value continuously increasing of
about 100MB/s.

This can be reproduced with master branch + your patches, then:
bitbake python3-uv-build-native -c do_create_spdx

Some failed builds for reference:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/110/builds/1605
https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/5058
https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/5060
https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/5039
https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/5061

Can you have a look at the issue?

Thanks,
Mathieu
diff mbox series

Patch

diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index e40716a6df..441df709a4 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -526,16 +526,24 @@  def get_cargo_scopes(d):
 
 
 def add_dependency_scopes(d, objset, build, source_files):
-    """Give crate downloads a lifecycle scope, see get_cargo_scopes()."""
+    """Give crate downloads a lifecycle scope, see get_cargo_scopes().
+
+    Returns the crate downloads that are not built into the target: crates
+    that are not used on it, build scripts, proc-macros and the crates
+    they use, and test-only crates.
+    """
     scopes = get_cargo_scopes(d)
     if scopes is None:
-        return
+        return set()
 
     by_scope = {}
+    not_in_target = set()
     for dl in source_files:
         if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"):
             continue
         scope = scopes.get((dl.name, dl.software_packageVersion))
+        if scope != "runtime":
+            not_in_target.add(dl)
         if scope:
             by_scope.setdefault(scope, []).append(dl)
         else:
@@ -554,6 +562,8 @@  def add_dependency_scopes(d, objset, build, source_files):
             sorted(oe.sbom30.get_element_link_id(dl) for dl in dls),
         )
 
+    return not_in_target
+
 
 def add_download_files(d, objset):
     inputs = set()
@@ -1057,8 +1067,20 @@  def create_spdx(d):
 
     source_files = add_download_files(d, build_objset)
     build_inputs |= source_files
+
+    # The output packages inherit the PURLs of the downloads they are built
+    # from, as the code of statically linked dependencies is in them. Only
+    # the downloads known not to be built into the target are left out.
+    not_in_target = set()
     if bb.data.inherits_class("cargo", d):
-        add_dependency_scopes(d, build_objset, build, source_files)
+        not_in_target = add_dependency_scopes(d, build_objset, build, source_files)
+    dependency_purls = sorted(
+        {
+            dl.software_packageUrl
+            for dl in source_files
+            if getattr(dl, "software_packageUrl", None) and dl not in not_in_target
+        }
+    )
 
     recipe_spdx_license = add_license_expression(
         d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset]
@@ -1171,7 +1193,7 @@  def create_spdx(d):
                 d.getVar("SPDX_PACKAGE_URLS:%s" % package)
                 or d.getVar("SPDX_PACKAGE_URLS")
                 or ""
-            ).split()
+            ).split() + dependency_purls
 
             set_purls(spdx_package, purls)