mbox series

[scarthgap,v3,00/16] backport: SPDX 3.0 support to Scarthgap

Message ID 20251103162654.1714239-1-kamel.bouhara@bootlin.com
Headers show
Series backport: SPDX 3.0 support to Scarthgap | expand

Message

Kamel Bouhara Nov. 3, 2025, 4:26 p.m. UTC
Hello,

This is the v3 series to backport SPDX 3.0 support from upstream
Walnascar to Scarthgap (Yocto Project 5.0 LTS), it essentially
includes selftest fixes.

Changes since v2:
-----------------
Cherry-picked commits:

    c5b5136bf8: Exclude do_create_spdx from automatic sysroot extension.

    03c6ace6c6: Fix SPDX_VERSION handling in selftests.

    7be5a6e699: Add tar test for SPDX 2.2.

Local fix:

    7e14a5b966: Align DEPLOY_DIR_SPDX in create-spdx-2.2.bbclass with
	${SPDX_VERSION} layout, restoring successful SPDX 2.2 selftests.

Greetings,
Kamel

Hongxu Jia (1):
  oeqa/selftest: Add SPDX 3.0 include source case for work-share

Joshua Watt (10):
  classes-global/license: Move functions to library code
  classes-global/staging: Exclude do_create_spdx from automatic sysroot
    extension
  lib/license: Move package license skip to library
  lib: oe: license: Add missing import
  classes-recipe/image: Add image file manifest
  classes-recipe/baremetal-image: Add image file manifest
  selftest: spdx: Add SPDX 3.0 test cases
  lib: oeqa: spdx: Add tests for extra options
  spdx 3.0: Rework how SPDX aliases are linked
  oeqa: spdx: Add tar test for SPDX 2.2

Kamel Bouhara (Schneider Electric) (4):
  backport: SPDX 3.0 fixes and tasks from upstream version Walnascar
  spdx30_tasks: fix FetchData attribute in add_download_files
  classes/create-spdx-2.2: align DEPLOY_DIR_SPDX with SPDX_VERSION
    layout
  spdx30_tasks: adapt CVE handling to new cve-check API

Richard Purdie (1):
  selftest/spdx: Fix for SPDX_VERSION addition

 meta/classes-global/base.bbclass              |   41 +-
 meta/classes-global/license.bbclass           |  165 -
 meta/classes-global/staging.bbclass           |    9 +-
 meta/classes-recipe/baremetal-image.bbclass   |   32 +-
 .../create-spdx-image-3.0.bbclass             |   85 +
 .../create-spdx-sdk-3.0.bbclass               |   74 +
 meta/classes-recipe/image.bbclass             |   58 +
 meta/classes-recipe/license_image.bbclass     |   14 +-
 meta/classes-recipe/nospdx.bbclass            |   13 +
 meta/classes/create-spdx-2.2.bbclass          |    3 +-
 meta/classes/create-spdx-3.0.bbclass          |  200 +
 meta/classes/spdx-common.bbclass              |   99 +
 meta/lib/oe/__init__.py                       |    2 +-
 meta/lib/oe/license.py                        |  203 +
 meta/lib/oe/sbom30.py                         | 1096 ++++
 meta/lib/oe/spdx30.py                         | 5593 +++++++++++++++++
 meta/lib/oe/spdx30_tasks.py                   | 1343 ++++
 meta/lib/oe/spdx_common.py                    |  244 +
 meta/lib/oeqa/selftest/cases/spdx.py          |  256 +-
 19 files changed, 9307 insertions(+), 223 deletions(-)
 create mode 100644 meta/classes-recipe/create-spdx-image-3.0.bbclass
 create mode 100644 meta/classes-recipe/create-spdx-sdk-3.0.bbclass
 create mode 100644 meta/classes-recipe/nospdx.bbclass
 create mode 100644 meta/classes/create-spdx-3.0.bbclass
 create mode 100644 meta/classes/spdx-common.bbclass
 create mode 100644 meta/lib/oe/sbom30.py
 create mode 100644 meta/lib/oe/spdx30.py
 create mode 100644 meta/lib/oe/spdx30_tasks.py
 create mode 100644 meta/lib/oe/spdx_common.py

--
2.43.0

Comments

Steve Sakoman Nov. 5, 2025, 2:40 p.m. UTC | #1
I'm getting autobuilder errors with this series:

ERROR: baremetal-helloworld-0.1+git-r0 do_rootfs: Error executing a
python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:image_combine_spdx(d)
     0003:
File: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/classes/create-spdx-2.2.bbclass',
lineno: 980, function: image_combine_spdx
     0976:    imgdeploydir = Path(d.getVar("IMGDEPLOYDIR"))
     0977:    img_spdxid = oe.sbom.get_image_spdxid(image_name)
     0978:    packages = image_list_installed_packages(d)
     0979:
 *** 0980:    combine_spdx(d, image_name, imgdeploydir, img_spdxid,
packages, Path(d.getVar("SPDXIMAGEWORK")))
     0981:
     0982:    def make_image_link(target_path, suffix):
     0983:        if image_link_name:
     0984:            link = imgdeploydir / (image_link_name + suffix)
File: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/classes/create-spdx-2.2.bbclass',
lineno: 1048, function: combine_spdx
     1044:    image.supplier = d.getVar("SPDX_SUPPLIER")
     1045:
     1046:    doc.packages.append(image)
     1047:
 *** 1048:    for name in sorted(packages.keys()):
     1049:        if name not in providers:
     1050:            bb.fatal("Unable to find SPDX provider for '%s'" % name)
     1051:
     1052:        pkg_name, pkg_hashfn = providers[name]
Exception: AttributeError: 'str' object has no attribute 'keys'

Here's an error log with more details:

https://errors.yoctoproject.org/Errors/Details/886537/

Steve

On Mon, Nov 3, 2025 at 8:27 AM Kamel Bouhara via
lists.openembedded.org
<kamel.bouhara=bootlin.com@lists.openembedded.org> wrote:
>
> Hello,
>
> This is the v3 series to backport SPDX 3.0 support from upstream
> Walnascar to Scarthgap (Yocto Project 5.0 LTS), it essentially
> includes selftest fixes.
>
> Changes since v2:
> -----------------
> Cherry-picked commits:
>
>     c5b5136bf8: Exclude do_create_spdx from automatic sysroot extension.
>
>     03c6ace6c6: Fix SPDX_VERSION handling in selftests.
>
>     7be5a6e699: Add tar test for SPDX 2.2.
>
> Local fix:
>
>     7e14a5b966: Align DEPLOY_DIR_SPDX in create-spdx-2.2.bbclass with
>         ${SPDX_VERSION} layout, restoring successful SPDX 2.2 selftests.
>
> Greetings,
> Kamel
>
> Hongxu Jia (1):
>   oeqa/selftest: Add SPDX 3.0 include source case for work-share
>
> Joshua Watt (10):
>   classes-global/license: Move functions to library code
>   classes-global/staging: Exclude do_create_spdx from automatic sysroot
>     extension
>   lib/license: Move package license skip to library
>   lib: oe: license: Add missing import
>   classes-recipe/image: Add image file manifest
>   classes-recipe/baremetal-image: Add image file manifest
>   selftest: spdx: Add SPDX 3.0 test cases
>   lib: oeqa: spdx: Add tests for extra options
>   spdx 3.0: Rework how SPDX aliases are linked
>   oeqa: spdx: Add tar test for SPDX 2.2
>
> Kamel Bouhara (Schneider Electric) (4):
>   backport: SPDX 3.0 fixes and tasks from upstream version Walnascar
>   spdx30_tasks: fix FetchData attribute in add_download_files
>   classes/create-spdx-2.2: align DEPLOY_DIR_SPDX with SPDX_VERSION
>     layout
>   spdx30_tasks: adapt CVE handling to new cve-check API
>
> Richard Purdie (1):
>   selftest/spdx: Fix for SPDX_VERSION addition
>
>  meta/classes-global/base.bbclass              |   41 +-
>  meta/classes-global/license.bbclass           |  165 -
>  meta/classes-global/staging.bbclass           |    9 +-
>  meta/classes-recipe/baremetal-image.bbclass   |   32 +-
>  .../create-spdx-image-3.0.bbclass             |   85 +
>  .../create-spdx-sdk-3.0.bbclass               |   74 +
>  meta/classes-recipe/image.bbclass             |   58 +
>  meta/classes-recipe/license_image.bbclass     |   14 +-
>  meta/classes-recipe/nospdx.bbclass            |   13 +
>  meta/classes/create-spdx-2.2.bbclass          |    3 +-
>  meta/classes/create-spdx-3.0.bbclass          |  200 +
>  meta/classes/spdx-common.bbclass              |   99 +
>  meta/lib/oe/__init__.py                       |    2 +-
>  meta/lib/oe/license.py                        |  203 +
>  meta/lib/oe/sbom30.py                         | 1096 ++++
>  meta/lib/oe/spdx30.py                         | 5593 +++++++++++++++++
>  meta/lib/oe/spdx30_tasks.py                   | 1343 ++++
>  meta/lib/oe/spdx_common.py                    |  244 +
>  meta/lib/oeqa/selftest/cases/spdx.py          |  256 +-
>  19 files changed, 9307 insertions(+), 223 deletions(-)
>  create mode 100644 meta/classes-recipe/create-spdx-image-3.0.bbclass
>  create mode 100644 meta/classes-recipe/create-spdx-sdk-3.0.bbclass
>  create mode 100644 meta/classes-recipe/nospdx.bbclass
>  create mode 100644 meta/classes/create-spdx-3.0.bbclass
>  create mode 100644 meta/classes/spdx-common.bbclass
>  create mode 100644 meta/lib/oe/sbom30.py
>  create mode 100644 meta/lib/oe/spdx30.py
>  create mode 100644 meta/lib/oe/spdx30_tasks.py
>  create mode 100644 meta/lib/oe/spdx_common.py
>
> --
> 2.43.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#225667): https://lists.openembedded.org/g/openembedded-core/message/225667
> Mute This Topic: https://lists.openembedded.org/mt/116100659/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Kamel Bouhara Nov. 6, 2025, 3:30 p.m. UTC | #2
On Wed, Nov 05, 2025 at 06:40:57AM -0800, Steve Sakoman wrote:
> I'm getting autobuilder errors with this series:
>
> ERROR: baremetal-helloworld-0.1+git-r0 do_rootfs: Error executing a
> python function in exec_func_python() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:image_combine_spdx(d)
>      0003:
> File: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/classes/create-spdx-2.2.bbclass',
> lineno: 980, function: image_combine_spdx
>      0976:    imgdeploydir = Path(d.getVar("IMGDEPLOYDIR"))
>      0977:    img_spdxid = oe.sbom.get_image_spdxid(image_name)
>      0978:    packages = image_list_installed_packages(d)
>      0979:
>  *** 0980:    combine_spdx(d, image_name, imgdeploydir, img_spdxid,
> packages, Path(d.getVar("SPDXIMAGEWORK")))
>      0981:
>      0982:    def make_image_link(target_path, suffix):
>      0983:        if image_link_name:
>      0984:            link = imgdeploydir / (image_link_name + suffix)
> File: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/classes/create-spdx-2.2.bbclass',
> lineno: 1048, function: combine_spdx
>      1044:    image.supplier = d.getVar("SPDX_SUPPLIER")
>      1045:
>      1046:    doc.packages.append(image)
>      1047:
>  *** 1048:    for name in sorted(packages.keys()):
>      1049:        if name not in providers:
>      1050:            bb.fatal("Unable to find SPDX provider for '%s'" % name)
>      1051:
>      1052:        pkg_name, pkg_hashfn = providers[name]
> Exception: AttributeError: 'str' object has no attribute 'keys'
>
> Here's an error log with more details:
>
> https://errors.yoctoproject.org/Errors/Details/886537/

Thanks Steve, I’ll reach out to Joshua to see the best way to fix this
issue.

Kamel

>
> Steve
>
> On Mon, Nov 3, 2025 at 8:27 AM Kamel Bouhara via
> lists.openembedded.org
> <kamel.bouhara=bootlin.com@lists.openembedded.org> wrote:
> >
> > Hello,
> >
> > This is the v3 series to backport SPDX 3.0 support from upstream
> > Walnascar to Scarthgap (Yocto Project 5.0 LTS), it essentially
> > includes selftest fixes.
> >
> > Changes since v2:
> > -----------------
> > Cherry-picked commits:
> >
> >     c5b5136bf8: Exclude do_create_spdx from automatic sysroot extension.
> >
> >     03c6ace6c6: Fix SPDX_VERSION handling in selftests.
> >
> >     7be5a6e699: Add tar test for SPDX 2.2.
> >
> > Local fix:
> >
> >     7e14a5b966: Align DEPLOY_DIR_SPDX in create-spdx-2.2.bbclass with
> >         ${SPDX_VERSION} layout, restoring successful SPDX 2.2 selftests.
> >
> > Greetings,
> > Kamel
> >
> > Hongxu Jia (1):
> >   oeqa/selftest: Add SPDX 3.0 include source case for work-share
> >
> > Joshua Watt (10):
> >   classes-global/license: Move functions to library code
> >   classes-global/staging: Exclude do_create_spdx from automatic sysroot
> >     extension
> >   lib/license: Move package license skip to library
> >   lib: oe: license: Add missing import
> >   classes-recipe/image: Add image file manifest
> >   classes-recipe/baremetal-image: Add image file manifest
> >   selftest: spdx: Add SPDX 3.0 test cases
> >   lib: oeqa: spdx: Add tests for extra options
> >   spdx 3.0: Rework how SPDX aliases are linked
> >   oeqa: spdx: Add tar test for SPDX 2.2
> >
> > Kamel Bouhara (Schneider Electric) (4):
> >   backport: SPDX 3.0 fixes and tasks from upstream version Walnascar
> >   spdx30_tasks: fix FetchData attribute in add_download_files
> >   classes/create-spdx-2.2: align DEPLOY_DIR_SPDX with SPDX_VERSION
> >     layout
> >   spdx30_tasks: adapt CVE handling to new cve-check API
> >
> > Richard Purdie (1):
> >   selftest/spdx: Fix for SPDX_VERSION addition
> >
> >  meta/classes-global/base.bbclass              |   41 +-
> >  meta/classes-global/license.bbclass           |  165 -
> >  meta/classes-global/staging.bbclass           |    9 +-
> >  meta/classes-recipe/baremetal-image.bbclass   |   32 +-
> >  .../create-spdx-image-3.0.bbclass             |   85 +
> >  .../create-spdx-sdk-3.0.bbclass               |   74 +
> >  meta/classes-recipe/image.bbclass             |   58 +
> >  meta/classes-recipe/license_image.bbclass     |   14 +-
> >  meta/classes-recipe/nospdx.bbclass            |   13 +
> >  meta/classes/create-spdx-2.2.bbclass          |    3 +-
> >  meta/classes/create-spdx-3.0.bbclass          |  200 +
> >  meta/classes/spdx-common.bbclass              |   99 +
> >  meta/lib/oe/__init__.py                       |    2 +-
> >  meta/lib/oe/license.py                        |  203 +
> >  meta/lib/oe/sbom30.py                         | 1096 ++++
> >  meta/lib/oe/spdx30.py                         | 5593 +++++++++++++++++
> >  meta/lib/oe/spdx30_tasks.py                   | 1343 ++++
> >  meta/lib/oe/spdx_common.py                    |  244 +
> >  meta/lib/oeqa/selftest/cases/spdx.py          |  256 +-
> >  19 files changed, 9307 insertions(+), 223 deletions(-)
> >  create mode 100644 meta/classes-recipe/create-spdx-image-3.0.bbclass
> >  create mode 100644 meta/classes-recipe/create-spdx-sdk-3.0.bbclass
> >  create mode 100644 meta/classes-recipe/nospdx.bbclass
> >  create mode 100644 meta/classes/create-spdx-3.0.bbclass
> >  create mode 100644 meta/classes/spdx-common.bbclass
> >  create mode 100644 meta/lib/oe/sbom30.py
> >  create mode 100644 meta/lib/oe/spdx30.py
> >  create mode 100644 meta/lib/oe/spdx30_tasks.py
> >  create mode 100644 meta/lib/oe/spdx_common.py
> >
> > --
> > 2.43.0
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#225667): https://lists.openembedded.org/g/openembedded-core/message/225667
> > Mute This Topic: https://lists.openembedded.org/mt/116100659/3620601
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >

--
Kamel Bouhara, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com