diff mbox series

[v2] ref-manual: Document SPDX 3.0.1 variables

Message ID 20251114151255.74797-1-stondo@gmail.com
State Accepted
Headers show
Series [v2] ref-manual: Document SPDX 3.0.1 variables | expand

Commit Message

Stefano Tondo Nov. 14, 2025, 3:12 p.m. UTC
From: Stefano Tondo <stefano.tondo.ext@siemens.com>

Add comprehensive documentation for SPDX-related variables in the
Yocto reference manual. This includes documenting previously
undocumented variables and updating existing documentation with
SPDX 3.0.1 specific information.

New variables documented:
- SPDX_LICENSES: Path to SPDX license identifier mapping file
- SPDX_MULTILIB_SSTATE_ARCHS: Architecture list for dependency collection
- SPDX_UUID_NAMESPACE: Namespace for UUID generation in SPDX documents

Updated existing variables with SPDX 3.0.1 data:
- SPDX_INCLUDE_SOURCES: Added concrete SPDX 3.0.1 size data
  (~260 MB without sources, ~2.5-2.6 GB with sources)
- SPDX_INCLUDE_COMPILED_SOURCES: Updated with SPDX 3.0.1 format information

The size information is based on building core-image-minimal for
qemux86-64 with SPDX 3.0.1 JSON-LD format. The uncompressed SBOM
file is approximately 260 MB without sources and increases to
~2.5-2.6 GB when sources are included (approximately 10x increase).

Note: SPDX 3.0.1 uses uncompressed JSON-LD format by default,
unlike the tar.zst compression used in SPDX 2.2.

These variables are defined in meta/classes/spdx-common.bbclass.

Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---

Changes in v2:
- Fixed unintended removal of non-SPDX variables (CCACHE_DISABLE,
  IMAGE_EXTRA_PARTITION_FILES, REQUIRED_*_FEATURES)
- Fixed unintended change to LAYERDEPENDS description
- Only SPDX-related variables are now modified

 documentation/ref-manual/variables.rst | 72 +++++++++++++++++++++++---
 1 file changed, 66 insertions(+), 6 deletions(-)

Comments

Antonin Godard Nov. 24, 2025, 3:48 p.m. UTC | #1
Hi,

On Fri Nov 14, 2025 at 4:12 PM CET, Stefano Tondo via lists.yoctoproject.org wrote:
> From: Stefano Tondo <stefano.tondo.ext@siemens.com>
>
> Add comprehensive documentation for SPDX-related variables in the
> Yocto reference manual. This includes documenting previously
> undocumented variables and updating existing documentation with
> SPDX 3.0.1 specific information.
>
> New variables documented:
> - SPDX_LICENSES: Path to SPDX license identifier mapping file
> - SPDX_MULTILIB_SSTATE_ARCHS: Architecture list for dependency collection
> - SPDX_UUID_NAMESPACE: Namespace for UUID generation in SPDX documents
>
> Updated existing variables with SPDX 3.0.1 data:
> - SPDX_INCLUDE_SOURCES: Added concrete SPDX 3.0.1 size data
>   (~260 MB without sources, ~2.5-2.6 GB with sources)
> - SPDX_INCLUDE_COMPILED_SOURCES: Updated with SPDX 3.0.1 format information
>
> The size information is based on building core-image-minimal for
> qemux86-64 with SPDX 3.0.1 JSON-LD format. The uncompressed SBOM
> file is approximately 260 MB without sources and increases to
> ~2.5-2.6 GB when sources are included (approximately 10x increase).
>
> Note: SPDX 3.0.1 uses uncompressed JSON-LD format by default,
> unlike the tar.zst compression used in SPDX 2.2.
>
> These variables are defined in meta/classes/spdx-common.bbclass.
>
> Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
> ---
>
> Changes in v2:
> - Fixed unintended removal of non-SPDX variables (CCACHE_DISABLE,
>   IMAGE_EXTRA_PARTITION_FILES, REQUIRED_*_FEATURES)
> - Fixed unintended change to LAYERDEPENDS description
> - Only SPDX-related variables are now modified
>
>  documentation/ref-manual/variables.rst | 72 +++++++++++++++++++++++---
>  1 file changed, 66 insertions(+), 6 deletions(-)
>
> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index a80ef364e..c098227f2 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -8962,10 +8962,21 @@ system and gives an overview of their function and contents.
>  
>           SPDX_INCLUDE_COMPILED_SOURCES = "1"
>  
> -      According to our tests, building ``core-image-minimal`` for the
> -      ``qemux86-64`` machine, enabling this option compared with the
> -      :term:`SPDX_INCLUDE_SOURCES` reduces the size of the  ``tmp/deploy/spdx``
> -      directory from 2GB to 1.6GB.
> +      According to our tests on release 4.1 "langdale" (SPDX 2.2 format), building

Did you really test this on langdale, or did you add langdale here because it
was mentioned in the other variable definitions?

> +      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this
> +      option compared with the :term:`SPDX_INCLUDE_SOURCES` reduced the size
> +      of the ``tmp/deploy/spdx`` directory from 2GB to 1.6GB.
> +
> +      With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file

I'm not sure it's worth mentioning JSON-LD here, as it can add confusion. I
would just go with JSON.

> +      (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
> +      **260 MB without sources** (``SPDX_INCLUDE_SOURCES = "0"``), and increases to
> +      **~2.5-2.6 GB when sources are included** (either ``SPDX_INCLUDE_SOURCES = "1"``
> +      or ``SPDX_INCLUDE_COMPILED_SOURCES = "1"``). This represents approximately

So SPDX_INCLUDE_COMPILED_SOURCES has the same impact on size compared to
SPDX_INCLUDE_SOURCES? I thought the point of that variable was to optimize
what's included in the file.

> +      a **10x size increase** when including source files.
> +
> +      Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
> +      tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
> +      reduce the file size.

We could use a ..note :: block here and in the other notes:

"""
.. note::

   SPDX 3.0.1 JSON files are not compressed by default, unlike the tar.zst
   format used in SPDX 2.2. Compression (e.g. zstd) can significantly reduce the
   file size.
"""

Also, how can one enable compression? If you know, it would be worth adding it
the note. Either the way to do it or a link to the variable that controls this.

>  
>     :term:`SPDX_INCLUDE_SOURCES`
>        This option allows to add a description of the source files used to build
> @@ -8979,8 +8990,8 @@ system and gives an overview of their function and contents.
>  
>           SPDX_INCLUDE_SOURCES = "1"
>  
> -      According to our tests on release 4.1 "langdale", building
> -      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> +      According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
> +      building ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
>        this option multiplied the total size of the ``tmp/deploy/spdx``
>        directory by a factor of 3  (+291 MiB for this image),
>        and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in
> @@ -8988,6 +8999,42 @@ system and gives an overview of their function and contents.
>        image), compared to just using the :ref:`ref-classes-create-spdx` class
>        with no option.
>  
> +      With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
> +      (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
> +      **260 MB without sources** (default: ``SPDX_INCLUDE_SOURCES = "0"``),
> +      and increases to **~2.5-2.6 GB when sources are included**
> +      (``SPDX_INCLUDE_SOURCES = "1"``). This represents approximately a
> +      **10x size increase** when including source files.
> +
> +      Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
> +      tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
> +      reduce the file size.
> +
> +   :term:`SPDX_LICENSES`
> +      Path to the JSON file containing SPDX license identifier mappings. This
> +      file maps common license names to official SPDX license identifiers used
> +      during SBOM generation.
> +
> +      The default value is::
> +
> +         SPDX_LICENSES = "${COREBASE}/meta/files/spdx-licenses.json"

I would avoid hardcoding a path in the docs, to decrease the maintenance
effort. Perhaps:

"""
The default value points to a copy of the license mappings defined by SPDX
(https://github.com/spdx/license-list-data) stored in :term:`OpenEmbedded-Core
(OE-Core)`.
"""

Same remark for SPDX_MULTILIB_SSTATE_ARCHS and SPDX_UUID_NAMESPACE.

> +
> +      You can override this variable to use a custom license mapping file if
> +      your organization uses different license naming conventions.
> +
> +   :term:`SPDX_MULTILIB_SSTATE_ARCHS`
> +      The list of sstate architectures to consider when collecting SPDX
> +      dependencies. This includes multilib architectures when multilib is
> +      enabled.
> +
> +      The default value is::
> +
> +         SPDX_MULTILIB_SSTATE_ARCHS = "${SSTATE_ARCHS}"
> +
> +      This variable is used internally by the SPDX generation classes to
> +      ensure all relevant dependencies are included in the SBOM, regardless
> +      of whether multilib is enabled or not.
> +
>     :term:`SPDX_NAMESPACE_PREFIX`
>        This option could be used in order to change the prefix of ``spdxDocument``
>        and the prefix of ``documentNamespace``. It is set by default to
> @@ -9015,6 +9062,19 @@ system and gives an overview of their function and contents.
>        this option is recommended if you want to inspect the SPDX
>        output files with a text editor.
>  
> +   :term:`SPDX_UUID_NAMESPACE`
> +      The namespace used for generating UUIDs in SPDX documents. This should
> +      be a domain name or unique identifier for your organization to ensure
> +      globally unique SPDX IDs across different builds and organizations.
> +
> +      The default value is::
> +
> +         SPDX_UUID_NAMESPACE = "sbom.openembedded.org"
> +
> +      You can override this variable to use your organization's domain::
> +
> +         SPDX_UUID_NAMESPACE = "sbom.example.com"
> +
>     :term:`SPDXLICENSEMAP`
>        Maps commonly used license names to their SPDX counterparts found in
>        ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP`


Thanks!
Antonin
diff mbox series

Patch

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index a80ef364e..c098227f2 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8962,10 +8962,21 @@  system and gives an overview of their function and contents.
 
          SPDX_INCLUDE_COMPILED_SOURCES = "1"
 
-      According to our tests, building ``core-image-minimal`` for the
-      ``qemux86-64`` machine, enabling this option compared with the
-      :term:`SPDX_INCLUDE_SOURCES` reduces the size of the  ``tmp/deploy/spdx``
-      directory from 2GB to 1.6GB.
+      According to our tests on release 4.1 "langdale" (SPDX 2.2 format), building
+      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this
+      option compared with the :term:`SPDX_INCLUDE_SOURCES` reduced the size
+      of the ``tmp/deploy/spdx`` directory from 2GB to 1.6GB.
+
+      With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
+      (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
+      **260 MB without sources** (``SPDX_INCLUDE_SOURCES = "0"``), and increases to
+      **~2.5-2.6 GB when sources are included** (either ``SPDX_INCLUDE_SOURCES = "1"``
+      or ``SPDX_INCLUDE_COMPILED_SOURCES = "1"``). This represents approximately
+      a **10x size increase** when including source files.
+
+      Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
+      tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
+      reduce the file size.
 
    :term:`SPDX_INCLUDE_SOURCES`
       This option allows to add a description of the source files used to build
@@ -8979,8 +8990,8 @@  system and gives an overview of their function and contents.
 
          SPDX_INCLUDE_SOURCES = "1"
 
-      According to our tests on release 4.1 "langdale", building
-      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
+      According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
+      building ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
       this option multiplied the total size of the ``tmp/deploy/spdx``
       directory by a factor of 3  (+291 MiB for this image),
       and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in
@@ -8988,6 +8999,42 @@  system and gives an overview of their function and contents.
       image), compared to just using the :ref:`ref-classes-create-spdx` class
       with no option.
 
+      With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
+      (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
+      **260 MB without sources** (default: ``SPDX_INCLUDE_SOURCES = "0"``),
+      and increases to **~2.5-2.6 GB when sources are included**
+      (``SPDX_INCLUDE_SOURCES = "1"``). This represents approximately a
+      **10x size increase** when including source files.
+
+      Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
+      tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
+      reduce the file size.
+
+   :term:`SPDX_LICENSES`
+      Path to the JSON file containing SPDX license identifier mappings. This
+      file maps common license names to official SPDX license identifiers used
+      during SBOM generation.
+
+      The default value is::
+
+         SPDX_LICENSES = "${COREBASE}/meta/files/spdx-licenses.json"
+
+      You can override this variable to use a custom license mapping file if
+      your organization uses different license naming conventions.
+
+   :term:`SPDX_MULTILIB_SSTATE_ARCHS`
+      The list of sstate architectures to consider when collecting SPDX
+      dependencies. This includes multilib architectures when multilib is
+      enabled.
+
+      The default value is::
+
+         SPDX_MULTILIB_SSTATE_ARCHS = "${SSTATE_ARCHS}"
+
+      This variable is used internally by the SPDX generation classes to
+      ensure all relevant dependencies are included in the SBOM, regardless
+      of whether multilib is enabled or not.
+
    :term:`SPDX_NAMESPACE_PREFIX`
       This option could be used in order to change the prefix of ``spdxDocument``
       and the prefix of ``documentNamespace``. It is set by default to
@@ -9015,6 +9062,19 @@  system and gives an overview of their function and contents.
       this option is recommended if you want to inspect the SPDX
       output files with a text editor.
 
+   :term:`SPDX_UUID_NAMESPACE`
+      The namespace used for generating UUIDs in SPDX documents. This should
+      be a domain name or unique identifier for your organization to ensure
+      globally unique SPDX IDs across different builds and organizations.
+
+      The default value is::
+
+         SPDX_UUID_NAMESPACE = "sbom.openembedded.org"
+
+      You can override this variable to use your organization's domain::
+
+         SPDX_UUID_NAMESPACE = "sbom.example.com"
+
    :term:`SPDXLICENSEMAP`
       Maps commonly used license names to their SPDX counterparts found in
       ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP`