[RFC] ref-manual: document create-spdx class and variables

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/classes.rst   | 11 ++++
 documentation/ref-manual/variables.rst | 71 ++++++++++++++++++++++++++
 2 files changed, 82 insertions(+)

On Tue, Oct 25, 2022 at 11:00 AM <michael.opdenacker@bootlin.com> wrote:
>
> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  documentation/ref-manual/classes.rst   | 11 ++++
>  documentation/ref-manual/variables.rst | 71 ++++++++++++++++++++++++++
>  2 files changed, 82 insertions(+)
>
> diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
> index 1880e44486..5274dd20bd 100644
> --- a/documentation/ref-manual/classes.rst
> +++ b/documentation/ref-manual/classes.rst
> @@ -376,6 +376,17 @@ support.
>  The :ref:`create-spdx <ref-classes-create-spdx>` class provides support for automatically creating
>  SPDX SBoM documents based upon image and SDK contents.
>
> +This class is meant to be inherited globally from a configuration file::
> +
> +   INHERIT += "create-spdx"
> +
> +The output files are generated in ``tmp/deploy/spdx`` under the
> +:term:`Build Directory`.

The "important" SPDX document that gets generated is actually the
image document that lives along side the image you built, e.g.

 tmp/deploy/images/MACHINE/core-image-minimal-MACHINE.spdx.tar.zst
(not sure the correct way to explain this path in the documentation)

This archive is the compendium of SPDX that went into that image;
notably it culls anything that didn't actually contribute to the
images (e.g. packages built, but not installed are omitted).

It would be good to point users there as the first stop instead of the
individual SPDX documents since it's a "complete" SBoM for the image.
The individual files are useful for some things, but probably not as
interesting for people who "just want an SBoM" :)

> +
> +The exact behaviour of this class can be controlled by the :term:`SPDX_PRETTY`,
> +:term:`SPDX_ARCHIVE_PACKAGED`, :term:`SPDX_ARCHIVE_SOURCES` and
> +:term:`SPDX_INCLUDE_SOURCES` variables.
> +
>  .. _ref-classes-cross:
>
>  ``cross.bbclass``
> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index 71e8c272a7..60d801932e 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -7278,6 +7278,77 @@ system and gives an overview of their function and contents.
>
>           You can specify only a single URL in :term:`SOURCE_MIRROR_URL`.
>
> +   :term:`SPDX_ARCHIVE_PACKAGED`
> +      This option allows to add to the SPDX output directory
> +      compressed archives of the files in the generated target packages.
> +
> +      Such archives are available in
> +      ``tmp/deploy/spdx/MACHINE/packages/packagename.tar.zst``
> +      under the :term:`Build Directory`.
> +
> +      Enable this option as follows::
> +
> +         SPDX_ARCHIVE_PACKAGED = "1"
> +
> +      According to our tests on release 4.1 "langdale", building
> +      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> +      this option multiplied the size of the ``tmp/deploy/spdx``
> +      directory by a factor of 13 (+1.6 GiB for this image),
> +      compared to just using the :ref:`create-spdx <ref-classes-create-spdx>`
> +      class with no option.
> +
> +   :term:`SPDX_ARCHIVE_SOURCES`
> +      This option allows to add to the SPDX output directory compressed archives
> +      of the sources for packages installed on the target. It currently
> +      only works when :term:`SPDX_INCLUDE_SOURCES` is set.
> +
> +      Such archives are available in
> +      ``tmp/deploy/spdx/MACHINE/recipes/recipe-packagename.tar.zst``
> +      under the :term:`Build Directory`.
> +
> +      Enable this option as follows::
> +
> +        SPDX_INCLUDE_SOURCES = "1"
> +         SPDX_ARCHIVE_SOURCES = "1"
> +
> +      According to our tests on release 4.1 "langdale", building
> +      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> +      these options multiplied the size of the ``tmp/deploy/spdx``
> +      directory by a factor of 11 (+1.4 GiB for this image),
> +      compared to just using the :ref:`create-spdx <ref-classes-create-spdx>`
> +      class with no option.
> +
> +   :term:`SPDX_INCLUDE_SOURCES`
> +      This option allows to add a description of the source
> +      files handled by the target recipes, to the ``spdx.json`` files in
> +      ``tmp/deploy/spdx/MACHINE/recipes/`` under the
> +      :term:`Build Directory`. As a consequence, the ``spdx.json`` files
> +      under the ``by-namespace`` and ``packages`` subdirectories
> +      in ``tmp/deploy/spdx/MACHINE`` are also modified to include
> +      references to such source file descriptions.
> +
> +      Enable this option as follows::
> +
> +         SPDX_INCLUDE_SOURCES = "1"
> +
> +      According to our tests on release 4.1 "langdale", building
> +      ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> +      this option multiplied the size of the ``tmp/deploy/spdx``
> +      directory by a factor of 3  (+291 MiB for this image),
> +      compared to just using the :ref:`create-spdx <ref-classes-create-spdx>`
> +      class with no option.
> +
> +   :term:`SPDX_PRETTY`
> +      This option makes the SPDX output more human-readable, using
> +      identation and newlines, instead of the default output in a
> +      single line::
> +
> +         SPDX_PRETTY = "1"
> +
> +      The generated SPDX files are approximately 20% bigger, but
> +      this option is recommended if you want to inspect the SPDX
> +      output files with a text editor.
> +
>     :term:`SPDXLICENSEMAP`
>        Maps commonly used license names to their SPDX counterparts found in
>        ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP`
> --
> 2.34.1
>

Hello Joshua,

On 10/25/22 21:04, Joshua Watt wrote:
> On Tue, Oct 25, 2022 at 11:00 AM <michael.opdenacker@bootlin.com> wrote:
>> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>>
>> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>> ---
>>   documentation/ref-manual/classes.rst   | 11 ++++
>>   documentation/ref-manual/variables.rst | 71 ++++++++++++++++++++++++++
>>   2 files changed, 82 insertions(+)
>>
>> diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
>> index 1880e44486..5274dd20bd 100644
>> --- a/documentation/ref-manual/classes.rst
>> +++ b/documentation/ref-manual/classes.rst
>> @@ -376,6 +376,17 @@ support.
>>   The :ref:`create-spdx <ref-classes-create-spdx>` class provides support for automatically creating
>>   SPDX SBoM documents based upon image and SDK contents.
>>
>> +This class is meant to be inherited globally from a configuration file::
>> +
>> +   INHERIT += "create-spdx"
>> +
>> +The output files are generated in ``tmp/deploy/spdx`` under the
>> +:term:`Build Directory`.
> The "important" SPDX document that gets generated is actually the
> image document that lives along side the image you built, e.g.
>
>   tmp/deploy/images/MACHINE/core-image-minimal-MACHINE.spdx.tar.zst
> (not sure the correct way to explain this path in the documentation)
>
> This archive is the compendium of SPDX that went into that image;
> notably it culls anything that didn't actually contribute to the
> images (e.g. packages built, but not installed are omitted).
>
> It would be good to point users there as the first stop instead of the
> individual SPDX documents since it's a "complete" SBoM for the image.
> The individual files are useful for some things, but probably not as
> interesting for people who "just want an SBoM" :)


Many thanks for these details! Everything makes more sense now. I was 
indeed distracted by tmp/deploy/spdx/.

Thanks again
Michael.