| Message ID | 20221028101939.93195-4-michael.opdenacker@bootlin.com |
|---|---|
| State | New |
| Headers | show |
| Series | manuals: add documentation about SBOM/SPDX generation | expand |
Hi, On Fri, Oct 28, 2022 at 12:19:37PM +0200, Michael Opdenacker via lists.yoctoproject.org wrote: > From: Michael Opdenacker <michael.opdenacker@bootlin.com> > > Also stop refering to the meta-spdxscanner class, no longer > relevant. > > Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> > --- > documentation/dev-manual/common-tasks.rst | 96 ++++++++++++++--------- > 1 file changed, 61 insertions(+), 35 deletions(-) > > diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst > index 25d7edbc29..263a1d302b 100644 > --- a/documentation/dev-manual/common-tasks.rst > +++ b/documentation/dev-manual/common-tasks.rst > @@ -11236,8 +11236,6 @@ to be covered by assuming that there are three main areas of concern: > - Compilation scripts and modifications to the source code must be > provided. > > -- spdx files can be provided. > - > There are other requirements beyond the scope of these three and the > methods described in this section (e.g. the mechanism through which > source code is distributed). > @@ -11429,39 +11427,6 @@ layers (recipes, configuration files, and so forth) enables you to meet > your requirements to include the scripts to control compilation as well > as any modifications to the original source. > > -Providing spdx files > -~~~~~~~~~~~~~~~~~~~~~~~~~ > - > -The spdx module has been integrated to a layer named meta-spdxscanner. > -meta-spdxscanner provides several kinds of scanner. If you want to enable > -this function, you have to follow the following steps: > - > -1. Add meta-spdxscanner layer into ``bblayers.conf``. > - > -2. Refer to the README in meta-spdxscanner to setup the environment (e.g, > - setup a fossology server) needed for the scanner. > - > -3. Meta-spdxscanner provides several methods within the bbclass to create spdx files. > - Please choose one that you want to use and enable the spdx task. You have to > - add some config options in ``local.conf`` file in your :term:`Build > - Directory`. Here is an example showing how to generate spdx files > - during BitBake using the fossology-python.bbclass:: > - > - # Select fossology-python.bbclass. > - INHERIT += "fossology-python" > - # For fossology-python.bbclass, TOKEN is necessary, so, after setup a > - # Fossology server, you have to create a token. > - TOKEN = "eyJ0eXAiO..." > - # The fossology server is necessary for fossology-python.bbclass. > - FOSSOLOGY_SERVER = "http://xx.xx.xx.xx:8081/repo" > - # If you want to upload the source code to a special folder: > - FOLDER_NAME = "xxxx" //Optional > - # If you don't want to put spdx files in tmp/deploy/spdx, you can enable: > - SPDX_DEPLOY_DIR = "${DEPLOY_DIR}" //Optional > - > -For more usage information refer to :yocto_git:`the meta-spdxscanner repository > -</meta-spdxscanner/>`. > - > Compliance Limitations with Executables Built from Static Libraries > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > @@ -11711,6 +11676,67 @@ When analyzing CVEs, it is recommended to: > - follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>`__ for > discussions and advance notifications of CVE bugs and software releases with fixes. > > +Creating a Software Bill of Materials > +===================================== > + > +Once you are able to build an image for your project, once the licenses for > +each software component are all identified (see > +":ref:`dev-manual/common-tasks:working with licenses`") and once vulnerability > +fixes are applied (see ":ref:`dev-manual/common-tasks:checking > +for vulnerabilities`"), the OpenEmbedded build system can generate > +a description of all the components you used, their licenses, their dependencies, > +the changes that were applied and the known vulnerabilities that were fixed. > + > +This description is generated in the form of a *Software Bill of Materials* > +(:term:`SBOM`), using the :term:`SPDX` standard. > + > +When you release software, this is the most standard way to provide information > +about the Software Supply Chain of your software image and SDK, and allow your > +users and customers to perform vulnerability and license compliance assessments, > +using the tools of their choice. I'd mention also something like this: "The :term:`SBOM` tooling is often used to ensure open source license compliance by providing the license texts used in the product which legal departments and end users can read in standardized format." Then I'd somehow link this to the features which create source code tar balls, since those are needed to fulfill "source code access" license requirements. Cheers, -Mikko
Hi Mikko Many thanks for the review! On 10/28/22 13:05, Mikko Rapeli wrote: > Hi, > > On Fri, Oct 28, 2022 at 12:19:37PM +0200, Michael Opdenacker via lists.yoctoproject.org wrote: >> From: Michael Opdenacker <michael.opdenacker@bootlin.com> >> >> Also stop refering to the meta-spdxscanner class, no longer >> relevant. >> >> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> >> --- >> documentation/dev-manual/common-tasks.rst | 96 ++++++++++++++--------- >> 1 file changed, 61 insertions(+), 35 deletions(-) >> >> diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst >> index 25d7edbc29..263a1d302b 100644 >> --- a/documentation/dev-manual/common-tasks.rst >> +++ b/documentation/dev-manual/common-tasks.rst >> @@ -11236,8 +11236,6 @@ to be covered by assuming that there are three main areas of concern: >> - Compilation scripts and modifications to the source code must be >> provided. >> >> -- spdx files can be provided. >> - >> There are other requirements beyond the scope of these three and the >> methods described in this section (e.g. the mechanism through which >> source code is distributed). >> @@ -11429,39 +11427,6 @@ layers (recipes, configuration files, and so forth) enables you to meet >> your requirements to include the scripts to control compilation as well >> as any modifications to the original source. >> >> -Providing spdx files >> -~~~~~~~~~~~~~~~~~~~~~~~~~ >> - >> -The spdx module has been integrated to a layer named meta-spdxscanner. >> -meta-spdxscanner provides several kinds of scanner. If you want to enable >> -this function, you have to follow the following steps: >> - >> -1. Add meta-spdxscanner layer into ``bblayers.conf``. >> - >> -2. Refer to the README in meta-spdxscanner to setup the environment (e.g, >> - setup a fossology server) needed for the scanner. >> - >> -3. Meta-spdxscanner provides several methods within the bbclass to create spdx files. >> - Please choose one that you want to use and enable the spdx task. You have to >> - add some config options in ``local.conf`` file in your :term:`Build >> - Directory`. Here is an example showing how to generate spdx files >> - during BitBake using the fossology-python.bbclass:: >> - >> - # Select fossology-python.bbclass. >> - INHERIT += "fossology-python" >> - # For fossology-python.bbclass, TOKEN is necessary, so, after setup a >> - # Fossology server, you have to create a token. >> - TOKEN = "eyJ0eXAiO..." >> - # The fossology server is necessary for fossology-python.bbclass. >> - FOSSOLOGY_SERVER = "http://xx.xx.xx.xx:8081/repo" >> - # If you want to upload the source code to a special folder: >> - FOLDER_NAME = "xxxx" //Optional >> - # If you don't want to put spdx files in tmp/deploy/spdx, you can enable: >> - SPDX_DEPLOY_DIR = "${DEPLOY_DIR}" //Optional >> - >> -For more usage information refer to :yocto_git:`the meta-spdxscanner repository >> -</meta-spdxscanner/>`. >> - >> Compliance Limitations with Executables Built from Static Libraries >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> @@ -11711,6 +11676,67 @@ When analyzing CVEs, it is recommended to: >> - follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>`__ for >> discussions and advance notifications of CVE bugs and software releases with fixes. >> >> +Creating a Software Bill of Materials >> +===================================== >> + >> +Once you are able to build an image for your project, once the licenses for >> +each software component are all identified (see >> +":ref:`dev-manual/common-tasks:working with licenses`") and once vulnerability >> +fixes are applied (see ":ref:`dev-manual/common-tasks:checking >> +for vulnerabilities`"), the OpenEmbedded build system can generate >> +a description of all the components you used, their licenses, their dependencies, >> +the changes that were applied and the known vulnerabilities that were fixed. >> + >> +This description is generated in the form of a *Software Bill of Materials* >> +(:term:`SBOM`), using the :term:`SPDX` standard. >> + >> +When you release software, this is the most standard way to provide information >> +about the Software Supply Chain of your software image and SDK, and allow your >> +users and customers to perform vulnerability and license compliance assessments, >> +using the tools of their choice. > I'd mention also something like this: > > "The :term:`SBOM` tooling is often used to ensure open source license compliance > by providing the license texts used in the product which legal departments and end > users can read in standardized format." I like this wording. See how I took it into account in my next iteration. > > Then I'd somehow link this to the features which create source code tar > balls, since those are needed to fulfill "source code access" license > requirements. Done, in the common-tasks.rst file and in variables.rst in the description of the SPDX_ARCHIVE_SOURCES variable. See my V2 coming soon. Thanks again Michael.
diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst index 25d7edbc29..263a1d302b 100644 --- a/documentation/dev-manual/common-tasks.rst +++ b/documentation/dev-manual/common-tasks.rst @@ -11236,8 +11236,6 @@ to be covered by assuming that there are three main areas of concern: - Compilation scripts and modifications to the source code must be provided. -- spdx files can be provided. - There are other requirements beyond the scope of these three and the methods described in this section (e.g. the mechanism through which source code is distributed). @@ -11429,39 +11427,6 @@ layers (recipes, configuration files, and so forth) enables you to meet your requirements to include the scripts to control compilation as well as any modifications to the original source. -Providing spdx files -~~~~~~~~~~~~~~~~~~~~~~~~~ - -The spdx module has been integrated to a layer named meta-spdxscanner. -meta-spdxscanner provides several kinds of scanner. If you want to enable -this function, you have to follow the following steps: - -1. Add meta-spdxscanner layer into ``bblayers.conf``. - -2. Refer to the README in meta-spdxscanner to setup the environment (e.g, - setup a fossology server) needed for the scanner. - -3. Meta-spdxscanner provides several methods within the bbclass to create spdx files. - Please choose one that you want to use and enable the spdx task. You have to - add some config options in ``local.conf`` file in your :term:`Build - Directory`. Here is an example showing how to generate spdx files - during BitBake using the fossology-python.bbclass:: - - # Select fossology-python.bbclass. - INHERIT += "fossology-python" - # For fossology-python.bbclass, TOKEN is necessary, so, after setup a - # Fossology server, you have to create a token. - TOKEN = "eyJ0eXAiO..." - # The fossology server is necessary for fossology-python.bbclass. - FOSSOLOGY_SERVER = "http://xx.xx.xx.xx:8081/repo" - # If you want to upload the source code to a special folder: - FOLDER_NAME = "xxxx" //Optional - # If you don't want to put spdx files in tmp/deploy/spdx, you can enable: - SPDX_DEPLOY_DIR = "${DEPLOY_DIR}" //Optional - -For more usage information refer to :yocto_git:`the meta-spdxscanner repository -</meta-spdxscanner/>`. - Compliance Limitations with Executables Built from Static Libraries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -11711,6 +11676,67 @@ When analyzing CVEs, it is recommended to: - follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>`__ for discussions and advance notifications of CVE bugs and software releases with fixes. +Creating a Software Bill of Materials +===================================== + +Once you are able to build an image for your project, once the licenses for +each software component are all identified (see +":ref:`dev-manual/common-tasks:working with licenses`") and once vulnerability +fixes are applied (see ":ref:`dev-manual/common-tasks:checking +for vulnerabilities`"), the OpenEmbedded build system can generate +a description of all the components you used, their licenses, their dependencies, +the changes that were applied and the known vulnerabilities that were fixed. + +This description is generated in the form of a *Software Bill of Materials* +(:term:`SBOM`), using the :term:`SPDX` standard. + +When you release software, this is the most standard way to provide information +about the Software Supply Chain of your software image and SDK, and allow your +users and customers to perform vulnerability and license compliance assessments, +using the tools of their choice. + +The OpenEmbedded build system doesn't generate such information by default. +To make this happen, you must inherit the +:ref:`create-spdx <ref-classes-create-spdx>` class from a configuration file:: + + INHERIT += "create-spdx" + +You then get :term:`SPDX` output in JSON format as an +``IMAGE-MACHINE.spdx.json`` file in ``tmp/deploy/images/MACHINE/`` inside the +:term:`Build Directory`. + +This is a toplevel file accompanied by an ``IMAGE-MACHINE.spdx.index.json`` +containing an index of JSON :term:`SPDX` files for individual recipes, together +with an ``IMAGE-MACHINE.spdx.tar.zst`` compressed archive containing all such +files. + +The :ref:`create-spdx <ref-classes-create-spdx>` class offers options to include +more information in the output :term:`SPDX` data, such as making the generated +files more human readable (:term:`SPDX_PRETTY`), adding compressed archives of +the files in the generated target packages (:term:`SPDX_ARCHIVE_PACKAGED`), +adding a description of the source files handled by the target recipes +(:term:`SPDX_INCLUDE_SOURCES`) and adding archives of these source files +themselves (:term:`SPDX_ARCHIVE_SOURCES`). + +Though the toplevel :term:`SPDX` output is available in +``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`, ancillary +generated files are available in ``tmp/deploy/spdx/MACHINE`` too, such as: + +- The individual :term:`SPDX` JSON files in the ``IMAGE-MACHINE.spdx.tar.zst`` + archive. + +- Compressed archives of the files in the generated target packages, + in ``packages/packagename.tar.zst`` (when :term:`SPDX_ARCHIVE_PACKAGED` + is set). + +- Compressed archives of the source files used to build the host tools + and the target packages in ``recipes/recipe-packagename.tar.zst`` + (when :term:`SPDX_ARCHIVE_SOURCES` is set). + +See the `tools page <https://spdx.dev/resources/tools/>`__ on the :term:`SPDX` +project website for a list of tools to consume and transform the :term:`SPDX` +data generated by the OpenEmbedded build system. + Using the Error Reporting Tool ==============================