| Message ID | 20250716090517.481832-2-kamel.bouhara@bootlin.com |
|---|---|
| State | New |
| Headers | show |
| Series | spdx3: Export kernel configuration as build parameters in SPDX output | expand |
Hi, On Wed, Jul 16, 2025 at 11:05:17AM +0200, Kamel Bouhara via lists.openembedded.org wrote: > Enhances SPDX Document by extracting kernel build-time configuration settings from '${B}/.config'. > > Each CONFIG_* line is parsed and exported as a DictionaryEntry in the build_Build.build_parameter > section of the SPDX document. This provides better visibility into kernel build behavior and > configuration, in alignment with the SPDX3 metadata model. > > The feature is gated by a new tunable variable: > > SPDX_INCLUDE_KERNEL_CONFIG (default: "1") > > Setting this to "0" disables exporting the kernel configuration, which may be useful to improve > performance or reduce the size of generated SPDX documents. > > Example: > > CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } > > This complements existing metadata export features and enables a more complete audit trail of how > the kernel is built within a given build. Why is the kernel so special? All other SW components have build time configs too. For information harvesting, a lot of data can be extracted from the build system but to me it's important that the build system and tools benefit the users who actually do maintenance and development work. They need to be able to see what patches get applied, what they fix, what configs are used etc. Extracting all possible info into some IT management tooling which never directly feeds back to the build system or developers to actually improve the CVE patch status, enable security features and updates and fixes for real bugs, is not very useful. If some information is in the build system, then developers can read it from there also when doing reviews and audits. For kernel, the -dev binary package has the effective config after build has completed. Cheers, -Mikko > Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com> > --- > meta/classes/create-spdx-3.0.bbclass | 6 ++++++ > meta/lib/oe/spdx30_tasks.py | 32 ++++++++++++++++++++++++++++ > 2 files changed, 38 insertions(+) > > diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass > index c0a5436ad6..cdb9422f37 100644 > --- a/meta/classes/create-spdx-3.0.bbclass > +++ b/meta/classes/create-spdx-3.0.bbclass > @@ -50,6 +50,12 @@ SPDX_INCLUDE_TIMESTAMPS[doc] = "Include time stamps in SPDX output. This is \ > useful if you want to know when artifacts were produced and when builds \ > occurred, but will result in non-reproducible SPDX output" > > +SPDX_INCLUDE_KERNEL_CONFIG ??= "1" > +SPDX_INCLUDE_KERNEL_CONFIG[doc] = "If set to '1', the .config file for the kernel will be parsed \ > +and each CONFIG_* value will be included in the Build.build_parameter list as DictionaryEntry \ > +items. Set to '0' to disable exporting kernel configuration to improve performance or reduce \ > +SPDX document size." > + > SPDX_IMPORTS ??= "" > SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \ > reference external SPDX ids. Each import is defined as a key in this \ > diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py > index c352dab152..f87d079cb0 100644 > --- a/meta/lib/oe/spdx30_tasks.py > +++ b/meta/lib/oe/spdx30_tasks.py > @@ -18,6 +18,28 @@ from contextlib import contextmanager > from datetime import datetime, timezone > from pathlib import Path > > +def parse_kernel_config(config_path): > + entries = [] > + if not os.path.exists(config_path): > + bb.warn(f"Kernel config file not found at: {config_path}") > + return entries > + > + try: > + with open(config_path, 'r') as f: > + for line in f: > + line = line.strip() > + if not line or line.startswith("#"): > + continue > + if "=" in line: > + key, value = line.split("=", 1) > + entries.append(oe.spdx30.DictionaryEntry( > + key=key, > + value=value.strip('"') > + )) > + bb.note(f"Parsed {len(entries)} kernel config entries from {config_path}") > + except Exception as e: > + bb.error(f"Failed to parse kernel config file: {e}") > + return entries > > def walk_error(err): > bb.error(f"ERROR walking {err.filename}: {err}") > @@ -495,6 +517,8 @@ def create_spdx(d): > > build_objset.doc.rootElement.append(build) > > + build.build_parameter = [] > + > build_objset.set_is_native(is_native) > > for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): > @@ -815,6 +839,14 @@ def create_spdx(d): > sorted(list(build_inputs)) + sorted(list(debug_source_ids)), > ) > > + if d.getVar("SPDX_INCLUDE_KERNEL_CONFIG", True) != "0": > + if "virtual/kernel" in (d.getVar("PROVIDES") or "").split(): > + bb.note("Detected virtual/kernel provider, extracting kernel configuration") > + config_path = d.expand("${B}/.config") > + kernel_params = parse_kernel_config(config_path) > + if kernel_params: > + build.build_parameter.extend(kernel_params) > + > oe.sbom30.write_recipe_jsonld_doc(d, build_objset, "recipes", deploydir) > > > -- > 2.43.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#220439): https://lists.openembedded.org/g/openembedded-core/message/220439 > Mute This Topic: https://lists.openembedded.org/mt/114181881/7159507 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [mikko.rapeli@linaro.org] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, Jul 16, 2025 at 12:28:06PM +0300, Mikko Rapeli via lists.openembedded.org wrote: > Hi, > Hi Mikko, > On Wed, Jul 16, 2025 at 11:05:17AM +0200, Kamel Bouhara via lists.openembedded.org wrote: > > Enhances SPDX Document by extracting kernel build-time configuration settings from '${B}/.config'. > > > > Each CONFIG_* line is parsed and exported as a DictionaryEntry in the build_Build.build_parameter > > section of the SPDX document. This provides better visibility into kernel build behavior and > > configuration, in alignment with the SPDX3 metadata model. > > > > The feature is gated by a new tunable variable: > > > > SPDX_INCLUDE_KERNEL_CONFIG (default: "1") > > > > Setting this to "0" disables exporting the kernel configuration, which may be useful to improve > > performance or reduce the size of generated SPDX documents. > > > > Example: > > > > CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } > > > > This complements existing metadata export features and enables a more complete audit trail of how > > the kernel is built within a given build. > > Why is the kernel so special? All other SW components have build time configs too. > > For information harvesting, a lot of data can be extracted from the build system > but to me it's important that the build system and tools benefit the users > who actually do maintenance and development work. They need to be able to see > what patches get applied, what they fix, what configs are used etc. Extracting > all possible info into some IT management tooling which never directly feeds > back to the build system or developers to actually improve the CVE patch status, > enable security features and updates and fixes for real bugs, is not very useful. > You're absolutely right to question the special handling of the kernel. In this case, the choice was intentional: the kernel’s .config is well-structured, easy to locate, and critical to the system's security and behavior. It's also a central component of the BSP, making it a strong candidate for initial integration when experimenting with build-time metadata capture in SPDX. That said, I fully agree, many other software components also have meaningful build-time configuration (e.g., PACKAGECONFIG, EXTRA_OECONF, cmake flags), and the longer-term goal is to extend support to those as well. Starting with the kernel gives us a good foundation to validate the approach. Importantly, the patch makes this metadata inclusion optional and configurable. It introduces a variable to let users control the granularity of what gets included in the SPDX document. That means teams can choose to include or exclude details like kernel configuration based on their policy, use case, or audit needs. About usefulness to developers, I agree this info only brings value if it integrates back into yocto developers workflows. That’s exactly the direction I’m aiming for, for example, we’ve added support in our SBOM diff tool to compare kernel configuration between builds by parsing CONFIG_* entries. This mostly lets teams audit config changes over time and detect if a some security-hardening option was unintentionally disabled. > If some information is in the build system, then developers can read it from there > also when doing reviews and audits. For kernel, the -dev binary package has the > effective config after build has completed. > I believe, SPDX provides a more structured, standardized, and portable format that makes it easy to expose and reuse this data in CI pipelines, and when publishing SBOMs externally. It becomes a central source that avoids custom parsing or packaging assumptions and integrates well with tooling for compliance, vulnerability tracking and auditing. Thanks again for the feedback. -- Kamel Bouhara, Bootlin Embedded Linux and kernel engineering https://bootlin.com
On Wed, Jul 16, 2025 at 7:34 AM Kamel Bouhara via lists.openembedded.org <kamel.bouhara=bootlin.com@lists.openembedded.org> wrote: > On Wed, Jul 16, 2025 at 12:28:06PM +0300, Mikko Rapeli via > lists.openembedded.org wrote: > > Hi, > > > > Hi Mikko, > > > On Wed, Jul 16, 2025 at 11:05:17AM +0200, Kamel Bouhara via > lists.openembedded.org wrote: > > > Enhances SPDX Document by extracting kernel build-time configuration > settings from '${B}/.config'. > > > > > > Each CONFIG_* line is parsed and exported as a DictionaryEntry in the > build_Build.build_parameter > > > section of the SPDX document. This provides better visibility into > kernel build behavior and > > > configuration, in alignment with the SPDX3 metadata model. > > > > > > The feature is gated by a new tunable variable: > > > > > > SPDX_INCLUDE_KERNEL_CONFIG (default: "1") > > > > > > Setting this to "0" disables exporting the kernel configuration, which > may be useful to improve > > > performance or reduce the size of generated SPDX documents. > > > > > > Example: > > > > > > CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } > > > > > > This complements existing metadata export features and enables a more > complete audit trail of how > > > the kernel is built within a given build. > > > > Why is the kernel so special? All other SW components have build time > configs too. > > > > For information harvesting, a lot of data can be extracted from the > build system > > but to me it's important that the build system and tools benefit the > users > > who actually do maintenance and development work. They need to be able > to see > > what patches get applied, what they fix, what configs are used etc. > Extracting > > all possible info into some IT management tooling which never directly > feeds > > back to the build system or developers to actually improve the CVE patch > status, > > enable security features and updates and fixes for real bugs, is not > very useful. > > > > You're absolutely right to question the special handling of the kernel. > In this case, the choice was intentional: the kernel’s .config is > well-structured, > easy to locate, and critical to the system's security and behavior. > It's also a central component of the BSP, making it a strong candidate for > initial > integration when experimenting with build-time metadata capture in SPDX. > > That said, I fully agree, many other software components also have > meaningful > build-time configuration (e.g., PACKAGECONFIG, EXTRA_OECONF, cmake flags), > and the > longer-term goal is to extend support to those as well. > > Starting with the kernel gives us a good foundation to validate the > approach. > > Importantly, the patch makes this metadata inclusion optional and > configurable. > It introduces a variable to let users control the granularity of what gets > included in the SPDX document. That means teams can choose to include or > exclude > details like kernel configuration based on their policy, use case, or > audit needs. > > About usefulness to developers, I agree this info only brings value if it > integrates back into yocto developers workflows. That’s exactly the > direction > I’m aiming for, for example, we’ve added support in our SBOM diff tool to > compare > kernel configuration between builds by parsing CONFIG_* entries. > This mostly lets teams audit config changes over time and detect if a > some security-hardening option was unintentionally disabled. > I have the same concerns and opinion on this as Mikko As long as this doesn't add overhead, or becomes mandatory, my objections can be ignored. But generally speaking, I'm opposed to jamming everything into SPDX and using it as a tool for what you are describing above. There are already existing tools that are focused and better at comparing configurations of packages (and the kernel). Turning SPDX into some kind of one-stop-shop and everything-in-one place tool is something I'd object strongly to. (I do realize you are talking about using some sort of other tool reading the SPDX data, but my objection is just the same if it becomes an absolutely massive monolithic data source). Bruce > > > If some information is in the build system, then developers can read it > from there > > also when doing reviews and audits. For kernel, the -dev binary package > has the > > effective config after build has completed. > > > > I believe, SPDX provides a more structured, standardized, and portable > format that makes > it easy to expose and reuse this data in CI pipelines, and when publishing > SBOMs > externally. It becomes a central source that avoids custom parsing or > packaging > assumptions and integrates well with tooling for compliance, vulnerability > tracking > and auditing. > > Thanks again for the feedback. > > -- > Kamel Bouhara, Bootlin > Embedded Linux and kernel engineering > https://bootlin.com > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#220447): > https://lists.openembedded.org/g/openembedded-core/message/220447 > Mute This Topic: https://lists.openembedded.org/mt/114181881/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Wed, Jul 16, 2025 at 09:30:11AM -0400, Bruce Ashfield via lists.openembedded.org wrote: > On Wed, Jul 16, 2025 at 7:34 AM Kamel Bouhara via > [1]lists.openembedded.org > <kamel.bouhara=[2]bootlin.com@lists.openembedded.org> wrote: > > On Wed, Jul 16, 2025 at 12:28:06PM +0300, Mikko Rapeli via > [3]lists.openembedded.org wrote: > > Hi, > > > Hi Mikko, > > On Wed, Jul 16, 2025 at 11:05:17AM +0200, Kamel Bouhara via > [4]lists.openembedded.org wrote: > > > Enhances SPDX Document by extracting kernel build-time > configuration settings from '${B}/.config'. > > > > > > Each CONFIG_* line is parsed and exported as a DictionaryEntry > in the build_Build.build_parameter > > > section of the SPDX document. This provides better visibility > into kernel build behavior and > > > configuration, in alignment with the SPDX3 metadata model. > > > > > > The feature is gated by a new tunable variable: > > > > > > SPDX_INCLUDE_KERNEL_CONFIG (default: "1") > > > > > > Setting this to "0" disables exporting the kernel configuration, > which may be useful to improve > > > performance or reduce the size of generated SPDX documents. > > > > > > Example: > > > > > > CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } > > > > > > This complements existing metadata export features and enables a > more complete audit trail of how > > > the kernel is built within a given build. > > > > Why is the kernel so special? All other SW components have build > time configs too. > > > > For information harvesting, a lot of data can be extracted from > the build system > > but to me it's important that the build system and tools benefit > the users > > who actually do maintenance and development work. They need to be > able to see > > what patches get applied, what they fix, what configs are used > etc. Extracting > > all possible info into some IT management tooling which never > directly feeds > > back to the build system or developers to actually improve the CVE > patch status, > > enable security features and updates and fixes for real bugs, is > not very useful. > > > You're absolutely right to question the special handling of the > kernel. > In this case, the choice was intentional: the kernel’s .config is > well-structured, > easy to locate, and critical to the system's security and behavior. > It's also a central component of the BSP, making it a strong > candidate for initial > integration when experimenting with build-time metadata capture in > SPDX. > That said, I fully agree, many other software components also have > meaningful > build-time configuration (e.g., PACKAGECONFIG, EXTRA_OECONF, cmake > flags), and the > longer-term goal is to extend support to those as well. > Starting with the kernel gives us a good foundation to validate the > approach. > Importantly, the patch makes this metadata inclusion optional and > configurable. > It introduces a variable to let users control the granularity of > what gets > included in the SPDX document. That means teams can choose to > include or exclude > details like kernel configuration based on their policy, use case, > or audit needs. > About usefulness to developers, I agree this info only brings value > if it > integrates back into yocto developers workflows. That’s exactly the > direction > I’m aiming for, for example, we’ve added support in our SBOM diff > tool to compare > kernel configuration between builds by parsing CONFIG_* entries. > This mostly lets teams audit config changes over time and detect if > a > some security-hardening option was unintentionally disabled. > > I have the same concerns and opinion on this as Mikko > As long as this doesn't add overhead, or becomes mandatory, my > objections can > be ignored. > But generally speaking, I'm opposed to jamming everything into SPDX and > using > it as a tool for what you are describing above. There are already > existing tools that > are focused and better at comparing configurations of packages (and the > kernel). > Turning SPDX into some kind of one-stop-shop and everything-in-one > place tool > is something I'd object strongly to. (I do realize you are talking > about using some > sort of other tool reading the SPDX data, but my objection is just the > same if it > becomes an absolutely massive monolithic data source). > Bruce > Hi Bruce, Thanks for the feedback. Just to clarify, the goal here isn't to turn SPDX into a one-stop-shop or to overload it with every piece of build metadata. The intent is to capture a minimal and focused subset of build-time parameters (like kernel .config, PACKAGECONFIG, etc.) where: - The structure is already well-defined - The data is relevant for security or compliance - And the cost of extraction is very low This is fully gated behind an opt-in variable (and could default to disabled). The idea is not to replace existing tools but rather to provide a structured, portable format for key metadata, especially in SBOM-driven environments where SPDX is already part of the process. In my use case, this metadata is consumed by a targeted diff tool; not treating SPDX as the end-goal, but more like a transport format. This approach lets us reuse existing SPDX outputs without interfering with or duplicating the role of more purpose-built tools. So I hear you: this shouldn't become bloated, mandatory. The goal is simply to make sure when people do need to trace how something was built, especially in CI or audit contexts, they have a consistent hook, and it can be disabled entirely if not useful. Cheers, -- Kamel Bouhara, Bootlin Embedded Linux and kernel engineering https://bootlin.com
On Wed, Jul 16, 2025 at 10:30 AM Kamel Bouhara <kamel.bouhara@bootlin.com> wrote: > On Wed, Jul 16, 2025 at 09:30:11AM -0400, Bruce Ashfield via > lists.openembedded.org wrote: > > On Wed, Jul 16, 2025 at 7:34 AM Kamel Bouhara via > > [1]lists.openembedded.org > > <kamel.bouhara=[2]bootlin.com@lists.openembedded.org> wrote: > > > > On Wed, Jul 16, 2025 at 12:28:06PM +0300, Mikko Rapeli via > > [3]lists.openembedded.org wrote: > > > Hi, > > > > > Hi Mikko, > > > On Wed, Jul 16, 2025 at 11:05:17AM +0200, Kamel Bouhara via > > [4]lists.openembedded.org wrote: > > > > Enhances SPDX Document by extracting kernel build-time > > configuration settings from '${B}/.config'. > > > > > > > > Each CONFIG_* line is parsed and exported as a DictionaryEntry > > in the build_Build.build_parameter > > > > section of the SPDX document. This provides better visibility > > into kernel build behavior and > > > > configuration, in alignment with the SPDX3 metadata model. > > > > > > > > The feature is gated by a new tunable variable: > > > > > > > > SPDX_INCLUDE_KERNEL_CONFIG (default: "1") > > > > > > > > Setting this to "0" disables exporting the kernel configuration, > > which may be useful to improve > > > > performance or reduce the size of generated SPDX documents. > > > > > > > > Example: > > > > > > > > CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } > > > > > > > > This complements existing metadata export features and enables a > > more complete audit trail of how > > > > the kernel is built within a given build. > > > > > > Why is the kernel so special? All other SW components have build > > time configs too. > > > > > > For information harvesting, a lot of data can be extracted from > > the build system > > > but to me it's important that the build system and tools benefit > > the users > > > who actually do maintenance and development work. They need to be > > able to see > > > what patches get applied, what they fix, what configs are used > > etc. Extracting > > > all possible info into some IT management tooling which never > > directly feeds > > > back to the build system or developers to actually improve the CVE > > patch status, > > > enable security features and updates and fixes for real bugs, is > > not very useful. > > > > > You're absolutely right to question the special handling of the > > kernel. > > In this case, the choice was intentional: the kernel’s .config is > > well-structured, > > easy to locate, and critical to the system's security and behavior. > > It's also a central component of the BSP, making it a strong > > candidate for initial > > integration when experimenting with build-time metadata capture in > > SPDX. > > That said, I fully agree, many other software components also have > > meaningful > > build-time configuration (e.g., PACKAGECONFIG, EXTRA_OECONF, cmake > > flags), and the > > longer-term goal is to extend support to those as well. > > Starting with the kernel gives us a good foundation to validate the > > approach. > > Importantly, the patch makes this metadata inclusion optional and > > configurable. > > It introduces a variable to let users control the granularity of > > what gets > > included in the SPDX document. That means teams can choose to > > include or exclude > > details like kernel configuration based on their policy, use case, > > or audit needs. > > About usefulness to developers, I agree this info only brings value > > if it > > integrates back into yocto developers workflows. That’s exactly the > > direction > > I’m aiming for, for example, we’ve added support in our SBOM diff > > tool to compare > > kernel configuration between builds by parsing CONFIG_* entries. > > This mostly lets teams audit config changes over time and detect if > > a > > some security-hardening option was unintentionally disabled. > > > > > I have the same concerns and opinion on this as Mikko > > As long as this doesn't add overhead, or becomes mandatory, my > > objections can > > be ignored. > > But generally speaking, I'm opposed to jamming everything into SPDX > and > > using > > it as a tool for what you are describing above. There are already > > existing tools that > > are focused and better at comparing configurations of packages (and > the > > kernel). > > Turning SPDX into some kind of one-stop-shop and everything-in-one > > place tool > > is something I'd object strongly to. (I do realize you are talking > > about using some > > sort of other tool reading the SPDX data, but my objection is just the > > same if it > > becomes an absolutely massive monolithic data source). > > Bruce > > > > Hi Bruce, > > Thanks for the feedback. > > Just to clarify, the goal here isn't to turn SPDX into a one-stop-shop or > to > overload it with every piece of build metadata. The intent is to capture a > minimal > and focused subset of build-time parameters > I wouldn't call capturing the configuration like this patch is doing "minimal", so we'll agree to disagree on that point. > (like kernel .config, PACKAGECONFIG, etc.) where: > > - The structure is already well-defined > - The data is relevant for security or compliance > - And the cost of extraction is very low > And arguably not the best or most efficient format ... > This is fully gated behind an opt-in variable (and could default to > disabled). > The idea is not to replace existing tools but rather to provide a > structured, > portable format for key metadata, especially in SBOM-driven environments > where SPDX > is already part of the process. It must default to disabled in my opinion. But I can already hear the arguments that will say "it won't be tested if it isn't on by default, etc" ... and that is where we will have the conflict about whether it is on, or off, by default. > > > In my use case, this metadata is consumed by a targeted diff tool; not > treating > SPDX as the end-goal, but more like a transport format. This approach lets > us > reuse existing SPDX outputs without interfering with or duplicating the > role of > more purpose-built tools. > > So I hear you: this shouldn't become bloated, mandatory. The goal is > simply to make > sure when people do need to trace how something was built, especially in > CI or > audit contexts, they have a consistent hook, and it can be disabled > entirely if > not useful. > It is something that I can ignore (which is good), so I'll just keep watching to ensure that it doesn't become mandatory, or fail configurations on compliance and I'll be happy. I feel that we are moving more and more towards a "pipeline" or "loadbuild" use case for OE, and that's also what I guard against. I'm already not happy how long SPDX adds to developer builds, so at some point I need to revisit disabling it completely (it used to fail when I first tried that). Cheers, Bruce > > Cheers, > > -- > Kamel Bouhara, Bootlin > Embedded Linux and kernel engineering > https://bootlin.com >
On Wed, Jul 16, 2025 at 10:51:09AM -0400, Bruce Ashfield wrote: [...] > > I wouldn't call capturing the configuration like this patch is doing > "minimal", so > we'll agree to disagree on that point. > Sure, I meant it in the context of the entire space of possible build parameters in Yocto, compared to attempting to reflect all variables, overrides, dependencies, and task-level configuration required to fully reproduce a build. > (like kernel .config, PACKAGECONFIG, etc.) where: > - The structure is already well-defined > - The data is relevant for security or compliance > - And the cost of extraction is very low > > And arguably not the best or most efficient format ... > There’s already an inherent trade-off in Yocto between build performance and build reproducibility. This patch stays within that space by offering optional, low-cost metadata that helps improve traceability without impacting core functionality. > This is fully gated behind an opt-in variable (and could default to > disabled). > The idea is not to replace existing tools but rather to provide a > structured, > portable format for key metadata, especially in SBOM-driven > environments where SPDX > is already part of the process. > > It must default to disabled in my opinion. But I can already hear the > arguments > that will say "it won't be tested if it isn't on by default, etc" ... > and that is where we > will have the conflict about whether it is on, or off, by default. > OK fair point. > In my use case, this metadata is consumed by a targeted diff tool; > not treating > SPDX as the end-goal, but more like a transport format. This > approach lets us > reuse existing SPDX outputs without interfering with or duplicating > the role of > more purpose-built tools. > So I hear you: this shouldn't become bloated, mandatory. The goal is > simply to make > sure when people do need to trace how something was built, > especially in CI or > audit contexts, they have a consistent hook, and it can be disabled > entirely if > not useful. > > It is something that I can ignore (which is good), so I'll just keep > watching to > ensure that it doesn't become mandatory, or fail configurations on > compliance > and I'll be happy. > I feel that we are moving more and more towards a "pipeline" or > "loadbuild" use > case for OE, and that's also what I guard against. > I'm already not happy how long SPDX adds to developer builds, so at > some point > I need to revisit disabling it completely (it used to fail when I first > tried that). I agree SPDX can feel heavyweight when misused, we’re aiming to keep it scoped and useful, especially where it improves communication between devs and security teams. -- Kamel Bouhara, Bootlin Embedded Linux and kernel engineering https://bootlin.com
Nak, see inline comment below about erasing the recipe variables In general, I'm not too opposed to being able to express this sort of thing in the SPDX (provided it's optional), there already is precedence for it (see SPDX_INCLUDE_BUILD_VARIABLES), and it is more or less intended that SPDX _can_ express these sorts of things (although, just because we can doesn't mean we have to). However, your approach here is incorrect. The primary problem is that you are including the config settings in the recipe Build object; you can't really do that because that recipe Build object (more or less) represents the build of entire "recipe" (up to the point of the do_create_spdx task anyway), and already has a defined meaning for what its build parameters mean (they are the bitbake variables at the time the do_create_spdx is invoked). Adding in the kernel config options to this build changes that meaning and means that you can't infer the meaning of the parameters anymore from the buildType, which is wrong. Instead, what you need to do is create a new build that represents the compile (or configuration) of the kernel, and assign it a unique buildType (or if one already exists for the kernel build with the same meaning, you could use that.... but I doubt that is the case). Since you have a unique buildType, you can then define the kernel config parameters as you have done here for that build and it will be possible to correctly interpret them. The final step is to link the existing build to your new build using the ancestorOf relationship to indicate that your new build is a "sub build" of the parent recipe build. Ideally, this would not live in spdx30_tasks.py, since it's highly specific to the kernel build and I really want to avoid putting everyone's specific reporting requirements in the common code; they need to be more distributed than that (IMHO). On Wed, Jul 16, 2025 at 3:05 AM Kamel Bouhara <kamel.bouhara@bootlin.com> wrote: > Enhances SPDX Document by extracting kernel build-time configuration > settings from '${B}/.config'. > > Each CONFIG_* line is parsed and exported as a DictionaryEntry in the > build_Build.build_parameter > section of the SPDX document. This provides better visibility into kernel > build behavior and > configuration, in alignment with the SPDX3 metadata model. > > The feature is gated by a new tunable variable: > > SPDX_INCLUDE_KERNEL_CONFIG (default: "1") > > Setting this to "0" disables exporting the kernel configuration, which may > be useful to improve > performance or reduce the size of generated SPDX documents. > > Example: > > CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } > > This complements existing metadata export features and enables a more > complete audit trail of how > the kernel is built within a given build. > > Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com> > --- > meta/classes/create-spdx-3.0.bbclass | 6 ++++++ > meta/lib/oe/spdx30_tasks.py | 32 ++++++++++++++++++++++++++++ > 2 files changed, 38 insertions(+) > > diff --git a/meta/classes/create-spdx-3.0.bbclass > b/meta/classes/create-spdx-3.0.bbclass > index c0a5436ad6..cdb9422f37 100644 > --- a/meta/classes/create-spdx-3.0.bbclass > +++ b/meta/classes/create-spdx-3.0.bbclass > @@ -50,6 +50,12 @@ SPDX_INCLUDE_TIMESTAMPS[doc] = "Include time stamps in > SPDX output. This is \ > useful if you want to know when artifacts were produced and when > builds \ > occurred, but will result in non-reproducible SPDX output" > > +SPDX_INCLUDE_KERNEL_CONFIG ??= "1" > +SPDX_INCLUDE_KERNEL_CONFIG[doc] = "If set to '1', the .config file for > the kernel will be parsed \ > +and each CONFIG_* value will be included in the Build.build_parameter > list as DictionaryEntry \ > +items. Set to '0' to disable exporting kernel configuration to improve > performance or reduce \ > +SPDX document size." > + > SPDX_IMPORTS ??= "" > SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how > to \ > reference external SPDX ids. Each import is defined as a key in this \ > diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py > index c352dab152..f87d079cb0 100644 > --- a/meta/lib/oe/spdx30_tasks.py > +++ b/meta/lib/oe/spdx30_tasks.py > @@ -18,6 +18,28 @@ from contextlib import contextmanager > from datetime import datetime, timezone > from pathlib import Path > > +def parse_kernel_config(config_path): > + entries = [] > + if not os.path.exists(config_path): > + bb.warn(f"Kernel config file not found at: {config_path}") > + return entries > + > + try: > + with open(config_path, 'r') as f: > + for line in f: > + line = line.strip() > + if not line or line.startswith("#"): > + continue > + if "=" in line: > + key, value = line.split("=", 1) > + entries.append(oe.spdx30.DictionaryEntry( > + key=key, > + value=value.strip('"') > + )) > + bb.note(f"Parsed {len(entries)} kernel config entries from > {config_path}") > + except Exception as e: > + bb.error(f"Failed to parse kernel config file: {e}") > + return entries > > def walk_error(err): > bb.error(f"ERROR walking {err.filename}: {err}") > @@ -495,6 +517,8 @@ def create_spdx(d): > > build_objset.doc.rootElement.append(build) > > + build.build_parameter = [] > This will actually erase the recipe variables when SPDX_INCLUDE_BUILD_VARIABLES is present, so you can't do this. + > build_objset.set_is_native(is_native) > > for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): > @@ -815,6 +839,14 @@ def create_spdx(d): > sorted(list(build_inputs)) + sorted(list(debug_source_ids)), > ) > > + if d.getVar("SPDX_INCLUDE_KERNEL_CONFIG", True) != "0": > + if "virtual/kernel" in (d.getVar("PROVIDES") or "").split(): > + bb.note("Detected virtual/kernel provider, extracting kernel > configuration") > + config_path = d.expand("${B}/.config") > + kernel_params = parse_kernel_config(config_path) > + if kernel_params: > + build.build_parameter.extend(kernel_params) > + > oe.sbom30.write_recipe_jsonld_doc(d, build_objset, "recipes", > deploydir) > > > -- > 2.43.0 > >
On Fri, Jul 18, 2025 at 04:10:31PM -0600, Joshua Watt wrote: > Nak, see inline comment below about erasing the recipe variables > In general, I'm not too opposed to being able to express this sort of > thing in the SPDX (provided it's optional), there already is precedence > for it (see SPDX_INCLUDE_BUILD_VARIABLES), and it is more or less > intended that SPDX _can_ express these sorts of things (although, just > because we can doesn't mean we have to). However, your approach here is > incorrect. Hi Joshua, > The primary problem is that you are including the config settings in > the recipe Build object; you can't really do that because that recipe > Build object (more or less) represents the build of entire "recipe" (up > to the point of the do_create_spdx task anyway), and already has a > defined meaning for what its build parameters mean (they are the > bitbake variables at the time the do_create_spdx is invoked). Adding in > the kernel config options to this build changes that meaning and means > that you can't infer the meaning of the parameters anymore from the > buildType, which is wrong. OK thanks for clarifying the context around the Build object. Just to make sure I understand correctly: The Build object created in create_spdx(d) is intended to represent the entire bitbake recipe build (up to do_create_spdx), including any tasks like do_configure, do_compile, etc., but it's still scoped to that specific recipe, not to the entire image or overall build. So anything that applies to just a step (like parsing .config for a kernel) shouldn't be added directly to this object, and should instead go in a separate Build object linked via a relationship like ancestorOf. Is that the correct way to interpret it? > Instead, what you need to do is create a new build that represents the > compile (or configuration) of the kernel, and assign it a unique > buildType (or if one already exists for the kernel build with the same > meaning, you could use that.... but I doubt that is the case). Since > you have a unique buildType, you can then define the kernel config > parameters as you have done here for that build and it will be possible > to correctly interpret them. The final step is to link the existing > build to your new build using the ancestorOf relationship to indicate > that your new build is a "sub build" of the parent recipe build. OK, I've started reworking the patch with that structure in mind. To create the new build object representing the kernel configuration, I’m planning to do the following: 1. Use oe.spdx30.build_Build(...) to create a new Build, with a unique buildType like "kernel-configuration". 2. Populate build_parameter with the parsed .config CONFIG_* values as DictionaryEntrys. 3. Link it to the main recipe Build object via ancestorOf relationship like: build_objset.new_relationship( [build], # Parent: full recipe build oe.spdx30.RelationshipType.ancestorOf, [main_recipe_build], # Child: kernel config sub-build ) 4. Set timestamps (seems optional ?) Does that approach look correct to you, especially the use of ancestorOf and the new buildType? Below is an example of spdx snippet obtained with your suggestion: { "type": "build_Build", "spdxId": "http://spdx.org/spdxdocs/linux-yocto-.../kernel-config", "creationInfo": "_:CreationInfo220", "build_buildType": "kernel-configuration", "build_parameter": [ { "type": "DictionaryEntry", "key": "CONFIG_CC_VERSION_TEXT", "value": "x86_64-oe-linux-gcc (GCC) 13.3.0" }, ... } > Ideally, this would not live in spdx30_tasks.py, since it's highly > specific to the kernel build and I really want to avoid putting > everyone's specific reporting requirements in the common code; they > need to be more distributed than that (IMHO). I see your point about keeping spdx30_tasks.py free of recipe-specific logic, that makes sense from a long-term maintainability standpoint. That said, I have some doubts about creating a dedicated kernel-spdx.bbclass just for handling this one case (the .config). It feels a bit heavy unless we expect additional kernel-specific SPDX processing in the future. Would a lighter alternative be acceptable, keeping the kernel build injection behind SPDX_INCLUDE_KERNEL_CONFIG? It would keep the logic local without needing a new class. Curious what you'd prefer here, I want to respect the layering without introducing more complexity than necessary. Cheers, -- Kamel Bouhara, Bootlin Embedded Linux and kernel engineering https://bootlin.com
On Mon, Jul 21, 2025 at 8:53 AM Kamel Bouhara <kamel.bouhara@bootlin.com> wrote: > On Fri, Jul 18, 2025 at 04:10:31PM -0600, Joshua Watt wrote: > > Nak, see inline comment below about erasing the recipe variables > > In general, I'm not too opposed to being able to express this sort of > > thing in the SPDX (provided it's optional), there already is > precedence > > for it (see SPDX_INCLUDE_BUILD_VARIABLES), and it is more or less > > intended that SPDX _can_ express these sorts of things (although, just > > because we can doesn't mean we have to). However, your approach here > is > > incorrect. > > Hi Joshua, > > > The primary problem is that you are including the config settings in > > the recipe Build object; you can't really do that because that recipe > > Build object (more or less) represents the build of entire "recipe" > (up > > to the point of the do_create_spdx task anyway), and already has a > > defined meaning for what its build parameters mean (they are the > > bitbake variables at the time the do_create_spdx is invoked). Adding > in > > the kernel config options to this build changes that meaning and means > > that you can't infer the meaning of the parameters anymore from the > > buildType, which is wrong. > > > OK thanks for clarifying the context around the Build object. > > Just to make sure I understand correctly: > > The Build object created in create_spdx(d) is intended to represent the > entire bitbake recipe build (up to do_create_spdx), including any tasks > like do_configure, do_compile, etc., but it's still scoped to that > specific recipe, not to the entire image or overall build. > > So anything that applies to just a step (like parsing .config for a kernel) > shouldn't be added directly to this object, and should instead go in a > separate Build object linked via a relationship like ancestorOf. > > Is that the correct way to interpret it? > Yes, this is a correct interpretation. In the future, we may also create sub-builds for the various tasks that run (do_compile, et. al.) in the same manner, but it's not really needed at this point. > > > Instead, what you need to do is create a new build that represents the > > compile (or configuration) of the kernel, and assign it a unique > > buildType (or if one already exists for the kernel build with the same > > meaning, you could use that.... but I doubt that is the case). Since > > you have a unique buildType, you can then define the kernel config > > parameters as you have done here for that build and it will be > possible > > to correctly interpret them. The final step is to link the existing > > build to your new build using the ancestorOf relationship to indicate > > that your new build is a "sub build" of the parent recipe build. > > OK, I've started reworking the patch with that structure in mind. > > To create the new build object representing the kernel configuration, > I’m planning to do the following: > > 1. Use oe.spdx30.build_Build(...) to create a new Build, with a unique > buildType like "kernel-configuration". > Like many other things in SPDX, this is actually a URI which is used to namespace it (i.e. it doesn't need to be a resolvable _URL_, but it looks like one). As such, you'd probably want to do: "https://openembedded.org/kernel-configuration" or similar. This prevents someone else from squatting on the "kernel-configuration" name and accidentally giving it a different meaning. > > 2. Populate build_parameter with the parsed .config CONFIG_* values as > DictionaryEntrys. > > 3. Link it to the main recipe Build object via ancestorOf relationship > like: > > build_objset.new_relationship( > [build], # Parent: full recipe build > oe.spdx30.RelationshipType.ancestorOf, > [main_recipe_build], # Child: kernel config sub-build > ) > > 4. Set timestamps (seems optional ?) > Just make sure to use set_timestamp_now(). > > Does that approach look correct to you, especially the use of ancestorOf > and the new buildType? > > Below is an example of spdx snippet obtained with your suggestion: > > { > "type": "build_Build", > "spdxId": "http://spdx.org/spdxdocs/linux-yocto-.../kernel-config", > "creationInfo": "_:CreationInfo220", > "build_buildType": "kernel-configuration", > "build_parameter": [ > { > "type": "DictionaryEntry", > "key": "CONFIG_CC_VERSION_TEXT", > "value": "x86_64-oe-linux-gcc (GCC) 13.3.0" > }, > ... > } > > > Ideally, this would not live in spdx30_tasks.py, since it's highly > > specific to the kernel build and I really want to avoid putting > > everyone's specific reporting requirements in the common code; they > > need to be more distributed than that (IMHO). > > I see your point about keeping spdx30_tasks.py free of recipe-specific > logic, that makes sense from a long-term maintainability standpoint. > > That said, I have some doubts about creating a dedicated > kernel-spdx.bbclass > just for handling this one case (the .config). It feels a bit heavy unless > we expect additional kernel-specific SPDX processing in the future. > IMHO, it would hopefully live in kernel.bbclass and hook into the SPDX generation from there (see below). One could imagine similar things for meson.bbclass, cmake.bbclass et. al. > > Would a lighter alternative be acceptable, keeping the > kernel build injection behind SPDX_INCLUDE_KERNEL_CONFIG? It would keep > the logic local without needing a new class. > It needs the SPDX_INCLUDE_KERNEL_CONFIG regardless. One advantage of being able to split this out would be that you could add the sub builds using a bbclass in your own layer if you wanted, which would make it more flexible to incubate stuff like this in your own layer before moving to core. Unfortunately, I've not had a chance to think about the best way to do this, so I'm not exactly sure what makes sense > > Curious what you'd prefer here, I want to respect the layering without > introducing more complexity than necessary. > > Cheers, > > -- > Kamel Bouhara, Bootlin > Embedded Linux and kernel engineering > https://bootlin.com >
diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index c0a5436ad6..cdb9422f37 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -50,6 +50,12 @@ SPDX_INCLUDE_TIMESTAMPS[doc] = "Include time stamps in SPDX output. This is \ useful if you want to know when artifacts were produced and when builds \ occurred, but will result in non-reproducible SPDX output" +SPDX_INCLUDE_KERNEL_CONFIG ??= "1" +SPDX_INCLUDE_KERNEL_CONFIG[doc] = "If set to '1', the .config file for the kernel will be parsed \ +and each CONFIG_* value will be included in the Build.build_parameter list as DictionaryEntry \ +items. Set to '0' to disable exporting kernel configuration to improve performance or reduce \ +SPDX document size." + SPDX_IMPORTS ??= "" SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \ reference external SPDX ids. Each import is defined as a key in this \ diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index c352dab152..f87d079cb0 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -18,6 +18,28 @@ from contextlib import contextmanager from datetime import datetime, timezone from pathlib import Path +def parse_kernel_config(config_path): + entries = [] + if not os.path.exists(config_path): + bb.warn(f"Kernel config file not found at: {config_path}") + return entries + + try: + with open(config_path, 'r') as f: + for line in f: + line = line.strip() + if not line or line.startswith("#"): + continue + if "=" in line: + key, value = line.split("=", 1) + entries.append(oe.spdx30.DictionaryEntry( + key=key, + value=value.strip('"') + )) + bb.note(f"Parsed {len(entries)} kernel config entries from {config_path}") + except Exception as e: + bb.error(f"Failed to parse kernel config file: {e}") + return entries def walk_error(err): bb.error(f"ERROR walking {err.filename}: {err}") @@ -495,6 +517,8 @@ def create_spdx(d): build_objset.doc.rootElement.append(build) + build.build_parameter = [] + build_objset.set_is_native(is_native) for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): @@ -815,6 +839,14 @@ def create_spdx(d): sorted(list(build_inputs)) + sorted(list(debug_source_ids)), ) + if d.getVar("SPDX_INCLUDE_KERNEL_CONFIG", True) != "0": + if "virtual/kernel" in (d.getVar("PROVIDES") or "").split(): + bb.note("Detected virtual/kernel provider, extracting kernel configuration") + config_path = d.expand("${B}/.config") + kernel_params = parse_kernel_config(config_path) + if kernel_params: + build.build_parameter.extend(kernel_params) + oe.sbom30.write_recipe_jsonld_doc(d, build_objset, "recipes", deploydir)
Enhances SPDX Document by extracting kernel build-time configuration settings from '${B}/.config'. Each CONFIG_* line is parsed and exported as a DictionaryEntry in the build_Build.build_parameter section of the SPDX document. This provides better visibility into kernel build behavior and configuration, in alignment with the SPDX3 metadata model. The feature is gated by a new tunable variable: SPDX_INCLUDE_KERNEL_CONFIG (default: "1") Setting this to "0" disables exporting the kernel configuration, which may be useful to improve performance or reduce the size of generated SPDX documents. Example: CONFIG_FOO=y → { key: "CONFIG_FOO", value: "y" } This complements existing metadata export features and enables a more complete audit trail of how the kernel is built within a given build. Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com> --- meta/classes/create-spdx-3.0.bbclass | 6 ++++++ meta/lib/oe/spdx30_tasks.py | 32 ++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+)