| Message ID | 20251010094034.4107881-1-Maganuru.Jayasurya@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | populate_sdk_ext: include image SPDX tasks in locked signatures | expand |
On Fri, Oct 10, 2025 at 3:41 AM Jayasurya Maganuru via lists.openembedded.org <maganuru.jayasurya=windriver.com@lists.openembedded.org> wrote: > > From: Jayasurya Maganuru <Maganuru.Jayasurya@windriver.com> > > Fixes [YOCTO #15726] > Fixes [YOCTO #15853] > > After the switch to SPDX 3.0, eSDK installation can fail with errors like: > > gcc-source-1*:do_fetch attempted to execute unexpectedly > This is usually due to missing setscene tasks. > > This is caused by image-related SPDX tasks, such as do_create_image_sbom_spdx, > being excluded from the locked signatures. Without these, the corresponding > sstate-cache artifacts are missing during eSDK installation. > > Previously (under SPDX 2.2), these image SPDX/SBOM tasks were not > dependencies of do_populate_sdk_ext task, so their sstate artifacts > were not required at install time. > > Fix: > - do_create_image_sbom_spdx is added to do_sdk_depends[recrdeptask] to > ensure it is included in the eSDK build and corresponding sstate cache. > - SDK_TARGETS (and multilib variants) are removed from the excluded target > list in prepare_locked_cache(), so their image SPDX/SBOM tasks are retained > in locked-sigs.inc. > > With this fix, eSDK installs work without unexpected task runs, even > for custom distros or "tar" images. > > Signed-off-by: Jayasurya Maganuru <Maganuru.Jayasurya@windriver.com> > --- > meta/classes-recipe/populate_sdk_ext.bbclass | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass > index 36a3e9c836..769672ec80 100644 > --- a/meta/classes-recipe/populate_sdk_ext.bbclass > +++ b/meta/classes-recipe/populate_sdk_ext.bbclass > @@ -465,6 +465,15 @@ def prepare_locked_cache(d, baseoutpath, derivative, conf_initpath): > > # Filter the locked signatures file to just the sstate tasks we are interested in > excluded_targets = get_sdk_install_targets(d, images_only=True) > + sdk_targets = d.getVar('SDK_TARGETS') > + ext_sdk_target_set = set(multilib_pkg_extend(d, sdk_targets).split()) > + excluded_set = set(excluded_targets.split()) > + > + # Ensure SDK_TARGETS and their image SPDX/SBOM tasks are included in the locked signatures, > + # as they are required during eSDK installation. > + filtered_excluded_set = excluded_set - ext_sdk_target_set > + excluded_targets = ' '.join(filtered_excluded_set) > + > sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc' > lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc' > #nativesdk-only sigfile to merge into locked-sigs.inc > @@ -821,7 +830,7 @@ addtask sdk_depends > do_sdk_depends[dirs] = "${WORKDIR}" > do_sdk_depends[depends] = "${@get_ext_sdk_depends(d)} meta-extsdk-toolchain:do_populate_sysroot" > do_sdk_depends[recrdeptask] = "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}" > -do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy ${SDK_RECRDEP_TASKS}" > +do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy do_create_image_sbom_spdx ${SDK_RECRDEP_TASKS}" The SPDX dependencies are generally added in creates-spdx-sdk-3.0.bbclass so as not to cause problems when SPDX is disabled. Is it possible to add it there instead? > do_sdk_depends[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE} ' + x + ':do_packagedata' for x in d.getVar('TOOLCHAIN_HOST_TASK_ESDK').split()])}" > > do_populate_sdk_ext[dirs] = "${@d.getVarFlag('do_populate_sdk', 'dirs', False)}" > -- > 2.49.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#224675): https://lists.openembedded.org/g/openembedded-core/message/224675 > Mute This Topic: https://lists.openembedded.org/mt/115686400/3616693 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 20/10/2025 22:09, Joshua Watt wrote: > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and know the content is safe. > > On Fri, Oct 10, 2025 at 3:41 AM Jayasurya Maganuru via > lists.openembedded.org > <maganuru.jayasurya=windriver.com@lists.openembedded.org> wrote: >> From: Jayasurya Maganuru<Maganuru.Jayasurya@windriver.com> >> >> Fixes [YOCTO #15726] >> Fixes [YOCTO #15853] >> >> After the switch to SPDX 3.0, eSDK installation can fail with errors like: >> >> gcc-source-1*:do_fetch attempted to execute unexpectedly >> This is usually due to missing setscene tasks. >> >> This is caused by image-related SPDX tasks, such as do_create_image_sbom_spdx, >> being excluded from the locked signatures. Without these, the corresponding >> sstate-cache artifacts are missing during eSDK installation. >> >> Previously (under SPDX 2.2), these image SPDX/SBOM tasks were not >> dependencies of do_populate_sdk_ext task, so their sstate artifacts >> were not required at install time. >> >> Fix: >> - do_create_image_sbom_spdx is added to do_sdk_depends[recrdeptask] to >> ensure it is included in the eSDK build and corresponding sstate cache. >> - SDK_TARGETS (and multilib variants) are removed from the excluded target >> list in prepare_locked_cache(), so their image SPDX/SBOM tasks are retained >> in locked-sigs.inc. >> >> With this fix, eSDK installs work without unexpected task runs, even >> for custom distros or "tar" images. >> >> Signed-off-by: Jayasurya Maganuru<Maganuru.Jayasurya@windriver.com> >> --- >> meta/classes-recipe/populate_sdk_ext.bbclass | 11 ++++++++++- >> 1 file changed, 10 insertions(+), 1 deletion(-) >> >> diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass >> index 36a3e9c836..769672ec80 100644 >> --- a/meta/classes-recipe/populate_sdk_ext.bbclass >> +++ b/meta/classes-recipe/populate_sdk_ext.bbclass >> @@ -465,6 +465,15 @@ def prepare_locked_cache(d, baseoutpath, derivative, conf_initpath): >> >> # Filter the locked signatures file to just the sstate tasks we are interested in >> excluded_targets = get_sdk_install_targets(d, images_only=True) >> + sdk_targets = d.getVar('SDK_TARGETS') >> + ext_sdk_target_set = set(multilib_pkg_extend(d, sdk_targets).split()) >> + excluded_set = set(excluded_targets.split()) >> + >> + # Ensure SDK_TARGETS and their image SPDX/SBOM tasks are included in the locked signatures, >> + # as they are required during eSDK installation. >> + filtered_excluded_set = excluded_set - ext_sdk_target_set >> + excluded_targets = ' '.join(filtered_excluded_set) >> + >> sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc' >> lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc' >> #nativesdk-only sigfile to merge into locked-sigs.inc >> @@ -821,7 +830,7 @@ addtask sdk_depends >> do_sdk_depends[dirs] = "${WORKDIR}" >> do_sdk_depends[depends] = "${@get_ext_sdk_depends(d)} meta-extsdk-toolchain:do_populate_sysroot" >> do_sdk_depends[recrdeptask] ="${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}" >> -do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy ${SDK_RECRDEP_TASKS}" >> +do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy do_create_image_sbom_spdx ${SDK_RECRDEP_TASKS}" > The SPDX dependencies are generally added in > creates-spdx-sdk-3.0.bbclass so as not to cause problems when SPDX is > disabled. Is it possible to add it there instead? Hi Joshua, Following your review, I’ve moved the logic to |create-spdx-sdk-3.0.bbclass| and sent V2. https://lists.openembedded.org/g/openembedded-core/message/225440 <cid:part1.tomxmoFd.K1rkOQUa@windriver.com> While exploring |create-*spdx-image*-3.0.bbclass|, we could alternatively place the |do_create_image_sbom_spdx| task like this: -addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build +addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build *do_sdk_depends *Do you think the current V2 approach is sufficient, or would this alternate placement be preferable? Thanks, Jayasurya > >> do_sdk_depends[rdepends] ="${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE} ' + x + >> ':do_packagedata' for x in >> d.getVar('TOOLCHAIN_HOST_TASK_ESDK').split()])}" >> >> do_populate_sdk_ext[dirs] ="${@d.getVarFlag('do_populate_sdk', 'dirs', False)}" >> -- >> 2.49.0 >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#224675):https://lists.openembedded.org/g/openembedded-core/message/224675 >> Mute This Topic:https://lists.openembedded.org/mt/115686400/3616693 >> Group Owner:openembedded-core+owner@lists.openembedded.org >> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >>
On Wed, Oct 29, 2025 at 5:36 AM Maganuru Jayasurya <Maganuru.Jayasurya@windriver.com> wrote: > > > On 20/10/2025 22:09, Joshua Watt wrote: > > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and know the content is safe. > > On Fri, Oct 10, 2025 at 3:41 AM Jayasurya Maganuru via > lists.openembedded.org > <maganuru.jayasurya=windriver.com@lists.openembedded.org> wrote: > > From: Jayasurya Maganuru <Maganuru.Jayasurya@windriver.com> > > Fixes [YOCTO #15726] > Fixes [YOCTO #15853] > > After the switch to SPDX 3.0, eSDK installation can fail with errors like: > > gcc-source-1*:do_fetch attempted to execute unexpectedly > This is usually due to missing setscene tasks. > > This is caused by image-related SPDX tasks, such as do_create_image_sbom_spdx, > being excluded from the locked signatures. Without these, the corresponding > sstate-cache artifacts are missing during eSDK installation. > > Previously (under SPDX 2.2), these image SPDX/SBOM tasks were not > dependencies of do_populate_sdk_ext task, so their sstate artifacts > were not required at install time. > > Fix: > - do_create_image_sbom_spdx is added to do_sdk_depends[recrdeptask] to > ensure it is included in the eSDK build and corresponding sstate cache. > - SDK_TARGETS (and multilib variants) are removed from the excluded target > list in prepare_locked_cache(), so their image SPDX/SBOM tasks are retained > in locked-sigs.inc. > > With this fix, eSDK installs work without unexpected task runs, even > for custom distros or "tar" images. > > Signed-off-by: Jayasurya Maganuru <Maganuru.Jayasurya@windriver.com> > --- > meta/classes-recipe/populate_sdk_ext.bbclass | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass > index 36a3e9c836..769672ec80 100644 > --- a/meta/classes-recipe/populate_sdk_ext.bbclass > +++ b/meta/classes-recipe/populate_sdk_ext.bbclass > @@ -465,6 +465,15 @@ def prepare_locked_cache(d, baseoutpath, derivative, conf_initpath): > > # Filter the locked signatures file to just the sstate tasks we are interested in > excluded_targets = get_sdk_install_targets(d, images_only=True) > + sdk_targets = d.getVar('SDK_TARGETS') > + ext_sdk_target_set = set(multilib_pkg_extend(d, sdk_targets).split()) > + excluded_set = set(excluded_targets.split()) > + > + # Ensure SDK_TARGETS and their image SPDX/SBOM tasks are included in the locked signatures, > + # as they are required during eSDK installation. > + filtered_excluded_set = excluded_set - ext_sdk_target_set > + excluded_targets = ' '.join(filtered_excluded_set) > + > sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc' > lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc' > #nativesdk-only sigfile to merge into locked-sigs.inc > @@ -821,7 +830,7 @@ addtask sdk_depends > do_sdk_depends[dirs] = "${WORKDIR}" > do_sdk_depends[depends] = "${@get_ext_sdk_depends(d)} meta-extsdk-toolchain:do_populate_sysroot" > do_sdk_depends[recrdeptask] = "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}" > -do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy ${SDK_RECRDEP_TASKS}" > +do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy do_create_image_sbom_spdx ${SDK_RECRDEP_TASKS}" > > The SPDX dependencies are generally added in > creates-spdx-sdk-3.0.bbclass so as not to cause problems when SPDX is > disabled. Is it possible to add it there instead? > > > Hi Joshua, > > Following your review, I’ve moved the logic to create-spdx-sdk-3.0.bbclass and sent V2. > https://lists.openembedded.org/g/openembedded-core/message/225440 > > While exploring create-spdx-image-3.0.bbclass, we could alternatively place the do_create_image_sbom_spdx task like this: > > -addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build > +addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build do_sdk_depends Yes, this approach is preferred; I don't really like addtask in multiple locations. Thanks > > Do you think the current V2 approach is sufficient, or would this alternate placement be preferable? > > Thanks, > Jayasurya > > > > do_sdk_depends[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE} ' + x + ':do_packagedata' for x in d.getVar('TOOLCHAIN_HOST_TASK_ESDK').split()])}" > > do_populate_sdk_ext[dirs] = "${@d.getVarFlag('do_populate_sdk', 'dirs', False)}" > -- > 2.49.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#224675): https://lists.openembedded.org/g/openembedded-core/message/224675 > Mute This Topic: https://lists.openembedded.org/mt/115686400/3616693 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass index 36a3e9c836..769672ec80 100644 --- a/meta/classes-recipe/populate_sdk_ext.bbclass +++ b/meta/classes-recipe/populate_sdk_ext.bbclass @@ -465,6 +465,15 @@ def prepare_locked_cache(d, baseoutpath, derivative, conf_initpath): # Filter the locked signatures file to just the sstate tasks we are interested in excluded_targets = get_sdk_install_targets(d, images_only=True) + sdk_targets = d.getVar('SDK_TARGETS') + ext_sdk_target_set = set(multilib_pkg_extend(d, sdk_targets).split()) + excluded_set = set(excluded_targets.split()) + + # Ensure SDK_TARGETS and their image SPDX/SBOM tasks are included in the locked signatures, + # as they are required during eSDK installation. + filtered_excluded_set = excluded_set - ext_sdk_target_set + excluded_targets = ' '.join(filtered_excluded_set) + sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc' lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc' #nativesdk-only sigfile to merge into locked-sigs.inc @@ -821,7 +830,7 @@ addtask sdk_depends do_sdk_depends[dirs] = "${WORKDIR}" do_sdk_depends[depends] = "${@get_ext_sdk_depends(d)} meta-extsdk-toolchain:do_populate_sysroot" do_sdk_depends[recrdeptask] = "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}" -do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy ${SDK_RECRDEP_TASKS}" +do_sdk_depends[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy do_create_image_sbom_spdx ${SDK_RECRDEP_TASKS}" do_sdk_depends[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE} ' + x + ':do_packagedata' for x in d.getVar('TOOLCHAIN_HOST_TASK_ESDK').split()])}" do_populate_sdk_ext[dirs] = "${@d.getVarFlag('do_populate_sdk', 'dirs', False)}"