diff mbox series

[v3] populate_sdk_ext, create-spdx-image-3.0: ensure image SPDX/SBOM tasks are retained for eSDK installation

Message ID 20251105121432.3987350-1-Maganuru.Jayasurya@windriver.com
State New
Headers show
Series [v3] populate_sdk_ext, create-spdx-image-3.0: ensure image SPDX/SBOM tasks are retained for eSDK installation | expand

Commit Message

Maganuru Jayasurya Nov. 5, 2025, 12:14 p.m. UTC 
From: Jayasurya Maganuru <Maganuru.Jayasurya@windriver.com>

Fixes [YOCTO #15726]
Fixes [YOCTO #15853]

During eSDK installation, errors such as the following may occur:

  <task>:do_fetch attempted to execute unexpectedly
  This is usually due to missing setscene tasks.

Investigation showed that image SPDX/SBOM tasks
(e.g. do_create_image_spdx, do_create_image_sbom_spdx, do_create_spdx)
were missing from the locked signatures and therefore not present in the
eSDK sstate cache. As a result, these tasks attempted to run again
during installation.

This issue is resolved in two parts:

1. Task ordering fix:
   The do_create_image_spdx task now runs before do_sdk_depends by
   adding do_sdk_depends to its "before" list. This ensures the SPDX
   image generation tasks are completed before locked signatures are
   created, allowing their signatures to be captured.

2. Locked signatures filtering fix:
   In populate_sdk_ext.bbclass, SDK_TARGETS (e.g. core-image-base)
   were previously removed from the locked signatures. The logic has
   been updated so that SDK_TARGETS and their multilib variants are
   retained, ensuring their image SPDX/SBOM tasks are included in
   locked-sigs.inc and preserved in the sstate cache.

With these changes, all required image SPDX/SBOM tasks are captured in
the locked signatures, and eSDK installation completes successfully
without unexpected task executions.

Signed-off-by: Jayasurya Maganuru <Maganuru.Jayasurya@windriver.com>
Suggested-by: Joshua Watt <jpewhacker@gmail.com>
---
v3: Moved the logic from create-spdx-sdk-3.0 to create-spdx-image-3.0

 meta/classes-recipe/create-spdx-image-3.0.bbclass | 2 +-
 meta/classes-recipe/populate_sdk_ext.bbclass      | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

Comments

patchtest@automation.yoctoproject.org Nov. 5, 2025, 12:31 p.m. UTC | #1 
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/v3-populate_sdk_ext-create-spdx-image-3.0-ensure-image-SPDX-SBOM-tasks-are-retained-for-eSDK-installation.patch

FAIL: test shortlog length: Edit shortlog so that it is 90 characters or less (currently 104 characters) (test_mbox.TestMbox.test_shortlog_length)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
Randy MacLeod Dec. 1, 2025, 7:49 p.m. UTC | #2 
On 2025-11-05 7:14 a.m., Jayasurya Maganuru via lists.openembedded.org 
wrote:
> From: Jayasurya Maganuru<Maganuru.Jayasurya@windriver.com>
>
> Fixes [YOCTO #15726]
> Fixes [YOCTO #15853]
>
> During eSDK installation, errors such as the following may occur:
>
>    <task>:do_fetch attempted to execute unexpectedly
>    This is usually due to missing setscene tasks.
>
> Investigation showed that image SPDX/SBOM tasks
> (e.g. do_create_image_spdx, do_create_image_sbom_spdx, do_create_spdx)
> were missing from the locked signatures and therefore not present in the
> eSDK sstate cache. As a result, these tasks attempted to run again
> during installation.
>
> This issue is resolved in two parts:
>
> 1. Task ordering fix:
>     The do_create_image_spdx task now runs before do_sdk_depends by
>     adding do_sdk_depends to its "before" list. This ensures the SPDX
>     image generation tasks are completed before locked signatures are
>     created, allowing their signatures to be captured.
>
> 2. Locked signatures filtering fix:
>     In populate_sdk_ext.bbclass, SDK_TARGETS (e.g. core-image-base)
>     were previously removed from the locked signatures. The logic has
>     been updated
** this part **

See below.

> so that SDK_TARGETS and their multilib variants are
>     retained, ensuring their image SPDX/SBOM tasks are included in
>     locked-sigs.inc and preserved in the sstate cache.
>
> With these changes, all required image SPDX/SBOM tasks are captured in
> the locked signatures, and eSDK installation completes successfully
> without unexpected task executions.

Hi Jayasurya,

This work is coming along well and may in fact be  done
but would it make sense to split the work into 2 patches if the the 
changes are not completely inter-tangled?
The goal would be to make the changes a bit easier to review and focus 
the commit log even if the
problem reported isn't fixed unless you have both patches.

There's also a problem flagged by patchtest:
FAIL: test shortlog length: Edit shortlog so that it is 90 characters or 
less (currently 104 characters) 
(test_mbox.TestMbox.test_shortlog_length) and slitting the commit in two 
could help with making the shortlogs shorter! ;-)


Also, in your for commit log related to:
meta/classes-recipe/populate_sdk_ext.bbclass

please explain *why* the changes for populate_sdk_ext.bbclass are needed 
especially expand on part just before "** this part **" and what the 
impact of the changes is. This came out of a (too brief) patch review 
discussion so there may be additional follow-up questions from some of 
the other people involved.

Qi and/or Hongxu,
     Can one of you reply to this thread and agree to help Jayasurya to
review these changes and the  v4 commit logs to ensure that they are
so clear that they easier to quickly review please ?

Thanks,

../Randy


>
> Signed-off-by: Jayasurya Maganuru<Maganuru.Jayasurya@windriver.com>
> Suggested-by: Joshua Watt<jpewhacker@gmail.com>
> ---
> v3: Moved the logic from create-spdx-sdk-3.0 to create-spdx-image-3.0
>
>   meta/classes-recipe/create-spdx-image-3.0.bbclass | 2 +-
>   meta/classes-recipe/populate_sdk_ext.bbclass      | 9 +++++++++
>   2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/create-spdx-image-3.0.bbclass b/meta/classes-recipe/create-spdx-image-3.0.bbclass
> index 636ab14eb0..f070b7e697 100644
> --- a/meta/classes-recipe/create-spdx-image-3.0.bbclass
> +++ b/meta/classes-recipe/create-spdx-image-3.0.bbclass
> @@ -69,7 +69,7 @@ python do_create_image_sbom_spdx() {
>       import oe.spdx30_tasks
>       oe.spdx30_tasks.create_image_sbom_spdx(d)
>   }
> -addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build
> +addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build do_sdk_depends
>   SSTATETASKS += "do_create_image_sbom_spdx"
>   SSTATE_SKIP_CREATION:task-create-image-sbom = "1"
>   do_create_image_sbom_spdx[sstate-inputdirs] = "${SPDXIMAGEDEPLOYDIR}"
> diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass
> index 2859320ddf..2838ca1a03 100644
> --- a/meta/classes-recipe/populate_sdk_ext.bbclass
> +++ b/meta/classes-recipe/populate_sdk_ext.bbclass
> @@ -460,6 +460,15 @@ def prepare_locked_cache(d, baseoutpath, derivative, conf_initpath):
>   
>       # Filter the locked signatures file to just the sstate tasks we are interested in
>       excluded_targets = get_sdk_install_targets(d, images_only=True)
> +    sdk_targets = d.getVar('SDK_TARGETS')
> +    ext_sdk_target_set = set(multilib_pkg_extend(d, sdk_targets).split())
> +    excluded_set = set(excluded_targets.split())
> +
> +    # Ensure SDK_TARGETS and their image SPDX/SBOM tasks are included in the locked signatures,
> +    # as they are required during eSDK installation.
> +    filtered_excluded_set = excluded_set - ext_sdk_target_set
> +    excluded_targets = ' '.join(filtered_excluded_set)
> +
>       sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc'
>       lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc'
>       #nativesdk-only sigfile to merge into locked-sigs.inc
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#225754):https://lists.openembedded.org/g/openembedded-core/message/225754
> Mute This Topic:https://lists.openembedded.org/mt/116133120/3616765
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
diff mbox series

Patch

diff --git a/meta/classes-recipe/create-spdx-image-3.0.bbclass b/meta/classes-recipe/create-spdx-image-3.0.bbclass
index 636ab14eb0..f070b7e697 100644
--- a/meta/classes-recipe/create-spdx-image-3.0.bbclass
+++ b/meta/classes-recipe/create-spdx-image-3.0.bbclass
@@ -69,7 +69,7 @@  python do_create_image_sbom_spdx() {
     import oe.spdx30_tasks
     oe.spdx30_tasks.create_image_sbom_spdx(d)
 }
-addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build
+addtask do_create_image_sbom_spdx after do_create_rootfs_spdx do_create_image_spdx before do_build do_sdk_depends
 SSTATETASKS += "do_create_image_sbom_spdx"
 SSTATE_SKIP_CREATION:task-create-image-sbom = "1"
 do_create_image_sbom_spdx[sstate-inputdirs] = "${SPDXIMAGEDEPLOYDIR}"
diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass
index 2859320ddf..2838ca1a03 100644
--- a/meta/classes-recipe/populate_sdk_ext.bbclass
+++ b/meta/classes-recipe/populate_sdk_ext.bbclass
@@ -460,6 +460,15 @@  def prepare_locked_cache(d, baseoutpath, derivative, conf_initpath):
 
     # Filter the locked signatures file to just the sstate tasks we are interested in
     excluded_targets = get_sdk_install_targets(d, images_only=True)
+    sdk_targets = d.getVar('SDK_TARGETS')
+    ext_sdk_target_set = set(multilib_pkg_extend(d, sdk_targets).split())
+    excluded_set = set(excluded_targets.split())
+
+    # Ensure SDK_TARGETS and their image SPDX/SBOM tasks are included in the locked signatures,
+    # as they are required during eSDK installation.
+    filtered_excluded_set = excluded_set - ext_sdk_target_set
+    excluded_targets = ' '.join(filtered_excluded_set)
+
     sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc'
     lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc'
     #nativesdk-only sigfile to merge into locked-sigs.inc