| Message ID | 20260314094758.3929192-1-richard.purdie@linuxfoundation.org |
|---|---|
| State | Under Review |
| Headers | show |
| Series | [1/5] sstate: Drop unneeded SSTATE_MANMACH variable | expand |
On Sat, 2026-03-14 at 09:47 +0000, Richard Purdie via lists.openembedded.org wrote: > Currently, the dummy SDK packages are re-running for different SDKMACHINE values > when they should not. The usage of allarch is broken and not triggering the right > PACKAGE_ARCH value due to the deferred nature of nativesdk. This was probably > broken when we switched to add deferred classes. > > To try and make this all more explict and less prone to breakage, switch to calling > oe.utils.make_arch_independent() directly. > > Add the 'special' package architecture values to SSTATE_ARCHS so the system cna properly > track them. > > Remove the pointless tasks we don't need from the dummy recipes, mark the packagedata > as machine independent and then remove from the conflict list in sstate.bbclass. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> This patch fixes real issues with the way the dummy recipes are currently being handled. Unfortunately this change still isn't without issues, this is shown by the failure in meta-virtualization it introduces. It is starting to feel like I'm somehow targeting Bruce recently, sorry! The failure this patchset triggers in meta-virtualization is here: https://autobuilder.yoctoproject.org/valkyrie/#/builders/89/builds/3215 The reason is that it is doing this: recipes-core/meta/container-dummy-provides.bb:require recipes-core/meta/dummy-sdk-package.inc which is fine, I hadn't realised layers were using that but it should be something you can do. At first glance, the fix looks easy, just add: DUMMYSDK_PKGDATA_VARNAME = "PKGDATA_DIR" DUMMYSDK_EXTRASTAMP_VARNAME = "MACHINE" to the recipe. That doesn't account for the sstate.bbclass and sstatesig.py changes this patch is making though. I've never been happy at having to add these individually to the "feed" lists in the first place and another layer trying to do this illustrates why. I'm at a bit of a loss on what to do from here. Options could be: a) move the recipe to core and add another entry to sstate.bbclass/sstatesig.py . That doesn't help it anyone else does this b) Create some layer.conf variables which allow these to be defined/added generically. That would need careful manipulation of the hash variable dependencies to stop things rebuilding like crazy. c) Find a way for the core not to need this list hardcoded. That would be nicer and removes what looks like a horrible scaling issue we have right now I don't know how to do that. d) Something else I'm open to ideas, these changes are needed to unblock the spdx changes. Cheers, Richard
On Sat, 2026-03-14 at 10:00 +0000, Richard Purdie via lists.openembedded.org wrote: > Options could be: > > a) move the recipe to core and add another entry to > sstate.bbclass/sstatesig.py . That doesn't help it anyone else does > this > > b) Create some layer.conf variables which allow these to be > defined/added generically. That would need careful manipulation of the > hash variable dependencies to stop things rebuilding like crazy. Having said this all out loud, b) is the only realistic option so I've just sent a patch which does that. I don't like it but it is the least worst option I can see right now. Cheers, Richard
On Sat, Mar 14, 2026 at 6:00 AM Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > On Sat, 2026-03-14 at 09:47 +0000, Richard Purdie via > lists.openembedded.org wrote: > > Currently, the dummy SDK packages are re-running for different > SDKMACHINE values > > when they should not. The usage of allarch is broken and not triggering > the right > > PACKAGE_ARCH value due to the deferred nature of nativesdk. This was > probably > > broken when we switched to add deferred classes. > > > > To try and make this all more explict and less prone to breakage, switch > to calling > > oe.utils.make_arch_independent() directly. > > > > Add the 'special' package architecture values to SSTATE_ARCHS so the > system cna properly > > track them. > > > > Remove the pointless tasks we don't need from the dummy recipes, mark > the packagedata > > as machine independent and then remove from the conflict list in > sstate.bbclass. > > > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > This patch fixes real issues with the way the dummy recipes are > currently being handled. Unfortunately this change still isn't without > issues, this is shown by the failure in meta-virtualization it > introduces. It is starting to feel like I'm somehow targeting Bruce > recently, sorry! > It's probably more that I'm doing strange and niche things :) I see your patch, I'll pull it in shortly, thanks for the fixup, I would have taken ages to figure that out! Bruce > > The failure this patchset triggers in meta-virtualization is here: > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/89/builds/3215 > > The reason is that it is doing this: > > recipes-core/meta/container-dummy-provides.bb:require > recipes-core/meta/dummy-sdk-package.inc > > which is fine, I hadn't realised layers were using that but it should > be something you can do. At first glance, the fix looks easy, just add: > > DUMMYSDK_PKGDATA_VARNAME = "PKGDATA_DIR" > DUMMYSDK_EXTRASTAMP_VARNAME = "MACHINE" > > to the recipe. That doesn't account for the sstate.bbclass and > sstatesig.py changes this patch is making though. > > I've never been happy at having to add these individually to the "feed" > lists in the first place and another layer trying to do this > illustrates why. > > I'm at a bit of a loss on what to do from here. Options could be: > > a) move the recipe to core and add another entry to > sstate.bbclass/sstatesig.py . That doesn't help it anyone else does > this > > b) Create some layer.conf variables which allow these to be > defined/added generically. That would need careful manipulation of the > hash variable dependencies to stop things rebuilding like crazy. > > c) Find a way for the core not to need this list hardcoded. That would > be nicer and removes what looks like a horrible scaling issue we have > right now I don't know how to do that. > > d) Something else > > I'm open to ideas, these changes are needed to unblock the spdx > changes. > > Cheers, > > Richard >
diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass index 3b2e307134e..fec4d88c4cf 100644 --- a/meta/classes-global/sstate.bbclass +++ b/meta/classes-global/sstate.bbclass @@ -9,7 +9,7 @@ SSTATE_VERSION = "14" SSTATE_ZSTD_CLEVEL ??= "8" SSTATE_MANIFESTS ?= "${TMPDIR}/sstate-control" -SSTATE_MANFILEPREFIX = "${SSTATE_MANIFESTS}/manifest-${SSTATE_MANMACH}-${PN}" +SSTATE_MANFILEPREFIX = "${SSTATE_MANIFESTS}/manifest-${SSTATE_PKGARCH}-${PN}" def generate_sstatefn(spec, hash, taskname, siginfo, d): if taskname is None: @@ -96,8 +96,6 @@ SSTATE_ARCHS = " \ ${MACHINE_ARCH}" SSTATE_ARCHS[vardepsexclude] = "ORIGNATIVELSBSTRING" -SSTATE_MANMACH ?= "${SSTATE_PKGARCH}" - SSTATECREATEFUNCS += "sstate_hardcode_path" SSTATECREATEFUNCS[vardeps] = "SSTATE_SCAN_FILES" SSTATEPOSTCREATEFUNCS = "" @@ -148,8 +146,7 @@ python () { d.setVar('SSTATE_PKGARCH', d.expand("${SDK_ARCH}_${PACKAGE_ARCH}")) elif bb.data.inherits_class('allarch', d) and d.getVar("PACKAGE_ARCH") == "all": d.setVar('SSTATE_PKGARCH', "allarch") - else: - d.setVar('SSTATE_MANMACH', d.expand("${PACKAGE_ARCH}")) + # Fall back to the default of SSTATE_PKGARCH=PACKAGE_ARCH if bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d): d.setVar('SSTATE_EXTRAPATH', "${NATIVELSBSTRING}/") @@ -320,7 +317,7 @@ def sstate_install(ss, d): # Append to the list of manifests for this PACKAGE_ARCH - i = d2.expand("${SSTATE_MANIFESTS}/index-${SSTATE_MANMACH}") + i = d2.expand("${SSTATE_MANIFESTS}/index-${SSTATE_PKGARCH}") l = bb.utils.lockfile(i + ".lock") filedata = d.getVar("STAMP") + " " + d2.getVar("SSTATE_MANFILEPREFIX") + " " + d.getVar("WORKDIR") + "\n" manifests = [] @@ -353,7 +350,7 @@ def sstate_install(ss, d): for lock in locks: bb.utils.unlockfile(lock) -sstate_install[vardepsexclude] += "SSTATE_ALLOW_OVERLAP_FILES SSTATE_MANMACH SSTATE_MANFILEPREFIX STAMP" +sstate_install[vardepsexclude] += "SSTATE_ALLOW_OVERLAP_FILES SSTATE_PKGARCH SSTATE_MANFILEPREFIX STAMP" def sstate_installpkg(ss, d): from oe.gpg_sign import get_signer @@ -520,7 +517,7 @@ def sstate_clean(ss, d): stamp_clean = d.getVar("STAMPCLEAN") extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info') if extrainf: - d2.setVar("SSTATE_MANMACH", extrainf) + d2.setVar("SSTATE_PKGARCH", extrainf) wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf) else: wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task']) diff --git a/meta/classes/toaster.bbclass b/meta/classes/toaster.bbclass index af7c4578088..10c728885ae 100644 --- a/meta/classes/toaster.bbclass +++ b/meta/classes/toaster.bbclass @@ -343,15 +343,15 @@ python toaster_artifacts() { d2 = d.createCopy() d2.setVar('FILE', e.taskfile) # Use 'stamp-extra-info' if present, else use workaround - # to determine 'SSTATE_MANMACH' + # to determine 'SSTATE_PKGARCH' extrainf = d2.getVarFlag(e.taskname, 'stamp-extra-info') if extrainf: - d2.setVar('SSTATE_MANMACH', extrainf) + d2.setVar('SSTATE_PKGARCH', extrainf) else: if "do_populate_sdk" == e.taskname: - d2.setVar('SSTATE_MANMACH', d2.expand("${MACHINE}${SDKMACHINE}")) + d2.setVar('SSTATE_PKGARCH', d2.expand("${MACHINE}${SDKMACHINE}")) else: - d2.setVar('SSTATE_MANMACH', d2.expand("${MACHINE}")) + d2.setVar('SSTATE_PKGARCH', d2.expand("${MACHINE}")) manifest = oe.sstatesig.sstate_get_manifest_filename(e.taskname[3:], d2)[0] if os.access(manifest, os.R_OK): diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py index ef687f5d417..1e07c71e944 100644 --- a/meta/lib/oe/sstatesig.py +++ b/meta/lib/oe/sstatesig.py @@ -491,7 +491,7 @@ def sstate_get_manifest_filename(task, d): d2 = d.createCopy() extrainf = d.getVarFlag("do_" + task, 'stamp-extra-info') if extrainf: - d2.setVar("SSTATE_MANMACH", extrainf) + d2.setVar("SSTATE_PKGARCH", extrainf) return (d2.expand("${SSTATE_MANFILEPREFIX}.%s" % task), d2) @bb.parse.vardepsexclude("BBEXTENDCURR", "BBEXTENDVARIANT", "OVERRIDES", "PACKAGE_EXTRA_ARCHS")
This variable doesn't appear needed and just confuses the code, remove it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- meta/classes-global/sstate.bbclass | 13 +++++-------- meta/classes/toaster.bbclass | 8 ++++---- meta/lib/oe/sstatesig.py | 2 +- 3 files changed, 10 insertions(+), 13 deletions(-)