| Message ID | 20260804220456.2342710-4-alhe@linux.microsoft.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [1/4] libtool: strip build system triplet from installed libtool script | expand |
On Wed Aug 5, 2026 at 12:04 AM CEST, Alejandro Hernandez Samaniego via lists.openembedded.org wrote: > RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which > tells bitbake that rust recipes should hash-match regardless of the > build machine's triplet. In practice that is not true: > > rustc computes each crate's Strict Version Hash (SVH) using inputs > that include the *stage0/stage1 bootstrap compiler* fingerprint, which > in turn depends on the build host arch. That seed cascades through > every dependent crate's mangled symbols and the packing order of > rodata sections, so an sstate blob populated on one worker arch and > reused on a differently-arched worker produces byte-different (but > semantically identical) artifacts and fails the reproducibility > selftest causing autobuilder intermittent issues when the sstate > matches for the incorrect architecture. > > Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash > tracks the build triplet and mixed-arch autobuilder pools get an > sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and > RUST_TARGET_SYS stay excluded because they're already covered by the > target/host arch hash inputs. > > While this is not ideal, it should unblock the reproducible test case, > another solution would be to patch rust sources manually and attempt > to upstream that change. > > This also has the side-effect that multiple variants of the conflicting > rust recipes sstate artifacts are created, but they'll be correctly used > now in each architecture. > > Also add an explanatory comment next to the RUST_*_SYS[vardepvalue] > declarations in rust-common.bbclass so future readers don't re-add > the exclusion. > > Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces > different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto- > graphy/cargo/librsvg, where previously the hashes matched despite the > build machine change. > > [YOCTO #15554] > > Assisted-by: AI - OpenAI > Signed-off-by: Alejandro Hernandez <alhe@linux.microsoft.com> > --- Hi Alejandro, It looks like this is breaking some selftests: 2026-08-05 10:11:05,494 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_sdk_arch_same_hash (subunit.RemotedTestCase) 2026-08-05 10:11:05,494 - oe-selftest - INFO - ... FAIL ... 2026-08-05 10:11:05,495 - oe-selftest - INFO - 6: 27/69 496/763 (108.24s) (2 failed) (sstatetests.SStateHashSameSigs.test_sstate_sdk_arch_same_hash) 2026-08-05 10:11:05,495 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 416, in test_sstate_sdk_arch_same_hash self.sstate_hashtest("aarch64") ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^ File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f return func(*args, **kwargs) File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 401, in sstate_hashtest self.assertCountEqual(files1, files2) ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/unittest/case.py", line 1238, in assertCountEqual self.fail(msg) ~~~~~~~~~^^^^^ File "/usr/lib/python3.13/unittest/case.py", line 732, in fail raise self.failureException(msg) AssertionError: Element counts were not equal: First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_create_spdx.sigdata.469683d7e676bd4feadb3e363efaa9a55c47eb06559fcda0d3f5cf26473919c5' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_populate_sysroot.sigdata.c29ff418823da4b52189a4cf7395c310a3c14fed57eb24562cd0e387fe2235a3' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_create_package_spdx.sigdata.ede62d86abb84a871e239607d0a7173c6a9c45027aafffc624b37e2e1bb5111f' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_create_spdx.sigdata.76ed6652ba8e8eba4e8750240fa7a048fcc90e94b45e753068a44560ef2e29f0' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_create_package_spdx.sigdata.4234a33aff5f4b52703a74c1caec1893206476f908c70b32c78e46dcc3e4fb38' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_compile.sigdata.67ed98cf74ac06f829d6432f4fd904779b2f6e540f588baeab3dcf0ac35a9c75' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_populate_sysroot.sigdata.3203f6ea1fd6bb92ed523ba0e1ef0464166f1e5392207ea002c35e0007d435f9' First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_configure.sigdata.df7cc5717b4cbc2ccb19bdb823fb55193cc2f95049ac7e07d440bc2da5590c96' ... And same for sstatetests.SStateHashSameSigs.test_sstate_32_64_same_hash: 2026-08-05 10:07:29,974 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_32_64_same_hash (subunit.RemotedTestCase) 2026-08-05 10:07:29,975 - oe-selftest - INFO - ... FAIL ... https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4472 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4290 Can you have a look at the issue? Thanks, Mathieu
On Tue, 2026-08-04 at 22:04 +0000, Alejandro Hernandez Samaniego via lists.openembedded.org wrote: > RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which > tells bitbake that rust recipes should hash-match regardless of the > build machine's triplet. In practice that is not true: > > rustc computes each crate's Strict Version Hash (SVH) using inputs > that include the *stage0/stage1 bootstrap compiler* fingerprint, which > in turn depends on the build host arch. That seed cascades through > every dependent crate's mangled symbols and the packing order of > rodata sections, so an sstate blob populated on one worker arch and > reused on a differently-arched worker produces byte-different (but > semantically identical) artifacts and fails the reproducibility > selftest causing autobuilder intermittent issues when the sstate > matches for the incorrect architecture. > > Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash > tracks the build triplet and mixed-arch autobuilder pools get an > sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and > RUST_TARGET_SYS stay excluded because they're already covered by the > target/host arch hash inputs. > > While this is not ideal, it should unblock the reproducible test case, > another solution would be to patch rust sources manually and attempt > to upstream that change. > > This also has the side-effect that multiple variants of the conflicting > rust recipes sstate artifacts are created, but they'll be correctly used > now in each architecture. > > Also add an explanatory comment next to the RUST_*_SYS[vardepvalue] > declarations in rust-common.bbclass so future readers don't re-add > the exclusion. > > Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces > different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto- > graphy/cargo/librsvg, where previously the hashes matched despite the > build machine change. > > [YOCTO #15554] > > Assisted-by: AI - OpenAI > Signed-off-by: Alejandro Hernandez <alhe@linux.microsoft.com> > --- > meta/classes-recipe/rust-common.bbclass | 10 ++++++++++ > meta/conf/bitbake.conf | 2 +- > 2 files changed, 11 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass > index 6bc42016d1..98b46c4e3c 100644 > --- a/meta/classes-recipe/rust-common.bbclass > +++ b/meta/classes-recipe/rust-common.bbclass > @@ -114,6 +114,16 @@ RUST_HOST_SYS[vardepvalue] = "${RUST_HOST_SYS}" > RUST_TARGET_SYS = "${@rust_base_triple(d, 'TARGET')}" > RUST_TARGET_SYS[vardepvalue] = "${RUST_TARGET_SYS}" > > +# Note: RUST_BUILD_SYS is intentionally NOT on BB_BASEHASH_IGNORE_VARS > +# (see meta/conf/bitbake.conf). rustc's crate SVH (Strict Version Hash) > +# is seeded by the stage0/stage1 bootstrap compiler whose fingerprint > +# depends on the BUILD host arch; that seed cascades through every > +# dependent crate's mangled symbols and rodata packing order. Excluding > +# RUST_BUILD_SYS from task hashes let a mixed-arch autobuilder pool > +# populate sstate on one worker arch and get a cache hit on a differently- > +# arched worker, producing byte-different (but semantically identical) > +# artifacts and failing reproducibility tests. See Yocto bug #15554. > + > # wrappers to get around the fact that Rust needs a single > # binary but Yocto's compiler and linker commands have > # arguments. Technically the archiver is always one command but > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > index bdf37d0da2..ee23459506 100644 > --- a/meta/conf/bitbake.conf > +++ b/meta/conf/bitbake.conf > @@ -969,7 +969,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI > SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \ > OMP_NUM_THREADS BB_CURRENTTASK" > BB_BASEHASH_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_INCLUDE_PATHS BUILDHISTORY_DIR \ > - SSTATE_DIR SOURCE_DATE_EPOCH RUST_BUILD_SYS RUST_HOST_SYS RUST_TARGET_SYS" > + SSTATE_DIR SOURCE_DATE_EPOCH RUST_HOST_SYS RUST_TARGET_SYS" > BB_HASHCONFIG_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \ > SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_PASSTHROUGH_ADDITIONS DISABLE_SANITY_CHECKS \ > PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \ Just to be clear, the sstatetests fail for this change for good reason, it breaks the way "native" works in our system. We work on the principle that "native" things work the same way regardless of architecture and that the sstate hashes remain the same. If you change this as above all of the rust targets will rebuild depending on the build architecture, despite the fact they're meant to be build architecture independent. It might manage to fool the test but the output would still not be build architecture independent and hence doesn't fix the real issue, just hides it from the tests. Cheers, Richard
On 8/6/2026 6:53 AM, Richard Purdie via lists.openembedded.org wrote: > On Tue, 2026-08-04 at 22:04 +0000, Alejandro Hernandez Samaniego via lists.openembedded.org wrote: >> RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which >> tells bitbake that rust recipes should hash-match regardless of the >> build machine's triplet. In practice that is not true: >> >> rustc computes each crate's Strict Version Hash (SVH) using inputs >> that include the *stage0/stage1 bootstrap compiler* fingerprint, which >> in turn depends on the build host arch. That seed cascades through >> every dependent crate's mangled symbols and the packing order of >> rodata sections, so an sstate blob populated on one worker arch and >> reused on a differently-arched worker produces byte-different (but >> semantically identical) artifacts and fails the reproducibility >> selftest causing autobuilder intermittent issues when the sstate >> matches for the incorrect architecture. >> >> Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash >> tracks the build triplet and mixed-arch autobuilder pools get an >> sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and >> RUST_TARGET_SYS stay excluded because they're already covered by the >> target/host arch hash inputs. >> >> While this is not ideal, it should unblock the reproducible test case, >> another solution would be to patch rust sources manually and attempt >> to upstream that change. >> >> This also has the side-effect that multiple variants of the conflicting >> rust recipes sstate artifacts are created, but they'll be correctly used >> now in each architecture. >> >> Also add an explanatory comment next to the RUST_*_SYS[vardepvalue] >> declarations in rust-common.bbclass so future readers don't re-add >> the exclusion. >> >> Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces >> different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto- >> graphy/cargo/librsvg, where previously the hashes matched despite the >> build machine change. >> >> [YOCTO #15554] >> >> Assisted-by: AI - OpenAI >> Signed-off-by: Alejandro Hernandez<alhe@linux.microsoft.com> >> --- >> meta/classes-recipe/rust-common.bbclass | 10 ++++++++++ >> meta/conf/bitbake.conf | 2 +- >> 2 files changed, 11 insertions(+), 1 deletion(-) >> >> diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass >> index 6bc42016d1..98b46c4e3c 100644 >> --- a/meta/classes-recipe/rust-common.bbclass >> +++ b/meta/classes-recipe/rust-common.bbclass >> @@ -114,6 +114,16 @@ RUST_HOST_SYS[vardepvalue] = "${RUST_HOST_SYS}" >> RUST_TARGET_SYS = "${@rust_base_triple(d, 'TARGET')}" >> RUST_TARGET_SYS[vardepvalue] = "${RUST_TARGET_SYS}" >> >> +# Note: RUST_BUILD_SYS is intentionally NOT on BB_BASEHASH_IGNORE_VARS >> +# (see meta/conf/bitbake.conf). rustc's crate SVH (Strict Version Hash) >> +# is seeded by the stage0/stage1 bootstrap compiler whose fingerprint >> +# depends on the BUILD host arch; that seed cascades through every >> +# dependent crate's mangled symbols and rodata packing order. Excluding >> +# RUST_BUILD_SYS from task hashes let a mixed-arch autobuilder pool >> +# populate sstate on one worker arch and get a cache hit on a differently- >> +# arched worker, producing byte-different (but semantically identical) >> +# artifacts and failing reproducibility tests. See Yocto bug #15554. >> + >> # wrappers to get around the fact that Rust needs a single >> # binary but Yocto's compiler and linker commands have >> # arguments. Technically the archiver is always one command but >> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf >> index bdf37d0da2..ee23459506 100644 >> --- a/meta/conf/bitbake.conf >> +++ b/meta/conf/bitbake.conf >> @@ -969,7 +969,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI >> SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \ >> OMP_NUM_THREADS BB_CURRENTTASK" >> BB_BASEHASH_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_INCLUDE_PATHS BUILDHISTORY_DIR \ >> - SSTATE_DIR SOURCE_DATE_EPOCH RUST_BUILD_SYS RUST_HOST_SYS RUST_TARGET_SYS" >> + SSTATE_DIR SOURCE_DATE_EPOCH RUST_HOST_SYS RUST_TARGET_SYS" >> BB_HASHCONFIG_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \ >> SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_PASSTHROUGH_ADDITIONS DISABLE_SANITY_CHECKS \ >> PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \ > Just to be clear, the sstatetests fail for this change for good reason, > it breaks the way "native" works in our system. We work on the > principle that "native" things work the same way regardless of > architecture and that the sstate hashes remain the same. > > If you change this as above all of the rust targets will rebuild > depending on the build architecture, despite the fact they're meant to > be build architecture independent. It might manage to fool the test but > the output would still not be build architecture independent and hence > doesn't fix the real issue, just hides it from the tests. > > Cheers, > > Richard Yes, that makes sense, the good thing is I think the root cause is correct, but if we instead patch rusts sources we'll end up with the same problem since the packages would also be arch independent, correct? Is the correct thing here to add packages to an ignore/allow list? Alejandro > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#242942):https://lists.openembedded.org/g/openembedded-core/message/242942 > Mute This Topic:https://lists.openembedded.org/mt/120602088/4354175 > Group Owner:openembedded-core+owner@lists.openembedded.org > Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [alhe@linux.microsoft.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 8/5/2026 11:58 PM, Mathieu Dubois-Briand via lists.openembedded.org wrote: > On Wed Aug 5, 2026 at 12:04 AM CEST, Alejandro Hernandez Samaniego via lists.openembedded.org wrote: >> RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which >> tells bitbake that rust recipes should hash-match regardless of the >> build machine's triplet. In practice that is not true: >> >> rustc computes each crate's Strict Version Hash (SVH) using inputs >> that include the *stage0/stage1 bootstrap compiler* fingerprint, which >> in turn depends on the build host arch. That seed cascades through >> every dependent crate's mangled symbols and the packing order of >> rodata sections, so an sstate blob populated on one worker arch and >> reused on a differently-arched worker produces byte-different (but >> semantically identical) artifacts and fails the reproducibility >> selftest causing autobuilder intermittent issues when the sstate >> matches for the incorrect architecture. >> >> Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash >> tracks the build triplet and mixed-arch autobuilder pools get an >> sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and >> RUST_TARGET_SYS stay excluded because they're already covered by the >> target/host arch hash inputs. >> >> While this is not ideal, it should unblock the reproducible test case, >> another solution would be to patch rust sources manually and attempt >> to upstream that change. >> >> This also has the side-effect that multiple variants of the conflicting >> rust recipes sstate artifacts are created, but they'll be correctly used >> now in each architecture. >> >> Also add an explanatory comment next to the RUST_*_SYS[vardepvalue] >> declarations in rust-common.bbclass so future readers don't re-add >> the exclusion. >> >> Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces >> different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto- >> graphy/cargo/librsvg, where previously the hashes matched despite the >> build machine change. >> >> [YOCTO #15554] >> >> Assisted-by: AI - OpenAI >> Signed-off-by: Alejandro Hernandez<alhe@linux.microsoft.com> >> --- > Hi Alejandro, > > It looks like this is breaking some selftests: > > 2026-08-05 10:11:05,494 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_sdk_arch_same_hash (subunit.RemotedTestCase) > 2026-08-05 10:11:05,494 - oe-selftest - INFO - ... FAIL > ... > > 2026-08-05 10:11:05,495 - oe-selftest - INFO - 6: 27/69 496/763 (108.24s) (2 failed) (sstatetests.SStateHashSameSigs.test_sstate_sdk_arch_same_hash) > 2026-08-05 10:11:05,495 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): > File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 416, in test_sstate_sdk_arch_same_hash > self.sstate_hashtest("aarch64") > ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^ > File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f > return func(*args, **kwargs) > File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 401, in sstate_hashtest > self.assertCountEqual(files1, files2) > ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ > File "/usr/lib/python3.13/unittest/case.py", line 1238, in assertCountEqual > self.fail(msg) > ~~~~~~~~~^^^^^ > File "/usr/lib/python3.13/unittest/case.py", line 732, in fail > raise self.failureException(msg) > AssertionError: Element counts were not equal: > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_create_spdx.sigdata.469683d7e676bd4feadb3e363efaa9a55c47eb06559fcda0d3f5cf26473919c5' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_populate_sysroot.sigdata.c29ff418823da4b52189a4cf7395c310a3c14fed57eb24562cd0e387fe2235a3' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_create_package_spdx.sigdata.ede62d86abb84a871e239607d0a7173c6a9c45027aafffc624b37e2e1bb5111f' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_create_spdx.sigdata.76ed6652ba8e8eba4e8750240fa7a048fcc90e94b45e753068a44560ef2e29f0' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_create_package_spdx.sigdata.4234a33aff5f4b52703a74c1caec1893206476f908c70b32c78e46dcc3e4fb38' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_compile.sigdata.67ed98cf74ac06f829d6432f4fd904779b2f6e540f588baeab3dcf0ac35a9c75' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_populate_sysroot.sigdata.3203f6ea1fd6bb92ed523ba0e1ef0464166f1e5392207ea002c35e0007d435f9' > First has 1, Second has 0: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_configure.sigdata.df7cc5717b4cbc2ccb19bdb823fb55193cc2f95049ac7e07d440bc2da5590c96' > ... > > And same for sstatetests.SStateHashSameSigs.test_sstate_32_64_same_hash: > 2026-08-05 10:07:29,974 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_32_64_same_hash (subunit.RemotedTestCase) > 2026-08-05 10:07:29,975 - oe-selftest - INFO - ... FAIL > ... > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4472 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4290 > > Can you have a look at the issue? > > Thanks, > Mathieu Thanks Mathieu, yes this actually make sense, this is why I wasn't too convinced since the beginning and wanted to send an RFC, I'll attempt to fix this in some other way. Alejandro > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#242899):https://lists.openembedded.org/g/openembedded-core/message/242899 > Mute This Topic:https://lists.openembedded.org/mt/120602088/4354175 > Group Owner:openembedded-core+owner@lists.openembedded.org > Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [alhe@linux.microsoft.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Thu, 2026-08-06 at 09:50 -0600, Alejandro Hernandez wrote: > > On 8/6/2026 6:53 AM, Richard Purdie via lists.openembedded.org wrote: > > > On Tue, 2026-08-04 at 22:04 +0000, Alejandro Hernandez Samaniego via lists.openembedded.org wrote: > > > RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which > > > tells bitbake that rust recipes should hash-match regardless of the > > > build machine's triplet. In practice that is not true: > > > > > > rustc computes each crate's Strict Version Hash (SVH) using inputs > > > that include the *stage0/stage1 bootstrap compiler* fingerprint, which > > > in turn depends on the build host arch. That seed cascades through > > > every dependent crate's mangled symbols and the packing order of > > > rodata sections, so an sstate blob populated on one worker arch and > > > reused on a differently-arched worker produces byte-different (but > > > semantically identical) artifacts and fails the reproducibility > > > selftest causing autobuilder intermittent issues when the sstate > > > matches for the incorrect architecture. > > > > > > Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash > > > tracks the build triplet and mixed-arch autobuilder pools get an > > > sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and > > > RUST_TARGET_SYS stay excluded because they're already covered by the > > > target/host arch hash inputs. > > > > > > While this is not ideal, it should unblock the reproducible test case, > > > another solution would be to patch rust sources manually and attempt > > > to upstream that change. > > > > > > This also has the side-effect that multiple variants of the conflicting > > > rust recipes sstate artifacts are created, but they'll be correctly used > > > now in each architecture. > > > > > > Also add an explanatory comment next to the RUST_*_SYS[vardepvalue] > > > declarations in rust-common.bbclass so future readers don't re-add > > > the exclusion. > > > > > > Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces > > > different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto- > > > graphy/cargo/librsvg, where previously the hashes matched despite the > > > build machine change. > > > > > > [YOCTO #15554] > > > > > > Assisted-by: AI - OpenAI > > > Signed-off-by: Alejandro Hernandez <alhe@linux.microsoft.com> > > > --- > > > meta/classes-recipe/rust-common.bbclass | 10 ++++++++++ > > > meta/conf/bitbake.conf | 2 +- > > > 2 files changed, 11 insertions(+), 1 deletion(-) > > > > > > diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass > > > index 6bc42016d1..98b46c4e3c 100644 > > > --- a/meta/classes-recipe/rust-common.bbclass > > > +++ b/meta/classes-recipe/rust-common.bbclass > > > @@ -114,6 +114,16 @@ RUST_HOST_SYS[vardepvalue] = "${RUST_HOST_SYS}" > > > RUST_TARGET_SYS = "${@rust_base_triple(d, 'TARGET')}" > > > RUST_TARGET_SYS[vardepvalue] = "${RUST_TARGET_SYS}" > > > > > > +# Note: RUST_BUILD_SYS is intentionally NOT on BB_BASEHASH_IGNORE_VARS > > > +# (see meta/conf/bitbake.conf). rustc's crate SVH (Strict Version Hash) > > > +# is seeded by the stage0/stage1 bootstrap compiler whose fingerprint > > > +# depends on the BUILD host arch; that seed cascades through every > > > +# dependent crate's mangled symbols and rodata packing order. Excluding > > > +# RUST_BUILD_SYS from task hashes let a mixed-arch autobuilder pool > > > +# populate sstate on one worker arch and get a cache hit on a differently- > > > +# arched worker, producing byte-different (but semantically identical) > > > +# artifacts and failing reproducibility tests. See Yocto bug #15554. > > > + > > > # wrappers to get around the fact that Rust needs a single > > > # binary but Yocto's compiler and linker commands have > > > # arguments. Technically the archiver is always one command but > > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > > > index bdf37d0da2..ee23459506 100644 > > > --- a/meta/conf/bitbake.conf > > > +++ b/meta/conf/bitbake.conf > > > @@ -969,7 +969,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI > > > SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \ > > > OMP_NUM_THREADS BB_CURRENTTASK" > > > BB_BASEHASH_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_INCLUDE_PATHS BUILDHISTORY_DIR \ > > > - SSTATE_DIR SOURCE_DATE_EPOCH RUST_BUILD_SYS RUST_HOST_SYS RUST_TARGET_SYS" > > > + SSTATE_DIR SOURCE_DATE_EPOCH RUST_HOST_SYS RUST_TARGET_SYS" > > > BB_HASHCONFIG_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \ > > > SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_PASSTHROUGH_ADDITIONS DISABLE_SANITY_CHECKS \ > > > PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \ > > > > > > > > > > > > Just to be clear, the sstatetests fail for this change for good reason, > > it breaks the way "native" works in our system. We work on the > > principle that "native" things work the same way regardless of > > architecture and that the sstate hashes remain the same. > > > > If you change this as above all of the rust targets will rebuild > > depending on the build architecture, despite the fact they're meant to > > be build architecture independent. It might manage to fool the test but > > the output would still not be build architecture independent and hence > > doesn't fix the real issue, just hides it from the tests. > > Yes, that makes sense, the good thing is I think the root cause is correct, but if we instead patch rusts sources > we'll end up with the same problem since the packages would also be arch independent, correct? > > Is the correct thing here to add packages to an ignore/allow list? We probably need to patch rust itself not to put the host information into the SVH. Cheers, Richard
diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass index 6bc42016d1..98b46c4e3c 100644 --- a/meta/classes-recipe/rust-common.bbclass +++ b/meta/classes-recipe/rust-common.bbclass @@ -114,6 +114,16 @@ RUST_HOST_SYS[vardepvalue] = "${RUST_HOST_SYS}" RUST_TARGET_SYS = "${@rust_base_triple(d, 'TARGET')}" RUST_TARGET_SYS[vardepvalue] = "${RUST_TARGET_SYS}" +# Note: RUST_BUILD_SYS is intentionally NOT on BB_BASEHASH_IGNORE_VARS +# (see meta/conf/bitbake.conf). rustc's crate SVH (Strict Version Hash) +# is seeded by the stage0/stage1 bootstrap compiler whose fingerprint +# depends on the BUILD host arch; that seed cascades through every +# dependent crate's mangled symbols and rodata packing order. Excluding +# RUST_BUILD_SYS from task hashes let a mixed-arch autobuilder pool +# populate sstate on one worker arch and get a cache hit on a differently- +# arched worker, producing byte-different (but semantically identical) +# artifacts and failing reproducibility tests. See Yocto bug #15554. + # wrappers to get around the fact that Rust needs a single # binary but Yocto's compiler and linker commands have # arguments. Technically the archiver is always one command but diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index bdf37d0da2..ee23459506 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -969,7 +969,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \ OMP_NUM_THREADS BB_CURRENTTASK" BB_BASEHASH_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_INCLUDE_PATHS BUILDHISTORY_DIR \ - SSTATE_DIR SOURCE_DATE_EPOCH RUST_BUILD_SYS RUST_HOST_SYS RUST_TARGET_SYS" + SSTATE_DIR SOURCE_DATE_EPOCH RUST_HOST_SYS RUST_TARGET_SYS" BB_HASHCONFIG_IGNORE_VARS ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \ SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_PASSTHROUGH_ADDITIONS DISABLE_SANITY_CHECKS \ PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \
RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which tells bitbake that rust recipes should hash-match regardless of the build machine's triplet. In practice that is not true: rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch. That seed cascades through every dependent crate's mangled symbols and the packing order of rodata sections, so an sstate blob populated on one worker arch and reused on a differently-arched worker produces byte-different (but semantically identical) artifacts and fails the reproducibility selftest causing autobuilder intermittent issues when the sstate matches for the incorrect architecture. Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash tracks the build triplet and mixed-arch autobuilder pools get an sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and RUST_TARGET_SYS stay excluded because they're already covered by the target/host arch hash inputs. While this is not ideal, it should unblock the reproducible test case, another solution would be to patch rust sources manually and attempt to upstream that change. This also has the side-effect that multiple variants of the conflicting rust recipes sstate artifacts are created, but they'll be correctly used now in each architecture. Also add an explanatory comment next to the RUST_*_SYS[vardepvalue] declarations in rust-common.bbclass so future readers don't re-add the exclusion. Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto- graphy/cargo/librsvg, where previously the hashes matched despite the build machine change. [YOCTO #15554] Assisted-by: AI - OpenAI Signed-off-by: Alejandro Hernandez <alhe@linux.microsoft.com> --- meta/classes-recipe/rust-common.bbclass | 10 ++++++++++ meta/conf/bitbake.conf | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-)