| Message ID | d87597ebbafa1ec77c21c72c9d3183e00ad9985b.1734553652.git.steve@sakoman.com |
|---|---|
| State | RFC |
| Delegated to: | Steve Sakoman |
| Headers | show |
| Series | [scarthgap,01/10] binutils: Fix CVE-2024-53589 | expand |
Small reminder for people doing incremental builds and having similarly short memory as me. This might cause errors like: error[E0464]: multiple candidates for `rmeta` dependency `core` found | = note: candidate #1: lib32-python3-bcrypt/4.1.2/lib32-recipe-sysroot/usr/lib/rustlib/armv7-webosmllib32-linux-gnueabi/lib/libcore-0efa44efb479c482.rmeta = note: candidate #2: lib32-python3-bcrypt/4.1.2/lib32-recipe-sysroot/usr/lib/rustlib/armv7-webosmllib32-linux-gnueabi/lib/libcore-6dc2d356d81e0fd8.rmeta in all recipes using rust. If you see this, then easiest work around is to cleansstate libstd-rs (lib32-libstd-rs with multilib) and all failing recipes and start again. Ideally help to fix it properly. From: #yocto/2024-05-29.log 13:24 < JaMa> anyone seeing "error[E0464]: multiple candidates for `rmeta` dependency `core` found" for various dependencies in rust recipes (after the update from yesterday), libstd-rs now provides multiple versions in incremental build (unless you explicitly rebuild from scratch after -c cleansstate) kanavin_ have you seen this before? 13:24 < JaMa> buildhistory-diff -p buildhistory 13:24 < JaMa> packages/raspberrypi4_64-webos-linux/libstd-rs/libstd-rs-dev: PKGSIZE changed from 304061571 to 160359626 (-47%) 13:24 < kanavin_> JaMa, incremental builds in rust are broken 13:25 < JaMa> whole buildhistory-diff after rebuilding with cleansstate https://dpaste.com/EZGZ4PAYT 13:25 < kanavin_> it doesn't clean itself properly 13:25 < JaMa> ok, haven't seen this one before, so I guess I was lucky, thanks 13:36 < RP> kanavin_: do we know what isn't cleaned properly in the rust builds? 13:42 < kanavin_> RP: I'm not sure. If JaMa can provide a reproducer I can take a look. 14:07 < JaMa> in my case today it was libstd-rs (I had to cleansstate all its users as well after rebuilding it, but the culprint was duplicated files installed and staged by libstd-rs) 14:08 < RP> I've seen this kind of issue before but couldn't reproduce it :/ 14:09 < JaMa> I'll try, because I'm pretty sure I've built libstd-rs from scratch yesterday and today it was rebuilt because of the last 4 commits currently in master (so I guess if I checkout HEAD~5, build libstd-rs, checkout master, build it again than I might see the issue) 14:28 < JaMa> hmm this trivial reproducer didn't work 14:29 < JaMa> libstd-rs-1.75.0-r0 was rebuilt from do_fetch after updating oe-core (and cleaning dirs while doing so) 14:58 < RP> JaMa: sounds like what I ran into last time :/ Cheers, On Wed, Dec 18, 2024 at 11:03 PM Steve Sakoman via lists.openembedded.org <steve=sakoman.com@lists.openembedded.org> wrote: > > From: Alexander Kanavin <alex@linutronix.de> > > [YOCTO #15185] > > Signed-off-by: Alexander Kanavin <alex@linutronix.de> > Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > (cherry picked from commit 924df18b47e9a69fa295bafe37bdb39d8eaea2bb) > Signed-off-by: Steve Sakoman <steve@sakoman.com> > --- > ...te-host-information-into-compilation.patch | 51 +++++++++++++++++++ > meta/recipes-devtools/rust/rust-source.inc | 1 + > 2 files changed, 52 insertions(+) > create mode 100644 meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch > > diff --git a/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch b/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch > new file mode 100644 > index 0000000000..a6ee867605 > --- /dev/null > +++ b/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch > @@ -0,0 +1,51 @@ > +From 065d7c263091118437465d714d8a29dbb6296921 Mon Sep 17 00:00:00 2001 > +From: Alexander Kanavin <alex@linutronix.de> > +Date: Mon, 13 May 2024 14:57:54 +0200 > +Subject: [PATCH] cargo: do not write host information into compilation unit > + hashes > + > +This breaks reproducibility in cross-builds where the cross-target > +can be the same, but build hosts are different, as seen with > +"rustc --version -v": > +... > +host: x86_64-unknown-linux-gnu > + > +vs. > + > +host: aarch64-unknown-linux-gnu > + > +This can possibly be improved by only hashing host info if the build > +is a native one (e.g. there's no --target option passed to cargo > +invocation) but I'm not sure how. > + > +Upstream-Status: Inappropriate [reported at https://github.com/rust-lang/cargo/issues/13922] > +Signed-off-by: Alexander Kanavin <alex@linutronix.de> > +--- > + .../src/cargo/core/compiler/context/compilation_files.rs | 4 ++-- > + 1 file changed, 2 insertions(+), 2 deletions(-) > + > +diff --git a/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs b/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs > +index d83dbf10c..b2ad8d9f3 100644 > +--- a/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs > ++++ b/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs > +@@ -652,7 +652,7 @@ fn hash_rustc_version(bcx: &BuildContext<'_, '_>, hasher: &mut StableHasher) { > + if vers.pre.is_empty() || bcx.config.cli_unstable().separate_nightlies { > + // For stable, keep the artifacts separate. This helps if someone is > + // testing multiple versions, to avoid recompiles. > +- bcx.rustc().verbose_version.hash(hasher); > ++ //bcx.rustc().verbose_version.hash(hasher); > + return; > + } > + // On "nightly"/"beta"/"dev"/etc, keep each "channel" separate. Don't hash > +@@ -665,7 +665,7 @@ fn hash_rustc_version(bcx: &BuildContext<'_, '_>, hasher: &mut StableHasher) { > + // Keep "host" since some people switch hosts to implicitly change > + // targets, (like gnu vs musl or gnu vs msvc). In the future, we may want > + // to consider hashing `unit.kind.short_name()` instead. > +- bcx.rustc().host.hash(hasher); > ++ //bcx.rustc().host.hash(hasher); > + // None of the other lines are important. Currently they are: > + // binary: rustc <-- or "rustdoc" > + // commit-hash: 38114ff16e7856f98b2b4be7ab4cd29b38bed59a > +-- > +2.39.2 > + > diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc > index 8a8e48b8ca..55ea1b77e6 100644 > --- a/meta/recipes-devtools/rust/rust-source.inc > +++ b/meta/recipes-devtools/rust/rust-source.inc > @@ -13,6 +13,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n > file://target-build-value.patch;patchdir=${RUSTSRC} \ > file://0001-Handle-vendored-sources-when-remapping-paths.patch;patchdir=${RUSTSRC} \ > file://repro-issue-fix-with-v175.patch;patchdir=${RUSTSRC} \ > + file://0001-cargo-do-not-write-host-information-into-compilation.patch;patchdir=${RUSTSRC} \ > " > SRC_URI[rust.sha256sum] = "4526f786d673e4859ff2afa0bab2ba13c918b796519a25c1acce06dba9542340" > > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#208890): https://lists.openembedded.org/g/openembedded-core/message/208890 > Mute This Topic: https://lists.openembedded.org/mt/110188795/3617156 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [martin.jansa@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch b/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch new file mode 100644 index 0000000000..a6ee867605 --- /dev/null +++ b/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch @@ -0,0 +1,51 @@ +From 065d7c263091118437465d714d8a29dbb6296921 Mon Sep 17 00:00:00 2001 +From: Alexander Kanavin <alex@linutronix.de> +Date: Mon, 13 May 2024 14:57:54 +0200 +Subject: [PATCH] cargo: do not write host information into compilation unit + hashes + +This breaks reproducibility in cross-builds where the cross-target +can be the same, but build hosts are different, as seen with +"rustc --version -v": +... +host: x86_64-unknown-linux-gnu + +vs. + +host: aarch64-unknown-linux-gnu + +This can possibly be improved by only hashing host info if the build +is a native one (e.g. there's no --target option passed to cargo +invocation) but I'm not sure how. + +Upstream-Status: Inappropriate [reported at https://github.com/rust-lang/cargo/issues/13922] +Signed-off-by: Alexander Kanavin <alex@linutronix.de> +--- + .../src/cargo/core/compiler/context/compilation_files.rs | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs b/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs +index d83dbf10c..b2ad8d9f3 100644 +--- a/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs ++++ b/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs +@@ -652,7 +652,7 @@ fn hash_rustc_version(bcx: &BuildContext<'_, '_>, hasher: &mut StableHasher) { + if vers.pre.is_empty() || bcx.config.cli_unstable().separate_nightlies { + // For stable, keep the artifacts separate. This helps if someone is + // testing multiple versions, to avoid recompiles. +- bcx.rustc().verbose_version.hash(hasher); ++ //bcx.rustc().verbose_version.hash(hasher); + return; + } + // On "nightly"/"beta"/"dev"/etc, keep each "channel" separate. Don't hash +@@ -665,7 +665,7 @@ fn hash_rustc_version(bcx: &BuildContext<'_, '_>, hasher: &mut StableHasher) { + // Keep "host" since some people switch hosts to implicitly change + // targets, (like gnu vs musl or gnu vs msvc). In the future, we may want + // to consider hashing `unit.kind.short_name()` instead. +- bcx.rustc().host.hash(hasher); ++ //bcx.rustc().host.hash(hasher); + // None of the other lines are important. Currently they are: + // binary: rustc <-- or "rustdoc" + // commit-hash: 38114ff16e7856f98b2b4be7ab4cd29b38bed59a +-- +2.39.2 + diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc index 8a8e48b8ca..55ea1b77e6 100644 --- a/meta/recipes-devtools/rust/rust-source.inc +++ b/meta/recipes-devtools/rust/rust-source.inc @@ -13,6 +13,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n file://target-build-value.patch;patchdir=${RUSTSRC} \ file://0001-Handle-vendored-sources-when-remapping-paths.patch;patchdir=${RUSTSRC} \ file://repro-issue-fix-with-v175.patch;patchdir=${RUSTSRC} \ + file://0001-cargo-do-not-write-host-information-into-compilation.patch;patchdir=${RUSTSRC} \ " SRC_URI[rust.sha256sum] = "4526f786d673e4859ff2afa0bab2ba13c918b796519a25c1acce06dba9542340"