mbox series

[0/5] rust: reproducibility fixes for arm build hosts

Message ID 20260915141834.1212812-1-sunilkumar.dora@windriver.com
Headers show
Series rust: reproducibility fixes for arm build hosts | expand

Message

Dora, Sunil Kumar Sept. 15, 2026, 2:18 p.m. UTC 
From: Sunil Dora <sunilkumar.dora@windriver.com>

The reproducible selftest fails for rust on the arm workers
([YOCTO #16376]). This series fixes that.

rust: drop the cc object-name reproducibility patch
rust: hash cc object names relative to their source roots

  The patch we carried trimmed the build path by matching the string
  "rustc", and the shared-source rework broke that for one of the two
  sets of sources cc compiles. The replacement strips the roots the
  sources actually come from. The OUT_DIR part is proposed upstream as
  https://github.com/rust-lang/cc-rs/pull/1902.
  The RUST_COMPILER_RT_ROOT part is specific to rust's own build
  system, so it is not part of that pull request.

rust: keep the build host triple out of cargo unit metadata

  From Alejandro Enedino Hernandez Samaniego's strict-version-hash
  series, carried with his sign-off:
  https://lists.openembedded.org/g/openembedded-core/topic/120939810
  cargo mixes the "host:" line of "rustc -vV" into build script and
  proc-macro metadata, and target crates inherit it.
  Upstream report: https://github.com/rust-lang/cargo/issues/8140

rust: backport deterministic DocLinkResMap serialisation

  Upstream commit f9d9939 by jprochazk and mejrs. It is in rust 1.99,
  so this drops out at the next rust upgrade.

rust: sort hygiene data before it is encoded

  Also from Alejandro's series. Upstream is fixing the same problem
  with a different approach in
  https://github.com/rust-lang/rust/pull/161450, which is still open.

Alejandro's StableCrateId patch is not included: the value it pins is
already host-independent in our builds, and it broke three rust test
suite cases when the autobuilder tested it.

Tested on the autobuilder: the x86 reproducible builder green with
zero differing packages, the rust test suite green, and on both arm
builders rust, rust-dbg, rust-rustdoc and libstd-rs-dev gone from the
failing list.


Sunil Dora (5):
  rust: drop the cc object-name reproducibility patch
  rust: hash cc object names relative to their source roots
  rust: keep the build host triple out of cargo unit metadata
  rust: backport deterministic DocLinkResMap serialisation
  rust: sort hygiene data before it is encoded

 ...-host-triple-from-unit-metadata-hash.patch |  62 ++++++++++
 ...oc-link-metadata-order-deterministic.patch |  40 ++++++
 ...hygiene-encoding-order-deterministic.patch |  58 +++++++++
 ...ject-names-relative-to-source-roots.patch} | 116 +++++++++---------
 meta/recipes-devtools/rust/rust-source.inc    |   5 +-
 5 files changed, 224 insertions(+), 57 deletions(-)
 create mode 100644 meta/recipes-devtools/rust/files/0007-cargo-omit-host-triple-from-unit-metadata-hash.patch
 create mode 100644 meta/recipes-devtools/rust/files/0008-rustc-hir-make-doc-link-metadata-order-deterministic.patch
 create mode 100644 meta/recipes-devtools/rust/files/0009-rustc-span-make-hygiene-encoding-order-deterministic.patch
 rename meta/recipes-devtools/rust/files/{repro-issue-fix-with-cc-crate-hashmap.patch => cc-hash-object-names-relative-to-source-roots.patch} (70%)

Comments

Alejandro Hernandez Sept. 15, 2026, 3:53 p.m. UTC | #1 
Hopefully these fix reproducibility issue when built from different hosts.

Unfortunately these go in a bit of a different way than my next version, so I
will wait for the result before sending to avoid confusion, but I'm fine with these
if they work, lets just add my sign off to the actual patches based off of mine
instead of the "Patch from Alejandro + OE link".

Cheers

Alejandro

On 9/15/2026 8:18 AM, Dora, Sunil Kumar via lists.openembedded.org wrote:
> From: Sunil Dora<sunilkumar.dora@windriver.com>
>
> The reproducible selftest fails for rust on the arm workers
> ([YOCTO #16376]). This series fixes that.
>
> rust: drop the cc object-name reproducibility patch
> rust: hash cc object names relative to their source roots
>
>    The patch we carried trimmed the build path by matching the string
>    "rustc", and the shared-source rework broke that for one of the two
>    sets of sources cc compiles. The replacement strips the roots the
>    sources actually come from. The OUT_DIR part is proposed upstream as
>    https://github.com/rust-lang/cc-rs/pull/1902.
>    The RUST_COMPILER_RT_ROOT part is specific to rust's own build
>    system, so it is not part of that pull request.
>
> rust: keep the build host triple out of cargo unit metadata
>
>    From Alejandro Enedino Hernandez Samaniego's strict-version-hash
>    series, carried with his sign-off:
>    https://lists.openembedded.org/g/openembedded-core/topic/120939810
>    cargo mixes the "host:" line of "rustc -vV" into build script and
>    proc-macro metadata, and target crates inherit it.
>    Upstream report:https://github.com/rust-lang/cargo/issues/8140
>
> rust: backport deterministic DocLinkResMap serialisation
>
>    Upstream commit f9d9939 by jprochazk and mejrs. It is in rust 1.99,
>    so this drops out at the next rust upgrade.
>
> rust: sort hygiene data before it is encoded
>
>    Also from Alejandro's series. Upstream is fixing the same problem
>    with a different approach in
>    https://github.com/rust-lang/rust/pull/161450, which is still open.
>
> Alejandro's StableCrateId patch is not included: the value it pins is
> already host-independent in our builds, and it broke three rust test
> suite cases when the autobuilder tested it.
>
> Tested on the autobuilder: the x86 reproducible builder green with
> zero differing packages, the rust test suite green, and on both arm
> builders rust, rust-dbg, rust-rustdoc and libstd-rs-dev gone from the
> failing list.
>
>
> Sunil Dora (5):
>    rust: drop the cc object-name reproducibility patch
>    rust: hash cc object names relative to their source roots
>    rust: keep the build host triple out of cargo unit metadata
>    rust: backport deterministic DocLinkResMap serialisation
>    rust: sort hygiene data before it is encoded
>
>   ...-host-triple-from-unit-metadata-hash.patch |  62 ++++++++++
>   ...oc-link-metadata-order-deterministic.patch |  40 ++++++
>   ...hygiene-encoding-order-deterministic.patch |  58 +++++++++
>   ...ject-names-relative-to-source-roots.patch} | 116 +++++++++---------
>   meta/recipes-devtools/rust/rust-source.inc    |   5 +-
>   5 files changed, 224 insertions(+), 57 deletions(-)
>   create mode 100644 meta/recipes-devtools/rust/files/0007-cargo-omit-host-triple-from-unit-metadata-hash.patch
>   create mode 100644 meta/recipes-devtools/rust/files/0008-rustc-hir-make-doc-link-metadata-order-deterministic.patch
>   create mode 100644 meta/recipes-devtools/rust/files/0009-rustc-span-make-hygiene-encoding-order-deterministic.patch
>   rename meta/recipes-devtools/rust/files/{repro-issue-fix-with-cc-crate-hashmap.patch => cc-hash-object-names-relative-to-source-roots.patch} (70%)
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#245843):https://lists.openembedded.org/g/openembedded-core/message/245843
> Mute This Topic:https://lists.openembedded.org/mt/121261374/4354175
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [alhe@linux.microsoft.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Dora, Sunil Kumar Sept. 15, 2026, 4:10 p.m. UTC | #2 
On 9/15/2026 9:23 PM, Alejandro Hernandez wrote:
CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hopefully these fix reproducibility issue when built from different hosts.

Unfortunately these go in a bit of a different way than my next version, so I
will wait for the result before sending to avoid confusion, but I'm fine with these
if they work, lets just add my sign off to the actual patches based off of mine
instead of the "Patch from Alejandro + OE link".

Thanks Alejandro. Your series did the hard part here, finding where the host details get into the
rust builds in the first place. Mine builds on that map.

On the contrib branch run the arm reproducible builders came back with the rust family gone
from the failing list, so it looks good so far. The real confirmation will be the autobuilder runs on
master-next once the series gets picked up there.

On the sign-off: your Signed-off-by is already inside the two carried patch files, but you are right that it is not in
the commit messages, so git log only shows mine.

In v2 I will add your Signed-off-by to those two commit messages and drop the "Patch from + link" wording,
together with whatever else the review brings up.

Thanks,
Sunil Dora



Cheers

Alejandro

On 9/15/2026 8:18 AM, Dora, Sunil Kumar via lists.openembedded.org<https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW77Ut4JIQ$> wrote:

From: Sunil Dora <sunilkumar.dora@windriver.com><mailto:sunilkumar.dora@windriver.com>

The reproducible selftest fails for rust on the arm workers
([YOCTO #16376]). This series fixes that.

rust: drop the cc object-name reproducibility patch
rust: hash cc object names relative to their source roots

  The patch we carried trimmed the build path by matching the string
  "rustc", and the shared-source rework broke that for one of the two
  sets of sources cc compiles. The replacement strips the roots the
  sources actually come from. The OUT_DIR part is proposed upstream as
  https://github.com/rust-lang/cc-rs/pull/1902<https://urldefense.com/v3/__https://github.com/rust-lang/cc-rs/pull/1902__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW4fqdQsfQ$>.
  The RUST_COMPILER_RT_ROOT part is specific to rust's own build
  system, so it is not part of that pull request.

rust: keep the build host triple out of cargo unit metadata

  From Alejandro Enedino Hernandez Samaniego's strict-version-hash
  series, carried with his sign-off:
  https://lists.openembedded.org/g/openembedded-core/topic/120939810<https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/topic/120939810__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW7uALNnJw$>
  cargo mixes the "host:" line of "rustc -vV" into build script and
  proc-macro metadata, and target crates inherit it.
  Upstream report: https://github.com/rust-lang/cargo/issues/8140<https://urldefense.com/v3/__https://github.com/rust-lang/cargo/issues/8140__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW7EZ-2c1Q$>

rust: backport deterministic DocLinkResMap serialisation

  Upstream commit f9d9939 by jprochazk and mejrs. It is in rust 1.99,
  so this drops out at the next rust upgrade.

rust: sort hygiene data before it is encoded

  Also from Alejandro's series. Upstream is fixing the same problem
  with a different approach in
  https://github.com/rust-lang/rust/pull/161450<https://urldefense.com/v3/__https://github.com/rust-lang/rust/pull/161450__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW5_rpjPaQ$>, which is still open.

Alejandro's StableCrateId patch is not included: the value it pins is
already host-independent in our builds, and it broke three rust test
suite cases when the autobuilder tested it.

Tested on the autobuilder: the x86 reproducible builder green with
zero differing packages, the rust test suite green, and on both arm
builders rust, rust-dbg, rust-rustdoc and libstd-rs-dev gone from the
failing list.


Sunil Dora (5):
  rust: drop the cc object-name reproducibility patch
  rust: hash cc object names relative to their source roots
  rust: keep the build host triple out of cargo unit metadata
  rust: backport deterministic DocLinkResMap serialisation
  rust: sort hygiene data before it is encoded

 ...-host-triple-from-unit-metadata-hash.patch |  62 ++++++++++
 ...oc-link-metadata-order-deterministic.patch |  40 ++++++
 ...hygiene-encoding-order-deterministic.patch |  58 +++++++++
 ...ject-names-relative-to-source-roots.patch} | 116 +++++++++---------
 meta/recipes-devtools/rust/rust-source.inc<https://urldefense.com/v3/__http://rust-source.inc__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW5pCv2TAA$>    |   5 +-
 5 files changed, 224 insertions(+), 57 deletions(-)
 create mode 100644 meta/recipes-devtools/rust/files/0007-cargo-omit-host-triple-from-unit-metadata-hash.patch
 create mode 100644 meta/recipes-devtools/rust/files/0008-rustc-hir-make-doc-link-metadata-order-deterministic.patch
 create mode 100644 meta/recipes-devtools/rust/files/0009-rustc-span-make-hygiene-encoding-order-deterministic.patch
 rename meta/recipes-devtools/rust/files/{repro-issue-fix-with-cc-crate-hashmap.patch => cc-hash-object-names-relative-to-source-roots.patch} (70%)





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#245843): https://lists.openembedded.org/g/openembedded-core/message/245843<https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/message/245843__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW4QtYWkrg$>
Mute This Topic: https://lists.openembedded.org/mt/121261374/4354175<https://urldefense.com/v3/__https://lists.openembedded.org/mt/121261374/4354175__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW4RTLA2Tw$>
Group Owner: openembedded-core+owner@lists.openembedded.org<mailto:openembedded-core+owner@lists.openembedded.org>
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub<https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/unsub__;!!AjveYdw8EvQ!bAZjXNFQVd7FZGNUYCD78uEwFODIFOwfzeAgrtyk8STWe6WGkL_Vzz465VBcVThqLu12-qJFkYBIO_U1bW6OY1T4lQ$> [alhe@linux.microsoft.com<mailto:alhe@linux.microsoft.com>]
-=-=-=-=-=-=-=-=-=-=-=-
Dora, Sunil Kumar Sept. 15, 2026, 5:55 p.m. UTC | #3 
A small update: the upstream half of the cc fix is now merged in rust-lang/cc-rs:
https://github.com/rust-lang/cc-rs/commit/fe27e7b62629daddccf1bdc9e9a447a37de2cd06

In v2 I will change the Upstream-Status of the cc patch from Submitted to Backport with that commit.
The RUST_COMPILER_RT_ROOT part stays local, as the patch header explains.

Thanks,
Sunil Dora