| Message ID | 20251120121947.3473848-3-skandigraun@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | -fcanon-file-prefix and cc-rs compiler flag mixing | expand |
On Thu Nov 20, 2025 at 1:19 PM CET, Gyorgy Sarvari via lists.openembedded.org wrote: > This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976 > > cc-rs crate is used by a number of projects, including rust's bootstrap also > to invoke the systems c/c++ compiler. > > A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now > merges them with HOST_C*FLAGS and TARGET_C*FLAGS. > > This is a problem when a recipe is cross compiled, but it has a build dependency > which uses this crate to be built. In this case the C*FLAGS variable contains > target flags, while the HOST_C*FLAGS contains host-specific flags, and when the > two are mixed, the output is not what one expects. > > This change tries to filter out the incorrect flags: > - If the wrapper is invoked as a c or c++ compiler wrapper, > - then determines if it compiles for host or for target > - Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS > correct, and the C*FLAGS variable content incorrect. > - It subtracts the correct set from the incorrect set, and drops the > remainder from the compiler flags. (Which might be often an empty list, so > the flags are frequently unchanged) > > Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> > --- Hi Gyorgy, Thanks for your patch. I suspect one of the commit of this series is responsible of this rust test failure: 2025-11-20 21:59:48,098 - oe-selftest - INFO - FAIL: rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase) ... File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f return func(*args, **kwargs) File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/rust.py", line 131, in test_rust retval = runCmd(cmd) File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/rust-targets; export RUSTFLAGS='-C strip=debuginfo'; export TEST_DEVICE_ADDR="192.168.7.6:12345"; cd /srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3 src/bootstrap/bootstrap.py test --exclude src/bootstrap --exclude src/doc/rustc --exclude src/doc/rustdoc --exclude src/doc/unstable-book --exclude src/etc/test-float-parse --exclude src/librustdoc --exclude src/rustdoc-json-types --exclude src/tools/coverage-dump --exclude src/tools/jsondoclint --exclude src/tools/lint-docs --exclude src/tools/replace-version-placeholder --exclude src/tools/rust-analyzer --exclude src/tools/rustdoc-themes --exclude src/tools/rust-installer --exclude src/tools/test-float-parse --exclude src/tools/suggest-tests --exclude src/tools/tidy --exclude tests/assembly-llvm/asm/aarch64-outline-atomics.rs --exclude tests/codegen-llvm/issues/issue-122805.rs --exclude tests/codegen-llvm/thread-local.rs --exclude tests/mir-opt/ --exclude tests/run-make --exclude tests/run-make-fulldeps --exclude tests/rustdoc --exclude tests/rustdoc-json --exclude tests/rustdoc-js-std --exclude tests/ui/abi/stack-probes-lto.rs --exclude tests/ui/abi/stack-probes.rs --exclude tests/ui/codegen/mismatched-data-layouts.rs --exclude tests/codegen-llvm/rust-abi-arch-specific-adjustment.rs --exclude tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs --exclude tests/ui/feature-gates/version_check.rs --exclude tests/ui-fulldeps/ --exclude tests/ui/process/nofile-limit.rs --exclude tidyselftest --no-doc --no-fail-fast --bless --target x86_64-poky-linux-gnu' returned non-zero exit status 1: ... https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2691 Can you have a look at this error? Thanks, Mathieu
On 11/21/25 10:17, Mathieu Dubois-Briand wrote: > On Thu Nov 20, 2025 at 1:19 PM CET, Gyorgy Sarvari via lists.openembedded.org wrote: >> This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976 >> >> cc-rs crate is used by a number of projects, including rust's bootstrap also >> to invoke the systems c/c++ compiler. >> >> A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now >> merges them with HOST_C*FLAGS and TARGET_C*FLAGS. >> >> This is a problem when a recipe is cross compiled, but it has a build dependency >> which uses this crate to be built. In this case the C*FLAGS variable contains >> target flags, while the HOST_C*FLAGS contains host-specific flags, and when the >> two are mixed, the output is not what one expects. >> >> This change tries to filter out the incorrect flags: >> - If the wrapper is invoked as a c or c++ compiler wrapper, >> - then determines if it compiles for host or for target >> - Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS >> correct, and the C*FLAGS variable content incorrect. >> - It subtracts the correct set from the incorrect set, and drops the >> remainder from the compiler flags. (Which might be often an empty list, so >> the flags are frequently unchanged) >> >> Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> >> --- > Hi Gyorgy, > > Thanks for your patch. > > I suspect one of the commit of this series is responsible of this rust > test failure: > > 2025-11-20 21:59:48,098 - oe-selftest - INFO - FAIL: rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase) > ... > File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f > return func(*args, **kwargs) > File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/rust.py", line 131, in test_rust > retval = runCmd(cmd) > File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd > raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) > AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/rust-targets; export RUSTFLAGS='-C strip=debuginfo'; export TEST_DEVICE_ADDR="192.168.7.6:12345"; cd /srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3 src/bootstrap/bootstrap.py test --exclude src/bootstrap --exclude src/doc/rustc --exclude src/doc/rustdoc --exclude src/doc/unstable-book --exclude src/etc/test-float-parse --exclude src/librustdoc --exclude src/rustdoc-json-types --exclude src/tools/coverage-dump --exclude src/tools/jsondoclint --exclude src/tools/lint-docs --exclude src/tools/replace-version-placeholder --exclude src/tools/rust-analyzer --exclude src/tools/rustdoc-themes --exclude src/tools/rust-installer --exclude src/tools/test-float-parse --exclude src/tools/suggest-tests --exclude src/tools/tidy --exclude tests/assembly-llvm/asm/aarch64-outline-atomics.rs --exclude tests/codegen-llvm/issues/issue-122805.rs --exclude tests/codegen-llvm/thread-local.rs --exclude tests/mir-opt/ --exclude tests/run-make --exclude tests/run-make-fulldeps --exclude tests/rustdoc --exclude tests/rustdoc-json --exclude tests/rustdoc-js-std --exclude tests/ui/abi/stack-probes-lto.rs --exclude tests/ui/abi/stack-probes.rs --exclude tests/ui/codegen/mismatched-data-layouts.rs --exclude tests/codegen-llvm/rust-abi-arch-specific-adjustment.rs --exclude tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs --exclude tests/ui/feature-gates/version_check.rs --exclude tests/ui-fulldeps/ --exclude tests/ui/process/nofile-limit.rs --exclude tidyselftest --no-doc --no-fail-fast --bless --target x86_64-poky-linux-gnu' returned non-zero exit status 1: > ... > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2691 > > Can you have a look at this error? Yes, I did already: https://lists.openembedded.org/g/openembedded-core/message/226640 - I believe this should take care of this issue (but I have to admit that currently I struggle executing the selftest on my machine - both inside and outside of docker I get mysterious qemu startup errors). > Thanks, > Mathieu >
On Fri, 2025-11-21 at 11:51 +0100, Gyorgy Sarvari via lists.openembedded.org wrote: > On 11/21/25 10:17, Mathieu Dubois-Briand wrote: > > On Thu Nov 20, 2025 at 1:19 PM CET, Gyorgy Sarvari via lists.openembedded.org wrote: > > > This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976 > > > > > > cc-rs crate is used by a number of projects, including rust's bootstrap also > > > to invoke the systems c/c++ compiler. > > > > > > A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now > > > merges them with HOST_C*FLAGS and TARGET_C*FLAGS. > > > > > > This is a problem when a recipe is cross compiled, but it has a build dependency > > > which uses this crate to be built. In this case the C*FLAGS variable contains > > > target flags, while the HOST_C*FLAGS contains host-specific flags, and when the > > > two are mixed, the output is not what one expects. > > > > > > This change tries to filter out the incorrect flags: > > > - If the wrapper is invoked as a c or c++ compiler wrapper, > > > - then determines if it compiles for host or for target > > > - Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS > > > correct, and the C*FLAGS variable content incorrect. > > > - It subtracts the correct set from the incorrect set, and drops the > > > remainder from the compiler flags. (Which might be often an empty list, so > > > the flags are frequently unchanged) > > > > > > Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> > > > --- > > Hi Gyorgy, > > > > Thanks for your patch. > > > > I suspect one of the commit of this series is responsible of this rust > > test failure: > > > > 2025-11-20 21:59:48,098 - oe-selftest - INFO - FAIL: rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase) > > ... > > File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f > > return func(*args, **kwargs) > > File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/rust.py", line 131, in test_rust > > retval = runCmd(cmd) > > File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd > > raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) > > AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/rust-targets; export RUSTFLAGS='-C strip=debuginfo'; export TEST_DEVICE_ADDR="192.168.7.6:12345"; cd /srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3 src/bootstrap/bootstrap.py test --exclude src/bootstrap --exclude src/doc/rustc --exclude src/doc/rustdoc --exclude src/doc/unstable-book --exclude src/etc/test-float-parse --exclude src/librustdoc --exclude src/rustdoc-json-types --exclude src/tools/coverage-dump --exclude src/tools/jsondoclint --exclude src/tools/lint-docs --exclude src/tools/replace-version-placeholder --exclude src/tools/rust-analyzer --exclude src/tools/rustdoc-themes --exclude src/tools/rust-installer --exclude src/tools/test-float-parse --exclude src/tools/suggest-tests --exclude src/tools/tidy --exclude tests/assembly-llvm/asm/aarch64-outline-atomics.rs --exclude tests/codegen-llvm/issues/issue-122805.rs --exclude tests/codegen-llvm/thread-local.rs --exclude tests/mir-opt/ --exclude tests/run-make --exclude tests/run-make-fulldeps --exclude tests/rustdoc --exclude tests/rustdoc-json --exclude tests/rustdoc-js-std --exclude tests/ui/abi/stack-probes-lto.rs --exclude tests/ui/abi/stack-probes.rs --exclude tests/ui/codegen/mismatched-data-layouts.rs --exclude tests/codegen-llvm/rust-abi-arch-specific-adjustment.rs --exclude tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs --exclude tests/ui/feature-gates/version_check.rs --exclude tests/ui-fulldeps/ --exclude tests/ui/process/nofile-limit.rs --exclude tidyselftest --no-doc --no-fail-fast --bless --target x86_64-poky-linux-gnu' returned non-zero exit status 1: > > ... > > > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2691 > > > > Can you have a look at this error? > > Yes, I did already: > https://lists.openembedded.org/g/openembedded-core/message/226640 - I > believe this should take care of this issue > (but I have to admit that currently I struggle executing the selftest on > my machine - both inside and outside of docker I get mysterious qemu > startup errors). I think I saw that issue in selftest with your patch applied. I'm trying a simpler patch on master-next right now as a workaround. Cheers, Richard
diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass index 31331c7a26..c61143a56c 100644 --- a/meta/classes-recipe/rust-common.bbclass +++ b/meta/classes-recipe/rust-common.bbclass @@ -148,6 +148,29 @@ create_wrapper_rust () { binary = orig_binary.split()[0] args = orig_binary.split() + sys.argv[1:] + + # The following is trying to be a workaround for + # https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976 + # cc-rs crate passes both host AND target flags at the same time + # to this script, in case a recipe is cross-compiled, but a crate + # is a build dependency, and is compiled for the host. + # This tries to filter out the inappropriate flags. + if sys.argv[0].endswith("-cc"): + script_type = "C" + elif sys.argv[0].endswith("-cxx"): + script_type = "CXX" + else: + script_type = "other" + + flags_to_remove = [] + if script_type != "other": + host_or_target = "HOST" if "build-rust-" in sys.argv[0] else "TARGET" + incorrect_flags = os.getenv("%sFLAGS" % script_type).split() + correct_flags = os.getenv("%s_%sFLAGS" % (host_or_target, script_type)).split() + flags_to_remove = set(incorrect_flags) - set(correct_flags) + + args = list(filter(lambda flag: flag not in flags_to_remove, args)) + if extras: args.append(extras) os.execvp(binary, args)
This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976 cc-rs crate is used by a number of projects, including rust's bootstrap also to invoke the systems c/c++ compiler. A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now merges them with HOST_C*FLAGS and TARGET_C*FLAGS. This is a problem when a recipe is cross compiled, but it has a build dependency which uses this crate to be built. In this case the C*FLAGS variable contains target flags, while the HOST_C*FLAGS contains host-specific flags, and when the two are mixed, the output is not what one expects. This change tries to filter out the incorrect flags: - If the wrapper is invoked as a c or c++ compiler wrapper, - then determines if it compiles for host or for target - Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS correct, and the C*FLAGS variable content incorrect. - It subtracts the correct set from the incorrect set, and drops the remainder from the compiler flags. (Which might be often an empty list, so the flags are frequently unchanged) Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com> --- meta/classes-recipe/rust-common.bbclass | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)