| Message ID | 20251112155730.646159-4-andreas.stergiopoulos@smile.fr |
|---|---|
| State | New |
| Headers | show |
| Series | Better support for Rust integration tests | expand |
On Wed Nov 12, 2025 at 4:57 PM CET, Andreas Stergiopoulos via lists.openembedded.org wrote: > This commit takes the installation paths collected during package installation and > sets them up as environment variables in the run-ptest script. This is necessary as > the integration tests require the executable binary to run successfully. > > The paths collected are not those of the artifacts created during cargo test, but the > paths where the artifacts in the system. The intention for this choice was to ensure > that the tested binaries are those who are actually executed by the user and vice > versa. > > Env has been used instead of export, as export explicitly does not support dashes "-" > when setting up environment variables. The execution environment is set up inside the > function exec_in_env. For every binary executable artifact collected, one environment > variable is added irregardles of whether it is needed or not. > > Signed-off-by: Andreas Stergiopoulos <andreas.stergiopoulos@smile.fr> > --- Hi Andreas, Thanks for your patch. It looks like this is breaking some ptests: AssertionError: Failed ptests: {'rpm-sequoia': ['/usr/lib/rpm-sequoia/ptest/symbols-4b3369365cdc8879']} https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2612 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/732 Ptest logs can be found here: https://valkyrie.yocto.io/pub/non-release/20251113-72/testresults/qemux86-64-ptest/core-image-ptest-rpm-sequoia/ https://valkyrie.yocto.io/pub/non-release/20251113-72/testresults/qemuriscv64-ptest/core-image-ptest-rpm-sequoia/ Can you have a look at these? Thanks, Mathieu
Hello Mathieu, Thank you for reporting this. I'm looking into it. Best regards, Andreas On Thu, Nov 13, 2025 at 5:31 PM Mathieu Dubois-Briand < mathieu.dubois-briand@bootlin.com> wrote: > On Wed Nov 12, 2025 at 4:57 PM CET, Andreas Stergiopoulos via > lists.openembedded.org wrote: > > This commit takes the installation paths collected during package > installation and > > sets them up as environment variables in the run-ptest script. This is > necessary as > > the integration tests require the executable binary to run successfully. > > > > The paths collected are not those of the artifacts created during cargo > test, but the > > paths where the artifacts in the system. The intention for this choice > was to ensure > > that the tested binaries are those who are actually executed by the user > and vice > > versa. > > > > Env has been used instead of export, as export explicitly does not > support dashes "-" > > when setting up environment variables. The execution environment is set > up inside the > > function exec_in_env. For every binary executable artifact collected, > one environment > > variable is added irregardles of whether it is needed or not. > > > > Signed-off-by: Andreas Stergiopoulos <andreas.stergiopoulos@smile.fr> > > --- > > Hi Andreas, > > Thanks for your patch. > > It looks like this is breaking some ptests: > > AssertionError: > Failed ptests: > {'rpm-sequoia': ['/usr/lib/rpm-sequoia/ptest/symbols-4b3369365cdc8879']} > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2612 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/732 > > Ptest logs can be found here: > > > https://valkyrie.yocto.io/pub/non-release/20251113-72/testresults/qemux86-64-ptest/core-image-ptest-rpm-sequoia/ > > https://valkyrie.yocto.io/pub/non-release/20251113-72/testresults/qemuriscv64-ptest/core-image-ptest-rpm-sequoia/ > > Can you have a look at these? > > Thanks, > Mathieu > > -- > Mathieu Dubois-Briand, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com > >
diff --git a/meta/classes-recipe/ptest-cargo.bbclass b/meta/classes-recipe/ptest-cargo.bbclass index 004abb40f7..f5dfe36c48 100644 --- a/meta/classes-recipe/ptest-cargo.bbclass +++ b/meta/classes-recipe/ptest-cargo.bbclass @@ -6,6 +6,11 @@ RUST_TEST_ARGS[doc] = "Arguments to give to the test binaries (e.g. --shuffle)" # I didn't find a cleaner way to share data between compile and install tasks CARGO_TEST_BINARIES_FILES ?= "${B}/test_binaries_list" +# Integration tests usually require the application binary to execute successfully. +# This file contains the paths in the image where all the crate executable artifacts +# reside. +CARGO_BIN_EXE_PATHS ?= "${B}/binary_exe_path_list" + # Sadly, generated test binaries have no deterministic names (https://github.com/rust-lang/cargo/issues/1924) # This forces us to parse the cargo output in json format to find those test binaries. python do_compile_ptest_cargo() { @@ -84,6 +89,7 @@ python do_install_ptest_cargo() { pn = d.getVar("PN") ptest_path = d.getVar("PTEST_PATH") cargo_test_binaries_file = d.getVar('CARGO_TEST_BINARIES_FILES') + cargo_bin_exe_paths = d.getVar('CARGO_BIN_EXE_PATHS') rust_test_args = d.getVar('RUST_TEST_ARGS') or "" ptest_dir = os.path.join(dest_dir, ptest_path.lstrip('/')) @@ -94,6 +100,14 @@ python do_install_ptest_cargo() { for line in f.readlines(): test_bins.append(line.strip('\n')) + # Read the file that contains the paths to the executable binaries. This step is + # necessary in order to successfully set up the execution environment for integration tests. + cargo_bin_exes = [] + if os.path.exists(cargo_bin_exe_paths): + with open(cargo_bin_exe_paths, "r") as f: + for line in f.readlines(): + cargo_bin_exes.append(line.strip('\n')) + test_paths = [] for test_bin in test_bins: shutil.copy2(test_bin, ptest_dir) @@ -106,11 +120,33 @@ python do_install_ptest_cargo() { f.write("#!/bin/sh\n") else: f.write(f"\necho \"\"\n") - f.write(f"echo \"## starting to run rust tests ##\"\n") + f.write(f"echo \"## starting to run rust tests ##\"\n") + + # For the integration tests, cargo uses the CARGO_BIN_EXE_<name> environment variable to + # communicate the location of the binary executable. More info on this variable can be + # found in the link: (https://doc.rust-lang.org/cargo/reference/environment-variables.html). + + # This environment variable is exported by cargo automatically during compilation. Making it + # also available during runtime is a very powerful tool as it allows the ptest framework to + # run the integration tests on the actual executables installed in the target, no matter the + # installation location. + + # By using env we ensure that we will be able to create environment variables even for apps + # whose name contains dashes "-", which is not supported by export. For every binary executable + # produced by the rust compiler, one environment variable is added. + + f.write("exec_in_env () {\n") + f.write(" env - ") + for cargo_bin_exe in cargo_bin_exes: + f.write(f"CARGO_BIN_EXE_{os.path.basename(cargo_bin_exe)}={cargo_bin_exe} \\\n") + + f.write(" \"$@\"\n") + f.write("}\n") + f.write("if [ -z \"$rc\" ]; then rc=0; fi\n") for test_path in test_paths: script = textwrap.dedent(f"""\ - if ! {test_path} {rust_test_args} + if ! exec_in_env {test_path} {rust_test_args} then rc=1 echo "FAIL: {test_path}"
This commit takes the installation paths collected during package installation and sets them up as environment variables in the run-ptest script. This is necessary as the integration tests require the executable binary to run successfully. The paths collected are not those of the artifacts created during cargo test, but the paths where the artifacts in the system. The intention for this choice was to ensure that the tested binaries are those who are actually executed by the user and vice versa. Env has been used instead of export, as export explicitly does not support dashes "-" when setting up environment variables. The execution environment is set up inside the function exec_in_env. For every binary executable artifact collected, one environment variable is added irregardles of whether it is needed or not. Signed-off-by: Andreas Stergiopoulos <andreas.stergiopoulos@smile.fr> --- meta/classes-recipe/ptest-cargo.bbclass | 40 +++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-)