diff mbox series

[2/2] ptest-cargo: fix incorrect FAIL count when multiple tests are run

Message ID 20250424123217.885006-2-ines.kchelfi@smile.fr
State New
Headers show
Series [1/2] ptest-cargo: refactor run-ptest generation to remove redundancy | expand

Commit Message

ines.kchelfi@smile.fr April 24, 2025, 12:32 p.m. UTC
From: Ines KCHELFI <ines.kchelfi@smile.fr>

When using the ptest-cargo class with multiple Rust test binaries, ptest-runner
may report FAIL: 0 even if one of the tests fails, as long as the last test passes.

This happens because the run-ptest script, as generated by the class, does not
track failures and simply returns the exit code of the last test.
To fix this, each test binary is checked individually for failure. If any test fails,
a non-zero exit code is returned.

This ensures that test failures are not silently ignored and are properly reported
by ptest-runner in multi-test scenarios.

Signed-off-by: Ines KCHELFI <ines.kchelfi@smile.fr>
---
 meta/classes-recipe/ptest-cargo.bbclass | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

Comments

Yoann Congal April 24, 2025, 2:55 p.m. UTC | #1
Le jeudi 24 avril 2025, ines.kchelfi via lists.openembedded.org
<ines.kchelfi=smile.fr@lists.openembedded.org> a écrit :
> From: Ines KCHELFI <ines.kchelfi@smile.fr>
>
> When using the ptest-cargo class with multiple Rust test binaries,
ptest-runner
> may report FAIL: 0 even if one of the tests fails, as long as the last
test passes.
>
> This happens because the run-ptest script, as generated by the class,
does not
> track failures and simply returns the exit code of the last test.
> To fix this, each test binary is checked individually for failure. If any
test fails,
> a non-zero exit code is returned.
>
> This ensures that test failures are not silently ignored and are properly
reported
> by ptest-runner in multi-test scenarios.
>
> Signed-off-by: Ines KCHELFI <ines.kchelfi@smile.fr>

Reviewed-by: Yoann Congal <yoann.congal@smile.fr>

> ---
>  meta/classes-recipe/ptest-cargo.bbclass | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes-recipe/ptest-cargo.bbclass
b/meta/classes-recipe/ptest-cargo.bbclass
> index 6799a454ac..390fcf0815 100644
> --- a/meta/classes-recipe/ptest-cargo.bbclass
> +++ b/meta/classes-recipe/ptest-cargo.bbclass
> @@ -101,14 +101,15 @@ python do_install_ptest_cargo() {
>      with open(ptest_script, "a") as f:
>          if not script_exists:
>              f.write("#!/bin/sh\n")
> -
> +            f.write("rc=0\n")
>          else:
>              f.write(f"\necho \"\"\n")
> -            f.write(f"echo \"## starting to run rust tests ##\"\n")
> -
> +            f.write(f"echo \"## starting to run rust tests ##\"\n")

>          for test_path in test_paths:
> -            f.write(f"{test_path} {rust_test_args}\n")
> +            f.write(f"if ! {test_path} {rust_test_args}; then rc=1;
fi\n")
>
> +        f.write("exit $rc\n")
> +
>      if not script_exists:
>          os.chmod(ptest_script, 0o755)
>
> --
> 2.43.0
>
>
diff mbox series

Patch

diff --git a/meta/classes-recipe/ptest-cargo.bbclass b/meta/classes-recipe/ptest-cargo.bbclass
index 6799a454ac..390fcf0815 100644
--- a/meta/classes-recipe/ptest-cargo.bbclass
+++ b/meta/classes-recipe/ptest-cargo.bbclass
@@ -101,14 +101,15 @@  python do_install_ptest_cargo() {
     with open(ptest_script, "a") as f:
         if not script_exists:
             f.write("#!/bin/sh\n")
-                
+            f.write("rc=0\n")                
         else:
             f.write(f"\necho \"\"\n")
-            f.write(f"echo \"## starting to run rust tests ##\"\n")
-                
+            f.write(f"echo \"## starting to run rust tests ##\"\n")               
         for test_path in test_paths:
-            f.write(f"{test_path} {rust_test_args}\n")
+            f.write(f"if ! {test_path} {rust_test_args}; then rc=1; fi\n")
         
+        f.write("exit $rc\n")
+
     if not script_exists:
         os.chmod(ptest_script, 0o755)