diff mbox series

[3/3] oeqa: print stderr in the AssertionError as well

Message ID 20230411090348.2434686-3-Martin.Jansa@gmail.com
State New
Headers show
Series [1/3] populate_sdk_ext.bbclass: set METADATA_REVISION with an DISTRO override | expand

Commit Message

Martin Jansa April 11, 2023, 9:03 a.m. UTC
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/lib/oeqa/utils/commands.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Luca Ceresoli April 11, 2023, 12:28 p.m. UTC | #1
Hello Martin,

On Tue, 11 Apr 2023 11:03:48 +0200
"Martin Jansa" <Martin.Jansa@gmail.com> wrote:

> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  meta/lib/oeqa/utils/commands.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
> index c1f533802e..7b2f6ec124 100644
> --- a/meta/lib/oeqa/utils/commands.py
> +++ b/meta/lib/oeqa/utils/commands.py
> @@ -209,7 +209,7 @@ def runCmd(command, ignore_status=False, timeout=None, assert_error=True, sync=T
>                  exc_output = "\n... (last %d lines of output)\n" % limit_exc_output + \
>                               '\n'.join(split[-limit_exc_output:])
>          if assert_error:
> -            raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
> +            raise AssertionError("Command '%s' returned non-zero exit status %d:\nstderr: %s\nstdout:\n%s" % (command, result.status, result.error, exc_output))

Testing with this series produces errors on the selftest builds:

---------------------------------8<---------------------------------

AssertionError: Command 'echo foobar >&2; false' returned non-zero exit status 1:
stderr: 
stdout:
foobar

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/runcmd.py", line 61, in test_result_assertion
    self.assertRaisesRegexp(AssertionError, "Command 'echo .* false' returned non-zero exit status 1:\nfoobar",
  File "/usr/lib/python3.9/unittest/case.py", line 1329, in deprecated_func
    return original_func(*args, **kwargs)
  File "/usr/lib/python3.9/unittest/case.py", line 1276, in assertRaisesRegex
    return context.handle('assertRaisesRegex', args, kwargs)
  File "/usr/lib/python3.9/unittest/case.py", line 201, in handle
    callable_obj(*args, **kwargs)
  File "/usr/lib/python3.9/unittest/case.py", line 239, in __exit__
    self._raiseFailure('"{}" does not match "{}"'.format(
  File "/usr/lib/python3.9/unittest/case.py", line 163, in _raiseFailure
    raise self.test_case.failureException(msg)
AssertionError: "Command 'echo .* false' returned non-zero exit status 1:
foobar" does not match "Command 'echo foobar >&2; false' returned non-zero exit status 1:
stderr: 
stdout:
foobar"

---------------------------------8<---------------------------------

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/5002/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5044/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5053/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/5083/steps/15/logs/stdio
Martin Jansa April 11, 2023, 1:59 p.m. UTC | #2
Hello Luca,

I'm sorry, I've updated the runcmd.RunCmdTests.test_result_assertion test
case to pass with this change just to realize that:
"stderr": subprocess.STDOUT
in self.defaultopts does work and even the stderr output is ending in
stdout, so this change isn't very useful (as I don't see any runcmd users
passing different options than defaultopts).

Please drop 3/3 other 2 changes are independent on it.

I have one more change in this area which is the real fix for esdk tests
with uninative disabled, but that has some weird side-effect which is
triggering:

NOTE: Running task 3807 of 3809
(/OE/build/poky/meta/recipes-core/images/core-image-minimal.bb:
do_populate_lic_deploy)
ERROR: When reparsing
/OE/build/poky/meta/recipes-core/images/core-image-minimal.bb:do_populate_sdk_ext,
the basehash value changed from
8249821d6529e111a2f7f1101632fe140639df54a700e2c604a99ea150541a6d to
86086c51c409799a0d4d2a4fcac9e7e245cb36d86f236dcf7093e428a7c37ac9. The
metadata is not deterministic and this needs to be fixed.
ERROR: The following commands may help:
ERROR: $ bitbake core-image-minimal -cdo_populate_sdk_ext -Snone
ERROR: Then:
ERROR: $ bitbake core-image-minimal -cdo_populate_sdk_ext -Sprintdiff

which I haven't resolved yet, so it might be a while before I send this 4th
change.

Regards,

On Tue, Apr 11, 2023 at 2:28 PM Luca Ceresoli <luca.ceresoli@bootlin.com>
wrote:

> Hello Martin,
>
> On Tue, 11 Apr 2023 11:03:48 +0200
> "Martin Jansa" <Martin.Jansa@gmail.com> wrote:
>
> > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> > ---
> >  meta/lib/oeqa/utils/commands.py | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/lib/oeqa/utils/commands.py
> b/meta/lib/oeqa/utils/commands.py
> > index c1f533802e..7b2f6ec124 100644
> > --- a/meta/lib/oeqa/utils/commands.py
> > +++ b/meta/lib/oeqa/utils/commands.py
> > @@ -209,7 +209,7 @@ def runCmd(command, ignore_status=False,
> timeout=None, assert_error=True, sync=T
> >                  exc_output = "\n... (last %d lines of output)\n" %
> limit_exc_output + \
> >                               '\n'.join(split[-limit_exc_output:])
> >          if assert_error:
> > -            raise AssertionError("Command '%s' returned non-zero exit
> status %d:\n%s" % (command, result.status, exc_output))
> > +            raise AssertionError("Command '%s' returned non-zero exit
> status %d:\nstderr: %s\nstdout:\n%s" % (command, result.status,
> result.error, exc_output))
>
> Testing with this series produces errors on the selftest builds:
>
> ---------------------------------8<---------------------------------
>
> AssertionError: Command 'echo foobar >&2; false' returned non-zero exit
> status 1:
> stderr:
> stdout:
> foobar
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>   File
> "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/runcmd.py",
> line 61, in test_result_assertion
>     self.assertRaisesRegexp(AssertionError, "Command 'echo .* false'
> returned non-zero exit status 1:\nfoobar",
>   File "/usr/lib/python3.9/unittest/case.py", line 1329, in deprecated_func
>     return original_func(*args, **kwargs)
>   File "/usr/lib/python3.9/unittest/case.py", line 1276, in
> assertRaisesRegex
>     return context.handle('assertRaisesRegex', args, kwargs)
>   File "/usr/lib/python3.9/unittest/case.py", line 201, in handle
>     callable_obj(*args, **kwargs)
>   File "/usr/lib/python3.9/unittest/case.py", line 239, in __exit__
>     self._raiseFailure('"{}" does not match "{}"'.format(
>   File "/usr/lib/python3.9/unittest/case.py", line 163, in _raiseFailure
>     raise self.test_case.failureException(msg)
> AssertionError: "Command 'echo .* false' returned non-zero exit status 1:
> foobar" does not match "Command 'echo foobar >&2; false' returned non-zero
> exit status 1:
> stderr:
> stdout:
> foobar"
>
> ---------------------------------8<---------------------------------
>
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/5002/steps/14/logs/stdio
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5044/steps/14/logs/stdio
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5053/steps/14/logs/stdio
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/5083/steps/15/logs/stdio
>
>
> --
> Luca Ceresoli, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
diff mbox series

Patch

diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index c1f533802e..7b2f6ec124 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -209,7 +209,7 @@  def runCmd(command, ignore_status=False, timeout=None, assert_error=True, sync=T
                 exc_output = "\n... (last %d lines of output)\n" % limit_exc_output + \
                              '\n'.join(split[-limit_exc_output:])
         if assert_error:
-            raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
+            raise AssertionError("Command '%s' returned non-zero exit status %d:\nstderr: %s\nstdout:\n%s" % (command, result.status, result.error, exc_output))
         else:
             raise CommandError(result.status, command, exc_output)