diff mbox series

[2/2] oeqa/logparser: ignore comments in the test log

Message ID 20260205161243.3884987-2-ross.burton@arm.com
State Under Review
Headers show
Series [1/2] python3-unittest-automake-output: upgrade to 0.4 | expand

Commit Message

Ross Burton Feb. 5, 2026, 4:12 p.m. UTC 
Whilst the log format is normally pretty simple:

  PASS: foo
  SKIP: bar

It's entirely possible for there to be an explanatory comment:

  SKIP: bar # only runs under Windows

We currently use the entire string after the test state as the test
name, which includes the comment.  This can lead to long test names, for
example:

  test_dtype.py:TestStructuredObjectRefcounting.test_structured_object_
  create_delete[ones-1-<subarray>]_#_SKIP_Python_3.12_has_immortal_
  refcounts,_this_test_will_no_longer_work._See_gh-23986

Whilst these test names are very long it isn't normally a problem, but
some packages have non-deterministic skip messages:

  test_ufunc.py:TestUfunc.test_identityless_reduction_huge_array_#_
  SKIP_6.442450944_GB_memory_required,_but_3.366531072_GB_available

This leads to churn in the test reports.

The comment isn't needed, so strip it out when computing the test name.

Note that this will result in a number of tests disappearing in the test
reports, with an identical number of new tests appearing.

[1] https://www.gnu.org/software/automake/manual/automake.html#Scripts_002dbased-Testsuites-1

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/lib/oeqa/utils/logparser.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
diff mbox series

Patch

diff --git a/meta/lib/oeqa/utils/logparser.py b/meta/lib/oeqa/utils/logparser.py
index 496d9e0c90..c479864162 100644
--- a/meta/lib/oeqa/utils/logparser.py
+++ b/meta/lib/oeqa/utils/logparser.py
@@ -77,10 +77,18 @@  class PtestParser(object):
                 for t in test_regex:
                     result = test_regex[t].search(line)
                     if result:
+                        section = result.group(1)
+                        # It's possible that the output contains a comment, eg:
+                        #   SKIP: test_something # only needed for Windows
+                        # If there's a comment, remove it.
+                        if "#" in section:
+                            section = section.partition("#")[0]
+                        section = section.strip()
+
                         try:
-                            self.results[current_section['name']][result.group(1).strip()] = t
+                            self.results[current_section['name']][section] = t
                         except KeyError:
-                            bb.warn("Result with no section: %s - %s" % (t, result.group(1).strip()))
+                            bb.warn("Result with no section: %s - %s" % (t, section))
 
         # Python performance for repeatedly joining long strings is poor, do it all at once at the end.
         # For 2.1 million lines in a log this reduces 18 hours to 12s.