From patchwork Wed Sep 23 19:04:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99094 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 692E0C9830B for ; Wed, 23 Sep 2026 19:58:44 +0000 (UTC) Received: from mail-vs2-f41.google.com (mail-vs2-f41.google.com [74.125.227.41]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3027.1790190266833243408 for ; Wed, 23 Sep 2026 12:04:27 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=BPLutAFw; spf=pass (domain: baylibre.com, ip: 74.125.227.41, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f41.google.com with SMTP id ada2fe7eead31-78564421fdbso459062137.1 for ; Wed, 23 Sep 2026 12:04:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190266; x=1790795066; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=ay/n7RaNb0iEBG7Y6xqdGxgqCR9jiV+cs/eLe0uckpU=; b=BPLutAFwezybLaZAZF7HE7rIIdr07xEA4q++kxVHBUz9LEjnt5doti04sbRYaMLHN2 jz1fZEQzjL3YNQ1GFMuEzN6muYGAKzVbnzijR3s3IQAAynfMjH/ei+2A0wZRWDnyFvrY BYcgJnD+Q4k8GEJdxrrqUf1Lkrt+uCVHoPvLdQEY5BVOrn42+PM3RE/JHfnqPAweJXm7 InvHIkLzMMDNBskkNO6kqyndanOSaZHP/J3rJ1SNVqBb+QjkjmwTk7INvr3lZHxa9XRu ZnBM9mIiuefq0UJ6YMS+KXLYBlDq24v6l3vgqoMZA4rDz1r8MJopj3evo17Obhj4rOoV UG5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190266; x=1790795066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=ay/n7RaNb0iEBG7Y6xqdGxgqCR9jiV+cs/eLe0uckpU=; b=bicYjPVpXIlfcuqJ8EK/W0AOR+lnGoWpxT7ladh91ZJMiohp71e6U3Q3eTZ0ZvM1C2 gAC582kF7k4oKTogJsgmL4+OESRSvWUd+0sD9peE0zcS4s9tslXVZEu0fKAkJI2A+TZB QfLv5V3+re+4/IT+YoubZpPKVjhd4EYdy9fkBnttDaRkNoxgZc7uAhsPRVVTUfK/X3Ug hqP80jmzfZYKQ/tkCDrkwDp29FtRt30AKL0c9xn3CJlZ0nnuZSENQI0kTE/hV6NmvPTv 5CZ3hNKhmDVMfWKHt8fhMkLbK22n4yPioUgGHF2DHABTPMq2F5fI6zMVRXc3zU/QKKe+ O6aQ== X-Gm-Message-State: AFuF++lpVBTZsYcUfaiWFv//nMyG/CmwkGSSuQQPrFrMwBsMKneclleE KbhQ4nMl6iih2Q7Zz2o0xJzA3vYewlUzQWN+IHilKa9bApVaL24UU7NXNWHo3tzrTvCcesZXUrd GqTB0 X-Gm-Gg: AYBFou3x0Evtdtp9mpVK9X+kcgIhJlN83KDwUGPBZ7GE+8Jzamcg0UzQmBhEbVZACbi oLrcmRb2pkLiVIJv3VlagpmG4fIkxOdZ0W9I2I/ZxtlSsuS0DMDlwkEL03Jv1iJoN0nrA4jJpUC aIayt+enVy1JhJs1hdYYCnvdDTgbBjeC68UGS9KX9YOboa7CkumrO+VfqohF7gP069gKSMSv02j PcbbdZHVaRRZolFOMgc5zKPVux1zy5P7l9xax8F7uj0/EwrFAEaGlo5Pi4AJx5j7GMRR9PjHH1X 3vzBUBQAqPrvQJ9NgblWsth/nGOT4c8IdyRpoCa2JrBSUmn4SiaUXIJekQf8SdGF6GnADWvaS86 UkGUWKMwM1gGcq8N7Nh5xHc6Vxgov0S4fCsYay7DUhXVkfbPu5rZ0LaGl//bxr3yTexlUBxNz99 RcJ8sQQNAPFXTMlb/86yRWScGY2qMOXH7oI1RixSJ9ZaX0U6ggp16DTtYPoeU0NAiK48NbA3LOS zXNoVukwdQorYpZtuXRP6iYf+WgzqD0Cg== X-Received: by 2002:a05:6102:3e8e:b0:7a1:f7d2:e839 with SMTP id ada2fe7eead31-7af1dadc074mr136735137.25.1790190265315; Wed, 23 Sep 2026 12:04:25 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-7abf566d033sm4537450137.5.2026.09.23.12.04.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:24 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 01/13] scripts: resulttool: add 'durations' option Date: Wed, 23 Sep 2026 15:04:07 -0400 Message-ID: <20260923190419.353493-2-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:58:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246541 Extend resulttool with the ability to compare two or more sets of testresults for significant changes in runtime duration. The idea is to allow tracking of performance deltas in ptest/toolchain/etc. test suites over time, especially when upgrades or patches alter configurations for underlying tools/infrastructure (QEMU, meson, autotools, architecture tunings, and so on). This feature works with raw testresults.json files as reported by the Autobuilder and reports generated with 'resulttool report ', although the behaviour is different: - For raw testresults.json files, it reports every target and applies the options to the test cases run against them. - For report files it summarizes the overall runtimes (see the examples below). Note that it is not designed to make any judgments about the usefulness of a given comparison - it only reports the differences between sources provided. Options for the 'durations' command include: - '--labels' to customize the column labels (defaults: 'RUN1', 'RUN2', 'RUN1->RUN2', etc.) - '--sort-by-delta/-s' to list the largest differences first - '--threshold/-t' for omitting test results without a percent delta meeting this value - '--limit' to specify how a maximum number of rows (0 = no limit) - '--min-duration' to omit tests where durations in all sources were less than the specified value (default: 10, 0 = disable) - '--show-short' to print tests below --min-duration threshold in a follow-up table AI-Generated: Uses Claude Sonnet 5 Signed-off-by: Trevor Gamblin --- scripts/lib/resulttool/durations.py | 187 ++++++++++++++++++++++++++++ scripts/resulttool | 2 + 2 files changed, 189 insertions(+) create mode 100644 scripts/lib/resulttool/durations.py diff --git a/scripts/lib/resulttool/durations.py b/scripts/lib/resulttool/durations.py new file mode 100644 index 0000000000..94a86cd26d --- /dev/null +++ b/scripts/lib/resulttool/durations.py @@ -0,0 +1,187 @@ +# resulttool - compare test durations between two or more runs +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import os +import re +import resulttool.resultutils as resultutils + +# Matches the section titles "resulttool report" prints before each +# Recipe/Passed/Failed/Skipped/Time(s) table (see template/test_report_full_text.txt) +REPORT_SECTION_RE = re.compile( + r'^\S+ (?:PTest Result Summary \(Libc: [^)]+\)|Ltp Test Result Summary|Ltp Posix Result Summary)$') +REPORT_ROW_RE = re.compile( + r'^(?P\S.*?)\s*\|\s*\d+\s*\|\s*\d+\s*\|\s*\d+\s*\|\s*(?P\d+(?:\.\d+)?)\s*T?\s*$') + +def parse_report_log(path): + """Parse the Recipe/Time(s) tables out of a 'resulttool report' text log""" + durations = {} + section = None + with open(path) as f: + for line in f: + line = line.rstrip('\n') + if REPORT_SECTION_RE.match(line.strip()): + section = line.strip() + continue + if not section: + continue + m = REPORT_ROW_RE.match(line) + if m: + durations.setdefault(section, {})[m.group('name').strip()] = float(m.group('duration')) + return durations + +def load_durations(source): + if os.path.isfile(source): + with open(source, errors='ignore') as f: + head = f.read(512).lstrip() + if not head.startswith('{'): + return parse_report_log(source) + return get_durations(resultutils.load_resultsdata(source, configmap=resultutils.store_map)) + +def get_durations(results): + """Flatten a loaded results dict into {testpath: {test_key: duration}}""" + durations = {} + for path, _, _, result in resultutils.test_run_results(results): + d = durations.setdefault(path, {}) + for k, v in result.items(): + if not isinstance(v, dict): + continue + if k.endswith(".sections"): + # ptestresult.sections/ltpresult.sections/ltpposixresult.sections: + # duration is per suite, not per testcase + for suite, info in v.items(): + dur = info.get('duration') if isinstance(info, dict) else None + if dur is None: + continue + if isinstance(dur, str): + dur = dur.split()[0] + try: + d['%s.%s' % (k, suite)] = float(dur) + except ValueError: + continue + elif 'duration' in v: + d[k] = float(v['duration']) + return durations + +def build_rows(testkeys, path, alldurations): + rows = [] + for k in testkeys: + vals = [d[path][k] for d in alldurations] + # one delta/pct per consecutive pair: (run1,run2), (run2,run3), ... + deltas = [] + for base, target in zip(vals, vals[1:]): + delta = target - base + # a percentage between two runs that both round to 0s is meaningless noise + if round(base) == 0 and round(target) == 0: + pct = None + else: + pct = (delta / base * 100) if base else 0.0 + deltas.append((delta, pct)) + rows.append((k, vals, deltas)) + return rows + +def filter_sort_rows(rows, args): + """Apply --threshold/--sort-by-delta/--limit to a table's rows""" + if args.threshold: + rows = [r for r in rows if any(pct is not None and abs(pct) >= args.threshold for _, pct in r[2])] + + sortkey = (lambda r: max((abs(pct) for _, pct in r[2] if pct is not None), default=0)) if args.sort_by_delta else (lambda r: r[0]) + rows = sorted(rows, key=sortkey, reverse=args.sort_by_delta) + if args.limit: + rows = rows[:args.limit] + return rows + +def print_table(title, rows, labels): + if not rows: + return + + width = max(len("TESTCASE"), min(70, max(len(r[0]) for r in rows))) + 2 + pair_labels = ["%s->%s" % pair for pair in zip(labels, labels[1:])] + pair_widths = [max(12, len(pl)) for pl in pair_labels] + + print("\n=== %s ===" % title) + print("%-*s%s %s" % (width, "TESTCASE", "".join("%12s" % l for l in labels), + " ".join("%*s%9s" % (pw, pl, "%") for pw, pl in zip(pair_widths, pair_labels)))) + for k, vals, deltas in rows: + # round() avoids -0.0 rendering as "-0" for small fractional deltas + pairstrs = ["%*d%9s" % (pw, round(delta), round(pct) if pct is not None else "") + for pw, (delta, pct) in zip(pair_widths, deltas)] + print("%-*s%s %s" % (width, k[:70], "".join("%12.0f" % v for v in vals), " ".join(pairstrs))) + +def build_tables(args, logger): + """ + Compute every (title, rows) table 'durations' prints, already filtered/sorted/limited, + in the order they should be printed: all normal tables first, then (if --show-short) + one short-tests table per test configuration. + Returns (labels, tables), or None if args were invalid. + """ + if len(args.sources) < 2: + logger.error("At least two sources are needed to compare durations") + return None + + labels = args.labels.split(',') if args.labels else ['RUN%d' % (i + 1) for i in range(len(args.sources))] + if len(labels) != len(args.sources): + logger.error("Number of --labels must match number of sources") + return None + + alldurations = [load_durations(s) for s in args.sources] + + testpaths = set.intersection(*(set(d) for d in alldurations)) + if not testpaths: + return labels, [] + + tables = [] + short_tables = [] + for path in sorted(testpaths): + testkeys = set.intersection(*(set(d[path]) for d in alldurations)) + rows = build_rows(testkeys, path, alldurations) + + long_rows = [r for r in rows if max(r[1]) >= args.min_duration] + short_rows = [r for r in rows if max(r[1]) < args.min_duration] + + tables.append((path, filter_sort_rows(long_rows, args))) + if args.show_short: + short_tables.append(("%s (under %gs, all sources)" % (path, args.min_duration), + filter_sort_rows(short_rows, args))) + + return labels, tables + short_tables + +def durations(args, logger): + result = build_tables(args, logger) + if result is None: + return 1 + labels, tables = result + if not tables: + print("No common test configurations found between the provided sources") + return 1 + + for title, rows in tables: + print_table(title, rows, labels) + return 0 + +def register_commands(subparsers): + """Register subcommands from this plugin""" + parser_build = subparsers.add_parser('durations', help='compare test durations across two or more runs', + description='compare oeqa/ptest test and suite durations across two or ' + 'more result sets, highlighting the biggest changes', + group='analysis') + parser_build.set_defaults(func=durations) + parser_build.add_argument('sources', nargs='+', + help='two or more sources to compare, in the order given: each is either a ' + 'testresults.json file/directory/URL, or a text log produced by ' + '"resulttool report"') + parser_build.add_argument('--labels', default='', + help='comma separated labels for each source (default: RUN1, RUN2, ...)') + parser_build.add_argument('-s', '--sort-by-delta', action='store_true', + help='sort by largest absolute change first (default: sorted by test name)') + parser_build.add_argument('-t', '--threshold', type=float, default=0.0, + help='only show tests where at least one consecutive pair of sources changed ' + 'by at least this percentage') + parser_build.add_argument('-l', '--limit', type=int, default=0, + help='limit output to this many rows per test configuration (0 = no limit)') + parser_build.add_argument('-m', '--min-duration', type=float, default=10.0, + help='omit tests whose duration was under this many seconds in every source ' + '(default: 10, use 0 to disable)') + parser_build.add_argument('--show-short', action='store_true', + help='also print tests below --min-duration, in a separate table') diff --git a/scripts/resulttool b/scripts/resulttool index 66a6af9959..6e8124529d 100755 --- a/scripts/resulttool +++ b/scripts/resulttool @@ -44,6 +44,7 @@ import resulttool.merge import resulttool.store import resulttool.regression import resulttool.report +import resulttool.durations import resulttool.manualexecution import resulttool.log import resulttool.junit @@ -64,6 +65,7 @@ def main(): subparsers.add_subparser_group('analysis', 'analysis', 100) resulttool.regression.register_commands(subparsers) resulttool.report.register_commands(subparsers) + resulttool.durations.register_commands(subparsers) resulttool.log.register_commands(subparsers) resulttool.junit.register_commands(subparsers) From patchwork Wed Sep 23 19:04:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78125C98310 for ; Wed, 23 Sep 2026 19:58:44 +0000 (UTC) Received: from mail-vs2-f43.google.com (mail-vs2-f43.google.com [74.125.227.43]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3029.1790190268925546766 for ; Wed, 23 Sep 2026 12:04:29 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=NVmikPiI; spf=pass (domain: baylibre.com, ip: 74.125.227.43, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f43.google.com with SMTP id ada2fe7eead31-7856441f7d8so726240137.0 for ; Wed, 23 Sep 2026 12:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190268; x=1790795068; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=kvegQdOSotauanZ6ZVKTFC0hhoNKqEJIQOlMCUPeK1Q=; b=NVmikPiI2n0QjV2dczgahQ5J94O2nbfZOTAW5/E6g7okaMDQVMJVECuyC1TiNX7cuC kAHPRqjblpG3qm75vL7O8qSrV5VH6mYvT46euSv4uBr3MXTvkQ96rAlQrz1GMmelkzgx yFjLn6FBHyigiYfKvI68c9dS3CGu5pVUPI5CBAA58cWCYeODtXvEN57YW2/sPlvGW7yt NGP+tU9tRBjzDDE2Zq8xOcbN4fu50vmy4knvMxS2mr6/ZU9+nwjNEn9cJ/+NUaIh7Vw3 sWXZgKHchcgY6qIYKhnRT3+iIdfZ1KgwB093v/KDQN8Ix0s3l1g+99oOdBM5deF3kOdH mZEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190268; x=1790795068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=kvegQdOSotauanZ6ZVKTFC0hhoNKqEJIQOlMCUPeK1Q=; b=IVfHRFDli2nMQufYNsOvHo5d+EwaxpwNUxqI6/YDuOMhcX5Q3DzSAp72PnjihoNoIZ fGvUI5CKXs55LkveMLIFASB7rnwFCSe2d+1j6nMZ/diCpF/tonUZ2kR0O73Ap1z+zlWn CcGKT8Dk95NYOuRTyEBB5yO/e8MMvYbHIKB+kNCNWNm/HhB0nf+haif0SzbDf2Z3YR9i Wn7axUfN0FJ9uwChzvgLCR7wXCOY/5UXBqXAhD/yVnvO7RUJR03FuY5V2NEtVEqrpjSz ddeFjvh3Pw9xYsV3ptJbt7haekdse9kLXB3hFWLTYHGza0nP4bWo48lJHK71wzpT/+I5 M0jQ== X-Gm-Message-State: AFuF++ksZie+gxBclyYx7kxoYrqVsaSNTgCGYCxjlZB2gPMuLK3ywXfZ taqUHDDxhga7yns8Hy0jFbf8V4qMhmhPhY+u09WGtvQO4d9LvP+4FqE/XG12WL6w6TjNJYOnJw6 BHm1g X-Gm-Gg: AYBFou3ey+j/Hc+a+uX1R2eX6AW/qJBNyf60+t99wn1skGC5K7nGb/0/sF3wyk6jh2i kxQuEDyx9y2DDfrkLybyeiqeTdz1A1SdY0IzzDXGwcUDVk3RxdpjycNCcivOEoTFO4WlP4b8zxL QhY/AhC8iTiFfDz2ZKREHG/zOEStRH0/gVnNCdOTM2WTuCwsEwgaFLHjQRUl2em4BOfNu7FqMw4 /FPzOYWQPe58/pPnnwHVt9dRKkS2lTNEGAkeOEzmKzWYkkYNaNh5kqdc0m96qeDPBpLfF8Auakr YZI+qmE0+hxJBKfRBEDHcmW2mTW85KhwOQxXBBhrkeSmlSoOPAyXtr97muh+Ee7MvEu3xWr3xJu oxPdUgrxm6DMHLHVDOV94Q2VNhjQnb3mUJRn19KXWLcKNl0pF+ubPPn7p8wdTsIj6y2M+G4MLrL 4I4eLeEqi+QlPt8NeYt9P6iYj8ZHTcz7B+SVfy3dmsIIEV3SujFnq29Dw72nmTjpYXCG5BMUPH7 AK3Z2eWPMSqgR43CX7KCeam8eJR0V+HiKADKQQo3eeF X-Received: by 2002:a05:6102:3fa8:b0:7a1:f7e2:aae7 with SMTP id ada2fe7eead31-7af1edef629mr119869137.25.1790190267630; Wed, 23 Sep 2026 12:04:27 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-7abf566d033sm4537581137.5.2026.09.23.12.04.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:26 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 02/13] selftest: resulttooltests.py: add basic durations test Date: Wed, 23 Sep 2026 15:04:08 -0400 Message-ID: <20260923190419.353493-3-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:58:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246542 Add imports for the resulttools durations.py module, then add a basic test for checking that testcase names and durations are properly extracted. AI-Generated: Uses Claude Sonnet 5 Signed-off-by: Trevor Gamblin --- .../oeqa/selftest/cases/resulttooltests.py | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index b03a880b2b..a380d08156 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -11,6 +11,7 @@ lib_path = basepath + '/scripts/lib' sys.path = sys.path + [lib_path] from resulttool.report import ResultsTextReport from resulttool import regression as regression +from resulttool import durations as durations from resulttool import resultutils as resultutils from oeqa.selftest.case import OESelftestTestCase @@ -18,6 +19,8 @@ from resulttool.junit import junit_tree, PtestSummary import xml.etree.ElementTree as ET import logging import json +import tempfile +from types import SimpleNamespace class ResultToolTests(OESelftestTestCase): @@ -669,3 +672,28 @@ class ResultToolTests(OESelftestTestCase): # Package testsuite should not exist (no ptestresult.sections) package_suite = testsuites_node.find(".//testsuite[@name='Package Tests']") self.assertIsNone(package_suite) + + def test_durations_get_durations_extracts_testcase_and_suite_durations(self): + results_data = {'result1': {'configuration': {"TEST_TYPE": "runtime", + "TESTSERIES": "series1", + "IMAGE_BASENAME": "image", + "IMAGE_PKGTYPE": "ipk", + "DISTRO": "mydistro", + "MACHINE": "qemux86"}, + 'result': { + 'test.Foo.test_bar': {'status': 'PASSED', 'duration': 12.345}, + 'test.Foo.test_no_duration': {'status': 'PASSED'}, + 'ptestresult.sections': { + 'bash': {'duration': '5', 'log': 'xxx'}, + # a timed-out suite has " T" appended to its duration + 'timedout': {'duration': '30 T'}, + }, + }}} + results = {} + resultutils.append_resultsdata(results, results_data) + flattened = durations.get_durations(results) + path = 'runtime/mydistro/qemux86/image' + self.assertEqual(flattened[path]['test.Foo.test_bar'], 12.345) + self.assertNotIn('test.Foo.test_no_duration', flattened[path]) + self.assertEqual(flattened[path]['ptestresult.sections.bash'], 5.0) + self.assertEqual(flattened[path]['ptestresult.sections.timedout'], 30.0) From patchwork Wed Sep 23 19:04:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A63DC98304 for ; Wed, 23 Sep 2026 19:58:54 +0000 (UTC) Received: from mail-vs2-f40.google.com (mail-vs2-f40.google.com [74.125.227.40]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3030.1790190271602473449 for ; Wed, 23 Sep 2026 12:04:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=gBjIX6uq; spf=pass (domain: baylibre.com, ip: 74.125.227.40, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f40.google.com with SMTP id 71dfb90a1353d-5c9b0bdf5b5so571313e0c.0 for ; Wed, 23 Sep 2026 12:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190270; x=1790795070; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=PU9YjQx0iNzugWOiJuyXtJ5V8EbsG4G9b4A0qWeen14=; b=gBjIX6uqeGLqqc5bElmmVbI8XMmU5UAsPgiPeLQ/1GLU647rfbaqlfV32nqyEj7qpq dYHfJ3XJzLUDl2IVligQLewgfPCQMjLR6u8is8DOqe/aMgME2gOA6HtUzOmRRmM2dpdg 3WFO+WAcBCWmIfvXKnKVa8IbkX08h3xXNMqVsIetqKXgroJh3CzYGXaFZC73ARGgJPpH VIvbrjH8DRpEX82xkfCt13WgCGy2xZlOIe0FUvgNJL0I9Eps6MEKE5RYX0io9TJFn9bf S6tUXprTCqtRifZnwemY7CAIKjrZgNBJwaD7yaIJbNxqFMurkB00mvF2dKqQIR48uWNP So3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190270; x=1790795070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=PU9YjQx0iNzugWOiJuyXtJ5V8EbsG4G9b4A0qWeen14=; b=gTbjrGNBJ1qLtx9D3h92uKOZvEsrcKva8TpJS0NmggvKTQAkGflUVVMi3WXw9TWdSy 9G2kcjav7Fqo3tvKPlsWl8J9ZqSWeenH8tRG2HeSAuhk7qcUBmeuutHAOgriLzBdmBIv Tr46FiHHkzFghFlvFDMGJ8EdfpgA4EDZsFRwCe/8DVSrbdyEEzCAj6BEoB7FP+lhM0NU xyJcxzzLnBwVfkVj1r3/aZ+ld0xDDgmG5sJ/EV/PXRHIKQNpATnFuQkNedkV0KziHW7b jNSk1EtqwCdZwzN+d35p6qyZih2zGpIzYNAlym/mdAoxMcz6YHT2ch7mbkWz2WLNA7xD Xm0w== X-Gm-Message-State: AFuF++kricluZ3iHD/wEA+Lzl3ITDt6Is2u5AcVxID+FF4ATmWPcQ0jr /Vdo+k52kEDyIpVhh7H3mf8FRe0lCSDb6DwpqCJ4LHny/QQxka3ENtKlxc8cMZuUU8AdPKFnaUf AnwlE X-Gm-Gg: AYBFou3ZqWta6g8ZYVnd2elstoWpcAAY8Wul8W6Dnbuf4Ni6H4A2UsE+mugtG7wU83x qlhcaR529t/sklZQdLGyKwBgfZ9a2UDZhDjGjKan48Xv7S/l2A+Pc+vvE5VnSXiakJ6r0Jips5X 64pmS16S0V1B8wLxx4yx0HxUjmgpglnWU3aIHzCLRUQaSQn1LrCY2UIZSgx6FV+/5/0zXyiULMN FVFDdzMErKukXvrRuitjKnxYnboAXKv30o1TxwkH/bZYBYXYU+aY4JzA2rX0Y12oapgAFFJSrpi jMjH/tlxH3PeTLUTEYoG74ilrFT+2i4woYeeOaAeIaEEkz1XNo+Eznm+atm9k/sCpxkJ9YjcU23 Jg1PxqEVphhhrQHRasbu5xQ7i0UluubQUGXp+u+2spgrwlfLT/saM58QCYhDN/juojZXv6gB+Do kmrxcJQaDy/os3Nu5uQ+yDutLLS2BYWjAQbE5mxhmikJuCIjyP8HRREv7iso7Vv0CK1M6KBJQjG dPv+m8o0qN9Rfmjg8qr+TuMHDVFNK0FxA== X-Received: by 2002:a05:6122:2985:b0:5bd:71b1:d5c6 with SMTP id 71dfb90a1353d-5cb0807358bmr215347e0c.4.1790190270276; Wed, 23 Sep 2026 12:04:30 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-5c9f04c2a36sm4356028e0c.5.2026.09.23.12.04.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:28 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 03/13] selftest: resulttooltests.py: add report-log Time(s) column test Date: Wed, 23 Sep 2026 15:04:09 -0400 Message-ID: <20260923190419.353493-4-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:58:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246543 Add a test for resulttool.durations.parse_report_log(), covering correct Time(s) column extraction from a 'resulttool report' text log, and confirming the unrelated percentage-based status summary table above it is not mistaken for a Time(s) table. AI-Generated: Uses Claude Sonnet 5 Signed-off-by: Trevor Gamblin --- .../oeqa/selftest/cases/resulttooltests.py | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index a380d08156..b3c6f3329b 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -697,3 +697,36 @@ class ResultToolTests(OESelftestTestCase): self.assertNotIn('test.Foo.test_no_duration', flattened[path]) self.assertEqual(flattened[path]['ptestresult.sections.bash'], 5.0) self.assertEqual(flattened[path]['ptestresult.sections.timedout'], 30.0) + + def test_durations_parse_report_log_extracts_time_column(self): + content = ( + "==============================================================================================================\n" + "Test Result Status Summary (Counts/Percentages sorted by testseries, ID)\n" + "==============================================================================================================\n" + "qemux86 | someresultid | 6 (86%) | 0 (0%) | 1 (14%)\n" + "\n" + "==============================================================================================================\n" + "qemux86 PTest Result Summary (Libc: glibc)\n" + "==============================================================================================================\n" + "--------------------------------------------------------------------------------------------------------------\n" + "Recipe | Passed | Failed | Skipped | Time(s) \n" + "--------------------------------------------------------------------------------------------------------------\n" + "acl | 1 | 0 | 0 | 1\n" + "bash | 43 | 0 | 0 | 48\n" + "timedout-suite | 1 | 0 | 0 | 30 T\n" + "--------------------------------------------------------------------------------------------------------------\n" + ) + with tempfile.NamedTemporaryFile(mode='w', suffix='.log', delete=False) as f: + f.write(content) + logpath = f.name + try: + parsed = durations.parse_report_log(logpath) + finally: + os.remove(logpath) + section = 'qemux86 PTest Result Summary (Libc: glibc)' + self.assertIn(section, parsed) + self.assertEqual(parsed[section]['acl'], 1.0) + self.assertEqual(parsed[section]['bash'], 48.0) + self.assertEqual(parsed[section]['timedout-suite'], 30.0) + # the status-summary table above (percentages, not a Time(s) table) must not be picked up + self.assertNotIn('someresultid', str(parsed)) From patchwork Wed Sep 23 19:04:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C050C98310 for ; Wed, 23 Sep 2026 19:59:04 +0000 (UTC) Received: from mail-vs2-f42.google.com (mail-vs2-f42.google.com [74.125.227.42]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3031.1790190273458535601 for ; Wed, 23 Sep 2026 12:04:33 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=V3bOJPjP; spf=pass (domain: baylibre.com, ip: 74.125.227.42, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f42.google.com with SMTP id 71dfb90a1353d-5c67e4d9197so812399e0c.0 for ; Wed, 23 Sep 2026 12:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190272; x=1790795072; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=rq2PUkUgDF4QSQbcT7VM4ag8ooc0+5lyOlpRL8Qfk0c=; b=V3bOJPjPbFdZ48JPtAaYZWJtm8qs/NrhOWaGJqyBdVuAR/0yyJixDZs1d4zmkZuLZ+ TZn8IQV7ZaNjTrIeYbUUmCEFtBIYmbaf8ftiTukXOoyqh7qqt81MqfW6DkM1u4s4yjBD Dl/PxDUhODcgOCMOnxqzqdFEnk68k0xFik0U4liyfFOtx8zUvk66+AYbmIbWXQ3YIhxN x9SJBKTNFuLcvsPKLRGEm2398LBfeELLLHzd6MuiKb/ojQLPRWWplOHavFHMfLNjF4ZS pSPMizh+pmOugxD0EiFUcM4r8FkoMZHUTVPflLq+RvuYRDVp5V6YY85deWyAzv1+l5bW qNZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190272; x=1790795072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=rq2PUkUgDF4QSQbcT7VM4ag8ooc0+5lyOlpRL8Qfk0c=; b=TXg6RMYnOuLIjDpyiwe6n4Pv1/MFTm7ZJDC/8TQFhW3HdyDMGPl48ST9x1cjn7LCJc pMSw9khm6jQA9Ryo3ZM95ZlW9+twCdyLycrRAE2wQhMaZIcVuJrHXBkuEDMej1h3T5Sv nhO35EdafK6qq6GLwihTkOI3gZuXse4KfppBd/0dNOV7jN+fUfDRDJld0YgHevtf3I9r 3I7ECWYgy4z4PV2PeLgYPoW8J7vzRdEYJlOb1Ou6oMfLzdUVfh8QPIo8SJlFCpphFhJ6 AiemeEIXkTgDHZFk/OGVlIZ3ihWeBr6vL/Ib9UZFzN4teVEcbvcp+CIybolSYfjU/btf qy2A== X-Gm-Message-State: AFuF++kd1m0asu9mXsZ7PkzygAzs1KW+SfSDif7Eoweeb+mbwPyGxY2n ewXurOgApYqYoAya7vcJx3vrNJrGAwQxCkD8NSA6nIWNigZC92ebo28QsvW9t+XWpbbvohlKxUu rwNRR X-Gm-Gg: AYBFou1o4ToBQdzwQuMhVQjP7EjyabIb+CHBxWO3cUV9roji7cX0+vLzbxzy2peEgSD +aqIxtwXtjgRbyyky+gySp3zlgpnpf6mJJPl34LqOhcbHn+OQOvbC9USyWxVLPWPxrz9VTx9BlC Scds8i4bgMpupPBC0QY+hwixBXNBT1Z+yNrchE+ZJBmuMvkV+QH/BWOIPHin/juFWeyLTRnPPJ1 2K83r2j71Fn9kEM9w9yFCCa1d6pd9BeNbE1sN/k+R6p3bBHk7cOd2JX7yuHHlJOnvr4UBeSHwks M7gr2UDz2Z5RRWVez9jssV5w0UquA2FGB/G5YZt2zdrPOLz0hljRsweFGp9rwPF4X+qtRfbFrgm BLUsJbUzlRk+eDUICA7V56l26+hCAqvYezeHcnNDJ4aeKRxb8heBO2Xhawfa9ysXtfSIk2o+v/e AVd5QluhK+TRk7mtv9WT5cvI0lB6s56eOLc0J35A7sEVIVt1r7OLTt+/hkMCSlQ/cDDB1paqIHz r+tnO2KxluGVyzDJpRytMhmDS0AViRlRg== X-Received: by 2002:a05:6102:5e91:b0:796:1ed:ba47 with SMTP id ada2fe7eead31-7af197369admr189441137.0.1790190272186; Wed, 23 Sep 2026 12:04:32 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-98517a8ecaesm4015858241.12.2026.09.23.12.04.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:31 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 04/13] selftest: resulttooltests.py: add zero-round pct suppression test Date: Wed, 23 Sep 2026 15:04:10 -0400 Message-ID: <20260923190419.353493-5-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:04 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246544 Add a test for resulttool.durations.build_rows(), covering suppression of delta/percentage reporting when durations round to zero. Times are reported in seconds and fractional times can be calculated, so we don't want to show (for example) RUN1 = RUN2 = 0 but a large delta because one run was many fractions of a second faster. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index b3c6f3329b..dfaa4edbba 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -730,3 +730,15 @@ class ResultToolTests(OESelftestTestCase): self.assertEqual(parsed[section]['timedout-suite'], 30.0) # the status-summary table above (percentages, not a Time(s) table) must not be picked up self.assertNotIn('someresultid', str(parsed)) + + def test_durations_build_rows_suppresses_pct_when_both_runs_round_to_zero(self): + alldurations = [ + {'path': {'tiny': 0.2, 'big': 100.0}}, + {'path': {'tiny': 0.4, 'big': 150.0}}, + ] + rows = durations.build_rows(['tiny', 'big'], 'path', alldurations) + rowmap = {k: deltas for k, vals, deltas in rows} + # both runs round to 0s: no meaningful percentage to report + self.assertEqual(rowmap['tiny'], [(0.2, None)]) + # normal case: delta and percentage both reported + self.assertEqual(rowmap['big'], [(50.0, 50.0)]) From patchwork Wed Sep 23 19:04:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 789FFC9830B for ; Wed, 23 Sep 2026 19:59:04 +0000 (UTC) Received: from mail-vs2-f40.google.com (mail-vs2-f40.google.com [74.125.227.40]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.2998.1790190275282402111 for ; Wed, 23 Sep 2026 12:04:35 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=CZtGHy9h; spf=pass (domain: baylibre.com, ip: 74.125.227.40, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f40.google.com with SMTP id 71dfb90a1353d-5c9d8b323d9so1017862e0c.1 for ; Wed, 23 Sep 2026 12:04:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190274; x=1790795074; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=a9dwOogtV8XWeBI+Q/u4h9qkg5SSGOPlSuB6TmrZmQU=; b=CZtGHy9htA/0mKTbfuNZ5RE7g0tt4xaEa0iz9MwqnfI1HPql8lsthxpbROTivPzHgz Hyg3edkIugWKd2Qf51iYsGuse1bP0tyJ7KWq2LUrTbGd1djnZktwtmTdyuV2HpDAXJHS HBX0c6C7Rn02fFiNuQHsA83b3vIhz4czDYFhIMoQ/+r8cpx59iCWgAq7Z+UOGB4eYCSC DeQNwlxjBwnZdWVQt5jjBo9w1xB8wVBBlFHMV9mN9aMNNH29CdQ/Hrl4rZaWMp323tfZ w3yRXydoZGi7DCxyWw8BUtCWvgTSdAftijQ2zkDHajNVtR7l7O+SNrHLtacC/TeudQuA IynA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190274; x=1790795074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=a9dwOogtV8XWeBI+Q/u4h9qkg5SSGOPlSuB6TmrZmQU=; b=MF4KwoPo85q3jCBuJoLwK3UMQnhiVBc1glatid6XowyJe9I3QiKUX4actMD8JMbKJj YlGU34M7gXjFp1wAStiQ1u+yAgAMww/OYqFeBibSuRXZoU3T/fqjTur7ow6NX+XBh+2c KsOmk/ftJU4Eu9jegfu8hlZy8EEzZFNVbu4Qm4BR9/aRsV8dRBeJYXquunTTErOgUKpP /tTcCs/N2Z4rQhsGO10raBGOpLhh46J8psWT1BapEMTt7PZjpN7C9ryEHHhRNGo3gU/f eR85y+uHzF4gyr4xLLAw/7Ft2+Sc2rBFXgLKe94p2JwMRrAnstjHl3HO+6bJ6kreIebH Hgpw== X-Gm-Message-State: AFuF++m9lbm5VUH92PayKMnLtvEXyIJCGsRHRb4wsXqJ1Ap0o87UhKDk D92O/x3ksKapvoEA4lJkz+3yCSSaw65JtgGsqfbMd3UjHgCqejEYC7HWt68vV4P9vB5aK+xrwB2 y90c+ X-Gm-Gg: AYBFou2KfDkFmJQoMmeI0z/Qkz0wu+r9RigZBGZBlEvumjfJxXmaiqpXsVnzfN1a0Sj As6I2NsnlCiCOHdIjMKuy510gCivnsaZ680uhWLjEPCxHgezd2SPez+LFCQnAiVe6spMN3Woz7a p7yswUhl8fE1JaveAhLuuk3v946xvX1qPragEBHnjbl4YcnwMvPJtekj0Krp+oFmpqLeBAfqV6L HUxRlYXo39qhIHCgvwCAocrMxlTfje2DnYddH3alAGtBcc+NGBtYJ/AzagY+l+re68GTK1JDn3h 6j6+ny5CKhPJOWGR/zV1XrAmw8s1aISamc6yqn4LkOzMa0qgQ8HpuyPguT7yamdeLOSr7T4V3mk 6I7R4DFlpTK1EmNWuCpxlG+usxqdtOq9qdcI4RqlhS9TcHgSTmwt93Raupz5WAq/RIAqkilKzkc C30TbLnGG4IlqWRJtUddoH/rc+cASvA61WHtJxjf6YHEJKzZB2M98NQG+zAYE7SVRasmcPqfhQi pfNwfvPRr8Eh6l2IRdom7LQYl7Slosi5lM= X-Received: by 2002:a05:6102:3749:b0:7a7:19aa:8a87 with SMTP id ada2fe7eead31-7af1f208a87mr143638137.34.1790190274154; Wed, 23 Sep 2026 12:04:34 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-5c9f03c7d18sm4098903e0c.1.2026.09.23.12.04.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:33 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 05/13] selftest: resulttooltests.py: add consecutive-pair delta test Date: Wed, 23 Sep 2026 15:04:11 -0400 Message-ID: <20260923190419.353493-6-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:04 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246545 Add a test for resulttool.durations.build_rows(), covering that comparisons are only generated for consecutive sources (e.g. RUN1->RUN2, RUN2->RUN3), never something like RUN1->RUN3. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index dfaa4edbba..28acc939f1 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -742,3 +742,13 @@ class ResultToolTests(OESelftestTestCase): self.assertEqual(rowmap['tiny'], [(0.2, None)]) # normal case: delta and percentage both reported self.assertEqual(rowmap['big'], [(50.0, 50.0)]) + + def test_durations_build_rows_computes_consecutive_pairs_only(self): + # 3 sources: deltas should be (run1,run2) and (run2,run3), never (run1,run3) + alldurations = [{'path': {'t': 10.0}}, {'path': {'t': 100.0}}, {'path': {'t': 11.0}}] + rows = durations.build_rows(['t'], 'path', alldurations) + k, vals, deltas = rows[0] + self.assertEqual(vals, [10.0, 100.0, 11.0]) + self.assertEqual(len(deltas), 2) + self.assertEqual(deltas[0], (90.0, 900.0)) + self.assertEqual(deltas[1][0], -89.0) From patchwork Wed Sep 23 19:04:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AD31C98304 for ; Wed, 23 Sep 2026 19:59:04 +0000 (UTC) Received: from mail-vs2-f43.google.com (mail-vs2-f43.google.com [74.125.227.43]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3033.1790190277088612240 for ; Wed, 23 Sep 2026 12:04:37 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=DW/yJ1QD; spf=pass (domain: baylibre.com, ip: 74.125.227.43, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f43.google.com with SMTP id ada2fe7eead31-78564421fdbso459116137.1 for ; Wed, 23 Sep 2026 12:04:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190276; x=1790795076; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=OW7HqZBhcYssfenPdoPQSFJd/w/Rf16DhcBvS444WBM=; b=DW/yJ1QD0nnkWmVO06bRHAZGUfLbIsVZ7WAIxWvAcF1v6JATIiml8ei2HAO9+3foo4 uFRCD9XD74kKkNPdyU5MANXaqYET7P7NYbS7QHGa8G8rR6CyZ3ZOKCIUX3WC7pnHvSAY kzAoqK9o8/+Ah+YzUj0IW66z2VRkOVJifZE5+T6ZVbspjD5NL8n3t6p9lBc0ouWZRLtz VmcuepwT/sTU5dpLVvUnGeWNR6J4pjvUKK+opFABab9W3S8+G5fBLvvngtzUe9Ak8/Kv Ux8XN3aj0BfD/QWW0xkK3k0l+Q9HkyPd3++cv2MUVSwdFkmjWkmbICa/TlUuTayGwbAZ xqcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190276; x=1790795076; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=OW7HqZBhcYssfenPdoPQSFJd/w/Rf16DhcBvS444WBM=; b=h1AIvzFij7xo64v7g2FB+qR4FMmBdax9pVRO2p3X4Xz6QJa8kVthED/7cuLJfR27oA Rxa0JWpFEMZbnXRhUkQBGQRgR1mWnZguKUb+faZWUQaAvN1YbcD7jAdA6Kst4cRAhVS3 F58iILNr/grYUGuzTl84N11wlkgNXxTLNt8sibUPj6Ih0NhtpxCeiAJ/jd8/Dj+xIM3L iMhAEaoCdty0wYqsfSurEXDmnhkcjWu053nXPkU3sg/UjKYaP1tcJTtrl0EgXfCAVLQ+ /XFiEKgrR3DxAIXnbUfxqy3JJ7pY92QYPG9+4n3VCTmoqCORRxUwIBpdvCoGk5tsklAg XoVQ== X-Gm-Message-State: AFuF++lzd+OggirixJFHLcWiBozDvMbARt3JUo2WaqovCDje7BGCjKow Jah9tdtef5vJ5ueEbMOAhHcUfBIUGOIdGvD22+y8l62IGq8U4yFPZfk8kzYCl20BboniKc+ZrUn +Gh+C X-Gm-Gg: AYBFou1ulo3Yf75TNGU2oGWZB3GfXoEEtQJ/bjyVzMxAxouUnf4Q3qiFD1fEgjGVCyI WcbO6F7hX2NZK3+B52hicKcpvIg1R2YKjdYwV7NqZw9EPnr1mlt6sx2XtT7/WZebtn1SGi81jRQ ZRO6jZnwYlOt3lWJMBnZ61l8oM7ZwlgHWj6L6NMv6btoO8mhZ/yWAWIyBKNVLXYX61ZtMKHhUMx I691ZYYuKBvDc/GEZeLffUDVx2Fri/9XrbHU3Vbdv9n3OYxwiMLM3B01hgEEFyEfIgm1b6Lb6G4 1FrAi3BXCBlJGDmVgbcjpK7g4RZh0bjaD+gsoB2Et39MV8lST5qrHFxo+m9DbTM7E1pv6+TEfno Qbiy+k0Fo3J7/aSpqmbaCQui22bAUZar/4gx1hBUaNM42NUf8vcP3Y2GM7PE77oG48ceOcxG3oK kfpmytm5dW9QhhKH+6TgRsWklQ/QkW1Om9iHHjX9jVv2/mPndhEnQH4D35kqUc1fSW+wyOKB9J/ 4AkujWQbgman643Lfsrm87VqMPaG3Zjip9D1hVa7fN7 X-Received: by 2002:a05:6102:4b17:b0:7a6:a24e:8140 with SMTP id ada2fe7eead31-7af1bd77fbcmr156627137.14.1790190275980; Wed, 23 Sep 2026 12:04:35 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-9852407d1f2sm1063533241.6.2026.09.23.12.04.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:35 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 06/13] selftest: resulttooltests.py: add threshold filtering test Date: Wed, 23 Sep 2026 15:04:12 -0400 Message-ID: <20260923190419.353493-7-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:04 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246546 Add a test for resulttool.durations.filter_sort_rows(), covering correct filtering when the '-t/--threshold' percentage is used to ignore result rows below it. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index 28acc939f1..a884de3949 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -752,3 +752,13 @@ class ResultToolTests(OESelftestTestCase): self.assertEqual(len(deltas), 2) self.assertEqual(deltas[0], (90.0, 900.0)) self.assertEqual(deltas[1][0], -89.0) + + def test_durations_filter_sort_rows_threshold_keeps_rows_above_it(self): + rows = [ + ('big_change', [10.0, 20.0], [(10.0, 100.0)]), + ('small_change', [10.0, 11.0], [(1.0, 10.0)]), + ('no_change', [10.0, 10.0], [(0.0, 0.0)]), + ] + args = SimpleNamespace(threshold=50.0, sort_by_delta=False, limit=0) + kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] + self.assertEqual(kept, ['big_change']) From patchwork Wed Sep 23 19:04:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6960FC9830B for ; Wed, 23 Sep 2026 19:59:14 +0000 (UTC) Received: from mail-vs2-f40.google.com (mail-vs2-f40.google.com [74.125.227.40]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3035.1790190279627835827 for ; Wed, 23 Sep 2026 12:04:39 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=Uot4EL95; spf=pass (domain: baylibre.com, ip: 74.125.227.40, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f40.google.com with SMTP id ada2fe7eead31-78a4c5c582eso526297137.3 for ; Wed, 23 Sep 2026 12:04:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190278; x=1790795078; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=B/rcPkV5E6GfN4glogKw7ve5nZfAHdS8Seqwj0/4QZs=; b=Uot4EL95cm3aeTOlW4cz7FYbPTr1JFd/SQpD1YkBvJQ2cVhppEjozCE2OaX4ROksdg Hduqd57FdcvFFf433U6Eb/Vhc3ZzX7PKnyVmgZua92VeYvjLqLBtKDcU7lEF4u3rwD5+ iQZyACsHGf1RGY/I6mso+PzqfOwwqFsBK2swK7xpJnygPxdiZR0wq3USJK2ll2FtsYwz sY2DUz8JPWYv09c5bKSw8A5pWDcYFCzqdcZI3MbuW2jAei1icIDSKuYGoT6Hk65geA+Y 1xAsx8dEY1D4SeWiXiTy/u+atYp3U/6pi1nhx1TUHNp3IgBq2twy1r1ytKRykmFBhz+F 7N2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190278; x=1790795078; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=B/rcPkV5E6GfN4glogKw7ve5nZfAHdS8Seqwj0/4QZs=; b=D5dg8NcZ7cJjxW+6y1zrBr5cUhEdBpoF/uexOtXN1OaEu/i2YTlD1qi8u7rS+bsh1/ zwc9wFgZMzFbIN70yVFrwm07hQ3CG4R93JOvfiOtCYh05czChhAY73nOcpQWyY6hlNVt v7dlzXWehvOc9Q3d3aZy30azbHRti/Joo7b7pavTOxeNe7hJF8/vo5EUtKIU1mCmI2zE PvxFfRAM1Bi9xfR7gO01dP2lM84Cide1vw01OnDVP9EYYXRrLKqDZjNyAIhMTcDfv7Yw NRMrnevhreY6hSN1ouX19Ol4E2Bvmbr50/Mw0Bl6Dp60Y4XEiBWjDWTI0NBbTfM4sJG5 gVuw== X-Gm-Message-State: AFuF++kBR8RjOaDxU95ouKNv9wuWUigpVyHxzJkZmHpZWVRPKWVKQy30 v9dHzI4tOWBE6Jl1iI0Wv8eXEgd1SbFQKnsOCys/NrQo4krEqCT2C5shzhaoHl1/0ChVe1yEdjW XspSR X-Gm-Gg: AYBFou0h2DT4wouERdMeHq547C+lNVlCqGmtDoFR1V2VJQZrrKCgZdGIEqeWooETSrD iPxoWam0Dl5tyr+1vQVBorKW21mxZrm8yp8dnwgQQHmFpovsF8UCj8PxH81AzlSGMIp4J7saPuS /rnnHXmp9N4JIgu8SD/GCqtLGC2M58QbzkdYWvBkLwPB6ttZEwwswtHZhU/OAfbRtecj5trvfRp ClbnH/waI3Zg3UCHE2PxrQYZwnPNLmzXqXFq9xwzYzetijLAtJOqKJx1GgxUZBPTnd9JsA6bYyL ACouFGY4F7NIJ7W9KFhuH38/BSvLpKh+GDR2xylaWFUTX4/rhdRbLIQRN9z+5vpDKvbgSDRDtXM SMS/yZnU8NqHGyYSs5Q/ZmBjilUBUe8yPGaQk56xZM3qjFGO4BMnUSoQkEBgaQujHolTOzgRWQ1 1a/vrPn5AZsct0Uu3yzgloIEAguzPI/EqtcSZYi7WqGPY1ZG4PkdBnvJTk+FEQfUh533i3amuSa PWdNBx3trXNSdf85T1uhRkOu99K32KMtA== X-Received: by 2002:a05:6102:4b17:b0:7a6:a24e:8140 with SMTP id ada2fe7eead31-7af1bd77fbcmr156706137.14.1790190278421; Wed, 23 Sep 2026 12:04:38 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-5c9f053a418sm3999471e0c.13.2026.09.23.12.04.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:37 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 07/13] selftest: resulttooltests.py: add threshold-disabled test Date: Wed, 23 Sep 2026 15:04:13 -0400 Message-ID: <20260923190419.353493-8-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246547 Add a test for resulttool.durations.filter_sort_rows(), covering that a threshold of 0 (the default) disables filtering entirely, so all rows are kept regardless of their delta. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index a884de3949..76bcc53711 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -762,3 +762,12 @@ class ResultToolTests(OESelftestTestCase): args = SimpleNamespace(threshold=50.0, sort_by_delta=False, limit=0) kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] self.assertEqual(kept, ['big_change']) + + def test_durations_filter_sort_rows_threshold_zero_disables_filtering(self): + rows = [ + ('big_change', [10.0, 20.0], [(10.0, 100.0)]), + ('no_change', [10.0, 10.0], [(0.0, 0.0)]), + ] + args = SimpleNamespace(threshold=0.0, sort_by_delta=False, limit=0) + kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] + self.assertEqual(sorted(kept), ['big_change', 'no_change']) From patchwork Wed Sep 23 19:04:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99100 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69B6AC98304 for ; Wed, 23 Sep 2026 19:59:24 +0000 (UTC) Received: from mail-vs1-f48.google.com (mail-vs1-f48.google.com [209.85.217.48]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.3002.1790190282184619979 for ; Wed, 23 Sep 2026 12:04:42 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=aRHXPFzB; spf=pass (domain: baylibre.com, ip: 209.85.217.48, mailfrom: tgamblin@baylibre.com) Received: by mail-vs1-f48.google.com with SMTP id ada2fe7eead31-78fb1fb9508so158929137.1 for ; Wed, 23 Sep 2026 12:04:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190281; x=1790795081; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=Z1rHm0E3nzKP7ENhE5GmbSZVJf6tc/xv+MsIZK7AOMY=; b=aRHXPFzB63GFx3b6yIChrhwrjrVC1kJEct22P+gFYBMTgQ+8Z05lffi+Mei8bgTIFz oC7qZxR0gRtQ9ON0fE5xkEL3pKESXOznOBXUpxMN6EC1ckcwPP+yWMXQ9JYU51t6osIv +THRInzNtmxDZlNippOAhETyVo6RECygL6uNrVwPOVXyK9m0hsiICqOU9xAKuSPYnWPy b01Rk/wn6f7bKhQ/2sdESZffazh4EoewPOCKWLzii9xXhy7lBcAnbyk+Bia4bKrE9u80 kWiUqg0+Z9q5P6sRA7fioxyO7KptG0u1Tj4BFX49hHwCtUyp43+yA+50vQI3TecM7Tdk A74A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190281; x=1790795081; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=Z1rHm0E3nzKP7ENhE5GmbSZVJf6tc/xv+MsIZK7AOMY=; b=wJZnJfRoPSr2sgQONVO0bBkBLTxk9ZF6tDeWR86mQ8kg8M6O1h+oLWf8+3POrTMOoR CK7T2ro2sYpim4FxVQ7ZgxcAgE/dswmmAZ3T3AenC854lQCimjTwayUmJukWvl22o2+0 nplRiHBqqF7AJowB4fiZ1c61ZAy5pmtiFAlJ05B2fU7hsuuGDrJVY4RUlPC2IgFG7XLO ASXvIvSAQcPwcOOm+JvY9Chmm34FfR5XYhyQXTasvyRGIH/k/usvCE1DAaUpY5IANTQK HlQt+d6vF+Iyt0nRucFTRIj+xKBGnsQ5saffu/EaaA/Yhw0k9LOT3aY0GMkrMkLIymIJ VcWQ== X-Gm-Message-State: AFuF++nfMrBxAqtYGGAVXJLPFIRIv/JwCw091pC7exJBo85wYZ6PEbau LHIPUF5yBttRnd1jXIV0w4B7FCtdkZfkd9MPpLazGRiyjAsgj6OnZygzgXu1Gz3zlhLvLt7ax9l d2igz X-Gm-Gg: AYBFou3LP2sdjZ4R3hcN1IaGUo42gLM7A+OL87m+8sEn3D1L2lE7QfWYDrOGUaraZwa AXk36w3LL/XvKwqarbzQ1Wr3MOa2jxuoVBfwNQlJ/90CklENVjHyvya21aItGUiNI0BZsJSyKgu /sZ3Jrx6lk9u54QbIXH+utNImEJVvXfh5+i5JuCjK8X41v8ozqPqOo74QamIBExFH2bzMiXbDvQ skNGHyOt+6zlYpgopDxRNg82Ouw9MwgFj8n/m/DnZTykA7mn6Oqh9p9Dz/HzHekNU9P7evqq+qr Ah6kEJ5f8un/J4JVJ25X1IzJ9U6V8D2uxIkmEq8O4Kcl6V+ILdvN9s/m88zlynn3xRt4Crza0Tm boGSstztj8PXzabvokxw0Jxd4vTNkQAVjgtiP5hRX9xIpR4c2fUeQ6XJ8HiNnkOVR7CrerWU7dj JBusIA108CJ4B/spYZspu1gIUbUAoW2XpglXoqAxeCOaVN5ZkdzWQrjc8VrytHUlb5whShGR71d CcZqUoj05/7HRPlz0jbNZaO29tZLnql9g== X-Received: by 2002:a67:ee54:0:b0:7a5:3035:fdad with SMTP id ada2fe7eead31-7af1c68d908mr110089137.3.1790190281058; Wed, 23 Sep 2026 12:04:41 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-98516a6ffc7sm4313732241.8.2026.09.23.12.04.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:39 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 08/13] selftest: resulttooltests.py: add multi-pair threshold test Date: Wed, 23 Sep 2026 15:04:14 -0400 Message-ID: <20260923190419.353493-9-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:24 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246548 Add a test for resulttool.durations.filter_sort_rows(), covering that threshold filtering applies to all pairs when more than two sources are provided, not just the first pair. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index 76bcc53711..c80027bfed 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -771,3 +771,10 @@ class ResultToolTests(OESelftestTestCase): args = SimpleNamespace(threshold=0.0, sort_by_delta=False, limit=0) kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] self.assertEqual(sorted(kept), ['big_change', 'no_change']) + + def test_durations_filter_sort_rows_threshold_checks_every_pair_not_just_first(self): + # 3 sources: only the 2nd pair crosses the threshold, the 1st doesn't + rows = [('t', [10.0, 11.0, 100.0], [(1.0, 10.0), (89.0, 809.0)])] + args = SimpleNamespace(threshold=50.0, sort_by_delta=False, limit=0) + kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] + self.assertEqual(kept, ['t']) From patchwork Wed Sep 23 19:04:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B5BFC98312 for ; Wed, 23 Sep 2026 19:59:34 +0000 (UTC) Received: from mail-vs2-f40.google.com (mail-vs2-f40.google.com [74.125.227.40]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.3004.1790190284527479239 for ; Wed, 23 Sep 2026 12:04:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=MZYGSQNB; spf=pass (domain: baylibre.com, ip: 74.125.227.40, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f40.google.com with SMTP id 71dfb90a1353d-5c9b0bdf5b5so571390e0c.0 for ; Wed, 23 Sep 2026 12:04:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190283; x=1790795083; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=7pMoA2SXn2GIMloMAwGzAo9oEt0PkPrmvsX4rfKHKiI=; b=MZYGSQNBPI6iHkLMD9QyfY93XtmaAJCwqIDF87BevVDYJwSZZ+knO1tem7Z+4ZJ2tp 6gVoLS/JMnWgfDAMzGJfGohK0xTgmRUeCGQFqk1fqCyTU5o6AAIeRy6YmrTWEfMR2Dgi V28nW82bhsijaHHGsmrXeQxsvSUbjzByvxuRX1+RiogPwfI6goYsrpC1omGe+yrzXHrY np1P43oDsE1vMY4tvb3T69r0HZ4SxCpLJjtnBekLkPUat8OXNfC/mSOfPbazZD6jzjoz YYNUrGmAxS6i6M2gnlEu2WWVR86UTBrCUTMI288tdR2ZJw29ZtYfoZ7AGCMnBS+iSiUB ki9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190283; x=1790795083; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=7pMoA2SXn2GIMloMAwGzAo9oEt0PkPrmvsX4rfKHKiI=; b=mkUgq8VocJ+M7WWq/sXg5VAuhgCLDsCNiwIPtPRbOWj9lBWsNFazCdX9H9ZFEY3LCr NzFqeUTReVK8z6d+C+EBZpwE+9feAveQSzfdv+0zMxAM3me+r5FeIZytRhhkDbyfVj0d H0PuX48lSBHT515A5hm1DDN+TL9EgnXtRuC3HYBAwddLhubuD7T0kEJXB3Q/rrTScswM ppQRc8GIWCFB7YRf5Bx5u3u3s8XyivonZm/LLxs/hlGiSP7Jn/H2OTlwqDmK9R8CmiJi U3HMpYmWP/tfPGdJFWkZ5ALANRiq4B2JUlazjvApfL1iip2LyVb10Yt0zhSqFjnvQ4cv MLqg== X-Gm-Message-State: AFuF++lRJM7JcdJIQ/kMXYDhBDLFxrxRiSpq/g7zTheh5/797sDyBjdU 30PbK7p8a3y8g1A/mUV7usJC8HcelW8xkLbsadw3/9B4gS07SrpQp6PfZRrPiKvdZJk876T6Ns+ Yu5/H X-Gm-Gg: AYBFou2G+KVTepStIgdPENeHmZJLCott5fyhTIgIR6kqXy0RJW0Fsp/jnGrxElo48Fn 1Sge5HGTCia7Ml5DSlarILVoFugb9eknwIW2q2MXuGli7P9r8gZJn85s987WsFIYb3JZvKrArG2 KiAqUmlAhR2n/a0OpqPlHFoM4N6KqZDwO8NxiLzvz/EKQNppmog0XM+NlgP/bxgHjYwH0VjIFoO eR+sG9zNXbZRmN2GIb2xRqSreZIFtyYZjJsY27o9QTDm7Lvby4MGN4rAXTJP4WNR3IYsJ9hcjlA 6wBneS0bC1skhzYEiLEZRLrSR77Q5cRi3M91PDN/cCkomFRlXbR9C/eXsHPKr5+bZLnGt9lLKpj g8MlucIK/xcW4IcNEz9zOQ9xo1EV37eEkVxkgcLM99CwzQZPrdn5Pet5Jlz1lXNN5IwG3smk8Hc w7og5Rhw7Fth8XjPsQmfxaK3a2hrS1W2aMn/N1zrAsAf5PYJk1O1lQsIr3zStP+eph7G7Ux/Gz7 76dwcEebcVbKrV/qixBNUtRBsMFYTGQNsLdDeKQGNXLzw== X-Received: by 2002:a05:6122:d02:b0:5c9:a60b:e5ca with SMTP id 71dfb90a1353d-5cb0a6cb326mr177656e0c.13.1790190283394; Wed, 23 Sep 2026 12:04:43 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-5c9f04c161fsm3951356e0c.4.2026.09.23.12.04.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:42 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 09/13] selftest: resulttooltests.py: add None-pct threshold test Date: Wed, 23 Sep 2026 15:04:15 -0400 Message-ID: <20260923190419.353493-10-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:34 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246549 Add a test for resulttool.durations.filter_sort_rows(), ensuring that a 'None' percentage (i.e. when both runs round to 0) never satisfies a threshold. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index c80027bfed..e6e75dd9b9 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -778,3 +778,13 @@ class ResultToolTests(OESelftestTestCase): args = SimpleNamespace(threshold=50.0, sort_by_delta=False, limit=0) kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] self.assertEqual(kept, ['t']) + + def test_durations_filter_sort_rows_threshold_ignores_suppressed_none_pct(self): + # a None pct (both runs round to 0s, see build_rows) must never satisfy a threshold + rows = [ + ('tiny', [0.2, 0.4], [(0.2, None)]), + ('big', [10.0, 20.0], [(10.0, 100.0)]), + ] + args = SimpleNamespace(threshold=1.0, sort_by_delta=False, limit=0) + kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] + self.assertEqual(kept, ['big']) From patchwork Wed Sep 23 19:04:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99102 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A11F3C98313 for ; Wed, 23 Sep 2026 19:59:34 +0000 (UTC) Received: from mail-vs2-f12.google.com (mail-vs2-f12.google.com [74.125.227.12]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3038.1790190287050695604 for ; Wed, 23 Sep 2026 12:04:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=OzAJOF+n; spf=pass (domain: baylibre.com, ip: 74.125.227.12, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f12.google.com with SMTP id ada2fe7eead31-7856441f7d9so526047137.3 for ; Wed, 23 Sep 2026 12:04:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190286; x=1790795086; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=NjyO6MTjDM0qaFwSiDRwDreYDNMPiBsbrQ4+AJp7Sz0=; b=OzAJOF+nDjG+/DVJPqjfCZzXpo0ByNl3FaKSK9/lD2/HWaPu1BaCO8r3ptAXPhaYd7 o3vYAh2jJz5LMHb7W+Xb6wT8vehEYpkjiUx2Fs4lRuta442IY8y0dcc1F+IJErhPa6oc OVskEfRUcTEL7c8+wSDTOaV+oNTzWfHOWzTW4AMl3kv7EBCTiSxcO+5jNAUUmSK5chlH Pd/i2yHHgK/sm9QjNfEwp7TkR55CMICRxFC8Y7dJUpVLdS6sIZBL6YnL8xOUT3FqhpJh f7W91awUm4Z4IZXONSoE5TU5Jtqwl64Mb6AR0sstWVAjfxUErQ34B9MVTlYMPYZ6fjtl geLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190286; x=1790795086; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=NjyO6MTjDM0qaFwSiDRwDreYDNMPiBsbrQ4+AJp7Sz0=; b=lXupRX1+x5wfi82A7Jx2Qt0g7Oj+34BkocD1tPRvdGC5HV0mkEZLy+t/ixp4DnrCU3 K06hwE9ZrZ/zZUrde2tqFwqVy5+7Rhi6h9LDqBTdmrgAn8194ZlI+OVR59mfYBy7u+HJ kEnEEIwWYCnWaldhzx6WMnha46L+Wz19t1b4kr95qNbDh4UuvL+Zp8PDmT7rtDMdvbWm ciyXQEuYLtSPXw3x4LyB+hHOwGfjPTr69/v2eLvRe8CqnPMa6sH4XFb9CA6VPywxPHwP OVfE5SE4vdD0D7KzRka3kEOaNADS2TddVIqEORLhaR+yorOLKfvcpevfr+djEWyR+iJf I02A== X-Gm-Message-State: AFuF++nGWvi6eHoafccPwF//jb8/Nax/vU8KytZzEUmkSvzQbNTCy3Kj nuVt53zLG1THbAyKUaXbWqyjsEiz8abhktgQVsabQeP+dgkjp05ROKKDYqQDiPPOybDs+aCfehz tiJMi X-Gm-Gg: AYBFou1J2vJuim77xAHZkoaec+G/r+GSaPwm5vsn6iYSu/354pptNgtggMNJPzqqqId 1HAmNzmLKATcRzGKzt1X0OlAP4GOyHdhedtmDVNet3Ixf0obgTKaZ5jU7LRVnFhsdQLuO+rnMcv jABkeHeKpY+h3plTTuIAJhZQ6AFq/VLB07HYjx6ateckfHg9urXvWh24siR6PpvjIbYoU3uejT9 jusYKPNv0MkOlvunqwqa6LYSTRZJqe3bM1HZnt0+Oi02qtOsF9YxwKFvAW7C2YET4q9RwRWi5zk p8FD4kbF8TYz6n7tTB+8tmL1O7COhW+LffXp5Xa/c5A8+6TfY4QkLpbY5HrLyWrvvfRKTpd4eg5 hXLHi8b+TaZ5DHwWwMU5raeZL9UQSRNdbFziVn0lgOZRqDawUr9omd/hDSJklQRvVDJX36K+PlZ o31+VY1DdqEdv8TKXVFH0NMsxsqR09FW0p2/9xzfkQj46sIwsrZqFldC73vCWuN3iOik/saDmOg ZCiOGQkqTpatP2kY3rb/t7eKFxjE6M92G59B912etDK837Nk6SjnC4= X-Received: by 2002:a05:6102:a50:b0:7a6:d3f5:3551 with SMTP id ada2fe7eead31-7af1cea1e21mr154367137.14.1790190285827; Wed, 23 Sep 2026 12:04:45 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-985237bbf9asm1293997241.0.2026.09.23.12.04.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:44 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 10/13] selftest: resulttooltests.py: add sort-by-delta test Date: Wed, 23 Sep 2026 15:04:16 -0400 Message-ID: <20260923190419.353493-11-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:34 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246550 Add a test for resulttool.durations.filter_sort_rows(), covering that passing '-s/--sort-by-delta' correctly orders output by largest absolute change first. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index e6e75dd9b9..886c5bb3ed 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -788,3 +788,13 @@ class ResultToolTests(OESelftestTestCase): args = SimpleNamespace(threshold=1.0, sort_by_delta=False, limit=0) kept = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] self.assertEqual(kept, ['big']) + + def test_durations_filter_sort_rows_sort_by_delta_orders_by_largest_change(self): + rows = [ + ('small', [10.0, 11.0], [(1.0, 10.0)]), + ('big', [10.0, 30.0], [(20.0, 200.0)]), + ('medium', [10.0, 15.0], [(5.0, 50.0)]), + ] + args = SimpleNamespace(threshold=0.0, sort_by_delta=True, limit=0) + ordered = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] + self.assertEqual(ordered, ['big', 'medium', 'small']) From patchwork Wed Sep 23 19:04:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 693D5C98304 for ; Wed, 23 Sep 2026 19:59:44 +0000 (UTC) Received: from mail-vs2-f12.google.com (mail-vs2-f12.google.com [74.125.227.12]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3040.1790190289483072835 for ; Wed, 23 Sep 2026 12:04:49 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=VBi+VA9V; spf=pass (domain: baylibre.com, ip: 74.125.227.12, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f12.google.com with SMTP id 71dfb90a1353d-5c67e512ee9so543100e0c.0 for ; Wed, 23 Sep 2026 12:04:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190288; x=1790795088; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=Pr1nqBDsfLrONsaf6MBoEkWWC8vN2L3inm0wiSksSQY=; b=VBi+VA9VCoc6h6OLD5zxnE0Jp2rVRTVsVpTZVVZmcWmRMIpzEVZnrUqZIXXk8KxZAo UBepZWBSNGdLt0nX5Nc6gWtOSE0QQs/Z572TyNDH/cVgnKz47AYtUTvckf8eY2jG39LS DDTFwH2inBVqvukJi0YDrMELRQxSt32cDMmPnCkuggoz57rSpYS2ys/oS5Rlqs5EXLq0 y5G1VNLQoSJXbWbsMDiShFJEv0VAk1h0MG73Ns/BJsEJKvRkEXzrCT3JtyWpjJo7At22 r/PTQkaekJKRGdb5FIXwxyVO+i1toFNjkQtQvqYAE2DbBSkUkzTm2oqwbvVTZ0F0k86K UGgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190288; x=1790795088; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=Pr1nqBDsfLrONsaf6MBoEkWWC8vN2L3inm0wiSksSQY=; b=Gbxtai/lsZ7zyqz7GAcJfajNh9+LdifEl4EqHKYZd+ew4ng+w+jTeLZzhEHzsVSxMD VIe8uCMtaqBYUsMxHkmLPDphOYRXLAAtUdEZw7nfurApqNlUPHHZRQ60QYuORCmcekg7 uueBEgFypU4jQXlxgc8KRXPLiniNJYNA56IJZ0Udy0cWvK0FH8tUk3Ok5JRPJVhZxwdo owgcbM64d7/1R4VNsCxoJ9MA9VLUeMG6A3dZvF4pNaNdIvnaGn3Yz0uDQIF/NS+zxI0Q VksmIqGnmvpcjc2AUt0rFdGTuzwZ0PDY1G6O+sfsarAM7+cNXy78ujKyaJS9HNgF/euG Dzjg== X-Gm-Message-State: AFuF++l1muiIxIhBzI8WMzzpAuXGViQubzfOoeblcoydU73HdyU6N/Mg 0S6ibYIfFeQDRKSboYtrYTFSgPodmbfLXi5as3ffVrRJEoG+g5bYon0kZ2U8N0mzOEynR5xD3Kb 2woGP X-Gm-Gg: AYBFou08jimjoViBtwkM+H+nWccUW1zpbGhKIwKGI9zxe51U2dewUSJOa2w2DiK0UjU xlDr47k9HVnDnbJxEu4CSUvF3DN217Ib8l+xkItu/sIu/qcGFDi8lDKW270fy3w0EQiJNPN0ThB w1g8LBI8S9R/25N0gB+VSeylL4VYqfAC9G2tT1AI8keIjQeUlkXabSlVgH/HfyT+Iiz1ZmkLVin VE+N/w/tvVLIxXRiHTLiOKuV7rREdpl1w4KiAqECCmxU3NX0+3SpIo7s57HSaQ8ZnbhCuC4CT8I 1HjWEwPG8f9ELRHNcbLl/nRALAXNRccc8JQ+EeY99E6EMTpSzKMhAPhfPGVDtepI72GTSuXzfqX npFn6t41A4MK7eYPzvrUzO6Emjbg8byR/j/BNxzfO5ZTvLC1/FRVeJds1te4UjTy2tt2fFpFqZs l5TEW7e7VG18EVAo/c7EeFPvRRy+ofj5rSTkwDDmfA45W3Pwxejqs2fpF/fkvCl0MSR+OdEakNl MI9Vy9mxGUFnxrciwradOcuj4QhvpHZZw== X-Received: by 2002:a05:6122:289b:b0:5c9:a60d:3277 with SMTP id 71dfb90a1353d-5cb0b584feamr161991e0c.11.1790190288378; Wed, 23 Sep 2026 12:04:48 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id 71dfb90a1353d-5c9f04f02bfsm4059969e0c.7.2026.09.23.12.04.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:47 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 11/13] selftest: resulttooltests.py: add limit-after-sort test Date: Wed, 23 Sep 2026 15:04:17 -0400 Message-ID: <20260923190419.353493-12-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246551 Add a test for resulttool.durations.filter_sort_rows(), ensuring the '-l/--limit' option applies after '-s/--sort-by-delta', truncating the already-sorted rows rather than sorting a truncated set. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index 886c5bb3ed..d45213b5b4 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -798,3 +798,13 @@ class ResultToolTests(OESelftestTestCase): args = SimpleNamespace(threshold=0.0, sort_by_delta=True, limit=0) ordered = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] self.assertEqual(ordered, ['big', 'medium', 'small']) + + def test_durations_filter_sort_rows_limit_applies_after_sort(self): + rows = [ + ('small', [10.0, 11.0], [(1.0, 10.0)]), + ('big', [10.0, 30.0], [(20.0, 200.0)]), + ('medium', [10.0, 15.0], [(5.0, 50.0)]), + ] + args = SimpleNamespace(threshold=0.0, sort_by_delta=True, limit=2) + ordered = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] + self.assertEqual(ordered, ['big', 'medium']) From patchwork Wed Sep 23 19:04:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AF84C98304 for ; Wed, 23 Sep 2026 19:59:54 +0000 (UTC) Received: from mail-vs2-f42.google.com (mail-vs2-f42.google.com [74.125.227.42]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.3005.1790190291981664625 for ; Wed, 23 Sep 2026 12:04:52 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=Xo4Yk95+; spf=pass (domain: baylibre.com, ip: 74.125.227.42, mailfrom: tgamblin@baylibre.com) Received: by mail-vs2-f42.google.com with SMTP id 71dfb90a1353d-5c98e1942dfso1108388e0c.3 for ; Wed, 23 Sep 2026 12:04:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190291; x=1790795091; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=g3NwAytRnVtPHIJIPdjA/9dveqG1/DPvlkqDjOsY+R0=; b=Xo4Yk95+C+JupUVnaaTGKaagj/qUMvZB5xE8FxPPJN2h096+BlpAT2fxubhdB7KUmb 6VXYxnPlAPoZAwnoNu0f9p63sA35VSIWYoWHfnAqlSvpYCWVcyqKH2mBSXp/MSPOuLQY j6cfjwtpDeLWH9qEhX/Vla51cgRcm75tNL4NSe9jxOF2q9ZUWSj4kWKo9ow0t4QvsM6D IxSur6jTY+bUQ8thWZTdyHaBWjRaRX/Wh7LQKPpnrWPwNe17jPvK/ysLc6uzrLX2TaRp ucpMiEBBNnSx6EU7DImogHwEHZ8ERYHqMhzvQ22hjUDGbVRmSjMY8FZtv8pTmBBOMJ3/ e5mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190291; x=1790795091; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=g3NwAytRnVtPHIJIPdjA/9dveqG1/DPvlkqDjOsY+R0=; b=fJm0P/qqR2m9QX9NdIuPLc5DIqCS1qpFbz4wra/bIJ2LSbH/33gA7UVKnUgJcsFy79 o8XmW7RQOFc/5tk5KgQkEAA1Rmfh1vLck9svgHM31XonpewBZoHnfDsMGPRxKKrZXZhK yghERdvA01a4ldPOCZGEv24Q2FFBBOfhsnW2QfOxtD5ghQuoApk6GT2rXxEge4121liP MyKBZKTvJsnycqAzqNzd7BhYuMiGL9jnUEDd8ru3mtFqLMeMfOwS4D7i+bZtHlrQPO9S aVJWG66Wr90X4J+ZbCVxKN2mGHLB7QNkB+8AMHvEGnHl24iwb4DgepYfxKHB5eYw6UL5 /bkA== X-Gm-Message-State: AFuF++nCd/YK9KjJRhWQIoxlJWtyHg0q8JMebFvdcsgp4VWNwID/N5o/ RJi3PmAYwMkQ+U8s2EljFXf9+uqddT0if9/xar98obtn5Hl8+p/S7KYKKtD848iUz0hWLPR/SrR BQklc X-Gm-Gg: AYBFou0Y1woGjUW65xQLZuIxuFX8QJW9GmVnhskEB6VryfIzydGFFsD5GmxqxdzVrXQ yEEn4S7s/kzwMFw10OXx4GpRJ2GHzvsxAqr4y+OY5vjn9TZCyTMrJ007XggLL1W/kjTKpLBFbB9 JT/joWOJ/Ah4omTQ9PfKeDfOvaDF151i7depKcm218IHtgEh3uaFYEyeaztK0aZmI0eViBzsHJ1 5rzLyfLeQlj4HLuK21vhjFm4aHEBL+o8OyDxWIVGk6cs4za3x3YOBWYhhVSWvN0HXl9b2VgHD4W c4AJva6HH00tGTbe/qVpcY+pr3kCa+rjyfeAPPFW4ogwUTiYaWru6+PHXINQnK7GmsTgYCeqhda rEYswxtrDOKrOUuIc/DJC2p8ENYcm+aFVeC4Ii8TQG7KxSuNYctB7VXX3zXIQXaRkMhdXVWLfRT wV+FcG6jUacUUDILBJW8TsigYTw6DDmyx2iRYZexpvq4x5kQYCJfOWYNye1qKD2ZHPy5wyY6ZjT Qx3wVrbldhXxNPJduxQEG4gcXnd4h9lluRnNxmC7zz+UmQm/EoELOs= X-Received: by 2002:a05:6102:5344:b0:7a7:198a:c2b5 with SMTP id ada2fe7eead31-7af1dde1da0mr132625137.29.1790190290797; Wed, 23 Sep 2026 12:04:50 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-7abf566d033sm4538842137.5.2026.09.23.12.04.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:49 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 12/13] selftest: resulttooltests.py: add short-tables ordering test Date: Wed, 23 Sep 2026 15:04:18 -0400 Message-ID: <20260923190419.353493-13-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246552 Add a test for resulttool.durations.build_tables(), ensuring that the '--show-short' tables are returned after every other table, rather than interleaved per test configuration. AI-Generated: Uses Claude Sonnet 5 --- .../oeqa/selftest/cases/resulttooltests.py | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index d45213b5b4..d93d04aaad 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -808,3 +808,50 @@ class ResultToolTests(OESelftestTestCase): args = SimpleNamespace(threshold=0.0, sort_by_delta=True, limit=2) ordered = [k for k, vals, deltas in durations.filter_sort_rows(rows, args)] self.assertEqual(ordered, ['big', 'medium']) + + def _duration_result(self, machine, long_duration, short_duration): + return {'configuration': {"TEST_TYPE": "runtime", + "TESTSERIES": "series1", + "IMAGE_BASENAME": "image", + "IMAGE_PKGTYPE": "ipk", + "DISTRO": "mydistro", + "MACHINE": machine}, + 'result': { + 'ptestresult.sections': { + 'longsuite': {'duration': str(long_duration)}, + 'shortsuite': {'duration': str(short_duration)}, + }, + }} + + def _write_json(self, data): + f = tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) + json.dump(data, f) + f.close() + return f.name + + def test_durations_build_tables_orders_short_tables_after_all_main_tables(self): + # Two machines, each with one test >= min_duration and one under it + run1 = self._write_json({'qemux86_result': self._duration_result('qemux86', 50, 2), + 'qemux86-64_result': self._duration_result('qemux86-64', 50, 2)}) + run2 = self._write_json({'qemux86_result': self._duration_result('qemux86', 80, 3), + 'qemux86-64_result': self._duration_result('qemux86-64', 80, 3)}) + try: + args = SimpleNamespace(sources=[run1, run2], labels='', sort_by_delta=False, + threshold=0.0, limit=0, min_duration=10.0, show_short=True) + result = durations.build_tables(args, self.logger) + finally: + os.remove(run1) + os.remove(run2) + + self.assertIsNotNone(result) + labels, tables = result + titles = [title for title, rows in tables] + self.assertEqual(len(titles), 4, msg="expected one main and one short table per machine: %s" % titles) + main_titles, short_titles = titles[:2], titles[2:] + self.assertTrue(all("under" not in t for t in main_titles), msg=titles) + self.assertTrue(all("under" in t for t in short_titles), msg=titles) + # the main tables only contain the long test, the short tables only the short one + for title, rows in tables[:2]: + self.assertEqual([k for k, vals, deltas in rows], ['ptestresult.sections.longsuite']) + for title, rows in tables[2:]: + self.assertEqual([k for k, vals, deltas in rows], ['ptestresult.sections.shortsuite']) From patchwork Wed Sep 23 19:04:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Gamblin X-Patchwork-Id: 99104 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78693C9830B for ; Wed, 23 Sep 2026 19:59:54 +0000 (UTC) Received: from mail-ua2-f12.google.com (mail-ua2-f12.google.com [74.125.226.204]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.3043.1790190293848195891 for ; Wed, 23 Sep 2026 12:04:54 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre.com header.s=google header.b=Z+MUPzKI; spf=pass (domain: baylibre.com, ip: 74.125.226.204, mailfrom: tgamblin@baylibre.com) Received: by mail-ua2-f12.google.com with SMTP id a1e0cc1a2514c-97ea1a712deso471496241.3 for ; Wed, 23 Sep 2026 12:04:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1790190293; x=1790795093; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=vcn4Xk0RYHbovD1Lv13ZGM8D7UHyVOTf5efP4CNEAmg=; b=Z+MUPzKIhXN7+LAcBpFIw2wZvD/6FeIBxccB9ESXxdDxu6qx06szt0EC7rV0XV+hvS GKMsGbec7STeqWPGk8g0dOBuUNeyW50RcVgQ5lUalExCHUpehQHYTdXZUySpeS83GZCt xJLyLFWy4NjOBX1U0NBINJZp48Mte6o5gKtHA57had2UasZG02sICe4w9sAUkYD2OFrN bQieZnv7FnIJEpTSCuNhcvRICUaTvbDr9BaawgFJhyltX73a6RvVFLYzyM9S53fHYNhK BftzFflmWcOmlo1oQ+BW/KzM0ha6fvoR/sZSoUklyKQAi3jOJppYuj1SKhp6XYNRbcG5 TnzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790190293; x=1790795093; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=vcn4Xk0RYHbovD1Lv13ZGM8D7UHyVOTf5efP4CNEAmg=; b=UeylBdARev87jce6zf3ke3afMrQW4k35UO2Y7Dbp8yfkO3rkaGxGvTKY7PdLgc12kj Q4kvxOjLTC/iN2xowS/lnpZsWru8g1cWBkqwicX5O7s5HwSJl0vvOaI6KILZ68/cY81M nr9KlqiHRdyvFQowyHogrB5OrC8OxdC2ku4josN68wGg2tLnwvYKciBzxBb1FHtSWIx/ pZQ7X5ndF0lEq7mqNYh+RzEmWSVsje2BOUoh7/jM4Agk1/WMhQJHLywb//eXhHXOron+ scTFTjsGTb8euS+W3RT6TO+DCn6aJZuLa5IKNCouXK8q9z7264AVVpfFu425qp0oInGZ 1EUg== X-Gm-Message-State: AFuF++kK+68RztsJJ+z3fnQIV2ZFPUaZ3G7yJhDdqYYTf14hTGiuJne5 mThV4cBFPqcVHrlpuIF1uq3xWapYASH41M1q8H8v/VEj9f9ATYSVJidbuhauhzSF92+dz5OngZF rtj3v X-Gm-Gg: AYBFou2wTl9TtL+FVR4zo388oIWJjnd2rEpnHZTrzNLjE6SImxoMApBQZGhaNbMQeDJ j+tVbordKqj8K5Qyn9/usPjW5WFz7vvVHxYIV5w04PQVo6jujwSWFWNEKEOw2oBbCGGr3HtK+eu DBGMkE7DLDTb5ZzHe9qkz8c0GryiPX3sUT2mtUvCDpVh3aIjasPHcCZGh80TN3a9eJ1ZxgaUCVP qBX1YUBOa07+tP7v32kbsFIc+9qbSyLRpqBjzYasJ1H5QwGTlWHMNtuxxyPjYbX0bUenHv37Ve5 EMwADxAOEF/NUqekRN9zu9t2RTrlpu2YW0aoOUIMohEYLi+QU1qlFfjnFcRImlT4QrTWlRr6LRx OukpzY23nogD8yM5Xhp5JQh55Qzwv7ZXiUjP1W0CyYPMeyuHH3ToYsP+daxEoXDY58cdYfOggXL 4W12kX61N85Oxc2PI3Hs3ZBkvX3rescHViwIVsVvCtwsoSqvmlOAOmHOc4fA56Gcj8mSRiRCfXt IV7MpNySgdBSfTftZzMxxeTq+20VYBAdA== X-Received: by 2002:a05:6102:809c:b0:7a5:7282:1994 with SMTP id ada2fe7eead31-7af1c99c6c7mr176033137.7.1790190292710; Wed, 23 Sep 2026 12:04:52 -0700 (PDT) Received: from localhost ([2001:1970:3847:e000:e8bd:ca0f:c232:9f10]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-98516a42fe6sm4283536241.7.2026.09.23.12.04.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 12:04:52 -0700 (PDT) From: "Trevor Gamblin" To: openembedded-core@lists.openembedded.org Subject: [OE-core][PATCH 13/13] selftest: resulttooltests.py: add min-duration default test Date: Wed, 23 Sep 2026 15:04:19 -0400 Message-ID: <20260923190419.353493-14-tgamblin@baylibre.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923190419.353493-1-tgamblin@baylibre.com> References: <20260923190419.353493-1-tgamblin@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 23 Sep 2026 19:59:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/246553 Add a test for resulttool.durations.build_tables(), ensuring that short results are hidden by default according to the default '-m/--min-duration' threshold of 10s. AI-Generated: Uses Claude Sonnet 5 --- meta/lib/oeqa/selftest/cases/resulttooltests.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/resulttooltests.py b/meta/lib/oeqa/selftest/cases/resulttooltests.py index d93d04aaad..9c3b9bf869 100644 --- a/meta/lib/oeqa/selftest/cases/resulttooltests.py +++ b/meta/lib/oeqa/selftest/cases/resulttooltests.py @@ -855,3 +855,18 @@ class ResultToolTests(OESelftestTestCase): self.assertEqual([k for k, vals, deltas in rows], ['ptestresult.sections.longsuite']) for title, rows in tables[2:]: self.assertEqual([k for k, vals, deltas in rows], ['ptestresult.sections.shortsuite']) + + def test_durations_build_tables_hides_short_tests_by_default(self): + run1 = self._write_json({'result': self._duration_result('qemux86', 50, 2)}) + run2 = self._write_json({'result': self._duration_result('qemux86', 80, 3)}) + try: + args = SimpleNamespace(sources=[run1, run2], labels='', sort_by_delta=False, + threshold=0.0, limit=0, min_duration=10.0, show_short=False) + labels, tables = durations.build_tables(args, self.logger) + finally: + os.remove(run1) + os.remove(run2) + + self.assertEqual(len(tables), 1) + title, rows = tables[0] + self.assertEqual([k for k, vals, deltas in rows], ['ptestresult.sections.longsuite'])