From patchwork Thu Mar 7 16:42:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 40659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EE6BC54E4A for ; Thu, 7 Mar 2024 16:42:51 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.28263.1709829768348189573 for ; Thu, 07 Mar 2024 08:42:48 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BAAA11FB for ; Thu, 7 Mar 2024 08:43:24 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 61C883F738 for ; Thu, 7 Mar 2024 08:42:47 -0800 (PST) From: ross.burton@arm.com To: yocto@lists.yoctoproject.org Subject: [PATCH yocto-autobuilder-helper 1/3] scripts/abint: sort and organise imports Date: Thu, 7 Mar 2024 16:42:43 +0000 Message-Id: <20240307164245.783037-1-ross.burton@arm.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 16:42:51 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62695 From: Ross Burton Signed-off-by: Ross Burton --- scripts/abint/abint.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/scripts/abint/abint.py b/scripts/abint/abint.py index f4cb8b29..d4fbed7e 100755 --- a/scripts/abint/abint.py +++ b/scripts/abint/abint.py @@ -1,14 +1,15 @@ #! /usr/bin/env python3 -import re import collections -from pprint import pprint -import bugzilla -import pickle -import requests import dataclasses -import arrow import logging +import pickle +import pprint +import re + +import arrow +import bugzilla +import requests MOCK = False # logging.basicConfig(level=logging.DEBUG) @@ -83,7 +84,7 @@ def get_data(): for bug in bugs ] - pprint(bugs) + pprint.pprint(bugs) print(f"Found {len(bugs)} AB-INT bugs") comments_data = bz.get_comments([bug.id for bug in bugs])["bugs"] From patchwork Thu Mar 7 16:42:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 40660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 291FFC54798 for ; Thu, 7 Mar 2024 16:42:51 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.28264.1709829768702005303 for ; Thu, 07 Mar 2024 08:42:48 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 62E1AC15 for ; Thu, 7 Mar 2024 08:43:25 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 12C293F738 for ; Thu, 7 Mar 2024 08:42:47 -0800 (PST) From: ross.burton@arm.com To: yocto@lists.yoctoproject.org Subject: [PATCH yocto-autobuilder-helper 2/3] scripts/abint: add argument parser Date: Thu, 7 Mar 2024 16:42:44 +0000 Message-Id: <20240307164245.783037-2-ross.burton@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240307164245.783037-1-ross.burton@arm.com> References: <20240307164245.783037-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 16:42:51 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62696 From: Ross Burton Add an argument parser so that the use of the cache or verbose logging can be enabled/disabled without having to edit the source code. Signed-off-by: Ross Burton --- scripts/abint/abint.py | 45 ++++++++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/scripts/abint/abint.py b/scripts/abint/abint.py index d4fbed7e..1d358836 100755 --- a/scripts/abint/abint.py +++ b/scripts/abint/abint.py @@ -1,5 +1,6 @@ #! /usr/bin/env python3 +import argparse import collections import dataclasses import logging @@ -11,19 +12,6 @@ import arrow import bugzilla import requests -MOCK = False -# logging.basicConfig(level=logging.DEBUG) - - -def save_pickle(name, data): - with open(name, "wb") as f: - pickle.dump(data, f, pickle.HIGHEST_PROTOCOL) - - -def load_pickle(name): - with open(name, "rb") as f: - return pickle.load(f) - http = requests.Session() @@ -104,15 +92,9 @@ def get_data(): except requests.exceptions.HTTPError as e: logging.debug(f"Couldn't find build for {builder=} {build=}") - save_pickle("bugs.data", bugs) return bugs -if MOCK: - - def get_data(): - return load_pickle("bugs.data") - def last_seen_report(data): import jinja2 @@ -129,5 +111,26 @@ def last_seen_report(data): f.write(template.render(bugs=data, start=start, now=arrow.now())) -data = get_data() -last_seen_report(data) +CACHE_NAME = "bugs.data" + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") + parser.add_argument("--cached", action="store_true", help="Use the local cached data instead of fetching") + args = parser.parse_args() + + if args.verbose: + logging.basicConfig(level=logging.DEBUG) + + if args.cached: + logging.debug(f"Loading cached data from {CACHE_NAME}") + with open(CACHE_NAME, "rb") as f: + data = pickle.load(f) + else: + data = get_data() + # Always save a cache because it's quick + logging.debug(f"Saving data to cache {CACHE_NAME}") + with open(CACHE_NAME, "wb") as f: + pickle.dump(data, f, pickle.HIGHEST_PROTOCOL) + + last_seen_report(data) From patchwork Thu Mar 7 16:42:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 40661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 290E2C54798 for ; Thu, 7 Mar 2024 16:43:11 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.28238.1709829769372112121 for ; Thu, 07 Mar 2024 08:42:49 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 137001FB for ; Thu, 7 Mar 2024 08:43:26 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AF5393F738 for ; Thu, 7 Mar 2024 08:42:48 -0800 (PST) From: ross.burton@arm.com To: yocto@lists.yoctoproject.org Subject: [PATCH yocto-autobuilder-helper 3/3] scripts/abint: fix sorting on initial display Date: Thu, 7 Mar 2024 16:42:45 +0000 Message-Id: <20240307164245.783037-3-ross.burton@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240307164245.783037-1-ross.burton@arm.com> References: <20240307164245.783037-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 16:43:11 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62697 From: Ross Burton The default sorting was based on the list of seen times, but it should be based on the latest seen time. Signed-off-by: Ross Burton --- scripts/abint/abint.html.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/abint/abint.html.j2 b/scripts/abint/abint.html.j2 index f531649d..580a4a35 100644 --- a/scripts/abint/abint.html.j2 +++ b/scripts/abint/abint.html.j2 @@ -56,7 +56,7 @@ - {% for bug in bugs|sort(attribute="seen", reverse=True) %} + {% for bug in bugs|sort(attribute="latest", reverse=True) %} {{bug.id}}