From patchwork Tue Feb 7 14:12:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 19085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0C8DC636CD for ; Tue, 7 Feb 2023 14:12:31 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by mx.groups.io with SMTP id smtpd.web10.84070.1675779145146881512 for ; Tue, 07 Feb 2023 06:12:25 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=NmOKmhAH; spf=pass (domain: bootlin.com, ip: 217.70.183.193, mailfrom: alexis.lothore@bootlin.com) Received: (Authenticated sender: alexis.lothore@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id 39863240005; Tue, 7 Feb 2023 14:12:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1675779143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YvGw4+WlYIg4t1KJBUqVcUgG9EW2bve9KDwmeQBwq+s=; b=NmOKmhAHVKq0JbLkC1xN/WG8inFS4atYq6mleOYVX6dK9qzgnX0aW2ukQ/jc94f9Qs+aoq xffX0RdTLSesdMXmLi3VbbX/Pbv1l/v24jcOctd9Iv2/1YDfaLfDRY36gZNxgoZg5BPZ46 Jb2wNnWs9m6ZJt5fOJ3YmW++tEGBE65wlJlBxWomxZ7aT1vqt12o28Qje6ijaiLdnOGvWJ Q+OYZg2fole87SGbNytRC01aUe1Yl1axrq1fYRH99NLnAaYib3OIbfmj9o2CilC5u51iRc cHi3QgZ/9BYXaQLZiIDYmzz5JyWIcjQjDaCOxqTQRLLyp7Zo9o9cLOgAcvgL4A== From: Alexis Lothore To: yocto@lists.yoctoproject.org Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com Subject: [autobuilder][PATCH 1/3] scripts/send-qa-email: force current revision fetch Date: Tue, 7 Feb 2023 15:12:15 +0100 Message-Id: <20230207141217.33892-2-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230207141217.33892-1-alexis.lothore@bootlin.com> References: <20230207141217.33892-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 07 Feb 2023 14:12:31 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/59189 If we try to run send-qa-email to simulate past releases (for example, for development or debugging), the execution will very likely fail because the target revision to examine (ie: the poky revision) is too old, and as a consequence is not contained in the testresults shallow clone anymore (because testsresults history keeps moving forward as builds are triggered on autobuilder). As a consequence, force-fetch the "target" revision, as it is already done for the "base" revision Signed-off-by: Alexis Lothoré Signed-off-by: Alexis Lothore --- scripts/send_qa_email.py | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py index 199fe4e..f8ebe34 100755 --- a/scripts/send_qa_email.py +++ b/scripts/send_qa_email.py @@ -75,6 +75,7 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir, running resulttool """ fetch_testresults(resultdir, baserevision) + fetch_testresults(resultdir, comparerevision) regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir]) with open(outputdir + "/testresult-regressions-report.txt", "wb") as f: f.write(regreport) From patchwork Tue Feb 7 14:12:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 19084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF1CBC636CC for ; Tue, 7 Feb 2023 14:12:31 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by mx.groups.io with SMTP id smtpd.web11.84295.1675779146195787522 for ; Tue, 07 Feb 2023 06:12:26 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=O4iaenKj; spf=pass (domain: bootlin.com, ip: 217.70.183.193, mailfrom: alexis.lothore@bootlin.com) Received: (Authenticated sender: alexis.lothore@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id C9B8624000F; Tue, 7 Feb 2023 14:12:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1675779144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZOq11+9pkp6a0J0k8W9TdZ3jYm5j2usI0CnCa6TTfWU=; b=O4iaenKj470htUG15RVgXBXUUpKBgdlrPCAAQ/OdnGlJUieOvnpfh4Ft1BqGIpppYz1sPM Se8aBwxFtc0bNto0T2LcTsxrT5h0sLPR9MfPh4FRwFyJ01tuRRdGqkmvi6plmBP7tuE5oy lM6cNbvAyGfaQj68jda2RgblPz+oOD+e6NK9ZEhxuQsNphY+9MeV8Qhzaf+Yr8ibMA4flF GFHmnYh7ReMQPhYdq8ob/m9I08BLxQy/L/QNtLUcEe+Z4aVd3wrsVNwBIGEV6vdyzFsiK4 Eq5o6fTRlpu528VdgP4qx1kgwxDbHn4RXwNZMfNBP/pYXI9z6GgvkWhoxZU6Mw== From: Alexis Lothore To: yocto@lists.yoctoproject.org Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com Subject: [autobuilder][PATCH 2/3] scripts/send-qa-email: reduce testresults shallow clone depth Date: Tue, 7 Feb 2023 15:12:16 +0100 Message-Id: <20230207141217.33892-3-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230207141217.33892-1-alexis.lothore@bootlin.com> References: <20230207141217.33892-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 07 Feb 2023 14:12:31 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/59190 Since we are now force-fetching base revisions and target revisions for regression report generation, we can make testresults clone even more "shallow" to increase clone speed in CI pipelines Signed-off-by: Alexis Lothoré Signed-off-by: Alexis Lothore --- scripts/send_qa_email.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py index f8ebe34..b87e822 100755 --- a/scripts/send_qa_email.py +++ b/scripts/send_qa_email.py @@ -139,10 +139,10 @@ def send_qa_email(): elif basebranch: cloneopts = ["--branch", basebranch] try: - subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "5"] + cloneopts) + subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "1"] + cloneopts) except subprocess.CalledProcessError: print("No comparision branch found, falling back to master") - subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "5"]) + subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "1"]) # If the base comparision branch isn't present regression comparision won't work # at least until we can tell the tool to ignore internal branch information From patchwork Tue Feb 7 14:12:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 19086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2741C6379F for ; Tue, 7 Feb 2023 14:12:31 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by mx.groups.io with SMTP id smtpd.web10.84071.1675779147229321671 for ; Tue, 07 Feb 2023 06:12:27 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=pzZsFYGm; spf=pass (domain: bootlin.com, ip: 217.70.183.193, mailfrom: alexis.lothore@bootlin.com) Received: (Authenticated sender: alexis.lothore@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id 2C6B1240008; Tue, 7 Feb 2023 14:12:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1675779145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=am/uGiBL8aJ2BUzc0N2n6muGgkew+ERb45bMutj6Fwc=; b=pzZsFYGm+ZYAGe8U17e0vIL0mMFy41f5Uh4ieyY8AMUOVbMACkc7Sr+aZ1ezJUh6/EuULI MklL3umfCC98tASaCGKKe1pswVDf61lsV0N1EeYhNuJK+ugzNmnwxGZGdUqBIMyUxMBpBt n17sYixeqlq0PwCK3tfla36sIwnaxUJ8vyMiXwqtghYRz+e+zOwVRS1DJ6Br0+nRMr3o6N mnfKN4bn1FV3YnDB2oWfXfuuFf5B5CgG8cs+mUnwFhbKhNjim5XbdDygdEwMDmpW8RZonP hh8Kp4F542WzBTYYRB0EXLrDoH/svg/+VaCk5EJw6Cl22Ja7i7iGKsg4KpBaqw== From: Alexis Lothore To: yocto@lists.yoctoproject.org Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com Subject: [autobuilder][PATCH 3/3] scripts/send-qa-email: add target and base revisions in regression report Date: Tue, 7 Feb 2023 15:12:17 +0100 Message-Id: <20230207141217.33892-4-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230207141217.33892-1-alexis.lothore@bootlin.com> References: <20230207141217.33892-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 07 Feb 2023 14:12:31 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/59191 Current regression reports do not contain information about versions compared when generating reports. While it is still possible to get the information by searching the autobuilder log, it is not convenient. Moreover, future developments will allow to generate multiple reports (with different bases for comparison) in a single build. As a consequence, embed target and base revisions in the report header Signed-off-by: Alexis Lothoré Signed-off-by: Alexis Lothore --- scripts/send_qa_email.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py index b87e822..e2d7284 100755 --- a/scripts/send_qa_email.py +++ b/scripts/send_qa_email.py @@ -78,6 +78,10 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir, fetch_testresults(resultdir, comparerevision) regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir]) with open(outputdir + "/testresult-regressions-report.txt", "wb") as f: + f.write(str.encode("========================== Regression report ==============================\n")) + f.write(str.encode(f'{"=> Target:": <16}{basebranch: <16}({comparerevision})\n')) + f.write(str.encode(f'{"=> Base:": <16}{baseversion: <16}({baserevision})\n')) + f.write(str.encode("===========================================================================\n\n")) f.write(regreport) except subprocess.CalledProcessError as e: error = str(e)