From patchwork Tue Feb 7 15:29:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 19096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37A66C636CC for ; Tue, 7 Feb 2023 15:29:42 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx.groups.io with SMTP id smtpd.web10.86139.1675783781196740212 for ; Tue, 07 Feb 2023 07:29:41 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=AV0Z97MH; spf=pass (domain: bootlin.com, ip: 217.70.183.197, mailfrom: alexis.lothore@bootlin.com) Received: (Authenticated sender: alexis.lothore@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id 591EF1C0012; Tue, 7 Feb 2023 15:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1675783779; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NQTkPFjpLHqtfMv72qq2jRoYDfsUbVzRrUW/JLQhy+Q=; b=AV0Z97MH670yBoxgwBhJutW9fltblHXpzOJWjT4vw6uCF6sBHHgA8H+nFtp9LNzaVSjS9d nAmMlt+VqDX07CdEvOJ560BnCklrlSjOu7JSERsunhW9mgvyJaVRQgtdrW0sPNSDLMwg+b YkcXG8Eo/2er7FX3+ZX5i+FBRk6jRbukQVfChsin1/BqG0CoA8snpS3iZp0ze/Zgo5seNm jk2iy/zkVQmop6vz4XkiI+deFfQMJsEoF7p3pMHKcpjIgdy+MdMDyuUo6G3wGkZTj/MSiy 7f/9zQYyHR8YkGdSiEN2mt4QSudT2Oid7a5XUPprP6fockP+1MkkMtiLj6Ei/w== From: alexis.lothore@bootlin.com To: yocto@lists.yoctoproject.org Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com Subject: [autobuilder][PATCH v2 1/3] scripts/send-qa-email: force current revision fetch Date: Tue, 7 Feb 2023 16:29:47 +0100 Message-Id: <20230207152949.43487-2-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230207152949.43487-1-alexis.lothore@bootlin.com> References: <20230207152949.43487-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 07 Feb 2023 15:29:42 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/59196 From: Alexis Lothoré If we try to run send-qa-email to simulate past releases (for example, for development or debugging), the execution will very likely fail because the target revision to examine (ie: the poky revision) is too old, and as a consequence is not contained in the testresults shallow clone anymore (because testsresults history keeps moving forward as builds are triggered on autobuilder). As a consequence, force-fetch the "target" revision, as it is already done for the "base" revision Signed-off-by: Alexis Lothoré --- scripts/send_qa_email.py | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py index 199fe4e..f8ebe34 100755 --- a/scripts/send_qa_email.py +++ b/scripts/send_qa_email.py @@ -75,6 +75,7 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir, running resulttool """ fetch_testresults(resultdir, baserevision) + fetch_testresults(resultdir, comparerevision) regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir]) with open(outputdir + "/testresult-regressions-report.txt", "wb") as f: f.write(regreport) From patchwork Tue Feb 7 15:29:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 19097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2034EC636D6 for ; Tue, 7 Feb 2023 15:29:52 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx.groups.io with SMTP id smtpd.web11.86371.1675783781958752254 for ; Tue, 07 Feb 2023 07:29:42 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=gAJovxx1; spf=pass (domain: bootlin.com, ip: 217.70.183.197, mailfrom: alexis.lothore@bootlin.com) Received: (Authenticated sender: alexis.lothore@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id 0D16E1C0007; Tue, 7 Feb 2023 15:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1675783780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7MtPAxL/l/W4hudxqsOc2+kd8n/xVOV3CSNdRfIn3Iw=; b=gAJovxx1pH2yQ3yxuHWqRVeCl3Y1/QzbAMciIQYcIIZ4DeJ8Jyk2pAtXEwssybdtf0LIx8 Fylo3D9NJleuFfwoBwQbMuyqk1MO5F5dlEzsF+jV3XevHwAzHoNgdg3RDx2AAa74H+Wf0n vEuOzbZiJG42ebvjHwXs7PPQIsE+9diN5nm2ZUVpJB11JogPyh3kbiCGCzVH5gTxfFdHyQ 6w14mUI31vGswWbnVpWwxnKdiUNBB6KmptFuEMsZZBh+xCnc070lPnGDpEFPlXtHn+0Ca9 TvYSLKQhFIkSLnf3ymEa47d/YIc8/sZ1BkV8b6LNwJjDJjUkwNoIRbx4bT0p6g== From: alexis.lothore@bootlin.com To: yocto@lists.yoctoproject.org Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com Subject: [autobuilder][PATCH v2 2/3] scripts/send-qa-email: reduce testresults shallow clone depth Date: Tue, 7 Feb 2023 16:29:48 +0100 Message-Id: <20230207152949.43487-3-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230207152949.43487-1-alexis.lothore@bootlin.com> References: <20230207152949.43487-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 07 Feb 2023 15:29:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/59197 From: Alexis Lothoré Since we are now force-fetching base revisions and target revisions for regression report generation, we can make testresults clone even more "shallow" to increase clone speed in CI pipelines Signed-off-by: Alexis Lothoré --- scripts/send_qa_email.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py index f8ebe34..b87e822 100755 --- a/scripts/send_qa_email.py +++ b/scripts/send_qa_email.py @@ -139,10 +139,10 @@ def send_qa_email(): elif basebranch: cloneopts = ["--branch", basebranch] try: - subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "5"] + cloneopts) + subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "1"] + cloneopts) except subprocess.CalledProcessError: print("No comparision branch found, falling back to master") - subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "5"]) + subprocess.check_call(["git", "clone", "git@push.yoctoproject.org:yocto-testresults", tempdir, "--depth", "1"]) # If the base comparision branch isn't present regression comparision won't work # at least until we can tell the tool to ignore internal branch information From patchwork Tue Feb 7 15:29:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 19098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA12C636CD for ; Tue, 7 Feb 2023 15:29:52 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx.groups.io with SMTP id smtpd.web10.86140.1675783782137663758 for ; Tue, 07 Feb 2023 07:29:42 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=a8F4BVJD; spf=pass (domain: bootlin.com, ip: 217.70.183.197, mailfrom: alexis.lothore@bootlin.com) Received: (Authenticated sender: alexis.lothore@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id 8C2551C0004; Tue, 7 Feb 2023 15:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1675783780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tQV4cxnhSSHAa+qcquwpbKfsUuyY89Na5nIRtcYUHDM=; b=a8F4BVJDcpfpRo940ruOTXtAIcnK2DrjiXNickQMv5ynX6EVkDm9y2IU5BIHQk64GY2313 iGyrnRTcK/7RUYiD1io5Q4H7KAuxdAmS9j4h2aDL7GzzCbtVA+eORFf1G4yQI0WDRjSBwX CewbbD/itsZ8atz+nC5uCW/qN6ivWmHc3Bvb8kCc2mKQtpVtJnEzEi19fbhNGfhMeKLXbK ezLTTVJTeYJcq6X/0E6rXBhIlArBt3jyBw0GTyGt+bHyT7MOJC5AZ0bf2amQt//qjMqHCX xHm/FwS+F6Vn1UyuR+G8NJLh8WQ2eRCsM+ULpqTtFoaYqZ5UVrHWZBvkiLjYOA== From: alexis.lothore@bootlin.com To: yocto@lists.yoctoproject.org Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com Subject: [autobuilder][PATCH v2 3/3] scripts/send-qa-email: add target and base revisions in regression report Date: Tue, 7 Feb 2023 16:29:49 +0100 Message-Id: <20230207152949.43487-4-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230207152949.43487-1-alexis.lothore@bootlin.com> References: <20230207152949.43487-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 07 Feb 2023 15:29:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/59198 From: Alexis Lothoré Current regression reports do not contain information about versions compared when generating reports. While it is still possible to get the information by searching the autobuilder log, it is not convenient. Moreover, future developments will allow to generate multiple reports (with different bases for comparison) in a single build. As a consequence, embed target and base revisions in the report header Signed-off-by: Alexis Lothoré --- scripts/send_qa_email.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py index b87e822..e2d7284 100755 --- a/scripts/send_qa_email.py +++ b/scripts/send_qa_email.py @@ -78,6 +78,10 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir, fetch_testresults(resultdir, comparerevision) regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir]) with open(outputdir + "/testresult-regressions-report.txt", "wb") as f: + f.write(str.encode("========================== Regression report ==============================\n")) + f.write(str.encode(f'{"=> Target:": <16}{basebranch: <16}({comparerevision})\n')) + f.write(str.encode(f'{"=> Base:": <16}{baseversion: <16}({baserevision})\n')) + f.write(str.encode("===========================================================================\n\n")) f.write(regreport) except subprocess.CalledProcessError as e: error = str(e)