From patchwork Fri Aug 11 12:55:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 28692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20E0EC41513 for ; Fri, 11 Aug 2023 12:55:03 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by mx.groups.io with SMTP id smtpd.web11.41932.1691758500120642155 for ; Fri, 11 Aug 2023 05:55:00 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=hJ/XTi6B; spf=pass (domain: bootlin.com, ip: 217.70.183.196, mailfrom: alexis.lothore@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 67A1CE0010; Fri, 11 Aug 2023 12:54:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1691758498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9LA7mgoy2D2LVaOf40Ou8NmjHdiYBedZcADRBTswbOw=; b=hJ/XTi6B2/2M5BWvQAesPOX9rKZc6ndba5/MYREjJq8pBkNBffaKQWBUJih3AJybdLZWX3 DdKNzejXjvVI+si4oLGakHPgVbW7BWuNl+Tgq3TVO3eDZyMV/SZukJ5JjvQeWsL57QkGDv acWEYcHv5u2pzo7G/AGXE0oZN+JNUwml/M1zFjnOjSZ9dmewrI9J1Pm4IOv/FPycsMQH6H VB2tAYuJ3Z2m9seNVGm44jUUeAxO1vdECNL/gJWbWubCJcP/KkH9cXeQwAbG9EjHioE5/m riuCvv594CZ8XbneS0J5vgdlGSE96YNDar3hRXrwlcjOLhvWNvwzF91gaa48mQ== From: =?utf-8?q?Alexis_Lothor=C3=A9?= To: Cc: Thomas Petazzoni , Alexandre Belloni Subject: [OE-Core][PATCH 2/2] oeqa/utils/gitarchive: fix tag computation when creating archive Date: Fri, 11 Aug 2023 14:55:32 +0200 Message-ID: <20230811125532.9427-3-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230811125532.9427-1-alexis.lothore@bootlin.com> References: <20230811125532.9427-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: alexis.lothore@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 11 Aug 2023 12:55:03 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/185837 From: Alexis Lothoré Sporadic errors have been observed in autobuilder when trying to store new tests results: error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults' hint: Updates were rejected because the tag already exists in the remote. The new tag name is generated by gitarchive based on known tags from the repository (learnt with git tag). In autobuilder case, this repository is a shallow clone, so git tag only returns most recent tags, which mean we could miss some older tags which exist in remote but not locally. In this case, gitarchive will likely create a tag which already exists in remote, and so will fail to push Fix this tag duplication by using git ls-remote to learn about existing tags instead of git tag. Two places which wrongly read only local tags has been identified in gitarchive: expand_tag_strings and get_test_runs Fixes [YOCTO #15140] Signed-off-by: Alexis Lothoré --- meta/lib/oeqa/utils/gitarchive.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/utils/gitarchive.py b/meta/lib/oeqa/utils/gitarchive.py index 6e8040eb5c96..73beafecb5fb 100644 --- a/meta/lib/oeqa/utils/gitarchive.py +++ b/meta/lib/oeqa/utils/gitarchive.py @@ -116,7 +116,8 @@ def expand_tag_strings(repo, name_pattern, msg_subj_pattern, msg_body_pattern, tag_re = tag_re.format(tag_number='(?P[0-9]{1,5})') keyws['tag_number'] = 0 - for existing_tag in repo.run_cmd('tag').splitlines(): + tags_refs = repo.run_cmd(['ls-remote', '--refs', '--tags', '-q']) + for existing_tag in ["".join(d.split()[1].split('/', 2)[2:]) for d in tags_refs.splitlines()]: match = re.match(tag_re, existing_tag) if match and int(match.group('tag_number')) >= keyws['tag_number']: @@ -181,7 +182,8 @@ def get_test_runs(log, repo, tag_name, **kwargs): # Get a list of all matching tags tag_pattern = tag_name.format(**str_fields) - tags = repo.run_cmd(['tag', '-l', tag_pattern]).splitlines() + revs = repo.run_cmd(['ls-remote', '--refs', '--tags', 'origin', '-q', tag_pattern]).splitlines() + tags = ["".join(d.split()[1].split('/', 2)[2:]) for d in revs] log.debug("Found %d tags matching pattern '%s'", len(tags), tag_pattern) # Parse undefined fields from tag names