From patchwork Tue Apr 1 13:09:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 60463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12C16C36018 for ; Tue, 1 Apr 2025 13:09:17 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.18688.1743512954452202546 for ; Tue, 01 Apr 2025 06:09:14 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C2011424 for ; Tue, 1 Apr 2025 06:09:17 -0700 (PDT) Received: from cesw-amp-gbt-1s-m12830-04.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8333F3F694 for ; Tue, 1 Apr 2025 06:09:13 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 1/3] CI: always save the lockfile.yml in update-repos Date: Tue, 1 Apr 2025 14:09:07 +0100 Message-ID: <20250401130909.2924629-1-ross.burton@arm.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Apr 2025 13:09:17 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/6427 The update-repos job can "fail with warnings" if the reference repository fetch fails. This is intentionally a warning as the CI may have set KAS_PREMIRRORS and a stale cache is fine. However, by default artifacts are only saved on successful jobs, so if this happens the lockfile.yml isn't saved. Ensure the artifacts are always saved so the rest of the pipeline is successful. Signed-off-by: Ross Burton --- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6d613e45..2337f957 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -114,6 +114,7 @@ update-repos: exit $exit_code artifacts: name: "lockfile" + when: always paths: - lockfile.yml From patchwork Tue Apr 1 13:09:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 60464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19C19C3601C for ; Tue, 1 Apr 2025 13:09:17 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.18689.1743512955103777135 for ; Tue, 01 Apr 2025 06:09:15 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 176D714BF for ; Tue, 1 Apr 2025 06:09:18 -0700 (PDT) Received: from cesw-amp-gbt-1s-m12830-04.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3F5BC3F694 for ; Tue, 1 Apr 2025 06:09:14 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 2/3] CI: disable KAS_REPO_REF_DIR by default Date: Tue, 1 Apr 2025 14:09:08 +0100 Message-ID: <20250401130909.2924629-2-ross.burton@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250401130909.2924629-1-ross.burton@arm.com> References: <20250401130909.2924629-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Apr 2025 13:09:17 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/6428 Having local repo caches is a little fiddly to manage, and by definition we're running CI inside GitLab which supports mirroring repositories automatically. As these mirrors are always available and update automatically, make Kas reference directories opt-in and instead expect that the site is either fine with full fetches, or is using KAS_PREMIRRORS. Signed-off-by: Ross Burton --- .gitlab-ci.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2337f957..2575807e 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -36,7 +36,8 @@ stages: KUBERNETES_CPU_REQUEST: $CPU_REQUEST KAS_WORK_DIR: $CI_PROJECT_DIR/work KAS_BUILD_DIR: $KAS_WORK_DIR/build - KAS_REPO_REF_DIR: $CACHE_DIR/repos + # Set this in the environment to enable local repository caches + KAS_REPO_REF_DIR: "" SSTATE_DIR: $CACHE_DIR/sstate DL_DIR: $CACHE_DIR/downloads BB_LOGCONFIG: $CI_PROJECT_DIR/ci/logging.yml From patchwork Tue Apr 1 13:09:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 60465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19BEAC36014 for ; Tue, 1 Apr 2025 13:09:17 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.18684.1743512955828079118 for ; Tue, 01 Apr 2025 06:09:15 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C7DAF1424 for ; Tue, 1 Apr 2025 06:09:18 -0700 (PDT) Received: from cesw-amp-gbt-1s-m12830-04.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F00563F694 for ; Tue, 1 Apr 2025 06:09:14 -0700 (PDT) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 3/3] CI: dump all environment variables in update-repos Date: Tue, 1 Apr 2025 14:09:09 +0100 Message-ID: <20250401130909.2924629-3-ross.burton@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250401130909.2924629-1-ross.burton@arm.com> References: <20250401130909.2924629-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Apr 2025 13:09:17 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/6429 Print all of the environment variables in the update-repos task for introspection, instead of a subset. Signed-off-by: Ross Burton --- .gitlab-ci.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2575807e..557d48d7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -45,10 +45,6 @@ stages: IMAGE_DIR: $KAS_BUILD_DIR/tmp/deploy/images TOOLCHAIN_LINK_DIR: $KAS_BUILD_DIR/toolchains before_script: - - echo KAS_WORK_DIR = $KAS_WORK_DIR - - echo SSTATE_DIR = $SSTATE_DIR - - echo DL_DIR = $DL_DIR - - echo KAS_PREMIRRORS = $KAS_PREMIRRORS - rm -rf $KAS_WORK_DIR - mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR @@ -99,6 +95,11 @@ update-repos: script: - | exit_code=0 + + # Dump the environment for reference + printenv + + # Update the reference repositories if needed if [ -n "$KAS_REPO_REF_DIR" ]; then flock --verbose --timeout 60 $KAS_REPO_REF_DIR --command ./ci/update-repos || exit_code=$? # Exit now if that failed, unless the status was 128 (fetch failed)