From patchwork Thu Oct 9 10:23:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 71921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4095CCD188 for ; Thu, 9 Oct 2025 10:24:16 +0000 (UTC) Received: from smtp-42ae.mail.infomaniak.ch (smtp-42ae.mail.infomaniak.ch [84.16.66.174]) by mx.groups.io with SMTP id smtpd.web10.8912.1760005453701236369 for ; Thu, 09 Oct 2025 03:24:14 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: 0leil.net, ip: 84.16.66.174, mailfrom: foss+yocto@0leil.net) Received: from smtp-3-0000.mail.infomaniak.ch (smtp-3-0000.mail.infomaniak.ch [10.4.36.107]) by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4cj5Zh0q6RzhPn; Thu, 9 Oct 2025 12:24:12 +0200 (CEST) Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA id 4cj5Zg5LBwzs9l; Thu, 9 Oct 2025 12:24:11 +0200 (CEST) From: Quentin Schulz Date: Thu, 09 Oct 2025 12:23:59 +0200 Subject: [PATCH v2 1/3] tools/build-docs-container: guarantee the image to run matches the just-built image MIME-Version: 1.0 Message-Id: <20251009-iid-file-v2-1-715d527457f0@cherry.de> References: <20251009-iid-file-v2-0-715d527457f0@cherry.de> In-Reply-To: <20251009-iid-file-v2-0-715d527457f0@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz X-Mailer: b4 0.14.3 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 09 Oct 2025 10:24:16 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/7676 From: Quentin Schulz We aren't that interested in tags actually, the only thing it brings is a belief that we are going to run exactly what we just built. The issue is that this is incorrect. Indeed, one could simply run the script in parallel for the same image. Script runtime A will build the image A and tag it X, Script runtime B will build the image B and tag it X, Script runtime B will run the tag X (image B), Script runtime A will run the tag X (image B). Note that this problem exists whether we are building from the same tree concurrently or from different yocto-docs trees concurrently. One way to fix this could be to introduce random numbers in the tag so that it's always unique, but we would be flooding the system with useless tags. Instead, we can use the sha of the generated image and run that sha directly. If it's the same across rebuilds, it'll stay the same. If it's different, the sha will be different and thus we are safe from concurrent use. The only downside is that we cannot infer from the image sha the underlying distro we're testing. Signed-off-by: Quentin Schulz --- documentation/tools/build-docs-container | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/documentation/tools/build-docs-container b/documentation/tools/build-docs-container index 70e05f295..831d357fb 100755 --- a/documentation/tools/build-docs-container +++ b/documentation/tools/build-docs-container @@ -64,10 +64,6 @@ main () OCI=$(which "$CONTAINERCMD") - # docker build doesn't accept 2 colons, so "sanitize" the name - local sanitized_dockername - sanitized_dockername=$(echo "$image" | tr ':.' '-') - local version version=$(echo "$image" | awk -F: '{print $NF}') @@ -139,8 +135,13 @@ main () ;; esac + local image_sha + image_id_file=$(mktemp) + # Don't clutter tmpfs on fails + trap 'rm -f "$image_id_file"' EXIT + $OCI build \ - --tag "yocto-docs-$sanitized_dockername:latest" \ + --iidfile "$image_id_file" \ --build-arg ARG_FROM="docker.io/$image" \ --build-arg DOCS="$docs" \ --build-arg DOCS_PDF="$docs_pdf" \ @@ -148,6 +149,9 @@ main () --file "$SCRIPT_DIR/$containerfile" \ "$SH_DIR/" + image_sha="$(< "$image_id_file")" + rm "$image_id_file" + local -a args_run=( --rm --interactive @@ -171,7 +175,7 @@ main () $OCI run \ "${args_run[@]}" \ - "yocto-docs-$sanitized_dockername" \ + "$image_sha" \ "$@" } From patchwork Thu Oct 9 10:24:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 71922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B310DCCA470 for ; Thu, 9 Oct 2025 10:24:16 +0000 (UTC) Received: from smtp-190a.mail.infomaniak.ch (smtp-190a.mail.infomaniak.ch [185.125.25.10]) by mx.groups.io with SMTP id smtpd.web10.8914.1760005454265619216 for ; Thu, 09 Oct 2025 03:24:14 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: 0leil.net, ip: 185.125.25.10, mailfrom: foss+yocto@0leil.net) Received: from smtp-3-0000.mail.infomaniak.ch (unknown [IPv6:2001:1600:4:17::246b]) by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4cj5Zh39bWzfv9; Thu, 9 Oct 2025 12:24:12 +0200 (CEST) Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA id 4cj5Zh0hFRzqyB; Thu, 9 Oct 2025 12:24:12 +0200 (CEST) From: Quentin Schulz Date: Thu, 09 Oct 2025 12:24:00 +0200 Subject: [PATCH v2 2/3] Makefile: allow to specify build directory MIME-Version: 1.0 Message-Id: <20251009-iid-file-v2-2-715d527457f0@cherry.de> References: <20251009-iid-file-v2-0-715d527457f0@cherry.de> In-Reply-To: <20251009-iid-file-v2-0-715d527457f0@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz X-Mailer: b4 0.14.3 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 09 Oct 2025 10:24:16 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/7679 From: Quentin Schulz This can be useful to compare the output of two runs or separate the build directory of 2+ distro builds from tools/build-docs-container. Note that make clean cannot remove BUILDDIR from previous runs if they differ as it wouldn't be able to know what it was named in the past. Signed-off-by: Quentin Schulz --- documentation/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/Makefile b/documentation/Makefile index bade78fe8..94d19e350 100644 --- a/documentation/Makefile +++ b/documentation/Makefile @@ -11,7 +11,7 @@ SOURCEDIR = . VALEDOCS ?= $(SOURCEDIR) SPHINXLINTDOCS ?= $(SOURCEDIR) IMAGEDIRS = */svg -BUILDDIR = _build +BUILDDIR ?= _build DESTDIR = final SVG2PNG = rsvg-convert SVG2PDF = rsvg-convert From patchwork Thu Oct 9 10:24:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 71923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE672CCD18A for ; Thu, 9 Oct 2025 10:24:16 +0000 (UTC) Received: from smtp-190c.mail.infomaniak.ch (smtp-190c.mail.infomaniak.ch [185.125.25.12]) by mx.groups.io with SMTP id smtpd.web10.8913.1760005454249939447 for ; Thu, 09 Oct 2025 03:24:14 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: 0leil.net, ip: 185.125.25.12, mailfrom: foss+yocto@0leil.net) Received: from smtp-3-0000.mail.infomaniak.ch (smtp-3-0000.mail.infomaniak.ch [10.4.36.107]) by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4cj5Zh69N4zdXV; Thu, 9 Oct 2025 12:24:12 +0200 (CEST) Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA id 4cj5Zh33S7zrVH; Thu, 9 Oct 2025 12:24:12 +0200 (CEST) From: Quentin Schulz Date: Thu, 09 Oct 2025 12:24:01 +0200 Subject: [PATCH v2 3/3] tools/build-docs-container: build in separate directory for each distro MIME-Version: 1.0 Message-Id: <20251009-iid-file-v2-3-715d527457f0@cherry.de> References: <20251009-iid-file-v2-0-715d527457f0@cherry.de> In-Reply-To: <20251009-iid-file-v2-0-715d527457f0@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz X-Mailer: b4 0.14.3 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 09 Oct 2025 10:24:16 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/7678 From: Quentin Schulz This allows to compare build output from different distros if necessary and make it easier to make sure we start building from a clean slate. Signed-off-by: Quentin Schulz --- documentation/tools/build-docs-container | 2 ++ 1 file changed, 2 insertions(+) diff --git a/documentation/tools/build-docs-container b/documentation/tools/build-docs-container index 831d357fb..47320e182 100755 --- a/documentation/tools/build-docs-container +++ b/documentation/tools/build-docs-container @@ -60,6 +60,7 @@ main () fi local image="$1" + local orig_image=$image shift OCI=$(which "$CONTAINERCMD") @@ -159,6 +160,7 @@ main () --volume="$DOCS_DIR:/docs:rw" --workdir=/docs --security-opt label=disable + --env BUILDDIR="_build-$orig_image-$image_sha" ) if [ "$OCI" = "docker" ]; then