From patchwork Tue Feb 24 16:32:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 81814 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8DC5F3C9B9 for ; Tue, 24 Feb 2026 16:33:04 +0000 (UTC) Received: from smtp-42ab.mail.infomaniak.ch (smtp-42ab.mail.infomaniak.ch [84.16.66.171]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.24551.1771950777985246838 for ; Tue, 24 Feb 2026 08:32:58 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@0leil.net header.s=20231125 header.b=nYEl2QfM; spf=pass (domain: 0leil.net, ip: 84.16.66.171, mailfrom: foss+yocto@0leil.net) Received: from smtp-4-0001.mail.infomaniak.ch (smtp-4-0001.mail.infomaniak.ch [10.7.10.108]) by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4fL3DS1RjmzgXm; Tue, 24 Feb 2026 17:32:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=0leil.net; s=20231125; t=1771950776; bh=CfYxGmPxIknzfbIgB2IxJDRQJtI4Dk3qfj2S9vBFgW0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nYEl2QfMI8xN5I5FxLBxQekb/zKl+i12t2yS3cTJ1pEGGzmftk/DL56tbQRRiHcuR eJNBebNEtHrw1VuKmx/0tZpAKO6NZ4AjNMP3cG9R0cLDrS4dQC/41gQpAvcdvldfnT 4My5kRmpSf5dZQufUmkQjMHO38cKeTlZTbEYqd+9vld0wLmUUy6XKLXqlxZ3fMjfQ/ 3GfelYWfPxekF+RhyuqP9WdxVuIbDXayESTWbGqG50u75cbPTJDc0EiBpxRtKSXRU+ hGQgQmx8k82ISwWHvKXSodEiSLPQWZrWNZ+MySLlE3h1ibUC/Z2YfjQJkRnm0qmJtA 7JithyIProrsw== Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4fL3DR5f6zzYQD; Tue, 24 Feb 2026 17:32:55 +0100 (CET) From: Quentin Schulz Date: Tue, 24 Feb 2026 17:32:36 +0100 Subject: [PATCH v5 1/4] Makefile: use python3 -m sphinx instead of sphinx-build MIME-Version: 1.0 Message-Id: <20260224-fix-make-multi-target-v5-1-7ebf281618b4@cherry.de> References: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> In-Reply-To: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz X-Mailer: b4 0.15-dev-47773 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 24 Feb 2026 16:33:04 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/8977 From: Quentin Schulz sphinx-build is a "script"[1][2] which is then made available on the system as a binary, upon installing the sphinx package. You can have multiple instances of sphinx installed, e.g. system-wide via the traditional distro package manager or in a venv. In the case of a venv, if you have sphinx already installed in your host system, source the venv activate script and then install sphinx via pip inside the venv, sphinx-build command will still point at the sphinx from the host system. The work-around is either to run the venv activate script again, run `hash -r` or `rehash` in your current shell, or call python3 -m sphinx instead of the Python script. Let's do the latter as it's the easiest/least-user-facing solution. This works because python3 interpreter will be the one from the venv, and it sets the appropriate paths for loading modules (use only packages from the venv by default, or prefer them if --system-site-packages is passed). [1] https://github.com/sphinx-doc/sphinx/blob/v9.1.0/pyproject.toml#L99 [2] https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#creating-executable-scripts Signed-off-by: Quentin Schulz --- documentation/Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/documentation/Makefile b/documentation/Makefile index e144a50b4..5d9fda501 100644 --- a/documentation/Makefile +++ b/documentation/Makefile @@ -4,7 +4,7 @@ # You can set these variables from the command line, and also # from the environment for the first two. SPHINXOPTS ?= -W --keep-going -j auto -SPHINXBUILD ?= sphinx-build +SPHINXBUILD ?= python3 -m sphinx # Release notes are excluded because they contain contributor names and commit messages which can't be modified VALEOPTS ?= --no-wrap --glob '!migration-guides/release-notes-*.rst' SOURCEDIR = . @@ -16,8 +16,8 @@ DESTDIR = final SVG2PNG = rsvg-convert SVG2PDF = rsvg-convert -ifeq ($(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi),0) -$(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed") +ifeq ($(shell if $(SPHINXBUILD) --version >/dev/null 2>&1; then echo 1; else echo 0; fi),0) +$(error "Sphinx was not found. Make sure you have it installed") endif # Put it first so that "make" without argument is like "make help". From patchwork Tue Feb 24 16:32:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 81816 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAA4CE9B278 for ; Tue, 24 Feb 2026 16:33:04 +0000 (UTC) Received: from smtp-8fad.mail.infomaniak.ch (smtp-8fad.mail.infomaniak.ch [83.166.143.173]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.24391.1771950778313975515 for ; Tue, 24 Feb 2026 08:32:59 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@0leil.net header.s=20231125 header.b=epid6TIO; spf=pass (domain: 0leil.net, ip: 83.166.143.173, mailfrom: foss+yocto@0leil.net) Received: from smtp-4-0001.mail.infomaniak.ch (smtp-4-0001.mail.infomaniak.ch [10.7.10.108]) by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4fL3DS5596zg1w; Tue, 24 Feb 2026 17:32:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=0leil.net; s=20231125; t=1771950776; bh=LkSlzmQaWmm8qm6rb2Ly1Z+YDE+5KB/5Z+FFl0qY030=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=epid6TIOPy1ZgxwyNQcSP0tutreg4oPvgWJZ3SGvJ2cgqopoAOAc1+gj7JV4M3l4A I6siVFOODy8SPGIw2VxNSOvd+ncugeTi0b6QWKkMPNCLu0pfB0N8Mhd0aWGSe8Bo/M +QRsCfx6Gc1LI/2vjLuBHSlAnMF/g5nPwmtws9bDtGkX9iDSQFsqsHsKttWnq9Ymn5 W01i9iYM7tfZCTIY13ON4Gucu1BzYAjb3Ob4TDBQOaQEHfzTGBPCRNWWamdfKHOa7M JXw3tyUZ3Y8Jl/QR+BYKajEYW1ElgQ/xUEJDWtOXC62F2AjdhdrDUB1Sj+pnmdKLdp ibVpiltBmDrMw== Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4fL3DS29wWzXwV; Tue, 24 Feb 2026 17:32:56 +0100 (CET) From: Quentin Schulz Date: Tue, 24 Feb 2026 17:32:37 +0100 Subject: [PATCH v5 2/4] use a venv for installing packages with pip MIME-Version: 1.0 Message-Id: <20260224-fix-make-multi-target-v5-2-7ebf281618b4@cherry.de> References: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> In-Reply-To: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz X-Mailer: b4 0.15-dev-47773 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 24 Feb 2026 16:33:04 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/8979 From: Quentin Schulz In a few commits, we'll migrate Debian/Ubuntu to use pip packages. However, Debian rejects pip commands outside of virtual environments. It is generally best practice to use virtual environments with pip, so let's migrate distros that currently use pip to use virtual environments as well. Because BitBake is a Python project and requires a few Python packages that we currently install with distro package managers, it would be best to also include system packages via --system-site-packages when creating the virtual environment such that the user would either be able to build the docs or run BitBake, from the same venv. Unfortunately, one of the packages[1] we'll include in an upcoming commit breaks sphinxcontrib namespace package and thus we cannot have a mix of sphinxcontrib-namespaced packages in and outside the venv. Let's create a completely empty venv for now until the offending package is fixed (see pending merge request[2]). Let's instruct the user to install the docs venv in a relative directory (likely the root of the yocto-docs git repo) instead of guessing a safe location (/tmp may or may not be an actual volatile directory depending on the distro). [1] https://github.com/missinglinkelectronics/sphinxcontrib-svg2pdfconverter/ [2] https://github.com/missinglinkelectronics/sphinxcontrib-svg2pdfconverter/pull/34 Signed-off-by: Quentin Schulz --- documentation/tools/containerfiles/Containerfile.dnf | 11 +++++++++++ documentation/tools/containerfiles/Containerfile.zypper | 11 +++++++++++ documentation/tools/host_packages_scripts/pip3_docs.sh | 4 +++- 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/documentation/tools/containerfiles/Containerfile.dnf b/documentation/tools/containerfiles/Containerfile.dnf index fd47fde34..12d97ef6c 100644 --- a/documentation/tools/containerfiles/Containerfile.dnf +++ b/documentation/tools/containerfiles/Containerfile.dnf @@ -1,6 +1,17 @@ ARG ARG_FROM=fedora:40 # default value to avoid warning FROM $ARG_FROM +# RUN . ./yocto-docs-venv/bin/activate doesn't work because env variables set +# in RUN don't persist, so set up the virtualenv manually ahead of time. +# Unsetting env variables in Containerfile is difficult, so we hope it works +# fine without doing that for PYTHONHOME (see activate script content). +# Note that the actual location depends on WORKDIR at this layer in the +# container, and it must match the pwd when pip3_docs.sh is run in a later +# layer. This *must* be outside of any mount point otherwise it won't be +# available within the container. +ENV VIRTUAL_ENV=$PWD/yocto-docs-venv +ENV PATH="$VIRTUAL_ENV/bin:$PATH" + ARG INCLUDE_ESSENTIAL_PACKAGES=0 ARG ESSENTIAL=fedora_essential.sh ARG DOCS=fedora_docs.sh diff --git a/documentation/tools/containerfiles/Containerfile.zypper b/documentation/tools/containerfiles/Containerfile.zypper index 3850b9ff9..6e667967c 100644 --- a/documentation/tools/containerfiles/Containerfile.zypper +++ b/documentation/tools/containerfiles/Containerfile.zypper @@ -1,6 +1,17 @@ ARG ARG_FROM=opensuse/leap:15.4 # default value to avoid warning FROM $ARG_FROM +# RUN . ./yocto-docs-venv/bin/activate doesn't work because env variables set +# in RUN don't persist, so set up the virtualenv manually ahead of time. +# Unsetting env variables in Containerfile is difficult, so we hope it works +# fine without doing that for PYTHONHOME (see activate script content). +# Note that the actual location depends on WORKDIR at this layer in the +# container, and it must match the pwd when pip3_docs.sh is run in a later +# layer. This *must* be outside of any mount point otherwise it won't be +# available within the container. +ENV VIRTUAL_ENV=$PWD/yocto-docs-venv +ENV PATH="$VIRTUAL_ENV/bin:$PATH" + ARG INCLUDE_ESSENTIAL_PACKAGES=0 ARG ESSENTIAL=opensuse_essential.sh ARG DOCS=opensuse_docs.sh diff --git a/documentation/tools/host_packages_scripts/pip3_docs.sh b/documentation/tools/host_packages_scripts/pip3_docs.sh index 907ecec55..b7f19b7ad 100644 --- a/documentation/tools/host_packages_scripts/pip3_docs.sh +++ b/documentation/tools/host_packages_scripts/pip3_docs.sh @@ -1 +1,3 @@ -sudo pip3 install sphinx sphinx_rtd_theme pyyaml sphinx-copybutton +python3 -m venv --clear ./yocto-docs-venv +. ./yocto-docs-venv/bin/activate +python3 -m pip install sphinx sphinx_rtd_theme pyyaml sphinx-copybutton From patchwork Tue Feb 24 16:32:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 81817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA82FEEC2B5 for ; Tue, 24 Feb 2026 16:33:04 +0000 (UTC) Received: from smtp-bc09.mail.infomaniak.ch (smtp-bc09.mail.infomaniak.ch [45.157.188.9]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.24392.1771950778916101292 for ; Tue, 24 Feb 2026 08:32:59 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@0leil.net header.s=20231125 header.b=WTl4gjY9; spf=pass (domain: 0leil.net, ip: 45.157.188.9, mailfrom: foss+yocto@0leil.net) Received: from smtp-4-0001.mail.infomaniak.ch (unknown [IPv6:2001:1600:7:10::a6c]) by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4fL3DT29Z0zfsJ; Tue, 24 Feb 2026 17:32:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=0leil.net; s=20231125; t=1771950777; bh=6dNsCMOiJRiigo0ZKnmpc3wIHjKRuCY7d/l8YwVSxnM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=WTl4gjY9a/Va32t1mB9P3kE2cku9Cj2uIbZmWH/Cpl1GZ1kAP/bolq4dshR35wL2I DH7sY2aAaCIig5T28Gn0C2u7TOC7w/FLudxYYcnZr+EQ2+mqXuX827iKMQNlNo++Y0 womgJ0i0UijgxRe+qSau5c1v3Tyk1W20vgitaXY+NLkPHMQEsHLk3JPc9MsCrP/a9R g3ymTuUzz365Hpo+bxV3kNo0tiZByfgy422geqiUEEPFF9toMNfjDHdq53bHBZkxAH mrEQNaycjsH0aWbDapUMLv0LVnvpv/leSHxIqwXi/K0poA1D1NV65jeJ8cId4zsT0g 3AvatAF/6iazQ== Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4fL3DS5q7MzYMl; Tue, 24 Feb 2026 17:32:56 +0100 (CET) From: Quentin Schulz Date: Tue, 24 Feb 2026 17:32:38 +0100 Subject: [PATCH v5 3/4] consistently use pip packages for all distros MIME-Version: 1.0 Message-Id: <20260224-fix-make-multi-target-v5-3-7ebf281618b4@cherry.de> References: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> In-Reply-To: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz , Antonin Godard X-Mailer: b4 0.15-dev-47773 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 24 Feb 2026 16:33:04 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/8980 From: Quentin Schulz Until now, Debian and Ubuntu had special instructions for installing Python packages to build the docs. Specifically, the packages that all other distros are installing via pip were installed from the package feed (apt). In the next commit, a new Python module will be required, which isn't available in the package feed. We thus have two options, have a pip3 Debian/Ubuntu-specific set of instructions to install this one package, or migrate packages currently installed through the package manager to use pip like other distros. To lower maintenance burden and the possibility of missing to update both "generic" and "Debian/Ubuntu" variants of pip instructions, the latter was chosen. Suggested-by: Antonin Godard Signed-off-by: Quentin Schulz --- documentation/ref-manual/system-requirements.rst | 3 +++ documentation/tools/build-docs-container | 4 +++- documentation/tools/containerfiles/Containerfile.apt | 14 ++++++++++++++ documentation/tools/host_packages_scripts/ubuntu_docs.sh | 2 +- 4 files changed, 21 insertions(+), 2 deletions(-) diff --git a/documentation/ref-manual/system-requirements.rst b/documentation/ref-manual/system-requirements.rst index f9f32f2a8..e7c1c3916 100644 --- a/documentation/ref-manual/system-requirements.rst +++ b/documentation/ref-manual/system-requirements.rst @@ -171,6 +171,9 @@ Here are the packages needed to build Project documentation manuals: .. literalinclude:: ../tools/host_packages_scripts/ubuntu_docs.sh :language: shell +.. literalinclude:: ../tools/host_packages_scripts/pip3_docs.sh + :language: shell + In addition to the previous packages, here are the packages needed to build the documentation in PDF format: diff --git a/documentation/tools/build-docs-container b/documentation/tools/build-docs-container index 23c3dfe33..f2f54044c 100755 --- a/documentation/tools/build-docs-container +++ b/documentation/tools/build-docs-container @@ -115,6 +115,7 @@ main () essential=ubuntu_essential.sh docs=ubuntu_docs.sh docs_pdf=ubuntu_docs_pdf.sh + pip3=pip3_docs.sh ;; "fedora:39"*|\ "fedora:40"*|\ @@ -151,6 +152,7 @@ main () essential=ubuntu_essential.sh docs=ubuntu_docs.sh docs_pdf=ubuntu_docs_pdf.sh + pip3=pip3_docs.sh ;; *) echo "$image not supported!" @@ -166,7 +168,7 @@ main () --build-arg ESSENTIAL="host_packages_scripts/$essential" \ --build-arg DOCS="host_packages_scripts/$docs" \ --build-arg DOCS_PDF="host_packages_scripts/$docs_pdf" \ - --build-arg PIP3="host_packages_scripts/${pip3:-}" \ + --build-arg PIP3="host_packages_scripts/$pip3" \ --file "$SCRIPT_DIR/containerfiles/$containerfile" \ "$SCRIPT_DIR" diff --git a/documentation/tools/containerfiles/Containerfile.apt b/documentation/tools/containerfiles/Containerfile.apt index a573786f0..7ee168be1 100644 --- a/documentation/tools/containerfiles/Containerfile.apt +++ b/documentation/tools/containerfiles/Containerfile.apt @@ -1,10 +1,22 @@ ARG ARG_FROM=debian:12 # default value to avoid warning FROM $ARG_FROM +# RUN . ./yocto-docs-venv/bin/activate doesn't work because env variables set +# in RUN don't persist, so set up the virtualenv manually ahead of time. +# Unsetting env variables in Containerfile is difficult, so we hope it works +# fine without doing that for PYTHONHOME (see activate script content). +# Note that the actual location depends on WORKDIR at this layer in the +# container, and it must match the pwd when pip3_docs.sh is run in a later +# layer. This *must* be outside of any mount point otherwise it won't be +# available within the container. +ENV VIRTUAL_ENV=$PWD/yocto-docs-venv +ENV PATH="$VIRTUAL_ENV/bin:$PATH" + ARG INCLUDE_ESSENTIAL_PACKAGES=0 ARG ESSENTIAL=ubuntu_essential.sh ARG DOCS=ubuntu_docs.sh ARG DOCS_PDF=ubuntu_docs_pdf.sh +ARG PIP3=pip3_docs.sh ENV DEBIAN_FRONTEND=noninteractive ARG TZ=Europe/Vienna @@ -13,6 +25,7 @@ ARG TZ=Europe/Vienna COPY --chmod=777 ${ESSENTIAL} /temp/host_packages_essential.sh COPY --chmod=777 ${DOCS} /temp/host_packages_docs.sh COPY --chmod=777 ${DOCS_PDF} /temp/host_packages_docs_pdf.sh +COPY --chmod=777 ${PIP3} /temp/pip3_docs.sh RUN ln -fs "/usr/share/zoneinfo/$TZ" /etc/localtime \ && apt-get update \ @@ -20,6 +33,7 @@ RUN ln -fs "/usr/share/zoneinfo/$TZ" /etc/localtime \ && if [ "$INCLUDE_ESSENTIAL_PACKAGES" = "1" ]; then yes | /temp/host_packages_essential.sh; fi \ && yes | /temp/host_packages_docs.sh \ && yes | /temp/host_packages_docs_pdf.sh \ + && yes | /temp/pip3_docs.sh \ && apt-get --yes autoremove \ && apt-get clean \ && rm -rf /temp diff --git a/documentation/tools/host_packages_scripts/ubuntu_docs.sh b/documentation/tools/host_packages_scripts/ubuntu_docs.sh index c322b61e4..db0743021 100644 --- a/documentation/tools/host_packages_scripts/ubuntu_docs.sh +++ b/documentation/tools/host_packages_scripts/ubuntu_docs.sh @@ -1 +1 @@ -sudo apt install git librsvg2-bin locales make python3-saneyaml python3-sphinx-rtd-theme python3-sphinx-copybutton sphinx +sudo apt install git librsvg2-bin locales make python3-saneyaml python3-pip python3-venv From patchwork Tue Feb 24 16:32:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 81815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6A85F3C9BC for ; Tue, 24 Feb 2026 16:33:04 +0000 (UTC) Received: from smtp-42a9.mail.infomaniak.ch (smtp-42a9.mail.infomaniak.ch [84.16.66.169]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.24553.1771950779538365363 for ; Tue, 24 Feb 2026 08:32:59 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@0leil.net header.s=20231125 header.b=Uc2aLu+L; spf=pass (domain: 0leil.net, ip: 84.16.66.169, mailfrom: foss+yocto@0leil.net) Received: from smtp-4-0001.mail.infomaniak.ch (smtp-4-0001.mail.infomaniak.ch [10.7.10.108]) by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4fL3DT5JwdzglC; Tue, 24 Feb 2026 17:32:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=0leil.net; s=20231125; t=1771950777; bh=udHgxuCU00MuR8+F1ZfZj+BCh+489o+pI+ezYAne8fw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Uc2aLu+Ly6dXqNooWkncOh3QUEcr2mzgV/EwAC88EdiAZjQdV/9A9tmxzDYISxhoL JN6koBKR/GYSNVOePUpbkbzKnQCKEjLF5cx+MgYn9Y2mhXRWm38kpR5eyFqlGq9JCe bdmrwFTO0VyL9fzS46jFGzX2XAGCpEty7u/xa1fS0QHvZE1A7GHUjWg7tLI5lUlSOn 4BR2mkVg+rFsw46/ktnFbxkG93yYoWTVqIcq+e17LMWJkGnOjDZxrAJnJT2YR4MffB 7wslP7ppXXBcg6B4z2xCNM9S/HPVEVVLwU04M0vfqYKnILbqBtc6AtfbDpAZJdtiBH pxT7VtMZaov9w== Received: from unknown by smtp-4-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4fL3DT2FLLzWQf; Tue, 24 Feb 2026 17:32:57 +0100 (CET) From: Quentin Schulz Date: Tue, 24 Feb 2026 17:32:39 +0100 Subject: [PATCH v5 4/4] convert SVGs to PDF and PNG using sphinxcontrib.rsvgconverter plugin MIME-Version: 1.0 Message-Id: <20260224-fix-make-multi-target-v5-4-7ebf281618b4@cherry.de> References: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> In-Reply-To: <20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de> To: docs@lists.yoctoproject.org Cc: Quentin Schulz X-Mailer: b4 0.15-dev-47773 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 24 Feb 2026 16:33:04 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/8981 From: Quentin Schulz The sphinxcontrib.rsvgconverter plugin allows to generate a PDF or PNG from an SVG. This is what we already do manually via Make targets but it isn't good enough. Sphinx generates a cache upon first parsing and stores it in a doctrees directory. Sphinx claims that it can be shared between all builders[1]. For glob patterns in image or figure directives, Sphinx will look for all possible matches in the source tree and store those in a "candidates" map for each file, along with the associated mimetype. When building, Sphinx will then look in this map to try to find a match in the current builder's supported_image_types. If none is found, the build will fail. The latexpdf (using the LaTeXBuilder) target does not support SVGs by default[2]. We used Make to generate PDFs from them before generating the doc PDF though (see PDFs variable and %.pdf in Makefile) and that type is supported by default[2]. The epub (via the Epub3Builder) target does support SVGs by default[3] but we disabled their support in commit ff3876ca4910 ("conf.py: use PNG first in EPUB output"). We used Make to generate PNGs from them before generating the doc epub though (see PNGs variable and %.png in Makefile) and that type is supported (c.f. Epub3Builder.supported_image_types in our conf.py). The issue is that this is done transparently from Sphinx. When we generate the PDFs or PNGs variants of the SVGs, we put them in-tree directly along their source file. Then, when caching, Sphinx will find both the source file and the appropriate variant. However, the cache isn't updated if there are new files in the tree and the source rST files aren't modified. So, the cache will not have its map updated and we won't be able to find the new variant when building for a non-SVG-compatible builder. Take the following scenario: - start from a clean source file (fresh clone or git clean -ffdx) - build the html target (which supports SVGs by default[4]) - sphinx will find all the files in the source tree matching the glob pattern in ".. image:: test.*", in our case only an SVG since the PDFs and PNGs are only generated for the latexpdf and epub targets respectively. The cache will only store the path to the SVG file because it is the only source file that matches the glob, - attempt to build the epub target (which doesn't support SVGs, only PNGs) - Sphinx checks for the file to include for '.. image:: test.*' and finds an SVG in the cache map and then check the list of supported image types for the Epub3Builder and find that it doesn't support that. It cannot find anything satisfying the dependency and thus fails to build. This scenario can easily be reproduced by running the `make all` command since the html builder will be used first, then epub and finally latexpdf. The `make publish` target works by chance, because the epub builder is built first and will cache SVG + PNG for each glob, then the latexpdf builder is built and supports PNGs so that's fine and then html, which supports SVG as well. To fix this issue, we could simply always generate PDFs and PNGs of all SVGs in the source tree, but this isn't ideal. Instead, let's use an ImageConverter from a Sphinx plugin. This allows to map a plugin as being able to generate a file of type Y from a file of type X. When Sphinx wants to build an image, it'll try to find the image with the type the current builder supports in the cache. If it cannot, it's going to try to find an ImageConverter plugin that is able to convert one of the image types in cache with one of the image types the current builder supports. Then Sphinx will call this plugin to generate the file and put it into the build directory (not in the source!). This allows to simplify the Makefile as well and is a much cleaner approach. The epub target is removed as the catch-all target contains the same instructions as the epub target we remove in this commit. sphinxcontrib.rsvgconverter is a plugin from sphinxcontrib-svg2pdfconverter python module. SVGs to PNGs is only supported since 2.0.0. [1] https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-d [2] https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.latex.LaTeXBuilder.supported_image_types [3] https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.epub3.Epub3Builder.supported_image_types [4] https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.html.StandaloneHTMLBuilder Signed-off-by: Quentin Schulz --- documentation/Makefile | 27 +++------------------- documentation/Pipfile | 2 ++ documentation/conf.py | 1 + .../tools/host_packages_scripts/pip3_docs.sh | 2 +- 4 files changed, 7 insertions(+), 25 deletions(-) diff --git a/documentation/Makefile b/documentation/Makefile index 5d9fda501..76be4f068 100644 --- a/documentation/Makefile +++ b/documentation/Makefile @@ -10,11 +10,8 @@ VALEOPTS ?= --no-wrap --glob '!migration-guides/release-notes-*.rst' SOURCEDIR = . VALEDOCS ?= $(SOURCEDIR) SPHINXLINTDOCS ?= $(SOURCEDIR) -IMAGEDIRS = */svg BUILDDIR = _build DESTDIR = final -SVG2PNG = rsvg-convert -SVG2PDF = rsvg-convert ifeq ($(shell if $(SPHINXBUILD) --version >/dev/null 2>&1; then echo 1; else echo 0; fi),0) $(error "Sphinx was not found. Make sure you have it installed") @@ -24,7 +21,7 @@ endif help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -.PHONY: all checks help Makefile clean stylecheck publish epub latexpdf +.PHONY: all checks help Makefile clean stylecheck publish latexpdf publish: Makefile checks epub latexpdf html singlehtml rm -rf $(BUILDDIR)/$(DESTDIR)/ @@ -35,22 +32,8 @@ publish: Makefile checks epub latexpdf html singlehtml cp $(BUILDDIR)/singlehtml/index.html $(BUILDDIR)/$(DESTDIR)/singleindex.html sed -i -e 's@index.html#@singleindex.html#@g' $(BUILDDIR)/$(DESTDIR)/singleindex.html -# Build a list of SVG files to convert to PDFs -PDFs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.pdf,$(wildcard $(SOURCEDIR)/$(dir)/*.svg))) - -# Build a list of SVG files to convert to PNGs -PNGs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.png,$(wildcard $(SOURCEDIR)/$(dir)/*.svg))) - -# Pattern rule for converting SVG to PDF -%.pdf : %.svg - $(SVG2PDF) --format=pdf --output=$@ $< - -# Pattern rule for converting SVG to PNG -%.png : %.svg - $(SVG2PNG) --format=png --output=$@ $< - clean: - @rm -rf $(BUILDDIR) $(PNGs) $(PDFs) poky.yaml sphinx-static/switchers.js releases.rst + @rm -rf $(BUILDDIR) poky.yaml sphinx-static/switchers.js releases.rst checks: $(SOURCEDIR)/tools/check-glossaries --docs-dir $(SOURCEDIR) @@ -62,14 +45,10 @@ stylecheck: sphinx-lint: sphinx-lint $(SPHINXLINTDOCS) -epub: $(PNGs) - $(SOURCEDIR)/set_versions.py - @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) - # Note: we need to pass buf_size here (which is also configurable from # texmf.cnf), to avoid following error: # Unable to read an entire line---bufsize=200000. Please increase buf_size in texmf.cnf. -latexpdf: $(PDFs) +latexpdf: $(SOURCEDIR)/set_versions.py buf_size=10000000 $(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/documentation/Pipfile b/documentation/Pipfile index 1fa2df31e..67fce078d 100644 --- a/documentation/Pipfile +++ b/documentation/Pipfile @@ -10,6 +10,8 @@ sphinx = "*" sphinx-rtd-theme = "*" pyyaml = "*" sphinx-copybutton = "*" +# SVG to PNG only supported since 2.0.0 +sphinxcontrib-svg2pdfconverter = ">=2.0.0" [requires] python_version = "3" diff --git a/documentation/conf.py b/documentation/conf.py index 3d04f9057..5a69977cf 100644 --- a/documentation/conf.py +++ b/documentation/conf.py @@ -67,6 +67,7 @@ extensions = [ 'sphinx.ext.extlinks', 'sphinx.ext.intersphinx', 'sphinx_copybutton', + 'sphinxcontrib.rsvgconverter', 'yocto-vars' ] autosectionlabel_prefix_document = True diff --git a/documentation/tools/host_packages_scripts/pip3_docs.sh b/documentation/tools/host_packages_scripts/pip3_docs.sh index b7f19b7ad..852845783 100644 --- a/documentation/tools/host_packages_scripts/pip3_docs.sh +++ b/documentation/tools/host_packages_scripts/pip3_docs.sh @@ -1,3 +1,3 @@ python3 -m venv --clear ./yocto-docs-venv . ./yocto-docs-venv/bin/activate -python3 -m pip install sphinx sphinx_rtd_theme pyyaml sphinx-copybutton +python3 -m pip install sphinx sphinx_rtd_theme pyyaml sphinx-copybutton 'sphinxcontrib-svg2pdfconverter>=2.0.0'