mbox series

[v5,0/4] venv migration for docs, migrate Debian/Ubuntu to it, fix epub and latexpdf targets not finding glob images

Message ID 20260224-fix-make-multi-target-v5-0-7ebf281618b4@cherry.de
Headers show
Series venv migration for docs, migrate Debian/Ubuntu to it, fix epub and latexpdf targets not finding glob images | expand

Message

Quentin Schulz Feb. 24, 2026, 4:32 p.m. UTC
This fixes images being improperly handled by our Make targets by
switching to a sphinx extension that generates a PNG or PDF of an SVG
upon request by Sphinx itself.

To do that, we need sphinxcontrib-svg2pdfconverter>=2.0.0 which is not
available for all supported distros, including Debian/Ubuntu which only
used packages from their official package feeds until now. This migrates
both distros to use pip packages for building the docs such that the
same instructions are used for all supported distros.
This however required to use a virtualenv to install packages, since
Debian doesn't allow using pip outside of a virtualenv since Bookworm
(12). So this migrates the pip package installation to use a virtualenv
as well. Since sourcing scripts in RUN commands to make variables
accessible in later container layers is not possible in Containerfile,
we do the same logic as in the venv bin/activate script manually in the
Containerfile.

Unlike agreed upon during review of v4, I actually decided to NOT do a
sed on ./yocto-docs-venv in pip3_docs.sh because we don't actually need
to. . ($PWD) currently seems to be resolved to / (the default location
for WORKDIR if unmodified by us or our base image (listed in FROM)) so
that means the venv will be outside of the /docs mount.

We may want to think about migrating the instructions in Containerfile
as being run with a non-root user to better match what the user will be
doing. I'm not sure this is straightforward (need to create a user
within the container likely with a stable UID/GID and make sure it's
mapped correctly to the user outside of the container). Just something I
thought when working on this.

This was tested by commenting out /temp/host_packages_docs_pdf.sh in
Containerfiles because ain't nobody got time for installing all those
packages and then running

for distro in almalinux:8 almalinux:9 centos:stream9 fedora:39 fedora:40 fedora:41 fedora:42 leap:15.5 leap:15.6 rockylinux:8 rockylinux:9 debian:12 debian:13 ubuntu:22.04 ubuntu:24.04 ubuntu:25.04 ubuntu:25.10; do
	echo "============$distro==============="
	git clean -ffdx
	CONTAINERCMD=podman tools/build-docs-container $distro html epub || echo $distro >> ../../fail.log
done

Nothing written to fail.log :)

Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
---
Changes in v5:
- switched to relative path for the venv,
- renamed venv to yocto-docs-venv,
- added patch to switch from sphinx-build to python3 -m sphinx for
  better working with venv,
- dropped --system-site-packages from venv as it isn't working due
  to sphinxcontrib-svg2pdfconverter v2.0.0 breaking sphinxcontrib
  namespace,
- slightly reworded commit logs to explain design choices (specifically
  why not using --system-site-packages for the venv),
- Link to v4: https://patch.msgid.link/20260216-fix-make-multi-target-v4-0-749e588cb7f5@cherry.de

Changes in v4:
- migrate all distros to use virtualenv for installing packages via pip,
- migrate Debian and Ubuntu to use pip installation steps for building
  the docs,
- tested all distros build correctly by running the `html epub` make
  targets (`all` doesn't work for almalinux, rockylinux and centos
  stream at the moment),
- Link to v3: https://patch.msgid.link/20260211-fix-make-multi-target-v3-1-1168a6f1525f@cherry.de

Changes in v3:
- rebased on top of master, rebuilt with `make all` no build error to be
  seen,
- Link to v2: https://patch.msgid.link/20251202-fix-make-multi-target-v2-1-7eea1d64cf6b@cherry.de

Changes in v2:
- switched to upstream sphinxcontrib-svg2pdfconverter since the
  necessary MR got merged and a new release made,
- removed IMAGEDIRS Make variable as it is now unused,
- added sphinxcontrib-svg2pdfconverter to Pipfile (not tested) and pip
  installation script,
- Link to v1: https://patch.msgid.link/20251030-fix-make-multi-target-v1-0-213616ed1f0a@cherry.de

---
Quentin Schulz (4):
      Makefile: use python3 -m sphinx instead of sphinx-build
      use a venv for installing packages with pip
      consistently use pip packages for all distros
      convert SVGs to PDF and PNG using sphinxcontrib.rsvgconverter plugin

 documentation/Makefile                             | 33 ++++------------------
 documentation/Pipfile                              |  2 ++
 documentation/conf.py                              |  1 +
 documentation/ref-manual/system-requirements.rst   |  3 ++
 documentation/tools/build-docs-container           |  4 ++-
 .../tools/containerfiles/Containerfile.apt         | 14 +++++++++
 .../tools/containerfiles/Containerfile.dnf         | 11 ++++++++
 .../tools/containerfiles/Containerfile.zypper      | 11 ++++++++
 .../tools/host_packages_scripts/pip3_docs.sh       |  4 ++-
 .../tools/host_packages_scripts/ubuntu_docs.sh     |  2 +-
 10 files changed, 55 insertions(+), 30 deletions(-)
---
base-commit: b70b4599a18ab6261f0c51f51c09f7f1c1bca093
change-id: 20251029-fix-make-multi-target-d2de9d4ff7ec

Best regards,
--  
Quentin Schulz <quentin.schulz@cherry.de>