diff mbox series

[v5,2/4] use a venv for installing packages with pip

Message ID 20260224-fix-make-multi-target-v5-2-7ebf281618b4@cherry.de
State New
Headers show
Series venv migration for docs, migrate Debian/Ubuntu to it, fix epub and latexpdf targets not finding glob images | expand

Commit Message

Quentin Schulz Feb. 24, 2026, 4:32 p.m. UTC 
From: Quentin Schulz <quentin.schulz@cherry.de>

In a few commits, we'll migrate Debian/Ubuntu to use pip packages.
However, Debian rejects pip commands outside of virtual environments.

It is generally best practice to use virtual environments with pip, so
let's migrate distros that currently use pip to use virtual environments
as well.

Because BitBake is a Python project and requires a few Python packages
that we currently install with distro package managers, it would be best
to also include system packages via --system-site-packages when creating
the virtual environment such that the user would either be able to build
the docs or run BitBake, from the same venv.

Unfortunately, one of the packages[1] we'll include in an upcoming
commit breaks sphinxcontrib namespace package and thus we cannot have a
mix of sphinxcontrib-namespaced packages in and outside the venv. Let's
create a completely empty venv for now until the offending package is
fixed (see pending merge request[2]).

Let's instruct the user to install the docs venv in a relative directory
(likely the root of the yocto-docs git repo) instead of guessing a safe
location (/tmp may or may not be an actual volatile directory depending
on the distro).

[1] https://github.com/missinglinkelectronics/sphinxcontrib-svg2pdfconverter/
[2] https://github.com/missinglinkelectronics/sphinxcontrib-svg2pdfconverter/pull/34
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
---
 documentation/tools/containerfiles/Containerfile.dnf    | 11 +++++++++++
 documentation/tools/containerfiles/Containerfile.zypper | 11 +++++++++++
 documentation/tools/host_packages_scripts/pip3_docs.sh  |  4 +++-
 3 files changed, 25 insertions(+), 1 deletion(-)
diff mbox series

Patch

diff --git a/documentation/tools/containerfiles/Containerfile.dnf b/documentation/tools/containerfiles/Containerfile.dnf
index fd47fde34..12d97ef6c 100644
--- a/documentation/tools/containerfiles/Containerfile.dnf
+++ b/documentation/tools/containerfiles/Containerfile.dnf
@@ -1,6 +1,17 @@ 
 ARG ARG_FROM=fedora:40 # default value to avoid warning
 FROM $ARG_FROM
 
+# RUN . ./yocto-docs-venv/bin/activate doesn't work because env variables set
+# in RUN don't persist, so set up the virtualenv manually ahead of time.
+# Unsetting env variables in Containerfile is difficult, so we hope it works
+# fine without doing that for PYTHONHOME (see activate script content).
+# Note that the actual location depends on WORKDIR at this layer in the
+# container, and it must match the pwd when pip3_docs.sh is run in a later
+# layer. This *must* be outside of any mount point otherwise it won't be
+# available within the container.
+ENV VIRTUAL_ENV=$PWD/yocto-docs-venv
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
 ARG INCLUDE_ESSENTIAL_PACKAGES=0
 ARG ESSENTIAL=fedora_essential.sh
 ARG DOCS=fedora_docs.sh
diff --git a/documentation/tools/containerfiles/Containerfile.zypper b/documentation/tools/containerfiles/Containerfile.zypper
index 3850b9ff9..6e667967c 100644
--- a/documentation/tools/containerfiles/Containerfile.zypper
+++ b/documentation/tools/containerfiles/Containerfile.zypper
@@ -1,6 +1,17 @@ 
 ARG ARG_FROM=opensuse/leap:15.4 # default value to avoid warning
 FROM $ARG_FROM
 
+# RUN . ./yocto-docs-venv/bin/activate doesn't work because env variables set
+# in RUN don't persist, so set up the virtualenv manually ahead of time.
+# Unsetting env variables in Containerfile is difficult, so we hope it works
+# fine without doing that for PYTHONHOME (see activate script content).
+# Note that the actual location depends on WORKDIR at this layer in the
+# container, and it must match the pwd when pip3_docs.sh is run in a later
+# layer. This *must* be outside of any mount point otherwise it won't be
+# available within the container.
+ENV VIRTUAL_ENV=$PWD/yocto-docs-venv
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
 ARG INCLUDE_ESSENTIAL_PACKAGES=0
 ARG ESSENTIAL=opensuse_essential.sh
 ARG DOCS=opensuse_docs.sh
diff --git a/documentation/tools/host_packages_scripts/pip3_docs.sh b/documentation/tools/host_packages_scripts/pip3_docs.sh
index 907ecec55..b7f19b7ad 100644
--- a/documentation/tools/host_packages_scripts/pip3_docs.sh
+++ b/documentation/tools/host_packages_scripts/pip3_docs.sh
@@ -1 +1,3 @@ 
-sudo pip3 install sphinx sphinx_rtd_theme pyyaml sphinx-copybutton
+python3 -m venv --clear ./yocto-docs-venv
+. ./yocto-docs-venv/bin/activate
+python3 -m pip install sphinx sphinx_rtd_theme pyyaml sphinx-copybutton