mbox series

[v3,0/2] add script for containers building the documentation

Message ID 20250127-instructions-shell-container-v3-0-9373d3f156f5@cherry.de
Headers show
Series add script for containers building the documentation | expand

Message

Quentin Schulz Jan. 27, 2025, 6:37 p.m. UTC
Based on initial work done by Antonin Godard[1].

This does two things:
- move all the installation steps to fulfil the system requirements for
  each distro into separate files,
- add a script for using those separate files for building a container
  which allows to build the documentation within,

Note that not all distros we say we support in the System Requirements
section work currently.

openSUSE Leap 15.6 somehow doesn't build but with not much interesting
context.

Almalinux instructions are missing a bunch of packages and I reached a
point where I couldn't find a package providing everysel.sty though
texlive-ms seems to be providing it according to pkgs.org.

How far I went:

"""
 diff --git a/documentation/tools/almalinux_host_packages_docs_pdf.sh b/documentation/tools/almalinux_host_packages_docs_pdf.sh
 index 8341eb8c2..19f45312b 100644
 --- a/documentation/tools/almalinux_host_packages_docs_pdf.sh
 +++ b/documentation/tools/almalinux_host_packages_docs_pdf.sh
 @@ -1 +1,5 @@
 -sudo dnf install latexmk texlive-collection-fontsrecommended texlive-collection-latex texlive-collection-latexrecommended texlive-collection-xetex texlive-fncychap texlive-gnu-freefont texlive-tex-gyre texlive-xetex
 +sudo dnf install -y epel-release
 +sudo yum install dnf-plugins-core
 +sudo dnf config-manager --set-enabled crb
 +sudo dnf makecache
 +sudo dnf install -y latexmk texlive-capt-of texlive-collection-fontsrecommended texlive-collection-latex texlive-collection-latexrecommended texlive-collection-xetex texlive-ctex texlive-fncychap texlive-framed texlive-gnu-freefont texlive-ms texlive-needspace texlive-tabulary texlive-tex-gyre texlive-upquote texlive-wrapfig texlive-xetex
"""

Note the dnf install -y for the packages is required for installing the
GPG key of one of the repos noninteractively.

Older Debian and Ubuntu both lack python3-saneyaml and similar Python
packages in their packagefeed.

Ubuntu 23.04 package feed isn't accessible anymore.

Note that this script cannot be run in parallel right now as all
containers would be sharing the same build directory. This could be
mitigated by allowing BUILDDIR to be overridden from the environment
before being used by make and have it overridden by the script to a
different location. There are probably other corner cases not handled
(likely concurrent set_versions.py for example?).

This was only build tested, I haven't looked at the build output.

[1] https://lore.kernel.org/yocto-docs/20241205-docs-build-dockerfiles-v2-1-047cb3245adf@bootlin.com/
[2] https://lore.kernel.org/yocto-docs/20241202110128.27711-1-guenael.muller@smile.fr/

Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
---
Changes in v3:
- add usage when the script is called with less than one argument or
  with a first argument that isn't supported by the script for now,
- use sed to add --non-interactive to zypper commands instead of adding
  them in the user instructions that would make it to the doc website,
- limit OCI images that can be used by hardcoding the tag (or start of
  the tag),
- remove ability to "build" on non-working distros (they are commented
  out with an explanation so someone can look at it if they feel like
  it),
- added vim modeline so people using vim will use the same indentation
  by default,
- rebase on master,
- Link to v2: https://lore.kernel.org/r/20241217-instructions-shell-container-v2-0-ce687b0cfac2@cherry.de

Changes in v2:
- rebase on master,
- fix issues for recent Ubuntu by setting the timezone,
- add support for building pdf docs from within container,
- fix support for passing multiple make targets to the script
- have all pip3-packages-requiring-distros use one file to make it
  easier to maintain,
- put scripts into a dedicated repo,
- rename variables and filenames to remove host_packages/packages from
  them to make things a bit more digest,
- fix incorrect comment in script about reading poky.yaml.in,

- Link to v1: https://lore.kernel.org/r/20241210-instructions-shell-container-v1-0-6a7cdc404ff4@cherry.de

---
Quentin Schulz (2):
      docs: use literalinclude for system requirements
      tools: add script for building documentation inside containers

 documentation/poky.yaml.in                         | 234 ---------------------
 documentation/ref-manual/system-requirements.rst   |  82 +++++---
 documentation/tools/Containerfile.almalinux        |   1 +
 documentation/tools/Containerfile.apt              |  26 +++
 documentation/tools/Containerfile.debian           |   1 +
 documentation/tools/Containerfile.dnf              |  25 +++
 documentation/tools/Containerfile.fedora           |   1 +
 documentation/tools/Containerfile.ubuntu           |   1 +
 documentation/tools/Containerfile.zypper           |  32 +++
 documentation/tools/build-docs-container           | 175 +++++++++++++++
 .../tools/host_packages_scripts/almalinux_docs.sh  |   1 +
 .../host_packages_scripts/almalinux_docs_pdf.sh    |   1 +
 .../host_packages_scripts/almalinux_essential.sh   |   5 +
 .../tools/host_packages_scripts/fedora_docs.sh     |   1 +
 .../tools/host_packages_scripts/fedora_docs_pdf.sh |   1 +
 .../host_packages_scripts/fedora_essential.sh      |   1 +
 .../tools/host_packages_scripts/opensuse_docs.sh   |   1 +
 .../host_packages_scripts/opensuse_docs_pdf.sh     |   1 +
 .../host_packages_scripts/opensuse_essential.sh    |   2 +
 .../tools/host_packages_scripts/pip3_docs.sh       |   1 +
 .../tools/host_packages_scripts/ubuntu_docs.sh     |   1 +
 .../tools/host_packages_scripts/ubuntu_docs_pdf.sh |   1 +
 .../host_packages_scripts/ubuntu_essential.sh      |   1 +
 23 files changed, 327 insertions(+), 269 deletions(-)
---
base-commit: 8bd2bc3c00ca80f4c000a2a8d618a9f8ea3aa54b
change-id: 20241210-instructions-shell-container-c5a84c043a35

Best regards,