From patchwork Tue Dec 10 15:26:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Quentin Schulz X-Patchwork-Id: 53885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19926E77180 for ; Tue, 10 Dec 2024 15:27:30 +0000 (UTC) Received: from smtp-bc0c.mail.infomaniak.ch (smtp-bc0c.mail.infomaniak.ch [45.157.188.12]) by mx.groups.io with SMTP id smtpd.web11.13054.1733844440498558127 for ; Tue, 10 Dec 2024 07:27:21 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: 0leil.net, ip: 45.157.188.12, mailfrom: foss+yocto@0leil.net) Received: from smtp-3-0000.mail.infomaniak.ch (unknown [IPv6:2001:1600:4:17::246b]) by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4Y72fH0YZ3zjHx; Tue, 10 Dec 2024 16:27:19 +0100 (CET) Received: from unknown by smtp-3-0000.mail.infomaniak.ch (Postfix) with ESMTPA id 4Y72fG5KNqzV7c; Tue, 10 Dec 2024 16:27:18 +0100 (CET) From: Quentin Schulz Date: Tue, 10 Dec 2024 16:26:45 +0100 Subject: [PATCH RFC 2/2] tools: add script for building documentation inside containers MIME-Version: 1.0 Message-Id: <20241210-instructions-shell-container-v1-2-6a7cdc404ff4@cherry.de> References: <20241210-instructions-shell-container-v1-0-6a7cdc404ff4@cherry.de> In-Reply-To: <20241210-instructions-shell-container-v1-0-6a7cdc404ff4@cherry.de> To: Quentin Schulz , docs@lists.yoctoproject.org X-Mailer: b4 0.14.2 X-Infomaniak-Routing: alpha List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 10 Dec 2024 15:27:30 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5926 From: Quentin Schulz This adds a script for building a container and building the documentation within that new container image. The openSUSE instructions now require a --non-interactive flag otherwise they fail to run. Sadly there doesn't seem to be a way to have this in an environment variable à-la DEBIAN_FRONTEND=noninteractive. Note that the Ubuntu and openSUSE instructions currently do not work. Signed-off-by: Quentin Schulz --- documentation/tools/Containerfile.almalinux | 1 + documentation/tools/Containerfile.apt | 21 +++++ documentation/tools/Containerfile.debian | 1 + documentation/tools/Containerfile.dnf | 19 ++++ documentation/tools/Containerfile.fedora | 1 + documentation/tools/Containerfile.ubuntu | 1 + documentation/tools/Containerfile.zypper | 18 ++++ documentation/tools/build-docs-container | 101 +++++++++++++++++++++ documentation/tools/opensuse_host_packages_docs.sh | 2 +- .../tools/opensuse_host_packages_essential.sh | 2 +- 10 files changed, 165 insertions(+), 2 deletions(-) diff --git a/documentation/tools/Containerfile.almalinux b/documentation/tools/Containerfile.almalinux new file mode 120000 index 0000000000000000000000000000000000000000..7237e9b99f4132957c0ad5b11fa6cefe9daaec74 --- /dev/null +++ b/documentation/tools/Containerfile.almalinux @@ -0,0 +1 @@ +Containerfile.dnf \ No newline at end of file diff --git a/documentation/tools/Containerfile.apt b/documentation/tools/Containerfile.apt new file mode 100644 index 0000000000000000000000000000000000000000..a601699ce4005b3bc5288990dbf2e7fa235ea431 --- /dev/null +++ b/documentation/tools/Containerfile.apt @@ -0,0 +1,21 @@ +ARG ARG_FROM=debian:12 # default value to avoid warning +FROM $ARG_FROM + +ARG HOST_DOCS_PACKAGES=ubuntu_host_packages_docs.sh + +ENV DEBIAN_FRONTEND=noninteractive + +# relative to the location of the dockerfile +COPY --chmod=777 ${HOST_DOCS_PACKAGES} /temp/host_packages_docs.sh + +RUN apt-get update \ + && apt-get install -y sudo \ + && yes | /temp/host_packages_docs.sh \ + && apt-get --yes autoremove \ + && apt-get clean \ + && rm -rf /temp + +RUN git config --global --add safe.directory /docs + +ENTRYPOINT ["/usr/bin/env", "make", "-C", "documentation/"] +CMD ["publish"] diff --git a/documentation/tools/Containerfile.debian b/documentation/tools/Containerfile.debian new file mode 120000 index 0000000000000000000000000000000000000000..5a7a425197afc2928802fcad5f34699b1ad3348a --- /dev/null +++ b/documentation/tools/Containerfile.debian @@ -0,0 +1 @@ +Containerfile.apt \ No newline at end of file diff --git a/documentation/tools/Containerfile.dnf b/documentation/tools/Containerfile.dnf new file mode 100644 index 0000000000000000000000000000000000000000..29ea670905ac011966d7bf5340c10be0660149a6 --- /dev/null +++ b/documentation/tools/Containerfile.dnf @@ -0,0 +1,19 @@ +ARG ARG_FROM=fedora:40 # default value to avoid warning +FROM $ARG_FROM + +ARG HOST_DOCS_PACKAGES=fedora_host_packages_docs.sh + +# relative to the location of the dockerfile +COPY --chmod=777 ${HOST_DOCS_PACKAGES} /temp/host_packages_docs.sh + +RUN dnf update -y \ + && dnf install -y sudo \ + && yes | /temp/host_packages_docs.sh \ + && dnf autoremove -y \ + && dnf clean all -y \ + && rm -rf /temp + +RUN git config --global --add safe.directory /docs + +ENTRYPOINT ["/usr/bin/env", "make", "-C", "documentation/"] +CMD ["publish"] diff --git a/documentation/tools/Containerfile.fedora b/documentation/tools/Containerfile.fedora new file mode 120000 index 0000000000000000000000000000000000000000..7237e9b99f4132957c0ad5b11fa6cefe9daaec74 --- /dev/null +++ b/documentation/tools/Containerfile.fedora @@ -0,0 +1 @@ +Containerfile.dnf \ No newline at end of file diff --git a/documentation/tools/Containerfile.ubuntu b/documentation/tools/Containerfile.ubuntu new file mode 120000 index 0000000000000000000000000000000000000000..5a7a425197afc2928802fcad5f34699b1ad3348a --- /dev/null +++ b/documentation/tools/Containerfile.ubuntu @@ -0,0 +1 @@ +Containerfile.apt \ No newline at end of file diff --git a/documentation/tools/Containerfile.zypper b/documentation/tools/Containerfile.zypper new file mode 100644 index 0000000000000000000000000000000000000000..567256d65b580aab45e27a7b7a386e7a8dbe107f --- /dev/null +++ b/documentation/tools/Containerfile.zypper @@ -0,0 +1,18 @@ +ARG ARG_FROM=opensuse/leap:15.4 # default value to avoid warning +FROM $ARG_FROM + +ARG HOST_DOCS_PACKAGES=opensuse_host_packages_docs.sh + +# relative to the location of the dockerfile +COPY --chmod=777 ${HOST_DOCS_PACKAGES} /temp/host_packages_docs.sh + +RUN zypper update -y \ + && zypper install -y sudo \ + && yes | /temp/host_packages_docs.sh \ + && zypper clean --all \ + && rm -rf /temp + +RUN git config --global --add safe.directory /docs + +ENTRYPOINT ["/usr/bin/env", "make", "-C", "documentation/"] +CMD ["publish"] diff --git a/documentation/tools/build-docs-container b/documentation/tools/build-docs-container new file mode 100755 index 0000000000000000000000000000000000000000..4a799bd8e795f1ab7d55eb99116fe859aff2692c --- /dev/null +++ b/documentation/tools/build-docs-container @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# +# Build a container ready to build the documentation be reading the dependencies +# listed in poky.yaml.in, and start a documentation build in this container. +# +# Usage: +# +# ./documentation/tools/build-docs-container [] +# +# e.g.: +# +# ./documentation/tools/build-docs-container ubuntu:24.04 html +# +# Will build the docs in an Ubuntu 24.04 container in html. +# +# The container engine can be selected by exporting CONTAINERCMD in the +# environment. The default is docker, but podman can also be used. + +set -eu -o pipefail + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) +CONTAINERCMD=${CONTAINERCMD:-docker} +DOCS_DIR="$SCRIPT_DIR/../.." + +main () +{ + local image="$1" + shift + + OCI=$(which "$CONTAINERCMD") + + # docker build doesn't accept 2 colons, so "sanitize" the name + local sanitized_dockername + sanitized_dockername=$(echo "$image" | tr ':.' '-') + + local version + version=$(echo "$image" | awk -F: '{print $NF}') + + case $image in + almalinux*) + containerfile=Containerfile.almalinux + host_packages_docs=almalinux_host_packages_docs.sh + ;; + debian*) + containerfile=Containerfile.debian + host_packages_docs=ubuntu_host_packages_docs.sh + ;; + fedora*) + containerfile=Containerfile.fedora + host_packages_docs=fedora_host_packages_docs.sh + ;; + opensuse* | leap*) + image=opensuse/leap:$version + containerfile=Containerfile.zypper + host_packages_docs=opensuse_host_packages_docs.sh + ;; + ubuntu*) + containerfile=Containerfile.ubuntu + host_packages_docs=ubuntu_host_packages_docs.sh + ;; + *) + echo "$image not supported" + exit 1 + ;; + esac + + $OCI build \ + --tag "yocto-docs-$sanitized_dockername:latest" \ + --build-arg ARG_FROM="docker.io/$image" \ + --build-arg HOST_DOCS_PACKAGES="$host_packages_docs" \ + --file "$SCRIPT_DIR/$containerfile" \ + "$SCRIPT_DIR/" + + local -a args_run=( + --rm + --interactive + --tty + --volume="$DOCS_DIR:/docs:rw" + --workdir=/docs + --security-opt label=disable + ) + + if [ "$OCI" = "docker" ]; then + args_run+=( + --user="$(id -u)":"$(id -g)" + ) + elif [ "$OCI" = "podman" ]; then + # we need net access to fetch bitbake terms + args_run+=( + --cap-add=NET_RAW + --userns=keep-id + ) + fi + + $OCI run \ + "${args_run[@]}" \ + "yocto-docs-$sanitized_dockername" \ + "$*" +} + +main "$@" diff --git a/documentation/tools/opensuse_host_packages_docs.sh b/documentation/tools/opensuse_host_packages_docs.sh index 90f52d576451ee0a7c481b31b7605e4a870496cd..18975bd8d2db93eaadc663fac738e9400fd11ffb 100644 --- a/documentation/tools/opensuse_host_packages_docs.sh +++ b/documentation/tools/opensuse_host_packages_docs.sh @@ -1,2 +1,2 @@ -sudo zypper install git make python3-pip which inkscape texlive-fncychap +sudo zypper --non-interactive install git make python3-pip which inkscape texlive-fncychap sudo pip3 install sphinx sphinx_rtd_theme pyyaml diff --git a/documentation/tools/opensuse_host_packages_essential.sh b/documentation/tools/opensuse_host_packages_essential.sh index ef24fcefd99c9ecc335c594965965bb36e43cd05..20d934c34b833f547463d2e635ee16db935de3dc 100644 --- a/documentation/tools/opensuse_host_packages_essential.sh +++ b/documentation/tools/opensuse_host_packages_essential.sh @@ -1,2 +1,2 @@ -sudo zypper install python gcc gcc-c++ git chrpath make wget python-xml diffstat makeinfo python-curses patch socat python3 python3-curses tar python3-pip python3-pexpect xz which python3-Jinja2 rpcgen zstd lz4 bzip2 gzip hostname libacl1 +sudo zypper --non-interactive install python gcc gcc-c++ git chrpath make wget python-xml diffstat makeinfo python-curses patch socat python3 python3-curses tar python3-pip python3-pexpect xz which python3-Jinja2 rpcgen zstd lz4 bzip2 gzip hostname libacl1 sudo pip3 install GitPython