diff mbox series

SSTATE_MIRRORS/SOURCE_MIRROR_URL: add instructions for mirror authentication

Message ID 20241224-sstate-mirror-user-password-v1-1-d33e424617be@bootlin.com
State Under Review
Headers show
Series SSTATE_MIRRORS/SOURCE_MIRROR_URL: add instructions for mirror authentication | expand

Commit Message

Antonin Godard Dec. 24, 2024, 3:35 p.m. UTC 
[ YOCTO #15218 ]

There are different ways of configuring the build host when
authentication to a mirror (SSTATE_MIRRORS or SOURCE_MIRROR_URL) is
required.

Document these methods, and begin with the preferred approach which is
to edit ~/.netrc appropriately. When that is not possible, specifying
the credentials from the URL is also possible, so document it as well.

Reported-by: peter.zsifkovits@at.bosch.com
Tested-by: Antonin Godard <antonin.godard@bootlin.com> # Debian 12 container
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Tested in a Debian 12 container like so:

  $ mkdir downloads
  $ cp .../{bash-5.2.21.tar.gz,bash-5.2.21.tar.gz.done} downloads/
  $ pip install sauth # python http server with basic auth
  $ sauth username password 127.0.0.1

In local.conf:

  INHERIT += "own-mirrors"
  SOURCE_MIRROR_URL = "http://127.0.0.1:8333/downloads"
  BB_ALLOWED_NETWORKS = "127.0.0.1"

In bitbake's shell:

  $ bitbake bash -c cleanall
  $ bitbake bash -c fetch

As expected, error in log.do_fetch:

  Connecting to 127.0.0.1:8333... connected.
  HTTP request sent, awaiting response... 401 Unauthorized

Now, configure the following in ~/.netrc:

  machine 127.0.0.1
  login username
  password password

Try again: "bitbake bash -c fetch" runs successfully.

Remove ~/.netrc, replace in local.conf:

  SOURCE_MIRROR_URL = "http://127.0.0.1:8333/downloads;user=username;pswd=password"

In bitbake's shell:

  $ bitbake bash -c cleanall
  $ bitbake bash -c fetch

"bitbake bash -c fetch" runs successfully.
---
 documentation/ref-manual/variables.rst | 53 ++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)


---
base-commit: 28850c974a3896895bc921c094071523218d6d07
change-id: 20241224-sstate-mirror-user-password-9743df197a3e

Best regards,

Comments

Antonin Godard Dec. 26, 2024, 8:48 a.m. UTC | #1 
On Tue Dec 24, 2024 at 4:35 PM CET, Antonin Godard wrote:
> [ YOCTO #15218 ]
>
> There are different ways of configuring the build host when
> authentication to a mirror (SSTATE_MIRRORS or SOURCE_MIRROR_URL) is
> required.
>
> Document these methods, and begin with the preferred approach which is
> to edit ~/.netrc appropriately. When that is not possible, specifying
> the credentials from the URL is also possible, so document it as well.
>
> Reported-by: peter.zsifkovits@at.bosch.com
> Tested-by: Antonin Godard <antonin.godard@bootlin.com> # Debian 12 container
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
> Tested in a Debian 12 container like so:
>
>   $ mkdir downloads
>   $ cp .../{bash-5.2.21.tar.gz,bash-5.2.21.tar.gz.done} downloads/
>   $ pip install sauth # python http server with basic auth
>   $ sauth username password 127.0.0.1
>
> In local.conf:
>
>   INHERIT += "own-mirrors"
>   SOURCE_MIRROR_URL = "http://127.0.0.1:8333/downloads"
>   BB_ALLOWED_NETWORKS = "127.0.0.1"
>
> In bitbake's shell:
>
>   $ bitbake bash -c cleanall
>   $ bitbake bash -c fetch
>
> As expected, error in log.do_fetch:
>
>   Connecting to 127.0.0.1:8333... connected.
>   HTTP request sent, awaiting response... 401 Unauthorized
>
> Now, configure the following in ~/.netrc:
>
>   machine 127.0.0.1
>   login username
>   password password

Forgot to mention: I also chowned this file to 600 and 400, and it works.
diff mbox series

Patch

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index e8db89f8c9642b1e36c574e1369093a7a2ce30a4..88eda960d0f06307d483ecd9c2f6d1c9cc6920f4 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8032,6 +8032,31 @@  system and gives an overview of their function and contents.
 
          You can specify only a single URL in :term:`SOURCE_MIRROR_URL`.
 
+      .. note::
+
+         If the mirror is protected behind a username and password, the
+         :term:`build host` needs to be configured so the :term:`build system
+         <OpenEmbedded Build System>` is able to fetch from the mirror.
+
+         The recommended way to do that is by setting the following parameters
+         in ``$HOME/.netrc`` (``$HOME`` being the :term:`build host` home
+         directory)::
+
+            machine example.com
+            login <user>
+            password <password>
+
+         This file requires permissions set to ``400`` or ``600`` to prevent
+         other users from reading the file::
+
+            chmod 600 "$HOME/.netrc"
+
+         Another method to configure the username and password is from the URL
+         in :term:`SOURCE_MIRROR_URL` directly, with the ``user`` and ``pswd``
+         parameters::
+
+            SOURCE_MIRROR_URL = "http://example.com/my_source_mirror;user=<user>;pswd=<password>"
+
    :term:`SPDX_ARCHIVE_PACKAGED`
       This option allows to add to :term:`SPDX` output compressed archives
       of the files in the generated target packages.
@@ -8439,6 +8464,34 @@  system and gives an overview of their function and contents.
              file://.* https://someserver.tld/share/sstate/PATH;downloadfilename=PATH \
              file://.* file:///some-local-dir/sstate/PATH"
 
+      .. note::
+
+         If the mirror is protected behind a username and password, the
+         :term:`build host` needs to be configured so the :term:`build system
+         <OpenEmbedded Build System>` is able to download the sstate cache using
+         authentication.
+
+         The recommended way to do that is by setting the following parameters
+         in ``$HOME/.netrc`` (``$HOME`` being the :term:`build host` home
+         directory)::
+
+            machine someserver.tld
+            login <user>
+            password <password>
+
+         This file requires permissions set to ``400`` or ``600`` to prevent
+         other users from reading the file::
+
+            chmod 600 "$HOME/.netrc"
+
+         Another method to configure the username and password is from the
+         URL in :term:`SSTATE_MIRRORS` directly, with the ``user`` and ``pswd``
+         parameters::
+
+            SSTATE_MIRRORS ?= "\
+                file://.* https://someserver.tld/share/sstate/PATH;user=<user>;pswd=<password>;downloadfilename=PATH \
+            "
+
       The Yocto Project actually shares the cache data objects built by its
       autobuilder::