[v2] ref-manual: structure.rst: document missing tmp/ dirs

Document `hosttools/`, `pkgdata/` and add some more information on
`work-shared/`.

Adresses [YOCTO #14543].

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
---
Changes in v2:
- hosttools: add missing "the"
- hosttools: Remove "pre-installed" packages and point to host
  requirements section instead.
- work-shared: s/as a way to make it common to/to make it available to/
- Link to v1: https://lore.kernel.org/r/20241014-bug-14543-v1-1-7c8519be9c9c@bootlin.com
---
 documentation/ref-manual/structure.rst | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)


---
base-commit: 1304b899cd4480f80eaeb6e58d57e87d3d2f6911
change-id: 20241014-bug-14543-56fb4089086f

Best regards,

Hi Antonin,

On 10/16/24 9:27 AM, Antonin Godard wrote:
> Document `hosttools/`, `pkgdata/` and add some more information on
> `work-shared/`.
> 
> Adresses [YOCTO #14543].
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
> ---
> Changes in v2:
> - hosttools: add missing "the"
> - hosttools: Remove "pre-installed" packages and point to host
>    requirements section instead.
> - work-shared: s/as a way to make it common to/to make it available to/
> - Link to v1: https://lore.kernel.org/r/20241014-bug-14543-v1-1-7c8519be9c9c@bootlin.com
> ---
>   documentation/ref-manual/structure.rst | 31 +++++++++++++++++++++++++++++--
>   1 file changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/documentation/ref-manual/structure.rst b/documentation/ref-manual/structure.rst
> index e4d8b54bb9e39ffe344d2db1798e4fe76c742f4e..738d4a6a8e1bc968abd4c0dcce8e0b6cde9db4e4 100644
> --- a/documentation/ref-manual/structure.rst
> +++ b/documentation/ref-manual/structure.rst
> @@ -484,6 +484,30 @@ the ":ref:`sdk-manual/appendix-obtain:building an sdk installer`"
>   section in the Yocto Project Application Development and the Extensible
>   Software Development Kit (eSDK) manual.
>   
> +.. _structure-build-tmp-hosttools:
> +
> +``build/tmp/hosttools/``
> +~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The OpenEmbedded build system uses this directory to create symbolic links to
> +some of the host components that are allowed to be called within tasks. These
> +are basic components listed in the :ref:`ref-manual/system-requirements:required
> +packages for the build host` section. These components are also listed in the
> +:term:`HOSTTOOLS` variable and are limited to this list to prevent host
> +contamination.
> +
> +.. _structure-build-tmp-pkgdata:
> +
> +``build/tmp/pkgdata/``
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +The OpenEmbedded build system uses this directory to store package metadata
> +generated during the :ref:`ref-tasks-packagedata` task. The files stored in this
> +directory contain information about each output package produced by the
> +OpenEmbedded build system, and are used in different ways by the build system
> +such as ":ref:`dev-manual/debugging:viewing package information with
> +\`\`oe-pkgdata-util\`\``".
> +
>   .. _structure-build-tmp-sstate-control:
>   
>   ``build/tmp/sstate-control/``
> @@ -657,8 +681,11 @@ Here are key subdirectories within each recipe work directory:
>   
>   For efficiency, the OpenEmbedded build system creates and uses this
>   directory to hold recipes that share a work directory with other
> -recipes. In practice, this is only used for ``gcc`` and its variants
> -(e.g. ``gcc-cross``, ``libgcc``, ``gcc-runtime``, and so forth).
> +recipes. This is for example used for ``gcc`` and its variants (e.g.
> +``gcc-cross``, ``libgcc``, ``gcc-runtime``, and so forth), or by the
> +:ref:`ref-classes-kernel` class to store the kernel source code and kernel build
> +artifacts to make it available to out-of-tree kernel modules or other

s/it/them/

I would also try to avoid one of the three "to" we have in the same 
sentence, so maybe:

"""
or by the :ref:`ref-classes-kernel` class to make the kernel source code 
and kernel build artifacts available to...
"""

Also, I'm wondering if we shouldn't expand a bit more on the 
"efficiency" part or what makes a recipe a candidate for using 
work-shared? I assume this is mostly because if we were to use the 
normal sysroot mechanism, it would be overgrown and take even more 
storage space than currently? I honestly do not know. Also, this doesn't 
need to be done in this commit, incremental changes are fine here :)

With the it/them swap or the second suggestion:

Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>

Thanks!
Quentin

Hi Quentin,

On Wed Oct 16, 2024 at 3:30 PM CEST, Quentin Schulz via lists.yoctoproject.org wrote:
> >   For efficiency, the OpenEmbedded build system creates and uses this
> >   directory to hold recipes that share a work directory with other
> > -recipes. In practice, this is only used for ``gcc`` and its variants
> > -(e.g. ``gcc-cross``, ``libgcc``, ``gcc-runtime``, and so forth).
> > +recipes. This is for example used for ``gcc`` and its variants (e.g.
> > +``gcc-cross``, ``libgcc``, ``gcc-runtime``, and so forth), or by the
> > +:ref:`ref-classes-kernel` class to store the kernel source code and kernel build
> > +artifacts to make it available to out-of-tree kernel modules or other
>
> s/it/them/
>
> I would also try to avoid one of the three "to" we have in the same
> sentence, so maybe:
>
> """
> or by the :ref:`ref-classes-kernel` class to make the kernel source code
> and kernel build artifacts available to...
> """

Better phrasing, agreed :)

> Also, I'm wondering if we shouldn't expand a bit more on the
> "efficiency" part or what makes a recipe a candidate for using
> work-shared? I assume this is mostly because if we were to use the
> normal sysroot mechanism, it would be overgrown and take even more
> storage space than currently? I honestly do not know. Also, this doesn't
> need to be done in this commit, incremental changes are fine here :)

Makes sense expanding to me too. I think reducing storage usage is one of the
reasons, another would be to share a workdir like gcc... But mentioning it's
rarity is a good thing IMO.

Since I'd rather not forget to add this, I will try to add a note on this it in
my v3.

> With the it/them swap or the second suggestion:
>
> Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
>
> Thanks!
> Quentin

Thank you!
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com