bitbake-user-manual-metadata: add a section about 'addfragments`

From: Alexander Kanavin <alex@linutronix.de>

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../bitbake-user-manual-metadata.rst          | 35 ++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

Hi Alex,

On 12/9/24 2:30 PM, Alexander Kanavin via lists.openembedded.org wrote:
> From: Alexander Kanavin <alex@linutronix.de>
> 
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>   .../bitbake-user-manual-metadata.rst          | 35 ++++++++++++++++++-
>   1 file changed, 34 insertions(+), 1 deletion(-)
> 
> diff --git a/bitbake-user-manual/bitbake-user-manual-metadata.rst b/bitbake-user-manual/bitbake-user-manual-metadata.rst
> index 40a0c6f02d9..16e4946b282 100644
> --- a/bitbake-user-manual/bitbake-user-manual-metadata.rst
> +++ b/bitbake-user-manual/bitbake-user-manual-metadata.rst
> @@ -754,7 +754,9 @@ share the task.
>   This section presents the mechanisms BitBake provides to allow you to
>   share functionality between recipes. Specifically, the mechanisms
>   include ``include``, ``inherit``, :term:`INHERIT`, and ``require``
> -directives.
> +directives. There is also a higher-level abstraction called
> +``configuration fragments`` that is enabled with ``addfragments``
> +directive.
>   

It'd be nice if we used links here too for the directives, but since 
it's not the only one with the issue, not a blocker.

>   Locating Include and Class Files
>   --------------------------------
> @@ -962,6 +964,37 @@ the ``autotools`` and ``pkgconfig`` classes::
>   
>      INHERIT += "autotools pkgconfig"
>   
> +``addfragments`` Directive
> +--------------------------
> +
> +This directive allows fine-tuning local configurations with configuration
> +snippets contained in layers in a structured, controlled way. Typically it would
> +go into ``bitbake.conf``, for example:

Missing second colon to make the below line a code-block.

> +
> +   addfragments conf/fragments OE_FRAGMENTS
> +
> +``addfragments`` takes two parameters:
> +
> +-  path prefix for fragment files inside the layer file tree
> +

I assume you can addfragments a fragment from another layer? So that 
would mean this is a path relative to any layer's LAYER_DIR dir?

> +-  name of variable that holds the list of enabled fragments
> +

+space-separated list

> +This allows listing enabled configuration fragments in ``OE_FRAGMENTS``
> +variable like this:
> +
> +   OE_FRAGMENTS = "core/domain/somefragment core/someotherfragment anotherlayer/anotherdomain/anotherfragment"
> +
> +Fragment names must be prefixed by the layer name where a fragment file is located,
> +defined by :term:`BBFILE_COLLECTIONS` in ``layer.conf``.
> +
> +The implementation then expands this list into
> +:ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>`
> +directives with full paths to respective layers:
> +

Missing second colon to make the below lines a code-block.

> +   require /path/to/core-layer/conf/fragments/domain/somefragment.conf
> +   require /path/to/core-layer/conf/fragments/someotherfragment.conf
> +   require /path/to/another-layer/conf/fragments/anotherdomain/anotherfragment.conf
> +

Can you tell us about a typical case this would be useful? I assume 
there's a reason this was implemented :)

Cheers,
Quentin

On Mon, 9 Dec 2024 at 15:35, Quentin Schulz <quentin.schulz@cherry.de> wrote:
> > +
> > +   addfragments conf/fragments OE_FRAGMENTS
> > +
> > +``addfragments`` takes two parameters:
> > +
> > +-  path prefix for fragment files inside the layer file tree
> > +
>
> I assume you can addfragments a fragment from another layer? So that
> would mean this is a path relative to any layer's LAYER_DIR dir?

addfragments in itself does not add fragments to a currently active
build. It declares two things:

- what is the variable name whose value contains enabled fragments in
an active build
- where to find the fragment files in layers when bitbake is going
over items in that variable and transforming them into require
directives

There's an example below that shows how the transformation from
fragment names to require directives happens.

addfragments is not something users of fragments need to worry about.
It's set just once in bitbake.conf, and the overwhelming majority of
yocto users would be using bitbake.conf from oe-core. But it needs to
be documented as a bitbake syntax item.

> > +   require /path/to/core-layer/conf/fragments/domain/somefragment.conf
> > +   require /path/to/core-layer/conf/fragments/someotherfragment.conf
> > +   require /path/to/another-layer/conf/fragments/anotherdomain/anotherfragment.conf
> > +
>
> Can you tell us about a typical case this would be useful? I assume
> there's a reason this was implemented :)

This has been discussed on various mailing lists many times, you can
search for fragments to find those dicsussions. In bitbake docs I'm
only saying 'This directive allows fine-tuning local configurations
with configuration snippets contained in layers in a structured,
controlled way. ' and not more. The actual user-facing documentation
for fragments will go into yocto manuals, once the remaining pieces
(tools and tests for them) merge into oe-core.

Alex

Hi Alex,

On 12/9/24 3:59 PM, Alexander Kanavin via lists.yoctoproject.org wrote:
> On Mon, 9 Dec 2024 at 15:35, Quentin Schulz <quentin.schulz@cherry.de> wrote:
>>> +
>>> +   addfragments conf/fragments OE_FRAGMENTS
>>> +
>>> +``addfragments`` takes two parameters:
>>> +
>>> +-  path prefix for fragment files inside the layer file tree
>>> +
>>
>> I assume you can addfragments a fragment from another layer? So that
>> would mean this is a path relative to any layer's LAYER_DIR dir?
> 
> addfragments in itself does not add fragments to a currently active
> build. It declares two things:
> 
> - what is the variable name whose value contains enabled fragments in
> an active build
> - where to find the fragment files in layers when bitbake is going
> over items in that variable and transforming them into require
> directives
> 

I'm suggesting to make the explanation a bit more thorough about what 
the "path prefix" actually is. My understanding is that it is the path 
relative to the root of any layer, i.e. relative to any LAYER_DIR.

This answers the question of "where should I put my fragment in my own 
layer so it can be used with addfragments directive". I could have a 
repo with multiple layers for example, where do I put it? Is it impacted 
by BBPATH?

> There's an example below that shows how the transformation from
> fragment names to require directives happens.
> 
> addfragments is not something users of fragments need to worry about.
> It's set just once in bitbake.conf, and the overwhelming majority of
> yocto users would be using bitbake.conf from oe-core. But it needs to
> be documented as a bitbake syntax item.
> 

But it's a directive, so it can be used anywhere if someone feels like 
it? Today there's only one user, bitbake.conf. If no other file is 
allowed to contain this directive or that only a specific kind of file 
is allowed to, we should document it. If it's usable in any file... a 
usecase would be most welcome.

>>> +   require /path/to/core-layer/conf/fragments/domain/somefragment.conf
>>> +   require /path/to/core-layer/conf/fragments/someotherfragment.conf
>>> +   require /path/to/another-layer/conf/fragments/anotherdomain/anotherfragment.conf
>>> +
>>
>> Can you tell us about a typical case this would be useful? I assume
>> there's a reason this was implemented :)
> 
> This has been discussed on various mailing lists many times, you can
> search for fragments to find those dicsussions. In bitbake docs I'm
> only saying 'This directive allows fine-tuning local configurations
> with configuration snippets contained in layers in a structured,
> controlled way. ' and not more. The actual user-facing documentation
> for fragments will go into yocto manuals, once the remaining pieces
> (tools and tests for them) merge into oe-core.
> 

Can't we have nice things in the BitBake docs as well? It is technically 
usable on its own without OE-Core even.

I don't understand why we should actively not document something in 
BitBake user manual only to have it in YP manual. Not my place to push 
for anything so will let maintainers decide for themselves.

Cheers,
Quentin

On Mon, 9 Dec 2024 at 16:50, Quentin Schulz <quentin.schulz@cherry.de> wrote:
> I'm suggesting to make the explanation a bit more thorough about what
> the "path prefix" actually is. My understanding is that it is the path
> relative to the root of any layer, i.e. relative to any LAYER_DIR.

I sent a v2. If you think further explanation for addfragment
parameters is needed, please be specific, and send followup patches.

> This answers the question of "where should I put my fragment in my own
> layer so it can be used with addfragments directive". I could have a
> repo with multiple layers for example, where do I put it? Is it impacted
> by BBPATH?

You place it in a layer which would be the most appropriate for the
fragment, considering its functionality. You can have any amount of
nested categories, as long as its subdirectories under conf/fragments
(or whatever location addfragments prescribes).

It isn't affected by BBPATH because the constructed paths to 'require'
are absolute from / on the build machine.

> Can't we have nice things in the BitBake docs as well? It is technically
> usable on its own without OE-Core even.

Technically, yes, it's usable without oe-core, and it can also be
placed elsewhere than bitbake.conf.

But I do not see a use case for either, and won't try to invent one in
a hand-wavy way. And so I'm describing the syntax, and only the
syntax, and leaving it at that. Actual instructions about using the
fragments are very much oe-core specific, and will be placed in
respective manuals. Including about where to place the fragment, and
how BBPATH is not involved and so on.

Alex