diff mbox series

[v3] bitbake-user-manual: Document override :append, :prepend, :remove order

Message ID 20220725155625.381756-1-marex@denx.de
State Accepted, archived
Commit 7379c2855b8497e56481d1ec7b5953e850550b85
Headers show
Series [v3] bitbake-user-manual: Document override :append, :prepend, :remove order | expand

Commit Message

Marek Vasut July 25, 2022, 3:56 p.m. UTC
The application order of override syntax :append, :prepend, :remove is
inobvious and undocumented. The order does not match variable parse
history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove
are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note
that this order does not match the variable parse history order. This
is mostly to prevent others from running into this and scratching their
heads, trying to find out what is going on.

Reviewed-by: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Martin Jansa <martin.jansa@gmail.com>
Cc: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Cc: Quentin Schulz <foss+yocto@0leil.net>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
---
V2: - Add RB from Quentin
    - Mention un-remove implementation
    - Reference :remove order in append/prepend override syntax
V3: - Reword the note per suggestion from Quentin
    - Use explicit FOOREMOVE per suggestion from Peter
---
 .../bitbake-user-manual-metadata.rst          | 26 +++++++++++++++++++
 1 file changed, 26 insertions(+)

Comments

Enrico Scholz July 25, 2022, 6:14 p.m. UTC | #1
"Marek Vasut" <marex@denx.de> writes:

>  
> +.. note::
> +
> +   The overrides are applied in this order, ":append", ":prepend", ":remove".
> +   This implies it is not possible to re-append previously removed
>     strings.

fwiw, this is possible by anonymous python functions which are evaluated
after ":remove"

| FOO = "123 abc XYZ"
| 
| python () {
|     v = d.getVar('FOO')
|     d.setVar('FOO', v + ' abc 789')
| }
| 
| FOO:remove = "abc XYZ"

results in

| FOO="123   abc 789"



Enrico
Marek Vasut July 25, 2022, 8:23 p.m. UTC | #2
On 7/25/22 20:14, Enrico Scholz wrote:
> "Marek Vasut" <marex@denx.de> writes:
> 
>>   
>> +.. note::
>> +
>> +   The overrides are applied in this order, ":append", ":prepend", ":remove".
>> +   This implies it is not possible to re-append previously removed
>>      strings.
> 
> fwiw, this is possible by anonymous python functions which are evaluated
> after ":remove"
> 
> | FOO = "123 abc XYZ"
> |
> | python () {
> |     v = d.getVar('FOO')
> |     d.setVar('FOO', v + ' abc 789')
> | }
> |
> | FOO:remove = "abc XYZ"
> 
> results in
> 
> | FOO="123   abc 789"
> 

IIRC anonymous python functions are frowned upon, aren't they ?
Richard Purdie July 25, 2022, 10:04 p.m. UTC | #3
On Mon, 2022-07-25 at 20:14 +0200, Enrico Scholz via
lists.yoctoproject.org wrote:
> "Marek Vasut" <marex@denx.de> writes:
> 
> >  
> > +.. note::
> > +
> > +   The overrides are applied in this order, ":append", ":prepend", ":remove".
> > +   This implies it is not possible to re-append previously removed
> >     strings.
> 
> fwiw, this is possible by anonymous python functions which are evaluated
> after ":remove"
> 
> > FOO = "123 abc XYZ"
> > 
> > python () {
> >     v = d.getVar('FOO')
> >     d.setVar('FOO', v + ' abc 789')
> > }
> > 
> > FOO:remove = "abc XYZ"
> 
> results in
> 
> > FOO="123   abc 789"

The reason for this isn't perhaps obvious and worth documenting - use
of setVar clears all append/prepend/remove operations against a
variable.

FOO = "A"
FOO:append = " B"

python () {
    d.setVar('FOO', 'C')
}

results in C.

Cheers,

Richard
Marek Vasut Aug. 19, 2022, 2:29 p.m. UTC | #4
On 7/25/22 17:56, Marek Vasut wrote:
> The application order of override syntax :append, :prepend, :remove is
> inobvious and undocumented. The order does not match variable parse
> history, i.e. output of "bitbake -e", either.
> 
> Add note documenting the exact order in which :append, :prepend, :remove
> are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note
> that this order does not match the variable parse history order. This
> is mostly to prevent others from running into this and scratching their
> heads, trying to find out what is going on.

Can this be applied now or is there something more to change ?
diff mbox series

Patch

diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
index 33782161..587c9120 100644
--- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
+++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
@@ -319,6 +319,10 @@  The variable ``D`` becomes "dvaladditional data".
 
    You must control all spacing when you use the override syntax.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+
 It is also possible to append and prepend to shell functions and
 BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"
 sections for examples.
@@ -351,6 +355,28 @@  The variable ``FOO`` becomes
 Like ":append" and ":prepend", ":remove" is applied at variable
 expansion time.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.
+   However, one can undo a ":remove" by using an intermediate variable whose
+   content is passed to the ":remove" so that modifying the intermediate
+   variable equals to keeping the string in::
+
+     FOOREMOVE = "123 456 789"
+     FOO:remove = "${FOOREMOVE}"
+     ...
+     FOOREMOVE = "123 789"
+
+   This expands to ``FOO:remove = "123 789"``.
+
+.. note::
+
+   Override application order may not match variable parse history, i.e.
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+
 Override Style Operation Advantages
 -----------------------------------