diff mbox series

[bitbake-devel,4/4] data_smart.py: clear expand_cache in _setvar_update_overridevars

Message ID 20250225064853.2406575-4-Qi.Chen@windriver.com
State New
Headers show
Series [bitbake-devel,1/4] data_smart.py: remove unnecessary ? from __expand_var_regexp__ | expand

Commit Message

ChenQi Feb. 25, 2025, 6:48 a.m. UTC 
From: Chen Qi <Qi.Chen@windriver.com>

At the end of this function, self.overrides is cleared, which means we'll
construct a new self.overrides after this call. And a new self.overrides
will give out different expand_cache, so the old expand_cache should also
be cleared to avoid any wrong value.

Currently, there's a problem revealed by recent recipe specific virtual
provider patch. If we enable multilib and set "OVERRIDES:prepend" in local.conf,
things don't work.

Here's the error message:

  ERROR: Nothing PROVIDES 'lib32-gcc-cross-x86_64'

Below are reproduce steps:
1. Add in local.conf the following lines:
   MACHINE ?= "qemux86-64"
   require conf/multilib.conf
   MULTILIBS ?= "multilib:lib32"
   DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"
   OVERRIDES:prepend = "some-override:"
   (Note that using :append and :remove also reproduces the issue.)
2. bitbake -n lib32-sysstat
   (bitbake -n core-image-minimal also reproduces the issue)

The expandWithRefs calls getVar, which fills expand_cache. So when setting
OVERRIDES:prepend, this will fill the expand_cache. When overridevars are updated,
if we don't clear expand_cache, we'll retrieve wrong values.

Previously, things happened to work because there's a call to expand
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc, which in turn expands
TARGET_VENDOR. Now what we expand is PREFERRED_PROVIDER_ virtual/cross-cc,
so the problem is revealed.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 bitbake/lib/bb/data_smart.py | 1 +
 1 file changed, 1 insertion(+)

Comments

ChenQi March 5, 2025, 1:57 a.m. UTC | #1 
kindly ping

On 2/25/25 14:48, Chen Qi via lists.openembedded.org wrote:
> From: Chen Qi <Qi.Chen@windriver.com>
>
> At the end of this function, self.overrides is cleared, which means we'll
> construct a new self.overrides after this call. And a new self.overrides
> will give out different expand_cache, so the old expand_cache should also
> be cleared to avoid any wrong value.
>
> Currently, there's a problem revealed by recent recipe specific virtual
> provider patch. If we enable multilib and set "OVERRIDES:prepend" in local.conf,
> things don't work.
>
> Here's the error message:
>
>    ERROR: Nothing PROVIDES 'lib32-gcc-cross-x86_64'
>
> Below are reproduce steps:
> 1. Add in local.conf the following lines:
>     MACHINE ?= "qemux86-64"
>     require conf/multilib.conf
>     MULTILIBS ?= "multilib:lib32"
>     DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"
>     OVERRIDES:prepend = "some-override:"
>     (Note that using :append and :remove also reproduces the issue.)
> 2. bitbake -n lib32-sysstat
>     (bitbake -n core-image-minimal also reproduces the issue)
>
> The expandWithRefs calls getVar, which fills expand_cache. So when setting
> OVERRIDES:prepend, this will fill the expand_cache. When overridevars are updated,
> if we don't clear expand_cache, we'll retrieve wrong values.
>
> Previously, things happened to work because there's a call to expand
> PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc, which in turn expands
> TARGET_VENDOR. Now what we expand is PREFERRED_PROVIDER_ virtual/cross-cc,
> so the problem is revealed.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>   bitbake/lib/bb/data_smart.py | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
> index caef93ff4f..1c867653f3 100644
> --- a/bitbake/lib/bb/data_smart.py
> +++ b/bitbake/lib/bb/data_smart.py
> @@ -635,6 +635,7 @@ class DataSmart(MutableMapping):
>                   nextnew.update(vardata.contains.keys())
>               new = nextnew
>           self.overrides = None
> +        self.expand_cache = {}
>   
>       def _setvar_update_overrides(self, var, **loginfo):
>           # aka pay the cookie monster
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#17301): https://lists.openembedded.org/g/bitbake-devel/message/17301
> Mute This Topic: https://lists.openembedded.org/mt/111373619/7304865
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Qi.Chen@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Richard Purdie March 5, 2025, 5:57 p.m. UTC | #2 
On Wed, 2025-03-05 at 09:57 +0800, Chen Qi via lists.openembedded.org wrote:
> kindly ping

Sorry, very briefly, 4/4 looks right, I merged 2/4, 3/4 removes a
cookie monster reference which needs to stay somehow. I need to double
check the rest of 3/4 but it probably is right after we switched
override characters. 1/4 is probably fine if all the tests pass.

Cheers,

Richard
diff mbox series

Patch

diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index caef93ff4f..1c867653f3 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -635,6 +635,7 @@  class DataSmart(MutableMapping):
                 nextnew.update(vardata.contains.keys())
             new = nextnew
         self.overrides = None
+        self.expand_cache = {}
 
     def _setvar_update_overrides(self, var, **loginfo):
         # aka pay the cookie monster