diff mbox series

[RESEND] oe-pkgdata-util: avoid glibc-locale being mapped out by glibc packages

Message ID 20250731073934.916831-1-Qi.Chen@windriver.com
State New
Headers show
Series [RESEND] oe-pkgdata-util: avoid glibc-locale being mapped out by glibc packages | expand

Commit Message

ChenQi July 31, 2025, 7:39 a.m. UTC 
From: Chen Qi <Qi.Chen@windriver.com>

oe-pkgdata-util's glob command will output different results depending
on whether glibc-locale is built or not. This in turn results in do_rootfs
error when multilib is enabled.

The error message is like below:

  No match for argument: lib32-glibc-locale-en-gb
  Error: Unable to find a match: lib32-glibc-locale-en-gb

Here are the steps to reproduce the issue:
1. Enable multilib and install a lib32 application by adding the following
   lines to local.conf:
   require conf/multilib.conf
   MULTILIBS ?= "multilib:lib32"
   DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"
   IMAGE_INSTALL:append = " lib32-sysstat"
2. bitbake lib32-glibc-locale
   Note that people usually don't deliberately build lib32-glibc-locale,
   but they will do world build. And when so do so, the following step fails.
3. bitbake core-image-full-cmdline

This above steps are how to reproduce the issue in oe-core. There are three
steps there. So it gives people the impression that it's not easy to reproduce
the issue. But in fact, with some image, it could be reproduced by just
doing `bitbake lib32-<image>', as long as that <image> brings glibc-locale
into its build dependency but not runtime dependency.

The problem happened because do_rootfs will first form its own repo by finding
dependencies. glibc-locale will be brought in because of IMAGE_LINGUAS. But
lib32-glibc-locale will not be brought in. And in install_complementary,
oe-pkgdata-util will use its glob command to find all complememtary packages
for IMAGE_LINGUAS, such as "*-locale-en-gb". Then we get the above error.

glibc-locale has its own recipe and which to install into an image is
determined by IMAGE_LINGUAS. So let's not map any glibc packages out
to glibc-locale in oe-pkgdata-util.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 scripts/oe-pkgdata-util | 4 ++++
 1 file changed, 4 insertions(+)

Comments

Randy MacLeod Aug. 4, 2025, 11:25 p.m. UTC | #1 
On 2025-07-31 3:39 a.m., Chen Qi via lists.openembedded.org wrote:
> From: Chen Qi<Qi.Chen@windriver.com>
>
> oe-pkgdata-util's glob command will output different results depending
> on whether glibc-locale is built or not. This in turn results in do_rootfs
> error when multilib is enabled.
>
> The error message is like below:
>
>    No match for argument: lib32-glibc-locale-en-gb
>    Error: Unable to find a match: lib32-glibc-locale-en-gb
>
> Here are the steps to reproduce the issue:
> 1. Enable multilib and install a lib32 application by adding the following
>     lines to local.conf:
>     require conf/multilib.conf
>     MULTILIBS ?= "multilib:lib32"
>     DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"
>     IMAGE_INSTALL:append = " lib32-sysstat"
> 2. bitbake lib32-glibc-locale
>     Note that people usually don't deliberately build lib32-glibc-locale,
>     but they will do world build. And when so do so, the following step fails.
> 3. bitbake core-image-full-cmdline
>
> This above steps are how to reproduce the issue in oe-core. There are three
> steps there. So it gives people the impression that it's not easy to reproduce
> the issue. But in fact, with some image, it could be reproduced by just
> doing `bitbake lib32-<image>', as long as that <image> brings glibc-locale
> into its build dependency but not runtime dependency.
>
> The problem happened because do_rootfs will first form its own repo by finding
> dependencies. glibc-locale will be brought in because of IMAGE_LINGUAS. But
> lib32-glibc-locale will not be brought in. And in install_complementary,
> oe-pkgdata-util will use its glob command to find all complememtary packages
> for IMAGE_LINGUAS, such as "*-locale-en-gb". Then we get the above error.
>
> glibc-locale has its own recipe and which to install into an image is
> determined by IMAGE_LINGUAS. So let's not map any glibc packages out
> to glibc-locale in oe-pkgdata-util.
>
> Signed-off-by: Chen Qi<Qi.Chen@windriver.com>
> ---
>   scripts/oe-pkgdata-util | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
> index 44ae40549a..9fb7952c58 100755
> --- a/scripts/oe-pkgdata-util
> +++ b/scripts/oe-pkgdata-util
> @@ -136,6 +136,10 @@ def glob(args):
>                           logger.debug("%s is not a valid package!" % (pkg))
>                           break
>   
> +                # we don't want glibc-locale to be mapped out by any glibc package, filter it out
> +                if "glibc-locale" in mappedpkg:
> +                    mappedpkg = ""
> +

Hi Qi,

During a brief discussion in the patch review meeting,
people didn't like the idea of changing a generic script to deal with a 
specific recipe problem
even if it's glibc and locales.

Can you review/re-explain the discussion from:
https://lore.kernel.org/openembedded-core/PH0PR11MB5611F2E7A0B97861D193A60AED132@PH0PR11MB5611.namprd11.prod.outlook.com/
in a follow-up email  and discuss with co-workers and the list to try to 
come up with an alternative approach.

I can eventually try to help brainstorm ideas once I've dug out from 
under this mountain of email that accumulated while I was on holiday.

../Randy


>                   if mappedpkg:
>                       logger.debug("%s (%s) -> %s" % (pkg, g, mappedpkg))
>                       mappedpkgs.add(mappedpkg)
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#221175):https://lists.openembedded.org/g/openembedded-core/message/221175
> Mute This Topic:https://lists.openembedded.org/mt/114463019/3616765
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
ChenQi Aug. 5, 2025, 1:52 a.m. UTC | #2 
Got it. I’ll revisit this issue and come up with some alternative solution.
Any suggestion is welcome.

Regards,
Qi

From: MacLeod, Randy <Randy.MacLeod@windriver.com>
Sent: Tuesday, August 5, 2025 7:25 AM
To: Chen, Qi <Qi.Chen@windriver.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core][RESEND][PATCH] oe-pkgdata-util: avoid glibc-locale being mapped out by glibc packages

On 2025-07-31 3:39 a.m., Chen Qi via lists.openembedded.org wrote:

From: Chen Qi <Qi.Chen@windriver.com><mailto:Qi.Chen@windriver.com>



oe-pkgdata-util's glob command will output different results depending

on whether glibc-locale is built or not. This in turn results in do_rootfs

error when multilib is enabled.



The error message is like below:



  No match for argument: lib32-glibc-locale-en-gb

  Error: Unable to find a match: lib32-glibc-locale-en-gb



Here are the steps to reproduce the issue:

1. Enable multilib and install a lib32 application by adding the following

   lines to local.conf:

   require conf/multilib.conf

   MULTILIBS ?= "multilib:lib32"

   DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"

   IMAGE_INSTALL:append = " lib32-sysstat"

2. bitbake lib32-glibc-locale

   Note that people usually don't deliberately build lib32-glibc-locale,

   but they will do world build. And when so do so, the following step fails.

3. bitbake core-image-full-cmdline



This above steps are how to reproduce the issue in oe-core. There are three

steps there. So it gives people the impression that it's not easy to reproduce

the issue. But in fact, with some image, it could be reproduced by just

doing `bitbake lib32-<image>', as long as that <image> brings glibc-locale

into its build dependency but not runtime dependency.



The problem happened because do_rootfs will first form its own repo by finding

dependencies. glibc-locale will be brought in because of IMAGE_LINGUAS. But

lib32-glibc-locale will not be brought in. And in install_complementary,

oe-pkgdata-util will use its glob command to find all complememtary packages

for IMAGE_LINGUAS, such as "*-locale-en-gb". Then we get the above error.



glibc-locale has its own recipe and which to install into an image is

determined by IMAGE_LINGUAS. So let's not map any glibc packages out

to glibc-locale in oe-pkgdata-util.



Signed-off-by: Chen Qi <Qi.Chen@windriver.com><mailto:Qi.Chen@windriver.com>

---

 scripts/oe-pkgdata-util | 4 ++++

 1 file changed, 4 insertions(+)



diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util

index 44ae40549a..9fb7952c58 100755

--- a/scripts/oe-pkgdata-util

+++ b/scripts/oe-pkgdata-util

@@ -136,6 +136,10 @@ def glob(args):

                         logger.debug("%s is not a valid package!" % (pkg))

                         break



+                # we don't want glibc-locale to be mapped out by any glibc package, filter it out

+                if "glibc-locale" in mappedpkg:

+                    mappedpkg = ""

+

Hi Qi,

During a brief discussion in the patch review meeting,
people didn't like the idea of changing a generic script to deal with a specific recipe problem
even if it's glibc and locales.

Can you review/re-explain the discussion from:
   https://lore.kernel.org/openembedded-core/PH0PR11MB5611F2E7A0B97861D193A60AED132@PH0PR11MB5611.namprd11.prod.outlook.com/
in a follow-up email  and discuss with co-workers and the list to try to come up with an alternative approach.

I can eventually try to help brainstorm ideas once I've dug out from under this mountain of email that accumulated while I was on holiday.

../Randy





                 if mappedpkg:

                     logger.debug("%s (%s) -> %s" % (pkg, g, mappedpkg))

                     mappedpkgs.add(mappedpkg)



-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.

View/Reply Online (#221175): https://lists.openembedded.org/g/openembedded-core/message/221175

Mute This Topic: https://lists.openembedded.org/mt/114463019/3616765

Group Owner: openembedded-core+owner@lists.openembedded.org<mailto:openembedded-core+owner@lists.openembedded.org>

Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com<mailto:randy.macleod@windriver.com>]

-=-=-=-=-=-=-=-=-=-=-=-





--

# Randy MacLeod

# Wind River Linux
diff mbox series

Patch

diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 44ae40549a..9fb7952c58 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -136,6 +136,10 @@  def glob(args):
                         logger.debug("%s is not a valid package!" % (pkg))
                         break
 
+                # we don't want glibc-locale to be mapped out by any glibc package, filter it out
+                if "glibc-locale" in mappedpkg:
+                    mappedpkg = ""
+
                 if mappedpkg:
                     logger.debug("%s (%s) -> %s" % (pkg, g, mappedpkg))
                     mappedpkgs.add(mappedpkg)