[master,scarthgap,v3] lib/oe/package-manager: skip processing installed-pkgs with empty globs

We can skip processing the installed-pkgs file if globs is empty.
This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY
returns an empty string. If globs is an empty string the result from
processing with empty glob in oe-pkgdata-util will always be 0 packages
to install.

Instead of return early on this we just skip and still generate the
locale archive if needed.

Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
---
 meta/lib/oe/package_manager/__init__.py | 76 ++++++++++++-------------
 1 file changed, 37 insertions(+), 39 deletions(-)

On Mon, 2024-10-07 at 22:39 +0200, Claus Stovgaard wrote:
> We can skip processing the installed-pkgs file if globs is empty.
> This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY
> returns an empty string. If globs is an empty string the result from
> processing with empty glob in oe-pkgdata-util will always be 0
> packages
> to install.
> 
> Instead of return early on this we just skip and still generate the
> locale archive if needed.
> 
> Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
> ---
>  meta/lib/oe/package_manager/__init__.py | 76 ++++++++++++-----------
> --
>  1 file changed, 37 insertions(+), 39 deletions(-)
> 
> diff --git a/meta/lib/oe/package_manager/__init__.py
> b/meta/lib/oe/package_manager/__init__.py
> index d3b2317894..2100a97c12 100644
> --- a/meta/lib/oe/package_manager/__init__.py
> +++ b/meta/lib/oe/package_manager/__init__.py
> @@ -365,45 +365,43 @@ class PackageManager(object,
> metaclass=ABCMeta):
>                  for complementary_linguas in
> (self.d.getVar('IMAGE_LINGUAS_COMPLEMENTARY') or "").split():
>                      globs += (" " + complementary_linguas) % lang
>  
> -        if globs is None:
> -            return
> -
> -        # we need to write the list of installed packages to a file
> because the
> -        # oe-pkgdata-util reads it from a file
> -        with tempfile.NamedTemporaryFile(mode="w+",
> prefix="installed-pkgs") as installed_pkgs:
> -            pkgs = self.list_installed()
> -
> -            provided_pkgs = set()
> -            for pkg in pkgs.values():
> -                provided_pkgs |= set(pkg.get('provs', []))
> -
> -            output = oe.utils.format_pkg_list(pkgs, "arch")
> -            installed_pkgs.write(output)
> -            installed_pkgs.flush()
> -
> -            cmd = ["oe-pkgdata-util",
> -                   "-p", self.d.getVar('PKGDATA_DIR'), "glob",
> installed_pkgs.name,
> -                   globs]
> -            exclude = self.d.getVar('PACKAGE_EXCLUDE_COMPLEMENTARY')
> -            if exclude:
> -                cmd.extend(['--exclude=' +
> '|'.join(exclude.split())])
> -            try:
> -                bb.note('Running %s' % cmd)
> -                proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
> stderr=subprocess.PIPE)
> -                stdout, stderr = proc.communicate()
> -                if stderr: bb.note(stderr.decode("utf-8"))
> -                complementary_pkgs = stdout.decode("utf-8")
> -                complementary_pkgs = set(complementary_pkgs.split())
> -                skip_pkgs = sorted(complementary_pkgs &
> provided_pkgs)
> -                install_pkgs = sorted(complementary_pkgs -
> provided_pkgs)
> -                bb.note("Installing complementary packages ... %s
> (skipped already provided packages %s)" % (
> -                    ' '.join(install_pkgs),
> -                    ' '.join(skip_pkgs)))
> -                self.install(install_pkgs, hard_depends_only=True)
> -            except subprocess.CalledProcessError as e:
> -                bb.fatal("Could not compute complementary packages
> list. Command "
> -                         "'%s' returned %d:\n%s" %
> -                         (' '.join(cmd), e.returncode,
> e.output.decode("utf-8")))
> +        if globs:
> +            # we need to write the list of installed packages to a
> file because the
> +            # oe-pkgdata-util reads it from a file
> +            with tempfile.NamedTemporaryFile(mode="w+",
> prefix="installed-pkgs") as installed_pkgs:
> +                pkgs = self.list_installed()
> +
> +                provided_pkgs = set()
> +                for pkg in pkgs.values():
> +                    provided_pkgs |= set(pkg.get('provs', []))
> +
> +                output = oe.utils.format_pkg_list(pkgs, "arch")
> +                installed_pkgs.write(output)
> +                installed_pkgs.flush()
> +
> +                cmd = ["oe-pkgdata-util",
> +                    "-p", self.d.getVar('PKGDATA_DIR'), "glob",
> installed_pkgs.name,
> +                    globs]
> +                exclude =
> self.d.getVar('PACKAGE_EXCLUDE_COMPLEMENTARY')
> +                if exclude:
> +                    cmd.extend(['--exclude=' +
> '|'.join(exclude.split())])
> +                try:
> +                    bb.note('Running %s' % cmd)
> +                    proc = subprocess.Popen(cmd,
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> +                    stdout, stderr = proc.communicate()
> +                    if stderr: bb.note(stderr.decode("utf-8"))
> +                    complementary_pkgs = stdout.decode("utf-8")
> +                    complementary_pkgs =
> set(complementary_pkgs.split())
> +                    skip_pkgs = sorted(complementary_pkgs &
> provided_pkgs)
> +                    install_pkgs = sorted(complementary_pkgs -
> provided_pkgs)
> +                    bb.note("Installing complementary packages ...
> %s (skipped already provided packages %s)" % (
> +                        ' '.join(install_pkgs),
> +                        ' '.join(skip_pkgs)))
> +                    self.install(install_pkgs,
> hard_depends_only=True)
> +                except subprocess.CalledProcessError as e:
> +                    bb.fatal("Could not compute complementary
> packages list. Command "
> +                            "'%s' returned %d:\n%s" %
> +                            (' '.join(cmd), e.returncode,
> e.output.decode("utf-8")))
>  
>          if self.d.getVar('IMAGE_LOCALES_ARCHIVE') == '1':
>              target_arch = self.d.getVar('TARGET_ARCH')


Hi Ross.

Now you commented on v2 of the patch. Hope you think v3 is okay.

/Claus

On Thu, 2024-10-10 at 23:13 +0200, Claus Stovgaard via
lists.openembedded.org wrote:
> 
> 
> Hi Ross.
> 
> Now you commented on v2 of the patch. Hope you think v3 is okay.
> 
> /Claus

Nevermind. I can see Richard har picked it up in OE-core and poky-
contrib.

/Claus