diff mbox series

license_image.bbclass: report all packages with incompatible license

Message ID 20260303234247.8034-1-martin.jansa@gmail.com
State Under Review
Headers show
Series license_image.bbclass: report all packages with incompatible license | expand

Commit Message

Martin Jansa March 3, 2026, 11:42 p.m. UTC 
From: Martin Jansa <martin.jansa@gmail.com>

When multiple packages cannot be installed it shows only first one it
finds, because of bb.fatal use. It might require many iterations to find
all packages to avoid.

e.g. with ptest enabled and GPL-3.0-or-later, GPL-3.0-only set as
incompatible licenses you might get list like this for relatively small
image:

Some packages cannot be installed into the image because they have incompatible licenses: bzip2-ptest (GPL-3.0-or-later), coreutils (GPL-3.0-or-later), coreutils-stdbuf (GPL-3.0-or-later), diffutils (GPL-3.0-or-later), findutils (GPL-3.0-or-later), gawk (GPL-3.0-or-later), gnutls-openssl (GPL-3.0-or-later), gnutls-ptest (GPL-3.0-or-later), grep (GPL-3.0-only), make (GPL-3.0-only), mpfr (LGPL-3.0-or-later), python3-dbusmock (GPL-3.0-only), readline (GPL-3.0-or-later), sed (GPL-3.0-or-later)

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
---
 meta/classes-recipe/license_image.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Comments

Peter Kjellerstedt March 4, 2026, 10:26 p.m. UTC | #1 
> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Martin Jansa via lists.openembedded.org
> Sent: den 4 mars 2026 00:43
> To: openembedded-core@lists.openembedded.org
> Cc: Martin Jansa <martin.jansa@gmail.com>
> Subject: [OE-core] [PATCH] license_image.bbclass: report all packages with incompatible license
> 
> From: Martin Jansa <martin.jansa@gmail.com>
> 
> When multiple packages cannot be installed it shows only first one it
> finds, because of bb.fatal use. It might require many iterations to find
> all packages to avoid.
> 
> e.g. with ptest enabled and GPL-3.0-or-later, GPL-3.0-only set as
> incompatible licenses you might get list like this for relatively small
> image:
> 
> Some packages cannot be installed into the image because they have incompatible licenses: bzip2-ptest (GPL-3.0-or-later), coreutils (GPL-3.0-or-later), coreutils-stdbuf (GPL-3.0-or-later), diffutils (GPL-3.0-or-later), findutils (GPL-3.0-or-later), gawk (GPL-3.0-or-later), gnutls-openssl (GPL-3.0-or-later), gnutls-ptest (GPL-3.0-or-later), grep (GPL-3.0-only), make (GPL-3.0-only), mpfr (LGPL-3.0-or-later), python3-dbusmock (GPL-3.0-only), readline (GPL-3.0-or-later), sed (GPL-3.0-or-later)
> 
> Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> ---
>  meta/classes-recipe/license_image.bbclass | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/license_image.bbclass b/meta/classes-recipe/license_image.bbclass
> index 8332905da5..252754727b 100644
> --- a/meta/classes-recipe/license_image.bbclass
> +++ b/meta/classes-recipe/license_image.bbclass
> @@ -116,6 +116,7 @@ def write_license_files(d, license_manifest, pkg_dic,
> rootfs):
>      bad_licenses = oe.license.expand_wildcard_licenses(d, bad_licenses)
>      pkgarchs = d.getVar("SSTATE_ARCHS").split()
>      pkgarchs.reverse()
> +    incompatible_packages = []
> 
>      exceptions = (d.getVar("INCOMPATIBLE_LICENSE_EXCEPTIONS") or "").split()
>      with open(license_manifest, "w") as license_file:
> @@ -123,7 +124,7 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs):
>              remaining_bad_licenses = oe.license.apply_pkg_license_exception(pkg, bad_licenses, exceptions)
>              incompatible_licenses = oe.license.incompatible_pkg_license(d, remaining_bad_licenses, pkg_dic[pkg]["LICENSE"])
>              if incompatible_licenses:
> -                bb.fatal("Package %s cannot be installed into the image because it has incompatible license(s): %s" %(pkg, ' '.join(incompatible_licenses)))
> +                incompatible_packages.append("%s (%s)" % (pkg, ' '.join(incompatible_licenses)))
>              else:
>                  incompatible_licenses = oe.license.incompatible_pkg_license(d, bad_licenses, pkg_dic[pkg]["LICENSE"])
>                  if incompatible_licenses:
> @@ -171,6 +172,9 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs):
>                                         "The license listed %s was not in the "\
>                                         "licenses collected for recipe %s"
>                                         % (lic, pkg_dic[pkg]["PN"]), d)
> +    if incompatible_packages:
> +        bb.fatal("Some packages cannot be installed into the image because they have incompatible licenses: %s" % (', '.join(incompatible_packages)))

May I suggest to use "\n\t" as separator instead. Especially if you expect 
a lot of packages to be shown, having them all on one line becomes messy 
very quickly. E.g.:

        bb.fatal("Some packages cannot be installed into the image because they have incompatible licenses:\n\t%s" % ("\n\t".join(incompatible_packages)))

> +
>      oe.qa.exit_if_errors(d)
> 
>      # Two options here:

//Peter
Martin Jansa March 4, 2026, 10:39 p.m. UTC | #2 
I don't mind changing it like that, I've kept it in one line, because
then it's easier to see them all when grepping for ERROR:, will send
v2 tomorrow.

On Wed, Mar 4, 2026 at 11:26 PM Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
>
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Martin Jansa via lists.openembedded.org
> > Sent: den 4 mars 2026 00:43
> > To: openembedded-core@lists.openembedded.org
> > Cc: Martin Jansa <martin.jansa@gmail.com>
> > Subject: [OE-core] [PATCH] license_image.bbclass: report all packages with incompatible license
> >
> > From: Martin Jansa <martin.jansa@gmail.com>
> >
> > When multiple packages cannot be installed it shows only first one it
> > finds, because of bb.fatal use. It might require many iterations to find
> > all packages to avoid.
> >
> > e.g. with ptest enabled and GPL-3.0-or-later, GPL-3.0-only set as
> > incompatible licenses you might get list like this for relatively small
> > image:
> >
> > Some packages cannot be installed into the image because they have incompatible licenses: bzip2-ptest (GPL-3.0-or-later), coreutils (GPL-3.0-or-later), coreutils-stdbuf (GPL-3.0-or-later), diffutils (GPL-3.0-or-later), findutils (GPL-3.0-or-later), gawk (GPL-3.0-or-later), gnutls-openssl (GPL-3.0-or-later), gnutls-ptest (GPL-3.0-or-later), grep (GPL-3.0-only), make (GPL-3.0-only), mpfr (LGPL-3.0-or-later), python3-dbusmock (GPL-3.0-only), readline (GPL-3.0-or-later), sed (GPL-3.0-or-later)
> >
> > Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> > ---
> >  meta/classes-recipe/license_image.bbclass | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes-recipe/license_image.bbclass b/meta/classes-recipe/license_image.bbclass
> > index 8332905da5..252754727b 100644
> > --- a/meta/classes-recipe/license_image.bbclass
> > +++ b/meta/classes-recipe/license_image.bbclass
> > @@ -116,6 +116,7 @@ def write_license_files(d, license_manifest, pkg_dic,
> > rootfs):
> >      bad_licenses = oe.license.expand_wildcard_licenses(d, bad_licenses)
> >      pkgarchs = d.getVar("SSTATE_ARCHS").split()
> >      pkgarchs.reverse()
> > +    incompatible_packages = []
> >
> >      exceptions = (d.getVar("INCOMPATIBLE_LICENSE_EXCEPTIONS") or "").split()
> >      with open(license_manifest, "w") as license_file:
> > @@ -123,7 +124,7 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs):
> >              remaining_bad_licenses = oe.license.apply_pkg_license_exception(pkg, bad_licenses, exceptions)
> >              incompatible_licenses = oe.license.incompatible_pkg_license(d, remaining_bad_licenses, pkg_dic[pkg]["LICENSE"])
> >              if incompatible_licenses:
> > -                bb.fatal("Package %s cannot be installed into the image because it has incompatible license(s): %s" %(pkg, ' '.join(incompatible_licenses)))
> > +                incompatible_packages.append("%s (%s)" % (pkg, ' '.join(incompatible_licenses)))
> >              else:
> >                  incompatible_licenses = oe.license.incompatible_pkg_license(d, bad_licenses, pkg_dic[pkg]["LICENSE"])
> >                  if incompatible_licenses:
> > @@ -171,6 +172,9 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs):
> >                                         "The license listed %s was not in the "\
> >                                         "licenses collected for recipe %s"
> >                                         % (lic, pkg_dic[pkg]["PN"]), d)
> > +    if incompatible_packages:
> > +        bb.fatal("Some packages cannot be installed into the image because they have incompatible licenses: %s" % (', '.join(incompatible_packages)))
>
> May I suggest to use "\n\t" as separator instead. Especially if you expect
> a lot of packages to be shown, having them all on one line becomes messy
> very quickly. E.g.:
>
>         bb.fatal("Some packages cannot be installed into the image because they have incompatible licenses:\n\t%s" % ("\n\t".join(incompatible_packages)))
>
> > +
> >      oe.qa.exit_if_errors(d)
> >
> >      # Two options here:
>
> //Peter
>
Mathieu Dubois-Briand March 5, 2026, 11:15 a.m. UTC | #3 
On Wed Mar 4, 2026 at 12:42 AM CET, Martin Jansa via lists.openembedded.org wrote:
> From: Martin Jansa <martin.jansa@gmail.com>
>
> When multiple packages cannot be installed it shows only first one it
> finds, because of bb.fatal use. It might require many iterations to find
> all packages to avoid.
>
> e.g. with ptest enabled and GPL-3.0-or-later, GPL-3.0-only set as
> incompatible licenses you might get list like this for relatively small
> image:
>
> Some packages cannot be installed into the image because they have incompatible licenses: bzip2-ptest (GPL-3.0-or-later), coreutils (GPL-3.0-or-later), coreutils-stdbuf (GPL-3.0-or-later), diffutils (GPL-3.0-or-later), findutils (GPL-3.0-or-later), gawk (GPL-3.0-or-later), gnutls-openssl (GPL-3.0-or-later), gnutls-ptest (GPL-3.0-or-later), grep (GPL-3.0-only), make (GPL-3.0-only), mpfr (LGPL-3.0-or-later), python3-dbusmock (GPL-3.0-only), readline (GPL-3.0-or-later), sed (GPL-3.0-or-later)
>
> Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> ---

Hi Martin,

Thanks for your patch.

I believe this is making some selftests fail:

2026-03-05 09:01:35,428 - oe-selftest - INFO - incompatible_lic.IncompatibleLicensePerImageTests.test_bash_and_license (subunit.RemotedTestCase)
2026-03-05 09:01:35,429 - oe-selftest - INFO -  ... FAIL
...
ERROR: core-image-minimal-1.0-r0 do_rootfs: QA Issue: The license listed SomeLicense was not in the licenses collected for recipe bash [license-file-missing]
ERROR: core-image-minimal-1.0-r0 do_rootfs: Some packages cannot be installed into the image because they have incompatible licenses: bash (GPL-3.0-or-later)
...
2026-03-05 09:02:33,821 - oe-selftest - INFO - incompatible_lic.IncompatibleLicensePerImageTests.test_bash_default (subunit.RemotedTestCase)
2026-03-05 09:02:33,821 - oe-selftest - INFO -  ... FAIL

https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3225
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3336

Can you have a look at these?

Thanks,
Mathieu
Martin Jansa March 5, 2026, 2:26 p.m. UTC | #4 
On Thu, Mar 5, 2026 at 12:15 PM Mathieu Dubois-Briand
<mathieu.dubois-briand@bootlin.com> wrote:
>
> On Wed Mar 4, 2026 at 12:42 AM CET, Martin Jansa via lists.openembedded.org wrote:
> > From: Martin Jansa <martin.jansa@gmail.com>
> >
> > When multiple packages cannot be installed it shows only first one it
> > finds, because of bb.fatal use. It might require many iterations to find
> > all packages to avoid.
> >
> > e.g. with ptest enabled and GPL-3.0-or-later, GPL-3.0-only set as
> > incompatible licenses you might get list like this for relatively small
> > image:
> >
> > Some packages cannot be installed into the image because they have incompatible licenses: bzip2-ptest (GPL-3.0-or-later), coreutils (GPL-3.0-or-later), coreutils-stdbuf (GPL-3.0-or-later), diffutils (GPL-3.0-or-later), findutils (GPL-3.0-or-later), gawk (GPL-3.0-or-later), gnutls-openssl (GPL-3.0-or-later), gnutls-ptest (GPL-3.0-or-later), grep (GPL-3.0-only), make (GPL-3.0-only), mpfr (LGPL-3.0-or-later), python3-dbusmock (GPL-3.0-only), readline (GPL-3.0-or-later), sed (GPL-3.0-or-later)
> >
> > Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
> > ---
>
> Hi Martin,
>
> Thanks for your patch.
>
> I believe this is making some selftests fail:
>
> 2026-03-05 09:01:35,428 - oe-selftest - INFO - incompatible_lic.IncompatibleLicensePerImageTests.test_bash_and_license (subunit.RemotedTestCase)
> 2026-03-05 09:01:35,429 - oe-selftest - INFO -  ... FAIL
> ...
> ERROR: core-image-minimal-1.0-r0 do_rootfs: QA Issue: The license listed SomeLicense was not in the licenses collected for recipe bash [license-file-missing]
> ERROR: core-image-minimal-1.0-r0 do_rootfs: Some packages cannot be installed into the image because they have incompatible licenses: bash (GPL-3.0-or-later)
> ...
> 2026-03-05 09:02:33,821 - oe-selftest - INFO - incompatible_lic.IncompatibleLicensePerImageTests.test_bash_default (subunit.RemotedTestCase)
> 2026-03-05 09:02:33,821 - oe-selftest - INFO -  ... FAIL
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3225
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3336
>
> Can you have a look at these?

Sure, will have a look. Needs the update for expected error_msg in
meta/lib/oeqa/selftest/cases/incompatible_lic.py. Will send v3 soon
(running selftest to confirm it works), please ignore v2 I've sent
earlier which just updates the formatting based on Peter's suggestion.

Regards,
diff mbox series

Patch

diff --git a/meta/classes-recipe/license_image.bbclass b/meta/classes-recipe/license_image.bbclass
index 8332905da5..252754727b 100644
--- a/meta/classes-recipe/license_image.bbclass
+++ b/meta/classes-recipe/license_image.bbclass
@@ -116,6 +116,7 @@  def write_license_files(d, license_manifest, pkg_dic, rootfs):
     bad_licenses = oe.license.expand_wildcard_licenses(d, bad_licenses)
     pkgarchs = d.getVar("SSTATE_ARCHS").split()
     pkgarchs.reverse()
+    incompatible_packages = []
 
     exceptions = (d.getVar("INCOMPATIBLE_LICENSE_EXCEPTIONS") or "").split()
     with open(license_manifest, "w") as license_file:
@@ -123,7 +124,7 @@  def write_license_files(d, license_manifest, pkg_dic, rootfs):
             remaining_bad_licenses = oe.license.apply_pkg_license_exception(pkg, bad_licenses, exceptions)
             incompatible_licenses = oe.license.incompatible_pkg_license(d, remaining_bad_licenses, pkg_dic[pkg]["LICENSE"])
             if incompatible_licenses:
-                bb.fatal("Package %s cannot be installed into the image because it has incompatible license(s): %s" %(pkg, ' '.join(incompatible_licenses)))
+                incompatible_packages.append("%s (%s)" % (pkg, ' '.join(incompatible_licenses)))
             else:
                 incompatible_licenses = oe.license.incompatible_pkg_license(d, bad_licenses, pkg_dic[pkg]["LICENSE"])
                 if incompatible_licenses:
@@ -171,6 +172,9 @@  def write_license_files(d, license_manifest, pkg_dic, rootfs):
                                        "The license listed %s was not in the "\
                                        "licenses collected for recipe %s"
                                        % (lic, pkg_dic[pkg]["PN"]), d)
+    if incompatible_packages:
+        bb.fatal("Some packages cannot be installed into the image because they have incompatible licenses: %s" % (', '.join(incompatible_packages)))
+
     oe.qa.exit_if_errors(d)
 
     # Two options here: