diff mbox series

[v2] reproducible: allow each layer to define its known non-repro packages

Message ID 20250609152757.1153027-1-yoann.congal@smile.fr
State New
Headers show
Series [v2] reproducible: allow each layer to define its known non-repro packages | expand

Commit Message

Yoann Congal June 9, 2025, 3:27 p.m. UTC 
From: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>

With this change, layers can use a conf/include/known_non_repro.inc file
to extend the OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable with their own known
non-reproducible packages to make them automatically excluded from the
reproducible test.

Example of usage, in conf/include/known_non_repro.inc:
  OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " \
        this-package-is-not-reproducible \
  "

This uses the new "include_all" in the reproducible test and will move
the maintenance burden from AB config[0] to the layers themselves.

Fixes [YOCTO #15699]

[0]: https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json#n343

Co-Developed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 8 ++++++++
 1 file changed, 8 insertions(+)

Comments

Yoann Congal June 9, 2025, 9:21 p.m. UTC | #1 
Le lun. 9 juin 2025 à 17:28, Yoann Congal <yoann.congal@smile.fr> a écrit :

> From: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
>
> With this change, layers can use a conf/include/known_non_repro.inc file
> to extend the OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable with their own
> known
> non-reproducible packages to make them automatically excluded from the
> reproducible test.
>
> Example of usage, in conf/include/known_non_repro.inc:
>   OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " \
>         this-package-is-not-reproducible \
>   "
>
> This uses the new "include_all" in the reproducible test and will move
> the maintenance burden from AB config[0] to the layers themselves.
>
> Fixes [YOCTO #15699]
>
> [0]:
> https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json#n343
>
> Co-Developed-by: Yoann Congal <yoann.congal@smile.fr>
> Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
> Signed-off-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
> ---
>

Oops, forgot the patch changelog:
v1->v2: Changed to OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable and removed
the now useless code.


>  meta/lib/oeqa/selftest/cases/reproducible.py | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py
> b/meta/lib/oeqa/selftest/cases/reproducible.py
> index f06027cb03..94d3db3677 100644
> --- a/meta/lib/oeqa/selftest/cases/reproducible.py
> +++ b/meta/lib/oeqa/selftest/cases/reproducible.py
> @@ -14,6 +14,7 @@ import shutil
>  import stat
>  import os
>  import datetime
> +import oeqa.utils.ftools as ftools
>
>  exclude_packages = [
>         ]
> @@ -158,6 +159,13 @@ class ReproducibleTests(OESelftestTestCase):
>      # reproducible, disable this in your derived test class
>      build_from_sstate = True
>
> +    @classmethod
> +    def setUpClass(cls):
> +        super().setUpClass()
> +        # include layer exclusion files (populate
> OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES)
> +        ftools.append_file(os.path.join(cls.builddir, "conf/local.conf"),
> \
> +             "\ninclude_all conf/include/known_non_repro.inc\n")
> +
>      def setUpLocal(self):
>          super().setUpLocal()
>          needed_vars = [
>
Quentin Schulz June 10, 2025, 10:57 a.m. UTC | #2 
Hi Yoann,

On 6/9/25 5:27 PM, Yoann Congal via lists.openembedded.org wrote:
> From: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
> 
> With this change, layers can use a conf/include/known_non_repro.inc file
> to extend the OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable with their own known
> non-reproducible packages to make them automatically excluded from the
> reproducible test.
> 
> Example of usage, in conf/include/known_non_repro.inc:
>    OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " \
>          this-package-is-not-reproducible \
>    "
> 
> This uses the new "include_all" in the reproducible test and will move
> the maintenance burden from AB config[0] to the layers themselves.
> 

Isn't this something that could already be handled by a simple

OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += "this-package-is-not-reproducible"

in the appropriate layer.conf file by any chance?

Cheers,
Quentin
Yoann Congal June 10, 2025, 12:50 p.m. UTC | #3 
Le mar. 10 juin 2025 à 12:57, Quentin Schulz <quentin.schulz@cherry.de> a
écrit :

> Hi Yoann,
>
> On 6/9/25 5:27 PM, Yoann Congal via lists.openembedded.org wrote:
> > From: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
> >
> > With this change, layers can use a conf/include/known_non_repro.inc file
> > to extend the OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable with their
> own known
> > non-reproducible packages to make them automatically excluded from the
> > reproducible test.
> >
> > Example of usage, in conf/include/known_non_repro.inc:
> >    OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " \
> >          this-package-is-not-reproducible \
> >    "
> >
> > This uses the new "include_all" in the reproducible test and will move
> > the maintenance burden from AB config[0] to the layers themselves.
> >
>
> Isn't this something that could already be handled by a simple
>
> OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += "this-package-is-not-reproducible"
>
> in the appropriate layer.conf file by any chance?
>

Yes, I guess that would work (and without adding new code).
I seen so much bad stuff put in layer.conf files that I always forgot that
you can! -_-

Richard, what do you think? Should we switch from
an include_all conf/include/known_non_repro.inc in each layer
to
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " ..." in layer.conf of each layer?

Regards,
Quentin Schulz June 10, 2025, 12:59 p.m. UTC | #4 
Hi Yoann,

On 6/10/25 2:50 PM, Yoann Congal wrote:
> Le mar. 10 juin 2025 à 12:57, Quentin Schulz <quentin.schulz@cherry.de> a
> écrit :
> 
>> Hi Yoann,
>>
>> On 6/9/25 5:27 PM, Yoann Congal via lists.openembedded.org wrote:
>>> From: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
>>>
>>> With this change, layers can use a conf/include/known_non_repro.inc file
>>> to extend the OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable with their
>> own known
>>> non-reproducible packages to make them automatically excluded from the
>>> reproducible test.
>>>
>>> Example of usage, in conf/include/known_non_repro.inc:
>>>     OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " \
>>>           this-package-is-not-reproducible \
>>>     "
>>>
>>> This uses the new "include_all" in the reproducible test and will move
>>> the maintenance burden from AB config[0] to the layers themselves.
>>>
>>
>> Isn't this something that could already be handled by a simple
>>
>> OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += "this-package-is-not-reproducible"
>>
>> in the appropriate layer.conf file by any chance?
>>
> 
> Yes, I guess that would work (and without adding new code).
> I seen so much bad stuff put in layer.conf files that I always forgot that
> you can! -_-
> 

Yeah, I've seen horrors too, so that's fair if we want to be a bit more 
selective with what can/should make it into the layer.conf :)

I assume we could have some sort (or maybe we already have?) of 
validator in scripts/yocto-check-layer that would only allow some 
variables to be set in layer.conf? in which case, we could either add 
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES to that list or rely on another conf 
file like you do here. This is not for this patch though, just an idea :)

Cheers,
Quentin
Richard Purdie June 10, 2025, 1 p.m. UTC | #5 
On Tue, 2025-06-10 at 14:50 +0200, Yoann Congal wrote:
> 
> 
> Le mar. 10 juin 2025 à 12:57, Quentin Schulz <quentin.schulz@cherry.de> a écrit :
> > Hi Yoann,
> > 
> > On 6/9/25 5:27 PM, Yoann Congal via lists.openembedded.org wrote:
> > > From: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr>
> > > 
> > > With this change, layers can use a conf/include/known_non_repro.inc file
> > > to extend the OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES variable with their own known
> > > non-reproducible packages to make them automatically excluded from the
> > > reproducible test.
> > > 
> > > Example of usage, in conf/include/known_non_repro.inc:
> > >     OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " \
> > >           this-package-is-not-reproducible \
> > >     "
> > > 
> > > This uses the new "include_all" in the reproducible test and will move
> > > the maintenance burden from AB config[0] to the layers themselves.
> > > 
> > 
> > Isn't this something that could already be handled by a simple
> > 
> > OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += "this-package-is-not-reproducible"
> > 
> > in the appropriate layer.conf file by any chance?
> > 
> 
> 
> Yes, I guess that would work (and without adding new code).
> I seen so much bad stuff put in layer.conf files that I always forgot that you can! -_-
> 
> Richard, what do you think? Should we switch from
> an include_all conf/include/known_non_repro.inc in each layer
> to
> OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES += " ..." in layer.conf of each layer?

I think I'd be ok with that...

Cheers,

Richard
diff mbox series

Patch

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index f06027cb03..94d3db3677 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -14,6 +14,7 @@  import shutil
 import stat
 import os
 import datetime
+import oeqa.utils.ftools as ftools
 
 exclude_packages = [
 	]
@@ -158,6 +159,13 @@  class ReproducibleTests(OESelftestTestCase):
     # reproducible, disable this in your derived test class
     build_from_sstate = True
 
+    @classmethod
+    def setUpClass(cls):
+        super().setUpClass()
+        # include layer exclusion files (populate OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES)
+        ftools.append_file(os.path.join(cls.builddir, "conf/local.conf"), \
+             "\ninclude_all conf/include/known_non_repro.inc\n")
+
     def setUpLocal(self):
         super().setUpLocal()
         needed_vars = [