| Message ID | 20250206131811.86935-3-yoann.congal@smile.fr |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [1/3] ref-manual/variables: document OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS | expand |
Hi Yoann, On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: > From: Yoann Congal <yoann.congal@smile.fr> > > Shows an example of the new variable OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS > and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that were > shown earlier. > > Signed-off-by: Yoann Congal <yoann.congal@smile.fr> > Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> > --- > .../test-manual/reproducible-builds.rst | 18 ++++++++++++++++++ > 1 file changed, 18 insertions(+) > > diff --git a/documentation/test-manual/reproducible-builds.rst b/documentation/test-manual/reproducible-builds.rst > index b9f671d68..909ac5216 100644 > --- a/documentation/test-manual/reproducible-builds.rst > +++ b/documentation/test-manual/reproducible-builds.rst > @@ -145,3 +145,21 @@ set of recipes before the test, meaning they are excluded from reproducibility > testing. As a practical example, you could set ``sstate_targets`` to > ``core-image-sato``, then setting ``targets`` to ``core-image-sato-sdk`` would > run reproducibility tests only on the targets belonging only to ``core-image-sato-sdk``. > + > +Using OEQA_REPRODUCIBLE_TEST_* variables I think it'd make sense here to tick-quote OEQA_REPRODUCIBLE_TEST_*? > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +If you want to test the reproducibility of a set of recipe, you can define s/recipe/recipes/ > +:term:`OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS`, in your local.conf:: > + > + OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS = "my-recipe" > + We align by three whitespaces, and I think it's four here if my eyes don't fail me? > +This will test the reproducibility of ``my-recipe`` but will use the > +:ref:`Shared State <overview-manual/concepts:Shared State>` for most its > +dependencies. > + > +You can have finer control on the test with: > + > +- :term:`OEQA_REPRODUCIBLE_TEST_TARGET`: lists recipes to be built, Reading this I'm wondering if this includes the recipes that will be built using sstate-cache as well, i.e. all recipes that will be built? Cheers, Quentin
Le 06/02/2025 à 17:08, Quentin Schulz a écrit : > Hi Yoann, > > On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: >> From: Yoann Congal <yoann.congal@smile.fr> >> >> Shows an example of the new variable OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS >> and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that were >> shown earlier. >> >> Signed-off-by: Yoann Congal <yoann.congal@smile.fr> >> Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> >> --- >> .../test-manual/reproducible-builds.rst | 18 ++++++++++++++++++ >> 1 file changed, 18 insertions(+) >> >> diff --git a/documentation/test-manual/reproducible-builds.rst b/ >> documentation/test-manual/reproducible-builds.rst >> index b9f671d68..909ac5216 100644 >> --- a/documentation/test-manual/reproducible-builds.rst >> +++ b/documentation/test-manual/reproducible-builds.rst >> @@ -145,3 +145,21 @@ set of recipes before the test, meaning they are >> excluded from reproducibility >> testing. As a practical example, you could set ``sstate_targets`` to >> ``core-image-sato``, then setting ``targets`` to ``core-image-sato- >> sdk`` would >> run reproducibility tests only on the targets belonging only to >> ``core-image-sato-sdk``. >> + >> +Using OEQA_REPRODUCIBLE_TEST_* variables > > I think it'd make sense here to tick-quote OEQA_REPRODUCIBLE_TEST_*? I agree, I'll modify for v2. FYI, on my build, it renders as italics. >> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> + >> +If you want to test the reproducibility of a set of recipe, you can >> define > > s/recipe/recipes/ Right, I'll modify for v2. > >> +:term:`OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS`, in your local.conf:: >> + >> + OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS = "my-recipe" >> + > > We align by three whitespaces, and I think it's four here if my eyes > don't fail me? Rhaa, nice catch, I did count these at some point. I'll modify for v2. >> +This will test the reproducibility of ``my-recipe`` but will use the >> +:ref:`Shared State <overview-manual/concepts:Shared State>` for most its >> +dependencies. >> + >> +You can have finer control on the test with: >> + >> +- :term:`OEQA_REPRODUCIBLE_TEST_TARGET`: lists recipes to be built, > > Reading this I'm wondering if this includes the recipes that will be > built using sstate-cache as well, i.e. all recipes that will be built? OEQA_REPRODUCIBLE_TEST_TARGET is the list of all recipes to build twice (once with sstate globally enabled, once with sstate globally disabled) Before the second build, the OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS set of recipes will be built using sstate so that the second build (without sstate) does not have to build them from scratch. We usually want OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to be a subset of the dependencies of OEQA_REPRODUCIBLE_TEST_TARGET but it not enforced by the test. Thanks for your attention to detail (seriously)! > Cheers, > Quentin
Hi Yoann, On 2/6/25 7:17 PM, Yoann Congal wrote: > > > Le 06/02/2025 à 17:08, Quentin Schulz a écrit : >> Hi Yoann, >> >> On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: >>> From: Yoann Congal <yoann.congal@smile.fr> >>> >>> Shows an example of the new variable OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS >>> and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that were >>> shown earlier. >>> >>> Signed-off-by: Yoann Congal <yoann.congal@smile.fr> >>> Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> Since this is your v1, I believe you're not allowed to provide Reviewed-by/Acked-by/Tested-by/etc... given privately. Guillaume should give their Reviewed-by on the mailing list before you can add it. [...] >>> +This will test the reproducibility of ``my-recipe`` but will use the >>> +:ref:`Shared State <overview-manual/concepts:Shared State>` for most its >>> +dependencies. >>> + >>> +You can have finer control on the test with: >>> + >>> +- :term:`OEQA_REPRODUCIBLE_TEST_TARGET`: lists recipes to be built, >> >> Reading this I'm wondering if this includes the recipes that will be >> built using sstate-cache as well, i.e. all recipes that will be built? > > OEQA_REPRODUCIBLE_TEST_TARGET is the list of all recipes to build twice > (once with sstate globally enabled, once with sstate globally disabled) > > Before the second build, the OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS set > of recipes will be built using sstate so that the second build (without > sstate) does not have to build them from scratch. > This is actually documented at the end of https://docs.yoctoproject.org/test-manual/reproducible-builds.html#can-we-prove-the-project-is-reproducible. I feel like a reminder wouldn't be too bad of an idea in this section even though it follows the one it's explained in, but not a requirement per-se, up to you. > We usually want OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to be a subset of > the dependencies of OEQA_REPRODUCIBLE_TEST_TARGET but it not enforced by > the test. > If my understand is correct, OEQA_REPRODUCIBLE_TEST_TARGET could be a recipe A and OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS could be dependencies of that recipe A, so it's not necessarily a subset? But they should relate otherwise they are noop? Thanks for the explanation! Cheers, Quentin
Le 07/02/2025 à 10:12, Quentin Schulz a écrit : > Hi Yoann, Hi, > On 2/6/25 7:17 PM, Yoann Congal wrote: >> Le 06/02/2025 à 17:08, Quentin Schulz a écrit : >>> Hi Yoann, >>> >>> On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: >>>> From: Yoann Congal <yoann.congal@smile.fr> >>>> >>>> Shows an example of the new variable >>>> OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS >>>> and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that >>>> were >>>> shown earlier. >>>> >>>> Signed-off-by: Yoann Congal <yoann.congal@smile.fr> >>>> Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> > > Since this is your v1, I believe you're not allowed to provide Reviewed- > by/Acked-by/Tested-by/etc... given privately. Guillaume should give > their Reviewed-by on the mailing list before you can add it. I did not know this was a thing. I did private reviews (with my Reviewed-By) for my coworkers for more than 2 years now and nobody told me. I'll remove it for v3. > [...] >>>> +This will test the reproducibility of ``my-recipe`` but will use the >>>> +:ref:`Shared State <overview-manual/concepts:Shared State>` for >>>> most its >>>> +dependencies. >>>> + >>>> +You can have finer control on the test with: >>>> + >>>> +- :term:`OEQA_REPRODUCIBLE_TEST_TARGET`: lists recipes to be built, >>> >>> Reading this I'm wondering if this includes the recipes that will be >>> built using sstate-cache as well, i.e. all recipes that will be built? >> >> OEQA_REPRODUCIBLE_TEST_TARGET is the list of all recipes to build twice >> (once with sstate globally enabled, once with sstate globally disabled) >> >> Before the second build, the OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS set >> of recipes will be built using sstate so that the second build (without >> sstate) does not have to build them from scratch. >> > > This is actually documented at the end of https://docs.yoctoproject.org/ > test-manual/reproducible-builds.html#can-we-prove-the-project-is- > reproducible. I feel like a reminder wouldn't be too bad of an idea in > this section even though it follows the one it's explained in, but not a > requirement per-se, up to you. My current ticket (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15701#c3) include to rework how the test is implemented. I will sent patch to the doc to explain the new implementation when merged. >> We usually want OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to be a subset of >> the dependencies of OEQA_REPRODUCIBLE_TEST_TARGET but it not enforced by >> the test. >> > > If my understand is correct, OEQA_REPRODUCIBLE_TEST_TARGET could be a > recipe A and OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS could be dependencies > of that recipe A, so it's not necessarily a subset? But they should > relate otherwise they are noop? If OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS is not a subset of the whole dependency tree of OEQA_REPRODUCIBLE_TEST_TARGET, they still will be built using sstate during the prebuild phase of reproducibleB but that won't decrease build time of reproducibleB, thus globally increasing build time (ie worse than noop). > Thanks for the explanation! > > Cheers, > Quentin
Hi Yoann, On Sun Feb 9, 2025 at 11:46 PM CET, Yoann Congal via lists.yoctoproject.org wrote: > Le 07/02/2025 à 10:12, Quentin Schulz a écrit : >> Hi Yoann, > > Hi, > >> On 2/6/25 7:17 PM, Yoann Congal wrote: >>> Le 06/02/2025 à 17:08, Quentin Schulz a écrit : >>>> Hi Yoann, >>>> >>>> On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: >>>>> From: Yoann Congal <yoann.congal@smile.fr> >>>>> >>>>> Shows an example of the new variable >>>>> OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS >>>>> and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that >>>>> were >>>>> shown earlier. >>>>> >>>>> Signed-off-by: Yoann Congal <yoann.congal@smile.fr> >>>>> Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> >> >> Since this is your v1, I believe you're not allowed to provide Reviewed- >> by/Acked-by/Tested-by/etc... given privately. Guillaume should give >> their Reviewed-by on the mailing list before you can add it. > > I did not know this was a thing. > I did private reviews (with my Reviewed-By) for my coworkers for more > than 2 years now and nobody told me. > > I'll remove it for v3. Also, if you add a cover letter to the series, and if Guillaume sends their Reviewed-by in reply to the cover-letter, it counts for the whole series. At least, that's how b4 behaves on my side when applying the patches. So Guillaume doesn't have to reply to each email. :) Antonin -- Antonin Godard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hi Yoann, On 2/9/25 11:46 PM, Yoann Congal wrote: > Le 07/02/2025 à 10:12, Quentin Schulz a écrit : >> Hi Yoann, > > Hi, > >> On 2/6/25 7:17 PM, Yoann Congal wrote: >>> Le 06/02/2025 à 17:08, Quentin Schulz a écrit : >>>> Hi Yoann, >>>> >>>> On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: >>>>> From: Yoann Congal <yoann.congal@smile.fr> >>>>> >>>>> Shows an example of the new variable >>>>> OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS >>>>> and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that >>>>> were >>>>> shown earlier. >>>>> >>>>> Signed-off-by: Yoann Congal <yoann.congal@smile.fr> >>>>> Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> >> >> Since this is your v1, I believe you're not allowed to provide Reviewed- >> by/Acked-by/Tested-by/etc... given privately. Guillaume should give >> their Reviewed-by on the mailing list before you can add it. > > I did not know this was a thing. > I did private reviews (with my Reviewed-By) for my coworkers for more > than 2 years now and nobody told me. > Coming from colleagues, it's less fishy but I've recently seen patches with Reviewed-by from maintainers in v1 of contributors where the maintainers didn't know anything about the patch. A Reviewed-by is a mark of trust that shouldn't be given lightly, so when it is given, it is helping the maintainer(s) making the decision whether to take the patch or not. Unless it is given publicly (hence "guaranteeing" the person who gives their Reviewed-by is the person they claim they are by the mail address they sent the mail from (though it's not a necessity, I sometimes give Reviewed-by: <personal-address> from my <company-address>)), there's nothing guaranteeing that this person actually reviewed the code. For the docs, the impact of impersonating someone in Reviewed-by is clearly not a big deal :) Cheers, Quentin
Le lun. 10 févr. 2025 à 18:52, Quentin Schulz <quentin.schulz@cherry.de> a écrit : > Hi Yoann, > > On 2/9/25 11:46 PM, Yoann Congal wrote: > > Le 07/02/2025 à 10:12, Quentin Schulz a écrit : > >> Hi Yoann, > > > > Hi, > > > >> On 2/6/25 7:17 PM, Yoann Congal wrote: > >>> Le 06/02/2025 à 17:08, Quentin Schulz a écrit : > >>>> Hi Yoann, > >>>> > >>>> On 2/6/25 2:18 PM, Yoann Congal via lists.yoctoproject.org wrote: > >>>>> From: Yoann Congal <yoann.congal@smile.fr> > >>>>> > >>>>> Shows an example of the new variable > >>>>> OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS > >>>>> and still mention OEQA_REPRODUCIBLE_TEST_TARGET/SSTATE_TARGETS that > >>>>> were > >>>>> shown earlier. > >>>>> > >>>>> Signed-off-by: Yoann Congal <yoann.congal@smile.fr> > >>>>> Reviewed-by: Guillaume Swaenepoel <guillaume.swaenepoel@smile.fr> > >> > >> Since this is your v1, I believe you're not allowed to provide Reviewed- > >> by/Acked-by/Tested-by/etc... given privately. Guillaume should give > >> their Reviewed-by on the mailing list before you can add it. > > > > I did not know this was a thing. > > I did private reviews (with my Reviewed-By) for my coworkers for more > > than 2 years now and nobody told me. > > > > Coming from colleagues, it's less fishy but I've recently seen patches > with Reviewed-by from maintainers in v1 of contributors where the > maintainers didn't know anything about the patch. A Reviewed-by is a > mark of trust that shouldn't be given lightly, so when it is given, it > is helping the maintainer(s) making the decision whether to take the > patch or not. Unless it is given publicly (hence "guaranteeing" the > person who gives their Reviewed-by is the person they claim they are by > the mail address they sent the mail from (though it's not a necessity, I > sometimes give Reviewed-by: <personal-address> from my > <company-address>)), there's nothing guaranteeing that this person > actually reviewed the code. For the docs, the impact of impersonating > someone in Reviewed-by is clearly not a big deal :) > Thanks for the explanation, I'll update the way we do that here. > Cheers, > Quentin >
diff --git a/documentation/test-manual/reproducible-builds.rst b/documentation/test-manual/reproducible-builds.rst index b9f671d68..909ac5216 100644 --- a/documentation/test-manual/reproducible-builds.rst +++ b/documentation/test-manual/reproducible-builds.rst @@ -145,3 +145,21 @@ set of recipes before the test, meaning they are excluded from reproducibility testing. As a practical example, you could set ``sstate_targets`` to ``core-image-sato``, then setting ``targets`` to ``core-image-sato-sdk`` would run reproducibility tests only on the targets belonging only to ``core-image-sato-sdk``. + +Using OEQA_REPRODUCIBLE_TEST_* variables +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you want to test the reproducibility of a set of recipe, you can define +:term:`OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS`, in your local.conf:: + + OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS = "my-recipe" + +This will test the reproducibility of ``my-recipe`` but will use the +:ref:`Shared State <overview-manual/concepts:Shared State>` for most its +dependencies. + +You can have finer control on the test with: + +- :term:`OEQA_REPRODUCIBLE_TEST_TARGET`: lists recipes to be built, +- :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS`: lists recipes that will + be built using :ref:`Shared State <overview-manual/concepts:Shared State>`.