diff mbox series

reproducible-builds.rst: show how to build a single package

Message ID 20241218200630.823299-1-tgamblin@baylibre.com
State Superseded
Headers show
Series reproducible-builds.rst: show how to build a single package | expand

Commit Message

Trevor Gamblin Dec. 18, 2024, 8:06 p.m. UTC
Add a bit to the "Can we prove the project is reproducible?" section to
show how OEQA_REPRODUCIBLE_TEST_TARGET can be used to test a single
package (rather than the world build), as this may be useful for
maintainers.

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
---
 .../test-manual/reproducible-builds.rst       | 22 +++++++++++++------
 1 file changed, 15 insertions(+), 7 deletions(-)

Comments

Yoann Congal Dec. 19, 2024, 12:27 p.m. UTC | #1
Le mer. 18 déc. 2024 à 21:06, Trevor Gamblin via lists.yoctoproject.org
<tgamblin=baylibre.com@lists.yoctoproject.org> a écrit :

> Add a bit to the "Can we prove the project is reproducible?" section to
> show how OEQA_REPRODUCIBLE_TEST_TARGET can be used to test a single
> package (rather than the world build), as this may be useful for
> maintainers.
>
> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
>

Thank you for doing this.

That is https://bugzilla.yoctoproject.org/show_bug.cgi?id=15619 (you may
want to take it and add a [YOCTO #15619] ref in this patch)

---
>  .../test-manual/reproducible-builds.rst       | 22 +++++++++++++------
>  1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/documentation/test-manual/reproducible-builds.rst
> b/documentation/test-manual/reproducible-builds.rst
> index 91f94a5c7..d581155eb 100644
> --- a/documentation/test-manual/reproducible-builds.rst
> +++ b/documentation/test-manual/reproducible-builds.rst
> @@ -91,13 +91,21 @@ run::
>
>     oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
>
> -This defaults to including a ``world`` build so, if other layers are
> added, it would
> -also run the tests for recipes in the additional layers. Different build
> targets
> -can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable
> in ``local.conf``.
> -The first build will be run using :ref:`Shared State
> <overview-manual/concepts:Shared State>` if
> -available, the second build explicitly disables
> -:ref:`Shared State <overview-manual/concepts:Shared State>` except for
> recipes defined in
> -the :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on
> the
> +This defaults to including a ``world`` build so, if other layers are
> added, it
> +would also run the tests for recipes in the additional layers. Different
> build
> +targets can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET`
> variable
> +in ``local.conf``. For example, running reproducibility tests for only the
> +``python3-numpy`` recipe can be done by setting::
> +
> +   OEQA_REPRODUCIBLE_TEST_TARGET = "python3-numpy"
> +
> +in local.conf before running the ``oe-selftest`` command shown above.
> +
> +Reproducibility builds the target list twice. The first build will be run
> using
> +:ref:`Shared State <overview-manual/concepts:Shared State>` if available,
> the
> +second build explicitly disables :ref:`Shared State
> +<overview-manual/concepts:Shared State>` except for recipes defined in the
> +:term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on the
>

What do you think of adding that a quick way to test the reproducibility of
a recipe is to do
OEQA_REPRODUCIBLE_TEST_TARGET = "recipe"
OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "DEPENDS of recipe"
?
(If not, leave the ticket assigned to me and I'll get to it eventually)


>  specific host the build is running on. This means we can test
> reproducibility
>  builds between different host distributions over time on the Autobuilder.
>
> --
> 2.39.5
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#5983):
> https://lists.yoctoproject.org/g/docs/message/5983
> Mute This Topic: https://lists.yoctoproject.org/mt/110186946/4316185
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [
> yoann.congal@smile.fr]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Trevor Gamblin Dec. 19, 2024, 2:50 p.m. UTC | #2
On 2024-12-19 07:27, Yoann Congal wrote:
> Le mer. 18 déc. 2024 à 21:06, Trevor Gamblin via 
> lists.yoctoproject.org <http://lists.yoctoproject.org> 
> <tgamblin=baylibre.com@lists.yoctoproject.org> a écrit :
>
>     Add a bit to the "Can we prove the project is reproducible?"
>     section to
>     show how OEQA_REPRODUCIBLE_TEST_TARGET can be used to test a single
>     package (rather than the world build), as this may be useful for
>     maintainers.
>
>     Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
>
>
> Thank you for doing this.
>
> That is https://bugzilla.yoctoproject.org/show_bug.cgi?id=15619 (you 
> may want to take it and add a [YOCTO #15619] ref in this patch)
Thanks, I'll do that.
>
>     ---
>      .../test-manual/reproducible-builds.rst       | 22
>     +++++++++++++------
>      1 file changed, 15 insertions(+), 7 deletions(-)
>
>     diff --git a/documentation/test-manual/reproducible-builds.rst
>     b/documentation/test-manual/reproducible-builds.rst
>     index 91f94a5c7..d581155eb 100644
>     --- a/documentation/test-manual/reproducible-builds.rst
>     +++ b/documentation/test-manual/reproducible-builds.rst
>     @@ -91,13 +91,21 @@ run::
>
>         oe-selftest -r
>     reproducible.ReproducibleTests.test_reproducible_builds
>
>     -This defaults to including a ``world`` build so, if other layers
>     are added, it would
>     -also run the tests for recipes in the additional layers.
>     Different build targets
>     -can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET`
>     variable in ``local.conf``.
>     -The first build will be run using :ref:`Shared State
>     <overview-manual/concepts:Shared State>` if
>     -available, the second build explicitly disables
>     -:ref:`Shared State <overview-manual/concepts:Shared State>`
>     except for recipes defined in
>     -the :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and
>     builds on the
>     +This defaults to including a ``world`` build so, if other layers
>     are added, it
>     +would also run the tests for recipes in the additional layers.
>     Different build
>     +targets can be defined using the
>     :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable
>     +in ``local.conf``. For example, running reproducibility tests for
>     only the
>     +``python3-numpy`` recipe can be done by setting::
>     +
>     +   OEQA_REPRODUCIBLE_TEST_TARGET = "python3-numpy"
>     +
>     +in local.conf before running the ``oe-selftest`` command shown above.
>     +
>     +Reproducibility builds the target list twice. The first build
>     will be run using
>     +:ref:`Shared State <overview-manual/concepts:Shared State>` if
>     available, the
>     +second build explicitly disables :ref:`Shared State
>     +<overview-manual/concepts:Shared State>` except for recipes
>     defined in the
>     +:term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and
>     builds on the
>
>
> What do you think of adding that a quick way to test the 
> reproducibility of a recipe is to do
> OEQA_REPRODUCIBLE_TEST_TARGET = "recipe"
> OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "DEPENDS of recipe"
> ?
> (If not, leave the ticket assigned to me and I'll get to it eventually)
>
>      specific host the build is running on. This means we can test
>     reproducibility
>      builds between different host distributions over time on the
>     Autobuilder.
>
>     -- 
>     2.39.5
>
>
>     -=-=-=-=-=-=-=-=-=-=-=-
>     Links: You receive all messages sent to this group.
>     View/Reply Online (#5983):
>     https://lists.yoctoproject.org/g/docs/message/5983
>     Mute This Topic: https://lists.yoctoproject.org/mt/110186946/4316185
>     Group Owner: docs+owner@lists.yoctoproject.org
>     <mailto:docs%2Bowner@lists.yoctoproject.org>
>     Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub
>     [yoann.congal@smile.fr]
>     -=-=-=-=-=-=-=-=-=-=-=-
>
>
>
> -- 
> Yoann Congal
> Smile ECS - Tech expert
Antonin Godard Dec. 20, 2024, 8:19 a.m. UTC | #3
Hi Yoann,

On Thu Dec 19, 2024 at 1:27 PM CET, Yoann Congal via lists.yoctoproject.org wrote:
> Le mer. 18 déc. 2024 à 21:06, Trevor Gamblin via lists.yoctoproject.org
> <tgamblin=baylibre.com@lists.yoctoproject.org> a écrit :
>
>> Add a bit to the "Can we prove the project is reproducible?" section to
>> show how OEQA_REPRODUCIBLE_TEST_TARGET can be used to test a single
>> package (rather than the world build), as this may be useful for
>> maintainers.
>>
>> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
>>
>
> Thank you for doing this.
>
> That is https://bugzilla.yoctoproject.org/show_bug.cgi?id=15619 (you may
> want to take it and add a [YOCTO #15619] ref in this patch)
>
> ---
>>  .../test-manual/reproducible-builds.rst       | 22 +++++++++++++------
>>  1 file changed, 15 insertions(+), 7 deletions(-)
>>
>> diff --git a/documentation/test-manual/reproducible-builds.rst
>> b/documentation/test-manual/reproducible-builds.rst
>> index 91f94a5c7..d581155eb 100644
>> --- a/documentation/test-manual/reproducible-builds.rst
>> +++ b/documentation/test-manual/reproducible-builds.rst
>> @@ -91,13 +91,21 @@ run::
>>
>>     oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
>>
>> -This defaults to including a ``world`` build so, if other layers are
>> added, it would
>> -also run the tests for recipes in the additional layers. Different build
>> targets
>> -can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable
>> in ``local.conf``.
>> -The first build will be run using :ref:`Shared State
>> <overview-manual/concepts:Shared State>` if
>> -available, the second build explicitly disables
>> -:ref:`Shared State <overview-manual/concepts:Shared State>` except for
>> recipes defined in
>> -the :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on
>> the
>> +This defaults to including a ``world`` build so, if other layers are
>> added, it
>> +would also run the tests for recipes in the additional layers. Different
>> build
>> +targets can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET`
>> variable
>> +in ``local.conf``. For example, running reproducibility tests for only the
>> +``python3-numpy`` recipe can be done by setting::
>> +
>> +   OEQA_REPRODUCIBLE_TEST_TARGET = "python3-numpy"
>> +
>> +in local.conf before running the ``oe-selftest`` command shown above.
>> +
>> +Reproducibility builds the target list twice. The first build will be run
>> using
>> +:ref:`Shared State <overview-manual/concepts:Shared State>` if available,
>> the
>> +second build explicitly disables :ref:`Shared State
>> +<overview-manual/concepts:Shared State>` except for recipes defined in the
>> +:term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on the
>>
>
> What do you think of adding that a quick way to test the reproducibility of
> a recipe is to do
> OEQA_REPRODUCIBLE_TEST_TARGET = "recipe"
> OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "DEPENDS of recipe"
> ?

I have a question on this:

What you are trying to say is that you need to set OEQA_REPRODUCIBLE_TEST_TARGET
to the recipe you want to test (`python3-numpy` in Trevor's example), and then
set OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to the DEPENDS of the recipe
(`bitbake-getvar -r python3-numpy DEPENDS` would be a way to obtain that value?
I would put that here as well).

My question is: is the reason to set OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to
avoid rebuilding all the dependencies of OEQA_REPRODUCIBLE_TEST_TARGET without
the sstate cache (and therefore save a lot of time)? If that's why, maybe
mention it explicitly here too.

Other question: can OEQA_REPRODUCIBLE_TEST_TARGET contain more than 1 recipe? If
so, do you need to concatenate the DEPENDS of each of them? The variable's doc is
not really clear on this.


Antonin
Yoann Congal Dec. 20, 2024, 9:46 a.m. UTC | #4
Le ven. 20 déc. 2024 à 09:19, Antonin Godard <antonin.godard@bootlin.com> a
écrit :

> Hi Yoann,
>
> On Thu Dec 19, 2024 at 1:27 PM CET, Yoann Congal via
> lists.yoctoproject.org wrote:
> > Le mer. 18 déc. 2024 à 21:06, Trevor Gamblin via lists.yoctoproject.org
> > <tgamblin=baylibre.com@lists.yoctoproject.org> a écrit :
> >
> >> Add a bit to the "Can we prove the project is reproducible?" section to
> >> show how OEQA_REPRODUCIBLE_TEST_TARGET can be used to test a single
> >> package (rather than the world build), as this may be useful for
> >> maintainers.
> >>
> >> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
> >>
> >
> > Thank you for doing this.
> >
> > That is https://bugzilla.yoctoproject.org/show_bug.cgi?id=15619 (you may
> > want to take it and add a [YOCTO #15619] ref in this patch)
> >
> > ---
> >>  .../test-manual/reproducible-builds.rst       | 22 +++++++++++++------
> >>  1 file changed, 15 insertions(+), 7 deletions(-)
> >>
> >> diff --git a/documentation/test-manual/reproducible-builds.rst
> >> b/documentation/test-manual/reproducible-builds.rst
> >> index 91f94a5c7..d581155eb 100644
> >> --- a/documentation/test-manual/reproducible-builds.rst
> >> +++ b/documentation/test-manual/reproducible-builds.rst
> >> @@ -91,13 +91,21 @@ run::
> >>
> >>     oe-selftest -r
> reproducible.ReproducibleTests.test_reproducible_builds
> >>
> >> -This defaults to including a ``world`` build so, if other layers are
> >> added, it would
> >> -also run the tests for recipes in the additional layers. Different
> build
> >> targets
> >> -can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable
> >> in ``local.conf``.
> >> -The first build will be run using :ref:`Shared State
> >> <overview-manual/concepts:Shared State>` if
> >> -available, the second build explicitly disables
> >> -:ref:`Shared State <overview-manual/concepts:Shared State>` except for
> >> recipes defined in
> >> -the :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds
> on
> >> the
> >> +This defaults to including a ``world`` build so, if other layers are
> >> added, it
> >> +would also run the tests for recipes in the additional layers.
> Different
> >> build
> >> +targets can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET`
> >> variable
> >> +in ``local.conf``. For example, running reproducibility tests for only
> the
> >> +``python3-numpy`` recipe can be done by setting::
> >> +
> >> +   OEQA_REPRODUCIBLE_TEST_TARGET = "python3-numpy"
> >> +
> >> +in local.conf before running the ``oe-selftest`` command shown above.
> >> +
> >> +Reproducibility builds the target list twice. The first build will be
> run
> >> using
> >> +:ref:`Shared State <overview-manual/concepts:Shared State>` if
> available,
> >> the
> >> +second build explicitly disables :ref:`Shared State
> >> +<overview-manual/concepts:Shared State>` except for recipes defined in
> the
> >> +:term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on
> the
> >>
> >
> > What do you think of adding that a quick way to test the reproducibility
> of
> > a recipe is to do
> > OEQA_REPRODUCIBLE_TEST_TARGET = "recipe"
> > OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "DEPENDS of recipe"
> > ?
>
> I have a question on this:
>
> What you are trying to say is that you need to set
> OEQA_REPRODUCIBLE_TEST_TARGET
> to the recipe you want to test (`python3-numpy` in Trevor's example), and
> then
> set OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to the DEPENDS of the recipe
> (`bitbake-getvar -r python3-numpy DEPENDS` would be a way to obtain that
> value?
> I would put that here as well).
>

Yes, that is what I usually do to test a single recipe's reproducibility.


> My question is: is the reason to set OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS
> to
> avoid rebuilding all the dependencies of OEQA_REPRODUCIBLE_TEST_TARGET
> without
> the sstate cache (and therefore save a lot of time)? If that's why, maybe
> mention it explicitly here too.
>

I agree. For the global project, it makes sense to rebuild everything. But,
for a single developer focused on only one recipe (and making the
hypothesis that all of the dependencies are already reproducible), it is
quicker for them to disable sstate only for the recipe.


> Other question: can OEQA_REPRODUCIBLE_TEST_TARGET contain more than 1
> recipe?


Yes.


> If so, do you need to concatenate the DEPENDS of each of them? The
> variable's doc is
> not really clear on this.
>

Also yes.

There is something to add here. The DEPENDS list is not enough. Some build
dependencies are dynamically computed. For example, I have this to test an
empty recipe :
OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "autoconf-native automake-native
libtool-native libtool-cross gettext-native ncurses bison-native
texinfo-dummy-native gcc-runtime glibc glibc opkg-utils gcc-cross-x86_64
linux-yocto"
I've gathered this list by looking at the build logs, but I always wanted
to find an automated way to get it.

Final thought, maybe all of this can be fully automated? Something like a
"OEQA_REPRODUCIBLE_ONLY_TEST_TARGET" that would :
* compute every dependencies of the recipe (dynamic deps included)
* build these using sstate
* build the recipe with and without sstate and compare output (normal
reproducible test)

I created https://bugzilla.yoctoproject.org/show_bug.cgi?id=15701 with this
idea.

Regards,
Antonin Godard Dec. 20, 2024, 11:05 a.m. UTC | #5
Hi Yoann,

On Fri Dec 20, 2024 at 10:46 AM CET, Yoann Congal wrote:
> Le ven. 20 déc. 2024 à 09:19, Antonin Godard <antonin.godard@bootlin.com> a
> écrit :
>
>> Hi Yoann,
>>
>> On Thu Dec 19, 2024 at 1:27 PM CET, Yoann Congal via
>> lists.yoctoproject.org wrote:
>> > Le mer. 18 déc. 2024 à 21:06, Trevor Gamblin via lists.yoctoproject.org
>> > <tgamblin=baylibre.com@lists.yoctoproject.org> a écrit :
>> >
>> >> Add a bit to the "Can we prove the project is reproducible?" section to
>> >> show how OEQA_REPRODUCIBLE_TEST_TARGET can be used to test a single
>> >> package (rather than the world build), as this may be useful for
>> >> maintainers.
>> >>
>> >> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
>> >>
>> >
>> > Thank you for doing this.
>> >
>> > That is https://bugzilla.yoctoproject.org/show_bug.cgi?id=15619 (you may
>> > want to take it and add a [YOCTO #15619] ref in this patch)
>> >
>> > ---
>> >>  .../test-manual/reproducible-builds.rst       | 22 +++++++++++++------
>> >>  1 file changed, 15 insertions(+), 7 deletions(-)
>> >>
>> >> diff --git a/documentation/test-manual/reproducible-builds.rst
>> >> b/documentation/test-manual/reproducible-builds.rst
>> >> index 91f94a5c7..d581155eb 100644
>> >> --- a/documentation/test-manual/reproducible-builds.rst
>> >> +++ b/documentation/test-manual/reproducible-builds.rst
>> >> @@ -91,13 +91,21 @@ run::
>> >>
>> >>     oe-selftest -r
>> reproducible.ReproducibleTests.test_reproducible_builds
>> >>
>> >> -This defaults to including a ``world`` build so, if other layers are
>> >> added, it would
>> >> -also run the tests for recipes in the additional layers. Different
>> build
>> >> targets
>> >> -can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable
>> >> in ``local.conf``.
>> >> -The first build will be run using :ref:`Shared State
>> >> <overview-manual/concepts:Shared State>` if
>> >> -available, the second build explicitly disables
>> >> -:ref:`Shared State <overview-manual/concepts:Shared State>` except for
>> >> recipes defined in
>> >> -the :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds
>> on
>> >> the
>> >> +This defaults to including a ``world`` build so, if other layers are
>> >> added, it
>> >> +would also run the tests for recipes in the additional layers.
>> Different
>> >> build
>> >> +targets can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET`
>> >> variable
>> >> +in ``local.conf``. For example, running reproducibility tests for only
>> the
>> >> +``python3-numpy`` recipe can be done by setting::
>> >> +
>> >> +   OEQA_REPRODUCIBLE_TEST_TARGET = "python3-numpy"
>> >> +
>> >> +in local.conf before running the ``oe-selftest`` command shown above.
>> >> +
>> >> +Reproducibility builds the target list twice. The first build will be
>> run
>> >> using
>> >> +:ref:`Shared State <overview-manual/concepts:Shared State>` if
>> available,
>> >> the
>> >> +second build explicitly disables :ref:`Shared State
>> >> +<overview-manual/concepts:Shared State>` except for recipes defined in
>> the
>> >> +:term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on
>> the
>> >>
>> >
>> > What do you think of adding that a quick way to test the reproducibility
>> of
>> > a recipe is to do
>> > OEQA_REPRODUCIBLE_TEST_TARGET = "recipe"
>> > OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "DEPENDS of recipe"
>> > ?
>>
>> I have a question on this:
>>
>> What you are trying to say is that you need to set
>> OEQA_REPRODUCIBLE_TEST_TARGET
>> to the recipe you want to test (`python3-numpy` in Trevor's example), and
>> then
>> set OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS to the DEPENDS of the recipe
>> (`bitbake-getvar -r python3-numpy DEPENDS` would be a way to obtain that
>> value?
>> I would put that here as well).
>>
>
> Yes, that is what I usually do to test a single recipe's reproducibility.
>
>
>> My question is: is the reason to set OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS
>> to
>> avoid rebuilding all the dependencies of OEQA_REPRODUCIBLE_TEST_TARGET
>> without
>> the sstate cache (and therefore save a lot of time)? If that's why, maybe
>> mention it explicitly here too.
>>
>
> I agree. For the global project, it makes sense to rebuild everything. But,
> for a single developer focused on only one recipe (and making the
> hypothesis that all of the dependencies are already reproducible), it is
> quicker for them to disable sstate only for the recipe.
>
>
>> Other question: can OEQA_REPRODUCIBLE_TEST_TARGET contain more than 1
>> recipe?
>
>
> Yes.
>
>
>> If so, do you need to concatenate the DEPENDS of each of them? The
>> variable's doc is
>> not really clear on this.
>>
>
> Also yes.
>
> There is something to add here. The DEPENDS list is not enough. Some build
> dependencies are dynamically computed. For example, I have this to test an
> empty recipe :
> OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS = "autoconf-native automake-native
> libtool-native libtool-cross gettext-native ncurses bison-native
> texinfo-dummy-native gcc-runtime glibc glibc opkg-utils gcc-cross-x86_64
> linux-yocto"
> I've gathered this list by looking at the build logs, but I always wanted
> to find an automated way to get it.
>
> Final thought, maybe all of this can be fully automated? Something like a
> "OEQA_REPRODUCIBLE_ONLY_TEST_TARGET" that would :
> * compute every dependencies of the recipe (dynamic deps included)
> * build these using sstate
> * build the recipe with and without sstate and compare output (normal
> reproducible test)
>
> I created https://bugzilla.yoctoproject.org/show_bug.cgi?id=15701 with this
> idea.

This seems like a way better implementation for sure!

Thanks a lot for all your answers.

For now, Trevor's patch is already an improvement, I'll take it. Then let's see
if there is movement on this thanks to your new bug, and update the doc or not.
:)


Antonin
diff mbox series

Patch

diff --git a/documentation/test-manual/reproducible-builds.rst b/documentation/test-manual/reproducible-builds.rst
index 91f94a5c7..d581155eb 100644
--- a/documentation/test-manual/reproducible-builds.rst
+++ b/documentation/test-manual/reproducible-builds.rst
@@ -91,13 +91,21 @@  run::
 
    oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
 
-This defaults to including a ``world`` build so, if other layers are added, it would
-also run the tests for recipes in the additional layers. Different build targets
-can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable in ``local.conf``.
-The first build will be run using :ref:`Shared State <overview-manual/concepts:Shared State>` if
-available, the second build explicitly disables
-:ref:`Shared State <overview-manual/concepts:Shared State>` except for recipes defined in
-the :term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on the
+This defaults to including a ``world`` build so, if other layers are added, it
+would also run the tests for recipes in the additional layers. Different build
+targets can be defined using the :term:`OEQA_REPRODUCIBLE_TEST_TARGET` variable
+in ``local.conf``. For example, running reproducibility tests for only the
+``python3-numpy`` recipe can be done by setting::
+
+   OEQA_REPRODUCIBLE_TEST_TARGET = "python3-numpy"
+
+in local.conf before running the ``oe-selftest`` command shown above.
+
+Reproducibility builds the target list twice. The first build will be run using
+:ref:`Shared State <overview-manual/concepts:Shared State>` if available, the
+second build explicitly disables :ref:`Shared State
+<overview-manual/concepts:Shared State>` except for recipes defined in the
+:term:`OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS` variable, and builds on the
 specific host the build is running on. This means we can test reproducibility
 builds between different host distributions over time on the Autobuilder.