| Message ID | 20241031-add-bblock-documentation-v1-5-32b89093bbda@baylibre.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Add bblock documentation | expand |
Hi Julien, great to have that documented! Just some typos I came across. On Thu, Oct 31 2024 at 17:56 +0100, "Julien Stephan" <jstephan@baylibre.com> wrote: > bblock is a helper tool to lock/unlock tasks and recipes to specific > signatures. Add a documentation page for it. > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > --- > documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++ > documentation/dev-manual/index.rst | 1 + > 2 files changed, 130 insertions(+) > > diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst > new file mode 100644 > index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf > --- /dev/null > +++ b/documentation/dev-manual/bblock.rst > @@ -0,0 +1,129 @@ > +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK > + > +Locking and Unlocking Recipes Using ``bblock`` > +********************************************** > + > +By design, the OpenEmbedded build system builds everything from scratch > +unless BitBake determines that specific parts do not require rebuilding. > +At startup, it computes a signature for each tasks, based on the task's input. s/for each tasks/for each task/ or perhaps s/for each tasks/for all tasks/ > +Then, it compares these signatures with the ones from sstate (if exist). Any s/if exist/if they exist/ or s/if exist/if it exists/ depending on if you meant the signatures or the sstate to exist. > +changes cause the task to rerun. > + > +During development, changes might trigger BitBake to rebuild certain > +recipes, even when we know they do not require rebuilding at that stage. > +For example, modifying a recipe can lead to the rebuilding of its native s/to the rebuilding/to rebuilding/ > +counterpart, which might proves unnecessary. Editing the ``python3`` recipe, s/which might proves/which might prove/ > +for instance, can prompt BitBake to rebuild ``python3-native`` along with any > +recipes that depend on it. > + > +To prevent this, use ``bblock`` to lock specific tasks or recipes to > +specific signatures, forcing BitBake to use the sstate cache for them. > + > +.. attention:: > + > + Use ``bblock`` only during the development phase. > + > + Forcing BitBake to use sstate cache, regardless of input changes, means > + the recipe metadata no longer directly reflect the output. Use this feature > + with caution. If you do not understand why signatures change, see the section > + on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__. > + > + > +Locking tasks and recipes > +------------------------- > + > +To lock a recipe, use:: > + > + $ bblock recipe > + > +You can also use a space-separated list of recipes to lock many recipes:: > + > + $ bblock recipe1 recipe2 > + > +Locking a recipe, means locking all tasks of the recipe. If you need to s/Locking a recipe, means locking/Locking a recipe means locking/ > +lock only particular tasks use the `-t` switch with a comma-separated > +list of tasks:: > + > + $ bblock -t task1,task2 recipe > + > + > +Unlocking tasks and recipes > +--------------------------- > + > +To unlock a recipe, use:: > + > + $ bblock -r recipe > + > +You can also use a space-separated list of recipes to unlock many recipes:: > + > + $ bblock -r recipe1 recipe2 > + > +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to s/Unlocking a recipe, means unlocking/Unlocking a recipe means unlocking/ > +unlock only particular tasks use the `-t` switch with a comma-separated > +list of tasks:: > + > + $ bblock -r -t task1,task2 recipe > + > +To unlock all recipes, do not specify any recipe:: > + > + $ bblock -r > + > + > +Configuration file > +------------------ > + > +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file, > +included by default in ``poky/meta/conf/bitbake.conf``. > + > +To dump it, use:: > + > + $ bblock -d > + > + > +Locking mechanism > +----------------- > + > +``bblock`` computes the signature(s) of the task(s) then uses the 3 following > +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES` > +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`. > + > +In particular, ``bblock`` sets:: > + > + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" > + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" > + > + SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>" > + > +This produces architecture specific locks and reminds user that some tasks s/reminds user/reminds the user/ Best regards Ulrich > +have locked signatures. > + > +Example > +------- > + > +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the > +following:: > + > + $ bblock python3-native > + $ bblock -d > + # Generated by bblock > + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" > + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" > + > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7" > + > diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst > index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644 > --- a/documentation/dev-manual/index.rst > +++ b/documentation/dev-manual/index.rst > @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual > error-reporting-tool > wayland > qemu > + bblock > > .. include:: /boilerplate.rst
Hi Julien, Thanks for taking the time to document your tool! On top of Ulrich's comments, here are my suggestions/corrections: On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote: > bblock is a helper tool to lock/unlock tasks and recipes to specific > signatures. Add a documentation page for it. > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > --- > documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++ > documentation/dev-manual/index.rst | 1 + > 2 files changed, 130 insertions(+) > > diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst > new file mode 100644 > index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf > --- /dev/null > +++ b/documentation/dev-manual/bblock.rst > @@ -0,0 +1,129 @@ > +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK > + > +Locking and Unlocking Recipes Using ``bblock`` > +********************************************** > + > +By design, the OpenEmbedded build system builds everything from scratch > +unless BitBake determines that specific parts do not require rebuilding. s/parts/tasks/? > +At startup, it computes a signature for each tasks, based on the task's input. > +Then, it compares these signatures with the ones from sstate (if exist). Any s/from sstate/from the sstate cache/? s/if exist/if they exist/ > +changes cause the task to rerun. > + > +During development, changes might trigger BitBake to rebuild certain > +recipes, even when we know they do not require rebuilding at that stage. > +For example, modifying a recipe can lead to the rebuilding of its native > +counterpart, which might proves unnecessary. Editing the ``python3`` recipe, > +for instance, can prompt BitBake to rebuild ``python3-native`` along with any > +recipes that depend on it. > + > +To prevent this, use ``bblock`` to lock specific tasks or recipes to > +specific signatures, forcing BitBake to use the sstate cache for them. > + > +.. attention:: > + > + Use ``bblock`` only during the development phase. > + > + Forcing BitBake to use sstate cache, regardless of input changes, means > + the recipe metadata no longer directly reflect the output. Use this feature s/reflect/reflects/ unless metadata is a plural by default? > + with caution. If you do not understand why signatures change, see the section > + on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__. You can use :yocto_wiki: here: :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`. (syntax needs double-checking ;)) > + > +Locking tasks and recipes > +------------------------- > + > +To lock a recipe, use:: > + > + $ bblock recipe > + > +You can also use a space-separated list of recipes to lock many recipes:: > + > + $ bblock recipe1 recipe2 > + > +Locking a recipe, means locking all tasks of the recipe. If you need to No comma "," between recipe and means, I think. > +lock only particular tasks use the `-t` switch with a comma-separated Comma after "tasks" I think would make more sense here (to me at least). > +list of tasks:: > + > + $ bblock -t task1,task2 recipe > + Maybe a real-life example would be great here! > +Unlocking tasks and recipes > +--------------------------- > + > +To unlock a recipe, use:: ...use the ``-r`` option:: > + > + $ bblock -r recipe > + > +You can also use a space-separated list of recipes to unlock many recipes:: s/many/multiple/ (many means "a lot of") > + > + $ bblock -r recipe1 recipe2 > + > +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to > +unlock only particular tasks use the `-t` switch with a comma-separated > +list of tasks:: > + > + $ bblock -r -t task1,task2 recipe > + > +To unlock all recipes, do not specify any recipe:: > + > + $ bblock -r > + > + > +Configuration file > +------------------ > + > +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file, > +included by default in ``poky/meta/conf/bitbake.conf``. Here you can use: :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>` Also, I believe that bblock.conf is a candidate for being in `documentation/ref-manual/structure.rst`, alongside local.conf and bblayers.conf in the section "The Build Directory --- ``build/``". Could you add a section there that explains this file briefly and that redirects the user to this document? > + > +To dump it, use:: s/To dump it/To dump the file/ > + > + $ bblock -d > + > + > +Locking mechanism > +----------------- > + > +``bblock`` computes the signature(s) of the task(s) then uses the 3 following s/then uses/and sets/? I.e. explain what it really does with them. > +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES` > +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`. > + > +In particular, ``bblock`` sets:: > + > + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" > + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" > + > + SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>" > + > +This produces architecture specific locks and reminds user that some tasks > +have locked signatures. > + > +Example > +------- > + > +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the s/on ``python3`` recipe/on the ``python3`` recipe/ s/pyton3-native/python3-native/ > +following:: > + > + $ bblock python3-native > + $ bblock -d > + # Generated by bblock > + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" > + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" > + > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28" > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7" > + > diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst > index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644 > --- a/documentation/dev-manual/index.rst > +++ b/documentation/dev-manual/index.rst > @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual > error-reporting-tool > wayland > qemu > + bblock > > .. include:: /boilerplate.rst Thanks! Antonin -- Antonin Godard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Le lun. 4 nov. 2024 à 14:32, Antonin Godard <antonin.godard@bootlin.com> a écrit : > > Hi Julien, > > Thanks for taking the time to document your tool! > On top of Ulrich's comments, here are my suggestions/corrections: > > On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote: > > bblock is a helper tool to lock/unlock tasks and recipes to specific > > signatures. Add a documentation page for it. > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > --- > > documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++ > > documentation/dev-manual/index.rst | 1 + > > 2 files changed, 130 insertions(+) > > > > diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst > > new file mode 100644 > > index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf > > --- /dev/null > > +++ b/documentation/dev-manual/bblock.rst > > @@ -0,0 +1,129 @@ > > +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK > > + > > +Locking and Unlocking Recipes Using ``bblock`` > > +********************************************** > > + > > +By design, the OpenEmbedded build system builds everything from scratch > > +unless BitBake determines that specific parts do not require rebuilding. > > s/parts/tasks/? > > > +At startup, it computes a signature for each tasks, based on the task's input. > > +Then, it compares these signatures with the ones from sstate (if exist). Any > > s/from sstate/from the sstate cache/? > s/if exist/if they exist/ > > > +changes cause the task to rerun. > > + > > +During development, changes might trigger BitBake to rebuild certain > > +recipes, even when we know they do not require rebuilding at that stage. > > +For example, modifying a recipe can lead to the rebuilding of its native > > +counterpart, which might proves unnecessary. Editing the ``python3`` recipe, > > +for instance, can prompt BitBake to rebuild ``python3-native`` along with any > > +recipes that depend on it. > > + > > +To prevent this, use ``bblock`` to lock specific tasks or recipes to > > +specific signatures, forcing BitBake to use the sstate cache for them. > > + > > +.. attention:: > > + > > + Use ``bblock`` only during the development phase. > > + > > + Forcing BitBake to use sstate cache, regardless of input changes, means > > + the recipe metadata no longer directly reflect the output. Use this feature > > s/reflect/reflects/ unless metadata is a plural by default? I tend to consider metadata plural by default. Maybe I'm wrong? > > > + with caution. If you do not understand why signatures change, see the section > > + on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__. > > You can use :yocto_wiki: here: > > :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`. > > (syntax needs double-checking ;)) > > > + > > +Locking tasks and recipes > > +------------------------- > > + > > +To lock a recipe, use:: > > + > > + $ bblock recipe > > + > > +You can also use a space-separated list of recipes to lock many recipes:: > > + > > + $ bblock recipe1 recipe2 > > + > > +Locking a recipe, means locking all tasks of the recipe. If you need to > > No comma "," between recipe and means, I think. > > > +lock only particular tasks use the `-t` switch with a comma-separated > > Comma after "tasks" I think would make more sense here (to me at least). > > > +list of tasks:: > > + > > + $ bblock -t task1,task2 recipe > > + > > Maybe a real-life example would be great here! I already put a real life example at the end. The idea was to avoid having an example in each section. Is it sufficient? > > > +Unlocking tasks and recipes > > +--------------------------- > > + > > +To unlock a recipe, use:: > > ...use the ``-r`` option:: > > > + > > + $ bblock -r recipe > > + > > +You can also use a space-separated list of recipes to unlock many recipes:: > > s/many/multiple/ (many means "a lot of") Yeah.. I will put "multiple" back.. But according to Vale: warning 'multiple' is too wordy. write-good.TooWordy warning 'several' is a weasel word! write-good.Weasel That's the reason why I put "many" ^^(but I agree it does not fit well here) > > > + > > + $ bblock -r recipe1 recipe2 > > + > > +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to > > +unlock only particular tasks use the `-t` switch with a comma-separated > > +list of tasks:: > > + > > + $ bblock -r -t task1,task2 recipe > > + > > +To unlock all recipes, do not specify any recipe:: > > + > > + $ bblock -r > > + > > + > > +Configuration file > > +------------------ > > + > > +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file, > > +included by default in ``poky/meta/conf/bitbake.conf``. > > Here you can use: > > :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>` > > Also, I believe that bblock.conf is a candidate for being in > `documentation/ref-manual/structure.rst`, alongside local.conf and > bblayers.conf in the section "The Build Directory --- ``build/``". Could you add > a section there that explains this file briefly and that redirects the user to > this document? > Sure! > > + > > +To dump it, use:: > > s/To dump it/To dump the file/ > > > + > > + $ bblock -d > > + > > + > > +Locking mechanism > > +----------------- > > + > > +``bblock`` computes the signature(s) of the task(s) then uses the 3 following > > s/then uses/and sets/? I.e. explain what it really does with them. > > > +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES` > > +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`. > > + > > +In particular, ``bblock`` sets:: > > + > > + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" > > + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" > > + > > + SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>" > > + > > +This produces architecture specific locks and reminds user that some tasks > > +have locked signatures. > > + > > +Example > > +------- > > + > > +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the > > s/on ``python3`` recipe/on the ``python3`` recipe/ > s/pyton3-native/python3-native/ > > > +following:: > > + > > + $ bblock python3-native > > + $ bblock -d > > + # Generated by bblock > > + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" > > + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" > > + > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28" > > + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7" > > + > > diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst > > index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644 > > --- a/documentation/dev-manual/index.rst > > +++ b/documentation/dev-manual/index.rst > > @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual > > error-reporting-tool > > wayland > > qemu > > + bblock > > > > .. include:: /boilerplate.rst > > Thanks! > Antonin > > -- > Antonin Godard, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com >
diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst new file mode 100644 index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf --- /dev/null +++ b/documentation/dev-manual/bblock.rst @@ -0,0 +1,129 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +Locking and Unlocking Recipes Using ``bblock`` +********************************************** + +By design, the OpenEmbedded build system builds everything from scratch +unless BitBake determines that specific parts do not require rebuilding. +At startup, it computes a signature for each tasks, based on the task's input. +Then, it compares these signatures with the ones from sstate (if exist). Any +changes cause the task to rerun. + +During development, changes might trigger BitBake to rebuild certain +recipes, even when we know they do not require rebuilding at that stage. +For example, modifying a recipe can lead to the rebuilding of its native +counterpart, which might proves unnecessary. Editing the ``python3`` recipe, +for instance, can prompt BitBake to rebuild ``python3-native`` along with any +recipes that depend on it. + +To prevent this, use ``bblock`` to lock specific tasks or recipes to +specific signatures, forcing BitBake to use the sstate cache for them. + +.. attention:: + + Use ``bblock`` only during the development phase. + + Forcing BitBake to use sstate cache, regardless of input changes, means + the recipe metadata no longer directly reflect the output. Use this feature + with caution. If you do not understand why signatures change, see the section + on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__. + + +Locking tasks and recipes +------------------------- + +To lock a recipe, use:: + + $ bblock recipe + +You can also use a space-separated list of recipes to lock many recipes:: + + $ bblock recipe1 recipe2 + +Locking a recipe, means locking all tasks of the recipe. If you need to +lock only particular tasks use the `-t` switch with a comma-separated +list of tasks:: + + $ bblock -t task1,task2 recipe + + +Unlocking tasks and recipes +--------------------------- + +To unlock a recipe, use:: + + $ bblock -r recipe + +You can also use a space-separated list of recipes to unlock many recipes:: + + $ bblock -r recipe1 recipe2 + +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to +unlock only particular tasks use the `-t` switch with a comma-separated +list of tasks:: + + $ bblock -r -t task1,task2 recipe + +To unlock all recipes, do not specify any recipe:: + + $ bblock -r + + +Configuration file +------------------ + +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file, +included by default in ``poky/meta/conf/bitbake.conf``. + +To dump it, use:: + + $ bblock -d + + +Locking mechanism +----------------- + +``bblock`` computes the signature(s) of the task(s) then uses the 3 following +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES` +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`. + +In particular, ``bblock`` sets:: + + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" + + SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>" + +This produces architecture specific locks and reminds user that some tasks +have locked signatures. + +Example +------- + +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the +following:: + + $ bblock python3-native + $ bblock -d + # Generated by bblock + SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" + SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" + + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28" + SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7" + diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644 --- a/documentation/dev-manual/index.rst +++ b/documentation/dev-manual/index.rst @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual error-reporting-tool wayland qemu + bblock .. include:: /boilerplate.rst
bblock is a helper tool to lock/unlock tasks and recipes to specific signatures. Add a documentation page for it. Signed-off-by: Julien Stephan <jstephan@baylibre.com> --- documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++ documentation/dev-manual/index.rst | 1 + 2 files changed, 130 insertions(+)