| Message ID | 20260429233224.562584-1-marek.vasut@mailbox.org |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | populate_sdk: Execute populate_sdk before rm_work if rm_work.bbclass active | expand |
On Thu, 2026-04-30 at 01:31 +0200, Marek Vasut wrote: > In case the rm_work.bbclass is inherited via INHERIT += "rm_work" into > a build, it is currently possible for an image build to execute both > rm_work and populate_sdk tasks in parallel, which usually leads to an > unusual failure of the populate_sdk task. > > This can be reproduced with oe-core and bitbake as follows: > $ . oe-init-build-env /path/to/build > $ bitbake core-image-base core-image-base:do_populate_sdk > > Fix this in a manner similar to 00b1911c65fa ("populate_sdk_ext.bbclass: > enhance compatibility with rm_work.bbclass"). Make sure do_populate_sdk > is executed before rm_work for both image and SDK only recipes. Use the > RM_WORK_BUILD_WITHOUT variable provided by rm_work bbclass to determine > whether rm_work is inherited or not. This way, in case rm_work is not > inherited, the dependencies remain unchanged, otherwise populate_sdk is > surely executed before the rm_work task. > > Signed-off-by: Marek Vasut <marek.vasut@mailbox.org> > --- > Cc: Adam Nilsson <Adam.X.Nilsson@axis.com> > Cc: Alexander Kanavin <alex@linutronix.de> > Cc: Khem Raj <raj.khem@gmail.com> > Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> > Cc: Patrick Ohly <patrick.ohly@intel.com> > Cc: Peter Kjellerstedt <peter.kjellerstedt@axis.com> > Cc: Philip Lorenz <philip.lorenz@bmw.de> > Cc: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > meta/classes-recipe/populate_sdk.bbclass | 9 ++++++++- > meta/classes-recipe/populate_sdk_base.bbclass | 7 +++++++ > 2 files changed, 15 insertions(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/populate_sdk.bbclass b/meta/classes-recipe/populate_sdk.bbclass > index caeef5d2b2a..3b89ff997f5 100644 > --- a/meta/classes-recipe/populate_sdk.bbclass > +++ b/meta/classes-recipe/populate_sdk.bbclass > @@ -10,4 +10,11 @@ > inherit populate_sdk_base > > addtask populate_sdk after do_install before do_build > - > +addhandler inject_sdk_rm_work > +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" > +python inject_sdk_rm_work() { > + if d.getVar('RM_WORK_BUILD_WITHOUT'): > + bb.build.addtask('do_populate_sdk', 'do_rm_work do_build', ' ', d) > + else: > + bb.build.addtask('do_populate_sdk', 'do_build', ' ', d) > +} > diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass > index b427ff27616..073949ebcde 100644 > --- a/meta/classes-recipe/populate_sdk_base.bbclass > +++ b/meta/classes-recipe/populate_sdk_base.bbclass > @@ -435,3 +435,10 @@ do_populate_sdk[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE > do_populate_sdk[recrdeptask] += "do_packagedata do_package_write_rpm do_package_write_ipk do_package_write_deb do_package_qa" > do_populate_sdk[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}" > addtask populate_sdk > + > +addhandler inject_sdk_rm_work > +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" > +python inject_sdk_rm_work() { > + if d.getVar('RM_WORK_BUILD_WITHOUT'): > + bb.build.addtask('do_populate_sdk', 'do_rm_work', ' ', d) > +} The solution is creative but I suspect it will cause problems with populate_sdk being injected into builds where it otherwise wasn't going to run. There is clearly an issue here but I'm not sure it can be solved with the way tehe task graph code currently works. It might be time to have some kind of bitbake level API to better handle a task with the requirements rm_work has. I've long wondered if we should do something like that and this is a further sign we may not be able to do much else. Cheers, Richard
On 4/30/26 10:46 AM, Richard Purdie wrote: Hello Richard, >> diff --git a/meta/classes-recipe/populate_sdk.bbclass b/meta/classes-recipe/populate_sdk.bbclass >> index caeef5d2b2a..3b89ff997f5 100644 >> --- a/meta/classes-recipe/populate_sdk.bbclass >> +++ b/meta/classes-recipe/populate_sdk.bbclass >> @@ -10,4 +10,11 @@ >> inherit populate_sdk_base >> >> addtask populate_sdk after do_install before do_build >> - >> +addhandler inject_sdk_rm_work >> +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" >> +python inject_sdk_rm_work() { >> + if d.getVar('RM_WORK_BUILD_WITHOUT'): >> + bb.build.addtask('do_populate_sdk', 'do_rm_work do_build', ' ', d) >> + else: >> + bb.build.addtask('do_populate_sdk', 'do_build', ' ', d) >> +} >> diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass >> index b427ff27616..073949ebcde 100644 >> --- a/meta/classes-recipe/populate_sdk_base.bbclass >> +++ b/meta/classes-recipe/populate_sdk_base.bbclass >> @@ -435,3 +435,10 @@ do_populate_sdk[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE >> do_populate_sdk[recrdeptask] += "do_packagedata do_package_write_rpm do_package_write_ipk do_package_write_deb do_package_qa" >> do_populate_sdk[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}" >> addtask populate_sdk >> + >> +addhandler inject_sdk_rm_work >> +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" >> +python inject_sdk_rm_work() { >> + if d.getVar('RM_WORK_BUILD_WITHOUT'): >> + bb.build.addtask('do_populate_sdk', 'do_rm_work', ' ', d) >> +} > > The solution is creative but I suspect it will cause problems with > populate_sdk being injected into builds where it otherwise wasn't going > to run. Currently, both populate_sdk.bbclass and populate_sdk_base.bbclass already contain "addtask populate_sdk", so aren't those tasks already injected into all such builds? My understanding is, that bb.build.addtask() only extends that addtask with optional " ... before rm_work" in case the rm_work bbclass is inherited in the build, so it shouldn't change the current behavior of these bbclasses, except for the rm_work case. > There is clearly an issue here but I'm not sure it can be solved with > the way tehe task graph code currently works. It might be time to have > some kind of bitbake level API to better handle a task with the > requirements rm_work has. I've long wondered if we should do something > like that and this is a further sign we may not be able to do much > else.
On Sat, 2026-05-02 at 19:29 +0200, Marek Vasut wrote: > On 4/30/26 10:46 AM, Richard Purdie wrote: > > Hello Richard, > > > > diff --git a/meta/classes-recipe/populate_sdk.bbclass b/meta/classes-recipe/populate_sdk.bbclass > > > index caeef5d2b2a..3b89ff997f5 100644 > > > --- a/meta/classes-recipe/populate_sdk.bbclass > > > +++ b/meta/classes-recipe/populate_sdk.bbclass > > > @@ -10,4 +10,11 @@ > > > inherit populate_sdk_base > > > > > > addtask populate_sdk after do_install before do_build > > > - > > > +addhandler inject_sdk_rm_work > > > +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" > > > +python inject_sdk_rm_work() { > > > + if d.getVar('RM_WORK_BUILD_WITHOUT'): > > > + bb.build.addtask('do_populate_sdk', 'do_rm_work do_build', ' ', d) > > > + else: > > > + bb.build.addtask('do_populate_sdk', 'do_build', ' ', d) > > > +} > > > diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass > > > index b427ff27616..073949ebcde 100644 > > > --- a/meta/classes-recipe/populate_sdk_base.bbclass > > > +++ b/meta/classes-recipe/populate_sdk_base.bbclass > > > @@ -435,3 +435,10 @@ do_populate_sdk[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE > > > do_populate_sdk[recrdeptask] += "do_packagedata do_package_write_rpm do_package_write_ipk do_package_write_deb do_package_qa" > > > do_populate_sdk[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}" > > > addtask populate_sdk > > > + > > > +addhandler inject_sdk_rm_work > > > +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" > > > +python inject_sdk_rm_work() { > > > + if d.getVar('RM_WORK_BUILD_WITHOUT'): > > > + bb.build.addtask('do_populate_sdk', 'do_rm_work', ' ', d) > > > +} > > > > The solution is creative but I suspect it will cause problems with > > populate_sdk being injected into builds where it otherwise wasn't going > > to run. > > Currently, both populate_sdk.bbclass and populate_sdk_base.bbclass > already contain "addtask populate_sdk", so aren't those tasks already > injected into all such builds? > > My understanding is, that bb.build.addtask() only extends that addtask > with optional " ... before rm_work" in case the rm_work bbclass is > inherited in the build, so it shouldn't change the current behavior of > these bbclasses, except for the rm_work case. When you run "bitbake XXX", it executes the default task, which is do_build. If you add: addtask do_populate_sdk before do_build it will always run do_populate_sdk when you run "bitbake XXX", which is a change of behaviour to what currently happens. Cheers, Richard
diff --git a/meta/classes-recipe/populate_sdk.bbclass b/meta/classes-recipe/populate_sdk.bbclass index caeef5d2b2a..3b89ff997f5 100644 --- a/meta/classes-recipe/populate_sdk.bbclass +++ b/meta/classes-recipe/populate_sdk.bbclass @@ -10,4 +10,11 @@ inherit populate_sdk_base addtask populate_sdk after do_install before do_build - +addhandler inject_sdk_rm_work +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" +python inject_sdk_rm_work() { + if d.getVar('RM_WORK_BUILD_WITHOUT'): + bb.build.addtask('do_populate_sdk', 'do_rm_work do_build', ' ', d) + else: + bb.build.addtask('do_populate_sdk', 'do_build', ' ', d) +} diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass index b427ff27616..073949ebcde 100644 --- a/meta/classes-recipe/populate_sdk_base.bbclass +++ b/meta/classes-recipe/populate_sdk_base.bbclass @@ -435,3 +435,10 @@ do_populate_sdk[rdepends] = "${@' '.join([x + ':do_package_write_${IMAGE_PKGTYPE do_populate_sdk[recrdeptask] += "do_packagedata do_package_write_rpm do_package_write_ipk do_package_write_deb do_package_qa" do_populate_sdk[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}" addtask populate_sdk + +addhandler inject_sdk_rm_work +inject_sdk_rm_work[eventmask] = "bb.event.RecipeTaskPreProcess" +python inject_sdk_rm_work() { + if d.getVar('RM_WORK_BUILD_WITHOUT'): + bb.build.addtask('do_populate_sdk', 'do_rm_work', ' ', d) +}
In case the rm_work.bbclass is inherited via INHERIT += "rm_work" into a build, it is currently possible for an image build to execute both rm_work and populate_sdk tasks in parallel, which usually leads to an unusual failure of the populate_sdk task. This can be reproduced with oe-core and bitbake as follows: $ . oe-init-build-env /path/to/build $ bitbake core-image-base core-image-base:do_populate_sdk Fix this in a manner similar to 00b1911c65fa ("populate_sdk_ext.bbclass: enhance compatibility with rm_work.bbclass"). Make sure do_populate_sdk is executed before rm_work for both image and SDK only recipes. Use the RM_WORK_BUILD_WITHOUT variable provided by rm_work bbclass to determine whether rm_work is inherited or not. This way, in case rm_work is not inherited, the dependencies remain unchanged, otherwise populate_sdk is surely executed before the rm_work task. Signed-off-by: Marek Vasut <marek.vasut@mailbox.org> --- Cc: Adam Nilsson <Adam.X.Nilsson@axis.com> Cc: Alexander Kanavin <alex@linutronix.de> Cc: Khem Raj <raj.khem@gmail.com> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Cc: Patrick Ohly <patrick.ohly@intel.com> Cc: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Cc: Philip Lorenz <philip.lorenz@bmw.de> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> --- meta/classes-recipe/populate_sdk.bbclass | 9 ++++++++- meta/classes-recipe/populate_sdk_base.bbclass | 7 +++++++ 2 files changed, 15 insertions(+), 1 deletion(-)