| Message ID | 20260815134722.497586-2-adrian.freihofer@siemens.com |
|---|---|
| State | New |
| Headers | show |
| Series | cooker/tinfoil: fix -b bbappend handling and add single-task prepared-task API | expand |
On Sat, 2026-08-15 at 15:46 +0200, Adrian Freihofer via lists.openembedded.org wrote: > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > "bitbake -b <recipe.bb>" builds the recipe without applying any of its > .bbappend files. > > buildFileInternal() resolves appends via > self.collections[mc].get_file_appends(fn), but self.collections[mc] is > only ever filled in by collect_bbfiles(), called from updateCache() - > a path -b deliberately skips. matchFiles(), the one -b-path function > that does call collect_bbfiles(), built a fresh CookerCollectFiles into > a throwaway local instead of self.collections[mc], so the append list > stayed empty (or, on a memory-resident server, stale from the last > full parse - e.g. missing a devtool/externalsrc workspace .bbappend > added since). Nothing warns that the built metadata differs from disk. > > Make matchFiles() refresh self.collections[mc] itself so the later > append lookup for the same fn sees the same fresh collection. > > AI-Generated: Uses GitHub Copilot > > Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> > --- > lib/bb/cooker.py | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) I've not checked but doesn't this only add limited collections data so whilst it fixes -b, it potentially corrupts the cache for the non -b cases? This is definitely something we should fix, I just want to make sure this doesn't corrupt something else. I suspect it deliberately doesn't write to self... Cheers, Richard
On Sat, 2026-08-15 at 22:39 +0100, Richard Purdie via lists.openembedded.org wrote: > On Sat, 2026-08-15 at 15:46 +0200, Adrian Freihofer via > lists.openembedded.org wrote: > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > "bitbake -b <recipe.bb>" builds the recipe without applying any of > > its > > .bbappend files. > > > > buildFileInternal() resolves appends via > > self.collections[mc].get_file_appends(fn), but self.collections[mc] > > is > > only ever filled in by collect_bbfiles(), called from updateCache() > > - > > a path -b deliberately skips. matchFiles(), the one -b-path > > function > > that does call collect_bbfiles(), built a fresh CookerCollectFiles > > into > > a throwaway local instead of self.collections[mc], so the append > > list > > stayed empty (or, on a memory-resident server, stale from the last > > full parse - e.g. missing a devtool/externalsrc workspace .bbappend > > added since). Nothing warns that the built metadata differs from > > disk. > > > > Make matchFiles() refresh self.collections[mc] itself so the later > > append lookup for the same fn sees the same fresh collection. > > > > AI-Generated: Uses GitHub Copilot > > > > Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> > > --- > > lib/bb/cooker.py | 6 ++++-- > > 1 file changed, 4 insertions(+), 2 deletions(-) > > I've not checked but doesn't this only add limited collections data > so > whilst it fixes -b, it potentially corrupts the cache for the non -b > cases? I tried to add a test case test_run_prepared_task_recipecache_self_heals see https://lists.openembedded.org/g/bitbake-devel/message/19965 which proves that this works as expected with v2: https://lists.openembedded.org/g/bitbake-devel/message/19964 Regards, Adrian > > This is definitely something we should fix, I just want to make sure > this doesn't corrupt something else. I suspect it deliberately > doesn't > write to self... > > Cheers, > > Richard > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#19957): > https://lists.openembedded.org/g/bitbake-devel/message/19957 > Mute This Topic: https://lists.openembedded.org/mt/120763373/4454582 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: > https://lists.openembedded.org/g/bitbake-devel/unsub [adrian.freihofer@gmail.com > ] > -=-=-=-=-=-=-=-=-=-=-=-
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 4b6ba3196..108551a60 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1322,8 +1322,10 @@ You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in si if bf.startswith("/") or bf.startswith("../"): bf = os.path.abspath(bf) - collections = {mc: CookerCollectFiles(self.bbfile_config_priorities, mc)} - filelist, masked, searchdirs = collections[mc].collect_bbfiles(self.databuilder.mcdata[mc], self.databuilder.mcdata[mc]) + # The only place the "bitbake -b" path fills in the bbappends which + # buildFileInternal() then reads back from self.collections[mc]. + self.collections[mc] = CookerCollectFiles(self.bbfile_config_priorities, mc) + filelist, masked, searchdirs = self.collections[mc].collect_bbfiles(self.databuilder.mcdata[mc], self.databuilder.mcdata[mc]) try: os.stat(bf) bf = os.path.abspath(bf)