| Message ID | 20250214211403.493573-1-chris.laplante@agilent.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] cooker/event: add cache validity checking events | expand |
> Subject: [bitbake-devel] [PATCH] bitbake: knotty: report cache validity checking > progress > > > > External Sender - Use caution opening files, clicking links, or responding to > requests. > > > From: Chris Laplante <chris.laplante@agilent.com> > > Signed-off-by: Chris Laplante <chris.laplante@agilent.com> > --- > bitbake/lib/bb/ui/knotty.py | 23 ++++++++++++++++++++++- > 1 file changed, 22 insertions(+), 1 deletion(-) > Sorry, accidentally sent this patch twice. Please disregard this one. Thanks, Chris
On Fri, 2025-02-14 at 16:14 -0500, Chris Laplante via lists.openembedded.org wrote: > From: Chris Laplante <chris.laplante@agilent.com> > > There's a "dead zone" between the "Loading cache" phase and the "Parsing > recipes" phase during which bitbake checks whether cache entries are actually > valid. Even at the highest (-DDD) debug level, bitbake produces no output. > > On my relatively powerful baremetal Yocto build PC, this phase takes 6 seconds > in the presence of 4 multiconfigs (including 'default') and 40 layers. > > This patch provides events to track the progress of cache validity > checking, so we can give the user some visual feedback. > > Signed-off-by: Chris Laplante <chris.laplante@agilent.com> > --- > lib/bb/cooker.py | 10 ++++++++++ > lib/bb/event.py | 15 +++++++++++++++ > 2 files changed, 25 insertions(+) Personally, I hate most of the progress events. Having a few seconds pause for a command line tool on something with 4 multiconfigs and 40 layers doesn't sound unreasonable. I've put them in the queue for discussion though as I know I'm in a minority on this, I just get to maintain it all and my own view counts for little. I'm just mentioning this to illustrate I don't get to simply do what I want as some people seem to think. Cheers, Richard
Hi Richard, > I've put them in the queue for discussion though as I know I'm in a minority on > this, I just get to maintain it all and my own view counts for little. I'm just > mentioning this to illustrate I don't get to simply do what I want as some > people seem to think. Is there a different model we could move to that would be easier to maintain? For example, instead of discrete SomethingStarted, SomethingProgress, and SomethingCompleted, maybe make use of the OperationStarted/Progress/Completed events directly? It would at a minimum deduplicate some code in the UIs. Thanks, Chris
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 5b885cddd7..9b78530ad3 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -2127,6 +2127,12 @@ class CookerParser(object): self.bb_caches = bb.cache.MulticonfigCache(self.cfgbuilder, self.cfghash, cooker.caches_array) self.fromcache = set() self.willparse = set() + + validate_count = sum(len(self.mcfilelist[mc]) for mc in self.cooker.multiconfigs) + validate_chunk = int(max(validate_count / 100, 1)) + + bb.event.fire(bb.event.CheckCacheValidityStarted(validate_count), self.cfgdata) + num_validated = 0 for mc in self.cooker.multiconfigs: for filename in self.mcfilelist[mc]: appends = self.cooker.collections[mc].get_file_appends(filename) @@ -2135,6 +2141,10 @@ class CookerParser(object): self.willparse.add((mc, self.bb_caches[mc], filename, appends, layername)) else: self.fromcache.add((mc, self.bb_caches[mc], filename, appends, layername)) + num_validated += 1 + if num_validated % validate_chunk == 0: + bb.event.fire(bb.event.CheckCacheValidityProgress(num_validated, validate_count), self.cfgdata) + bb.event.fire(bb.event.CheckCacheValidityCompleted(validate_count), self.cfgdata) self.total = len(self.fromcache) + len(self.willparse) self.toparse = len(self.willparse) diff --git a/lib/bb/event.py b/lib/bb/event.py index 952c85c0bd..e9834d01c7 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -639,6 +639,21 @@ class CacheLoadCompleted(OperationCompleted): OperationCompleted.__init__(self, total, "Loading cache Completed") self.num_entries = num_entries +class CheckCacheValidityStarted(OperationStarted): + """Started checking whether cached entries are valid""" + def __init__(self, total): + super().__init__("Checking cache validity started") + self.total = total + +class CheckCacheValidityProgress(OperationProgress): + def __init__(self, current, total): + super().__init__(current, total, "Checking cache validity") + +class CheckCacheValidityCompleted(OperationCompleted): + """Finished checking whether cached entries are valid""" + def __init__(self, total): + super().__init__(total, "Checking cache validity completed") + class TreeDataPreparationStarted(OperationStarted): """Tree data preparation started""" def __init__(self):