| Message ID | 20261020223910.539165-1-ricardo.salveti@oss.qualcomm.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | checklayer: run bitbake once per machine for the machine tests | expand |
On Wed Oct 21, 2026 at 12:39 AM CEST, Ricardo Salveti via lists.openembedded.org wrote: > With --machines, test_machine_signatures and test_machine_world each > run "bitbake -S lockedsigs world" for every machine, with and without > -k. When world can be built, -k makes no difference and both runs give > the same signatures, so bitbake runs twice per machine for nothing. On > meta-qcom, which tests 24 machines, these two tests take 1665 s of the > 1818 s of the check. > > Cache the outcome of the run without -k on the test context, which is > created for every tested layer, so that whichever test comes first runs > bitbake and the other one reuses the result. When world can't be built, > test_machine_signatures still runs bitbake again with -k for the > signatures: get_signatures(failsafe=True) ignores the error of that > run, which test_machine_world needs, so one -k run can't serve both > tests. test_world and test_signatures have the same duplication without > a machine; that is left for a separate change. > > Tested on meta-qcom (127450527f) with 16 bitbake threads, time in the > two tests: 24 machines 1664.8 s -> 969.9 s, three machines 229.9 s -> > 140.5 s, with the same results and logs. Adding a machine that breaks > world, one with different zlib CFLAGS or one that doesn't parse to the > three machines gives the same verdicts and errors as before. > > AI-Generated: Uses Claude Code (Claude Opus 5.5) > Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> > --- Hi Ricardo, Thanks for your patch. I can see it conflicts with the patch from Otavio Salvador sent a few days ago. I believe part of the points addressed by both patches are similar. Any thought on that? https://lore.kernel.org/openembedded-core/20261019143924.1521981-4-otavio@ossystems.com.br/ Thanks, Mathieu
Hi, On Wed, Oct 21, 2026 at 10:47 AM Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> wrote: > > On Wed Oct 21, 2026 at 12:39 AM CEST, Ricardo Salveti via lists.openembedded.org wrote: > > With --machines, test_machine_signatures and test_machine_world each > > run "bitbake -S lockedsigs world" for every machine, with and without > > -k. When world can be built, -k makes no difference and both runs give > > the same signatures, so bitbake runs twice per machine for nothing. On > > meta-qcom, which tests 24 machines, these two tests take 1665 s of the > > 1818 s of the check. > > > > Cache the outcome of the run without -k on the test context, which is > > created for every tested layer, so that whichever test comes first runs > > bitbake and the other one reuses the result. When world can't be built, > > test_machine_signatures still runs bitbake again with -k for the > > signatures: get_signatures(failsafe=True) ignores the error of that > > run, which test_machine_world needs, so one -k run can't serve both > > tests. test_world and test_signatures have the same duplication without > > a machine; that is left for a separate change. > > > > Tested on meta-qcom (127450527f) with 16 bitbake threads, time in the > > two tests: 24 machines 1664.8 s -> 969.9 s, three machines 229.9 s -> > > 140.5 s, with the same results and logs. Adding a machine that breaks > > world, one with different zlib CFLAGS or one that doesn't parse to the > > three machines gives the same verdicts and errors as before. > > > > AI-Generated: Uses Claude Code (Claude Opus 5.5) > > Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> > > --- > > Hi Ricardo, > > Thanks for your patch. > > I can see it conflicts with the patch from Otavio Salvador sent a few > days ago. I believe part of the points addressed by both patches are > similar. Any thought on that? > > https://lore.kernel.org/openembedded-core/20261019143924.1521981-4-otavio@ossystems.com.br/ Thanks, I missed this one, and it is indeed trying to solve the same issue, so we can drop mine and I will follow up with that one instead. Cheers, Ricardo
diff --git a/scripts/lib/checklayer/cases/bsp.py b/scripts/lib/checklayer/cases/bsp.py index b76163fb56..e9472fd512 100644 --- a/scripts/lib/checklayer/cases/bsp.py +++ b/scripts/lib/checklayer/cases/bsp.py @@ -5,7 +5,7 @@ import unittest -from checklayer import LayerType, get_signatures, check_command, get_depgraph +from checklayer import LayerType, check_command, get_depgraph from checklayer.case import OECheckLayerTestCase class BSPCheckLayer(OECheckLayerTestCase): @@ -45,10 +45,9 @@ class BSPCheckLayer(OECheckLayerTestCase): msg = [] for machine in self.td['machines']: # In contrast to test_machine_signatures() below, errors are fatal here. - try: - get_signatures(self.td['builddir'], failsafe=False, machine=machine) - except RuntimeError as ex: - msg.append(str(ex)) + error = self.tc.world_error(machine) + if error is not None: + msg.append(str(error)) if msg: msg.insert(0, 'The following machines broke a world build:') self.fail('\n'.join(msg)) @@ -80,7 +79,7 @@ class BSPCheckLayer(OECheckLayerTestCase): tunes = {} # Preserve ordering of machines as chosen by the user. for machine in self.td['machines']: - curr_sigs, tune2tasks = get_signatures(self.td['builddir'], failsafe=True, machine=machine) + curr_sigs, tune2tasks = self.tc.world_signatures(machine) # Invert the tune -> [tasks] mapping. tasks2tune = {} for tune, tasks in tune2tasks.items(): diff --git a/scripts/lib/checklayer/context.py b/scripts/lib/checklayer/context.py index 4de8f668fd..19bd59a709 100644 --- a/scripts/lib/checklayer/context.py +++ b/scripts/lib/checklayer/context.py @@ -9,9 +9,44 @@ import glob import re from oeqa.core.context import OETestContext +from checklayer import get_signatures class CheckLayerTestContext(OETestContext): def __init__(self, td=None, logger=None, layer=None, test_software_layer_signatures=True): super(CheckLayerTestContext, self).__init__(td, logger) self.layer = layer self.test_software_layer_signatures = test_software_layer_signatures + self._world_runs = {} + + def _world_run(self, machine): + ''' + Calls get_signatures(failsafe=False) once per machine and remembers + the outcome: the signatures, or the RuntimeError raised instead. + ''' + if machine not in self._world_runs: + try: + self._world_runs[machine] = get_signatures(self.td['builddir'], failsafe=False, machine=machine) + except RuntimeError as ex: + self._world_runs[machine] = ex + return self._world_runs[machine] + + def world_error(self, machine): + ''' + Returns the RuntimeError that get_signatures(failsafe=False) raises + for the given machine, or None if world can be built. + ''' + result = self._world_run(machine) + if isinstance(result, RuntimeError): + return result + return None + + def world_signatures(self, machine): + ''' + Returns (sigs, tune2tasks) like get_signatures(failsafe=True). When + world can be built, -k makes no difference, so the signatures of the + run without it are reused; otherwise bitbake runs again with -k. + ''' + result = self._world_run(machine) + if isinstance(result, RuntimeError): + return get_signatures(self.td['builddir'], failsafe=True, machine=machine) + return result
With --machines, test_machine_signatures and test_machine_world each run "bitbake -S lockedsigs world" for every machine, with and without -k. When world can be built, -k makes no difference and both runs give the same signatures, so bitbake runs twice per machine for nothing. On meta-qcom, which tests 24 machines, these two tests take 1665 s of the 1818 s of the check. Cache the outcome of the run without -k on the test context, which is created for every tested layer, so that whichever test comes first runs bitbake and the other one reuses the result. When world can't be built, test_machine_signatures still runs bitbake again with -k for the signatures: get_signatures(failsafe=True) ignores the error of that run, which test_machine_world needs, so one -k run can't serve both tests. test_world and test_signatures have the same duplication without a machine; that is left for a separate change. Tested on meta-qcom (127450527f) with 16 bitbake threads, time in the two tests: 24 machines 1664.8 s -> 969.9 s, three machines 229.9 s -> 140.5 s, with the same results and logs. Adding a machine that breaks world, one with different zlib CFLAGS or one that doesn't parse to the three machines gives the same verdicts and errors as before. AI-Generated: Uses Claude Code (Claude Opus 5.5) Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> --- scripts/lib/checklayer/cases/bsp.py | 11 +++++---- scripts/lib/checklayer/context.py | 35 +++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 6 deletions(-)