mbox series

[v2,0/3] yocto-check-layer: Faster BSP checks over many machines

Message ID 20261019143924.1521981-1-otavio@ossystems.com.br
Headers show
Series yocto-check-layer: Faster BSP checks over many machines | expand

Message

Otavio Salvador Oct. 19, 2026, 2:39 p.m. UTC 
Checking a BSP layer over many machines is dominated by world signature
dumps: two per machine, plus four for the common tests. In CI the
machines are usually split into several runs, which repeat the common
tests and their dumps in each.

This series lets such runs select their tests and dumps each
configuration once:

- --run-tests, as in oe-test, so the common tests run once and the BSP
  tests in each group of machines;
- no initial (pre-layer) dump when test_signatures is not selected;
- get_signatures() always runs with -k and returns the error with the
  signatures, so the common tests, and each machine of the BSP tests,
  dump once in setUpClass() and the world and signature tests share it.

A full run over N machines goes from 2N+4 dumps to N+3; a split run
does 3 dumps for the common tests and N for the BSP ones.

Changes in v2:
- Patch 3 replaces the signature cache in get_signatures() with one dump
  per configuration in setUpClass(), as suggested by Paul Barker. Failed
  dumps are no longer run a second time either.
- Patches 1 and 2 are unchanged.

Tested on meta-freescale against unpatched OE-core and against the
wrapper proposed for its CI, which does the same from outside
yocto-check-layer: with two and eight machines, a recipe breaking world
on one machine, and a recipe that does not parse. Every test gives the
same result in all of them, with the same machines reported as broken.
World dumps per run:

                                  unpatched  wrapper  v2
  full run, 2 machines                8          5      5
  BSP tests, 2 machines, 1 broken     -          3      2
  BSP tests, 8 machines, 1 broken     -          9      8
  parse error, 2 machines             7          7      5

  (-: unpatched yocto-check-layer cannot run the BSP tests alone)

The only visible change is in world failures: the dump now uses -k, so
it lists all the errors of the failing machine instead of the first one.

v1: https://lists.openembedded.org/g/openembedded-core/topic/121810615

Otavio Salvador (3):
  yocto-check-layer: Add --run-tests to select the tests to run
  yocto-check-layer: Skip initial signatures unless test_signatures runs
  checklayer: Share each world signature dump between the tests needing
    it

 scripts/lib/checklayer/__init__.py     | 28 +++++++++++++-------------
 scripts/lib/checklayer/cases/bsp.py    | 15 +++++++++-----
 scripts/lib/checklayer/cases/common.py | 20 +++++++++++-------
 scripts/yocto-check-layer              | 14 +++++++++----
 4 files changed, 47 insertions(+), 30 deletions(-)