mbox series

[layerindex-web,0/6] Get the RRS update scripts working again

Message ID 20261115003506.53654-1-tim.orling@konsulko.com
Headers show
Series Get the RRS update scripts working again | expand

Message

Tim Orling Nov. 15, 2026, 12:35 a.m. UTC
The RRS data on layers.openembedded.org has not been updated for a long
time: in a copy of its database the latest upstream history is
2024-12-28 and the latest recipe upgrade 2025-06-16, so every milestone
since 5.3 M1 shows 0%. These fix the problems that stopped each RRS
script, found by running them on a test instance loaded with that
database copy (layers.moto-timo.net).

1: Bug 16499: every RRS script crashed at startup in the Docker image,
   where TOOLS_LOG_DIR is empty.
2: Bug 16501: load_recipes() imported bb.fetch before init_parser() had
   checked out the right BitBake revision.
3: Bug 16500: upgrade history hung for ever on the OE-Core commit that
   changed the addfragments syntax (8dca5305); use a BitBake from before
   that change for older commits.
4: Bug 16502: upstream history and distros parsed each layer at
   whatever branch the layer index update had left it on (e.g.
   meta-openembedded at dunfell), so most of its recipes failed to
   parse.
5: Bug 16504: upgrade history hung on OE-Core 3c00ad5d3b, the one
   commit where OEEquivHash is the default but BB_HASHSERVE is not set.
   Parsing doesn't need hash equivalence, so use OEBasicHash.
6: Bug 16506: rrs_distros.py failed on the second layer branch, because
   OE-Core's distro_check leaves the build's PATH in the environment.

With these on the test instance, upgrade history has caught up on
OE-Core and meta-intel, upstream history has run for all eight layer
branches (0 recipes that couldn't be parsed, against 2051 before), and
rrs_distros.py has completed for all eight. The test suite passes in
the Docker image.

rrs_distros.py also needs the OE-Core distro_check fixes for bug 16503,
sent separately to openembedded-core:
https://lore.kernel.org/all/20261114200810.31078-1-tim.orling@konsulko.com/

Tim Orling (6):
  rrs/tools/common.py: don't write a log file without TOOLS_LOG_DIR
  rrs/tools/common.py: import BitBake after checking out its revision
  rrs_upgrade_history: use the right BitBake around the 2025 fragments
    change
  rrs/tools/common.py: check out the layer branch before parsing it
  conf/local.conf: use OEBasicHash for parsing
  rrs_distros: keep the environment when fetching distro package lists

 conf/local.conf                   |  4 ++
 docker/settings.py                |  2 +-
 rrs/tools/common.py               | 49 ++++++++++++++++---
 rrs/tools/rrs_distros.py          |  5 +-
 rrs/tools/rrs_upgrade_history.py  | 32 ++++++++++---
 settings.py                       |  2 +-
 tests/test_rrs_distros_environ.py | 45 +++++++++++++++++
 tests/test_rrs_load_recipes.py    | 80 +++++++++++++++++++++++++++++++
 tests/test_rrs_logger.py          | 38 +++++++++++++++
 tests/test_rrs_upgrade_history.py | 43 +++++++++++++++++
 10 files changed, 282 insertions(+), 18 deletions(-)
 create mode 100644 tests/test_rrs_distros_environ.py
 create mode 100644 tests/test_rrs_load_recipes.py
 create mode 100644 tests/test_rrs_logger.py
 create mode 100644 tests/test_rrs_upgrade_history.py