diff mbox series

oe-build-perf-report: filter used measurements for each commit

Message ID 20251022-mathieu-buildperf-multi-bitbake-commits-v1-1-180328529ea5@bootlin.com
State New
Headers show
Series oe-build-perf-report: filter used measurements for each commit | expand

Commit Message

Mathieu Dubois-Briand Oct. 22, 2025, 1:43 p.m. UTC 
As the poky repository is no longer used, measurements are indexed using
the oe-core commit. But as bitbake, oe-core and meta-yocto are now
retrieved from separate gits, while measuring performances for a given branch
at some time interval, we can get the same commit for oe-core but
different ones for bitbake or meta-yocto. As a consequence, metadata
associated with the same index (oe-core commit) might differ.

Today this is not supported, as we do expect all metadata for a given
version remain the same.

For each oe-core commit, filter the measurements, in order to only keep
the ones with the metadata matching the last measurement found for the
said commit.

Fixes [YOCTO #16014]

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
Fixing some buildperf issue [1] occurring after the bitbake-setup
switch.

I was tempted to send this as an RFC, as I'm not really sure if this the
best solution here. I decided to filter-out the entries with different
metadata, but we could also want to merge them or integrate them as
different results.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16014
---
 scripts/oe-build-perf-report | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)


---
base-commit: 7a900ff52d76490f42236f9dd898ded0da77058e
change-id: 20251022-mathieu-buildperf-multi-bitbake-commits-ba7909d894dc

Best regards,
diff mbox series

Patch

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index a36f3c1bca34..2d51e84444ba 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -133,12 +133,19 @@  def read_results(repo, tags, xml=True):
     if xml:
         git_objs = [tag + ':metadata.xml' for tag in tags] + [tag + ':results.xml' for tag in tags]
         data = parse_xml_stream(repo.run_cmd(['show'] + git_objs + ['--']))
-        return ([metadata_xml_to_json(e) for e in data[0:num_revs]],
-                [results_xml_to_json(e) for e in data[num_revs:]])
+        metadata, results = ([metadata_xml_to_json(e) for e in data[0:num_revs]],
+                             [results_xml_to_json(e) for e in data[num_revs:]])
     else:
         git_objs = [tag + ':metadata.json' for tag in tags] + [tag + ':results.json' for tag in tags]
         data = parse_json_stream(repo.run_cmd(['show'] + git_objs + ['--']))
-        return data[0:num_revs], data[num_revs:]
+        metadata, results = data[0:num_revs], data[num_revs:]
+
+    # Filter on results with version matching the last one
+    valid_bitbake = metadata[-1]['bitbake']['commit']
+    valid_meta_yocto = metadata[-1]['layers']['meta-poky']['commit']
+    return zip(*[(m, r) for m, r in zip(metadata, results)
+                 if m['bitbake']['commit'] == valid_bitbake
+                 and m['layers']['meta-poky']['commit'] == valid_meta_yocto])
 
 
 def get_data_item(data, key):