diff mbox series

[layerindex-web,1/5] layerindex/views.py: fix StatsView statistics page timeout

Message ID 20260510004706.81282-1-tim.orling@konsulko.com
State New
Headers show
Series [layerindex-web,1/5] layerindex/views.py: fix StatsView statistics page timeout | expand

Commit Message

Tim Orling May 10, 2026, 12:46 a.m. UTC
The per-branch statistics query used a single ORM queryset with five
Count(..., distinct=True) annotations spanning deep reverse FK chains
(layerbranch__recipe, layerbranch__bbclass, etc.).  Django translates
this into one SQL query with multiple LEFT OUTER JOINs across all those
tables simultaneously.  At production data volumes (~541 layers, ~25k
recipes) the resulting query exhausts gunicorn worker memory/time,
causing a 504 Gateway Time-out.

Fix by iterating over each visible branch individually and issuing simple
COUNT queries per table.  This replaces one huge cross-join with NĂ—5
small indexed WHERE-clause queries that are far cheaper and independently
cacheable.

The template is unchanged; Django's template engine resolves dict keys
and object attributes identically with dot notation.

[YOCTO #15391]

AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 layerindex/views.py | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
diff mbox series

Patch

diff --git a/layerindex/views.py b/layerindex/views.py
index 3cf91d2..23bafd0 100644
--- a/layerindex/views.py
+++ b/layerindex/views.py
@@ -1522,12 +1522,23 @@  class StatsView(TemplateView):
         context['class_count_distinct'] = BBClass.objects.values('name').distinct().count()
         context['machine_count_distinct'] = Machine.objects.values('name').distinct().count()
         context['distro_count_distinct'] = Distro.objects.values('name').distinct().count()
-        context['perbranch'] = Branch.objects.filter(hidden=False).order_by('sort_priority').annotate(
-                layer_count=Count('layerbranch', distinct=True),
-                recipe_count=Count('layerbranch__recipe', distinct=True),
-                class_count=Count('layerbranch__bbclass', distinct=True),
-                machine_count=Count('layerbranch__machine', distinct=True),
-                distro_count=Count('layerbranch__distro', distinct=True))
+        # Compute per-branch counts with individual queries rather than a single
+        # annotated queryset. The multi-Count(distinct=True) approach generates one
+        # large SQL query with many LEFT JOINs that causes a gunicorn worker timeout
+        # at production data volumes. Simple per-branch COUNT queries are far cheaper.
+        # See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15391
+        perbranch = []
+        for branch in Branch.objects.filter(hidden=False).order_by('sort_priority'):
+            perbranch.append({
+                'name': branch.name,
+                'updates_enabled': branch.updates_enabled,
+                'layer_count': LayerBranch.objects.filter(branch=branch).count(),
+                'recipe_count': Recipe.objects.filter(layerbranch__branch=branch).count(),
+                'class_count': BBClass.objects.filter(layerbranch__branch=branch).count(),
+                'machine_count': Machine.objects.filter(layerbranch__branch=branch).count(),
+                'distro_count': Distro.objects.filter(layerbranch__branch=branch).count(),
+            })
+        context['perbranch'] = perbranch
         return context