From patchwork Sat Nov 14 02:26:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 104025 X-Patchwork-Delegate: ticotimo@gmail.com Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.4534.1794623204535245827 for ; Fri, 13 Nov 2026 18:26:44 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=Ig1s8qaE; spf=pass (domain: gmail.com, ip: 209.85.216.48, mailfrom: ticotimo@gmail.com) Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-3b08cf16decso853265a91.0 for ; Fri, 13 Nov 2026 18:26:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1794623204; x=1795228004; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=F5lstnEuvfYsTpRFzgKSL108vqmoOUd4PKLeqnxUOWw=; b=Ig1s8qaEli26CffO4WN0fJ2CHj9eq2Ofz5tzPG9WaHas7/4EKJ//iDZ0LX/ynGWRcm t3+eatpOCGJr0j2lfULXq8WsdPFK6QVcqghlAKxi+EKk3Jv0s2zS8iG/YIRSscoL/nIW 3wpLKd4hR4eIfYmJwgk3g743qtqwyCQAlUXFeS8+iVBlj2ZrJiVZaBrN0+tG4SKPmgeq 2EI93a35FKqhuPOgIA3WOcsPA5Pv6JEWgOqLjpMVoJfS+tygSeeyMCI0Jfkv5ylFV3JT De8O61zuH2Jw++TzJ9JvndBwPWnT2IeOCZ4bA2GGl7955rf4DNkbbxWOxLMxeznt0ytK Aa2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794623204; x=1795228004; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=F5lstnEuvfYsTpRFzgKSL108vqmoOUd4PKLeqnxUOWw=; b=UqmminUHQVu+/o2NxTvZt3cMOGw+uNjbx9fMnE6t5Kx1cULQ1/oxz0qo6XtsCaqABd q/UbzuqmQDsfoyUUWPX/Qcz67oXONt8po/dcVduZishmW/xpYEv7xF16EK0bRcuwO4tx nqGdqkk0JYBeZRkSl/+GWdJMK41hau7zmp8JOhvIpObnrlc8o9F6KaU8GHtHK82Ec7/S kD+mEkV8v9WKnU6YDCUP6JqejIJd5zwhwUEkTWYYrt9LpIbKb4u2/NYFbqvanZWYUYF0 YavImD9Tfl/pNbVWbWpn67g1E4Tt2RIp5cn8NuhVetcKQI8KrP0KHu3GwnHgG8hzk1eZ 3TRg== X-Gm-Message-State: APuU4Z/EYZ1QsmtOBxXh9Gh3m8zgBkrr0t0jbV4aHwHs5iYmxl4FgYHO riaVxlMROMJoOMpAWmTK/ibIH/b1NK5nSIQPN9yCnit3PBtmCNRq5zQRHG7Slw== X-Gm-Gg: AYBFou1rTwNBP5h05JMOkX19nRsDo6JlbdboRu5U/1fLN0zZqqr5FGfDXleifkk9lkl ykv2QKPT3q9I9hWhA4fD463zgJ4tW19J/w0EYKycs8h2aV1Kfa+uGcogzKo6hi2V8ilXqNBumqi sPYU+miUTJaW3+bohNkC6w0f1Ud+vfL2aE9HcY73pCwoU/5DFOPJU5mBP/FQ1djwK7Jl2YwwMje G5Vs3PwRxQrZbCdsvcVpu7xjmpfTHVnST8kosPp9+UK90tHLcdYILBz8W+LfCTu+FTI/j+lGyIG cJiCGJKBYR1uTikQWVPHqfu45hqSzwwqJMvgt1rFiQU3/sRRgC5yisiUetmd4lGwEHWrLlIniy6 CoSTytcUAAfEcUZMaoWi69X3CbqtJSzhnY8Q6ffyZ2fsy2U/jjH+m923FSo/h2Q36mjmnYyoQNJ 4/kKazbNR/BY/fcXrcpdE8CYTR2Loc0jqNSOUofmG0+DTxUDAyuA5jRsOg5Xmjr9YV0teBYhgE0 NC2w246paQIbiyciXG+FODTuXoOCVYvB/TvRKRKpTi9HEgZYryeQ/z96Moiw6aUVhnWEMn90nO+ 9G09FUOESM1GKu6fdDRNS+g4vHTlcl97nszV32O5W+3CozVJrc1jBbpawzdkTtvaNfbsVUoW+vZ 71/Ll7xc= X-Received: by 2002:a17:90b:5627:b0:3ab:17b6:9a93 with SMTP id 98e67ed59e1d1-3b1b0ba8ec8mr4452696a91.52.1794623203879; Fri, 13 Nov 2026 18:26:43 -0800 (PST) Return-Path: Received: from localhost.localdomain ([69.9.132.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3b1aec622b6sm7433636a91.11.2026.11.13.18.26.42 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 13 Nov 2026 18:26:42 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [layerindex-web][PATCH 1/3] update: don't print the initial layer.conf values Date: Fri, 13 Nov 2026 18:26:35 -0800 Message-ID: <20261114022637.60789-2-tim.orling@konsulko.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261114022637.60789-1-tim.orling@konsulko.com> References: <20261114022637.60789-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Sat, 14 Nov 2026 02:26:45 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/5357 update.py gets BBFILE_COLLECTIONS, LAYERVERSION, LAYERDEPENDS and LAYERRECOMMENDS for each layer by running update_layer.py --initial, which printed them, so they appeared on the console and in the update log of every layer. Have update_layer.py write them to a file that update.py names instead; errors are still shown as before. [YOCTO #12130] AI-Generated: Claude Opus 5.5 Signed-off-by: Tim Orling --- layerindex/update.py | 23 +++++++++++++++-------- layerindex/update_layer.py | 12 +++++++----- tests/test_update.py | 11 +++++++++++ 3 files changed, 33 insertions(+), 13 deletions(-) diff --git a/layerindex/update.py b/layerindex/update.py index ca68e3e..ccf44a3 100755 --- a/layerindex/update.py +++ b/layerindex/update.py @@ -16,6 +16,8 @@ import optparse import codecs import logging import subprocess +import shlex +import tempfile from datetime import datetime, timedelta from packaging_legacy.version import parse as parse_version import utils @@ -36,7 +38,7 @@ except ImportError: sys.exit(1) -def prepare_update_layer_command(options, branch, layer, initial=False): +def prepare_update_layer_command(options, branch, layer, initial=None): """Prepare the update_layer.py command line""" if branch.update_environment: cmdprefix = branch.update_environment.get_command() @@ -54,7 +56,7 @@ def prepare_update_layer_command(options, branch, layer, initial=False): if options.dryrun: cmd += ' -n' if initial: - cmd += ' -i' + cmd += ' -i %s' % shlex.quote(initial) if options.loglevel == logging.DEBUG: cmd += ' -d' elif options.loglevel == logging.ERROR: @@ -448,10 +450,15 @@ def main(): logger.error("conf/layer.conf not found for layer %s (branch %s) - is subdirectory set correctly?" % (layer.name, branch)) continue - cmd = prepare_update_layer_command(options, branchobj, layer, initial=True) + fd, initial_fn = tempfile.mkstemp(prefix='layerindex-initial-') + os.close(fd) + cmd = prepare_update_layer_command(options, branchobj, layer, initial=initial_fn) logger.debug('Running layer update command: %s' % cmd) ret, output = utils.run_command_interruptible(cmd) logger.debug('output: %s' % output) + with open(initial_fn) as f: + initial_values = f.read() + os.remove(initial_fn) if ret == 254: # Interrupted by user, break out of loop logger.info('Update interrupted, exiting') @@ -470,14 +477,14 @@ def main(): layerupdate.save() continue - col = extract_value('BBFILE_COLLECTIONS', output) + col = extract_value('BBFILE_COLLECTIONS', initial_values) if not col: - logger.error('Unable to find BBFILE_COLLECTIONS value in initial output') + logger.error('Unable to find BBFILE_COLLECTIONS value in initial values') # Assume (perhaps naively) that it's an error specific to the layer continue - ver = extract_value('LAYERVERSION', output) - deps = extract_value('LAYERDEPENDS', output) - recs = extract_value('LAYERRECOMMENDS', output) + ver = extract_value('LAYERVERSION', initial_values) + deps = extract_value('LAYERDEPENDS', initial_values) + recs = extract_value('LAYERRECOMMENDS', initial_values) if not options.nocheckout: # We need to check this out because we're using stuff from bb.utils diff --git a/layerindex/update_layer.py b/layerindex/update_layer.py index 0f49371..c11c002 100644 --- a/layerindex/update_layer.py +++ b/layerindex/update_layer.py @@ -290,8 +290,8 @@ def main(): help = "Stop on first parsing error", action="store_true", default=False, dest="stop_on_error") parser.add_option("-i", "--initial", - help = "Print initial values parsed from layer.conf only", - action="store_true") + help = "Write the initial values parsed from layer.conf to FILE, and do nothing else", + action="store", metavar="FILE") parser.add_option("-a", "--actual-branch", help = "Specify actual_branch for git checkout", action="store", dest="actual_branch", default=None) @@ -454,9 +454,11 @@ def main(): sys.exit(1) utils.set_layerbranch_collection_version(layerbranch, layer_config_data, logger=logger) if options.initial: - # Use print() rather than logger.info() since "-q" makes it print nothing. - for i in ["BBFILE_COLLECTIONS", "LAYERVERSION", "LAYERDEPENDS", "LAYERRECOMMENDS"]: - print('%s = "%s"' % (i, utils.get_layer_var(layer_config_data, i, logger))) + # Write these to a file rather than the output, which is + # shown on the console and in the update log + with open(options.initial, 'w') as f: + for i in ["BBFILE_COLLECTIONS", "LAYERVERSION", "LAYERDEPENDS", "LAYERRECOMMENDS"]: + f.write('%s = "%s"\n' % (i, utils.get_layer_var(layer_config_data, i, logger))) sys.exit(0) # Set up for recording patch info diff --git a/tests/test_update.py b/tests/test_update.py index a0ac897..1e1db0d 100644 --- a/tests/test_update.py +++ b/tests/test_update.py @@ -163,6 +163,17 @@ def test_add_recipe(import_layer, repo, db_access_without_rollback_and_truncate) assert recipe.summary == "Brand new recipe" assert recipe.license == "MIT" +def test_initial_values_not_in_output(import_layer): + # [YOCTO #12130] update.py gets the values it needs from the first parse + # of conf/layer.conf through a file, not from the output it shows + result = subprocess.run("layerindex/update.py -r -n -l meta-layerindex-test --nofetch --nocheckout", + shell=True, cwd=basepath, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, text=True) + assert result.returncode == 0, result.stdout + assert 'BBFILE_COLLECTIONS = ' not in result.stdout + assert 'Unable to find BBFILE_COLLECTIONS' not in result.stdout + + # FIXME test recipe modify From patchwork Sat Nov 14 02:26:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 104026 X-Patchwork-Delegate: ticotimo@gmail.com Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.4397.1794623207737837535 for ; Fri, 13 Nov 2026 18:26:47 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=nJ0wrK0K; spf=pass (domain: gmail.com, ip: 209.85.215.176, mailfrom: ticotimo@gmail.com) Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-cdac6d19ddaso775757a12.2 for ; Fri, 13 Nov 2026 18:26:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1794623207; x=1795228007; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=lxpxJhmrTPKiUsWn+1bZKE9C8zKMHvbWmVyClj+3v6M=; b=nJ0wrK0KhKGW+/XNQcukuuRadLDJXxLdup9/hU0goQqbbsbaoaLgncrvo+ZQ1QtUsk oDhFMdWyv6IAH9ghxyanPmbFzaoDpks5sk+DBLBlM0qAC656H2NpQzTtKhPYF6v19xmz zpS4M4NHhPkRh8JY1LwYp+dUK1kV0FEGy4j3QbhP+/ktg/b5XK6p23v+DSWOMAfMscTC ZGmp3uDVh6UCkpaTV1Qz6oELela5WwbG1qIqafdvi/LVIIytSUHhqdfVGiaflSk2uwV0 Rp9cE49MJGG69DxIhdMiXEnWhCd8cbW4Ksc384wx+wR5SE7T7vW+AoNBYs4o6XBUPel+ KGyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794623207; x=1795228007; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=lxpxJhmrTPKiUsWn+1bZKE9C8zKMHvbWmVyClj+3v6M=; b=FW9kjstwUdZQoJI9s6wX5EJAqILD0Wia1tbCXrtPe7dyhU/8AxL/OoqaFiuHTZaGwg 9OnpD7jIAhRDN14YIqWjrvxG38SbVT44KN1EfDrz6o6TU33ABh/2R68zbrM3xTayH/UO IflU3PtleJnEVjGyw98Fq1tL0ejPYri0oJfDKDtCYVfXzs/iQtx58py12wnFv7XEzgDR wts4QXSIySqqJwERcE5G/peF1Y+gO33SzHPiM1qY8bwU6P6OcBSPIyF4scTJs3Ct9iMQ 0AVCY40SgDEimBUaO1Ngsr3dGtyit/Y2K08bWWNWsGCWPBeu26uyML3C65yhrIk0LcMV G/ww== X-Gm-Message-State: APuU4Z/GHWyAdWaUpnHO9DSqE5OczYF8CKJDhlaKvJfN9a+C/19mU6ej poJbg87kcqGT6wKovREF3sHkq64/mIeMO80GIMA4CPyjuLfoBmU4C8GKINX93iK8 X-Gm-Gg: AYBFou3YKTs/RUpKUZkfqNSVdMOLrFZ7whfRXlDGBJtxFkLXP3T864CDKAeUeP67i5D jtu6uVOWuaACF4yu3pmCrW5L4XBwkpgoslJGQHY6V8dlqTNCEU/X5lxhrnLkERfaKmI5pbb/Hex utTKu3l60m8shKet6BE+CmZgnC4OgS3cyPhIXa/cKtDPKekrbtyr5trIvGNS1l5qeLnY8ypTp6L Eln7+Na5HH06obNR4YuwDJnULHt9JnNdSDSR+uciR2bF5WAQlEj99IjAh4BOIY07Sl9tegcDYfJ OlMQ6318/8NKnEIAEsl3LU+PnRy5Gh8ZvNKqW0VSqHSknw3iE4zD3sMZBgiHnd1xKNJFAzvM/3P rKWN04Yb7N5wRcU+G0V66BW3233PpBxrSDjgMwubNZ1ef4I4loZ6NGOFt/T3XOvhjz7xKmjMb/k xKbFLMZCwn9T+G1WtJ8/VmqCdDeUGy3DCsI5awdtg/lvpv80eRFIdzWi3oFetIdpzLVHnfyfuWZ Ud5SQmnQrLDqPwfe1PANhpUcbemREKNtwQrk1erah9AlVfFyf8qUw/f9dNpCLR8R2Gj+Sg7rKwE 8Ab3HTqbMr5R0fHp/fh08OBf9Hz4dHqYqYzHtFANUlpqVk43yYPAjNkNOc+iabbGaudOXqFQIux j94IBqBo= X-Received: by 2002:a17:90b:1650:b0:3ab:3437:554f with SMTP id 98e67ed59e1d1-3b1b0a0db3bmr4354767a91.29.1794623206866; Fri, 13 Nov 2026 18:26:46 -0800 (PST) Return-Path: Received: from localhost.localdomain ([69.9.132.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3b1aec622b6sm7433636a91.11.2026.11.13.18.26.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 13 Nov 2026 18:26:45 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [layerindex-web][PATCH 2/3] update: record errors in a layer's first parse against the layer Date: Fri, 13 Nov 2026 18:26:36 -0800 Message-ID: <20261114022637.60789-3-tim.orling@konsulko.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261114022637.60789-1-tim.orling@konsulko.com> References: <20261114022637.60789-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Sat, 14 Nov 2026 02:26:54 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/5358 When update.py could not parse a layer's conf/layer.conf in its first pass (fetch failed, subdirectory or conf/layer.conf missing, no BBFILE_COLLECTIONS, LAYERDEPENDS not satisfied), it only logged the error in the overall update log, so nothing showed in the layer's own update history. Record these as layer updates with the error, as is already done when update_layer.py itself fails. [YOCTO #13490] AI-Generated: Claude Opus 5.5 Signed-off-by: Tim Orling --- layerindex/update.py | 40 +++++++++++++++++++++++++++++++--------- tests/test_update.py | 23 +++++++++++++++++++++++ 2 files changed, 54 insertions(+), 9 deletions(-) diff --git a/layerindex/update.py b/layerindex/update.py index ccf44a3..dd794dd 100755 --- a/layerindex/update.py +++ b/layerindex/update.py @@ -124,10 +124,12 @@ def print_subdir_error(newbranch, layername, vcs_subdir, branchdesc): # This will error out if the directory is completely invalid or had never existed at this point # If it previously existed but has since been deleted, you will get the revision where it was # deleted - so we need to handle that case separately later + # Returns the error, if it is one, for the caller to record if newbranch: logger.info("Skipping update of layer %s for branch %s - subdirectory %s does not exist on this branch" % (layername, branchdesc, vcs_subdir)) elif vcs_subdir: - logger.error("Subdirectory for layer %s does not exist on branch %s - if this is legitimate, the layer branch record should be deleted" % (layername, branchdesc)) + return "Subdirectory for layer %s does not exist on branch %s - if this is legitimate, the layer branch record should be deleted" % (layername, branchdesc) + return None def extract_value(valuename, output): res = re.search("^%s = \"(.*)\"" % valuename, output, re.M) @@ -277,6 +279,18 @@ def main(): update.reload = False if not options.dryrun: update.save() + + def record_layer_error(layer, branchobj, msg, log=True): + """Record an error that stops a layer being updated against the layer, + so that it shows in its update history, not only in this update's log""" + if log: + logger.error(msg) + layerupdate = LayerUpdate(update=update, layer=layer, branch=branchobj, + started=datetime.now(), finished=datetime.now(), + log='ERROR: %s' % msg, retcode=1) + if not options.dryrun: + layerupdate.save() + try: lockfn = os.path.join(fetchdir, "layerindex.lock") lockfile = utils.lock_file(lockfn, options.timeout, logger) @@ -363,7 +377,8 @@ def main(): for layer in layerquery: if layer.vcs_url in failedrepos: - logger.info("Skipping update of layer %s - fetch failed" % layer.name) + # fetch_repo() has already logged the error + record_layer_error(layer, branchobj, "Fetch of layer %s failed: %s" % (layer.name, failedrepos[layer.vcs_url]), log=False) continue layerbranch = layer.get_layerbranch(branch) @@ -400,7 +415,7 @@ def main(): repodir = os.path.join(fetchdir, urldir) repo = git.Repo(repodir) if repo.bare: - logger.error('Repository %s is bare, not supported' % repodir) + record_layer_error(layer, branchobj, 'Repository %s is bare, not supported' % repodir) continue try: # Allow stable branches to be created if actual_branch exists @@ -425,9 +440,11 @@ def main(): # A bit odd to do it this way but apparently there's no other way in the GitPython API topcommit = next(repo.iter_commits('origin/%s' % branchname, paths=layerbranch.vcs_subdir), None) if not topcommit: - print_subdir_error(newbranch, layer.name, layerbranch.vcs_subdir, branchdesc) + errmsg = print_subdir_error(newbranch, layer.name, layerbranch.vcs_subdir, branchdesc) + if errmsg: + record_layer_error(layer, branchobj, errmsg) if not (newbranch and layerbranch.vcs_subdir): - logger.error("Failed to get last revision for layer %s on branch %s" % (layer.name, branchdesc)) + record_layer_error(layer, branchobj, "Failed to get last revision for layer %s on branch %s" % (layer.name, branchdesc)) continue if layerbranch.vcs_last_rev == topcommit.hexsha and not update.reload: @@ -443,11 +460,13 @@ def main(): utils.checkout_layer_branch(layerbranch, repodir, actual_branch=options.actual_branch, logger=logger) layerdir = os.path.join(repodir, layerbranch.vcs_subdir) if layerbranch.vcs_subdir and not os.path.exists(layerdir): - print_subdir_error(newbranch, layer.name, layerbranch.vcs_subdir, branchdesc) + errmsg = print_subdir_error(newbranch, layer.name, layerbranch.vcs_subdir, branchdesc) + if errmsg: + record_layer_error(layer, branchobj, errmsg) continue if not os.path.exists(os.path.join(layerdir, 'conf/layer.conf')): - logger.error("conf/layer.conf not found for layer %s (branch %s) - is subdirectory set correctly?" % (layer.name, branch)) + record_layer_error(layer, branchobj, "conf/layer.conf not found for layer %s (branch %s) - is subdirectory set correctly?" % (layer.name, branch)) continue fd, initial_fn = tempfile.mkstemp(prefix='layerindex-initial-') @@ -479,7 +498,7 @@ def main(): col = extract_value('BBFILE_COLLECTIONS', initial_values) if not col: - logger.error('Unable to find BBFILE_COLLECTIONS value in initial values') + record_layer_error(layer, branchobj, 'Unable to find BBFILE_COLLECTIONS value for layer %s - is it set in conf/layer.conf?' % layer.name) # Assume (perhaps naively) that it's an error specific to the layer continue ver = extract_value('LAYERVERSION', initial_values) @@ -548,7 +567,10 @@ def main(): for layer, value in deps_dict_all.items(): logger.warn('%s: LAYERDEPENDS: %s LAYERRECOMMENDS: %s' % (layer.name, value['deps'], value['recs'])) if value['deps']: - failed_layers[branch].append('%s: Failed to add since LAYERDEPENDS [%s ...] is not satisfied' % (layer.name, next(iter(value['deps'])))) + failed_msg = '%s: Failed to add since LAYERDEPENDS [%s ...] is not satisfied' % (layer.name, next(iter(value['deps']))) + failed_layers[branch].append(failed_msg) + # Reported with the other issues at the end + record_layer_error(layer, branchobj, failed_msg, log=False) else: # Should never come here logger.error("Unexpected errors when sorting layers") diff --git a/tests/test_update.py b/tests/test_update.py index 1e1db0d..3266d44 100644 --- a/tests/test_update.py +++ b/tests/test_update.py @@ -173,6 +173,29 @@ def test_initial_values_not_in_output(import_layer): assert 'BBFILE_COLLECTIONS = ' not in result.stdout assert 'Unable to find BBFILE_COLLECTIONS' not in result.stdout +def set_layerbranch(subdir, last_rev): + # In a separate process, so that update.py sees the change: the tests run + # in a transaction + run_cmd('python3 manage.py shell -c "from layerindex.models import LayerBranch; ' + 'LayerBranch.objects.filter(layer__name=\'meta-layerindex-test\').update(' + 'vcs_subdir=\'%s\', vcs_last_rev=\'%s\')"' % (subdir, last_rev), cwd=basepath) + +def test_first_parse_error_recorded_against_layer(import_layer, repo, db_access_without_rollback_and_truncate): + # [YOCTO #13490] an error that stops a layer being parsed is recorded + # in the layer's update history, not only in the overall update log + from layerindex.models import LayerBranch, LayerUpdate + layerbranch = LayerBranch.objects.get(layer__name='meta-layerindex-test') + set_layerbranch('meta-layerindex-test/recipes-example', '') + try: + result = subprocess.run("layerindex/update.py -l meta-layerindex-test --nofetch --nocheckout", + shell=True, cwd=basepath, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, text=True) + layerupdate = LayerUpdate.objects.filter(layer=layerbranch.layer).order_by('-id').first() + assert 'conf/layer.conf not found for layer meta-layerindex-test' in layerupdate.log, result.stdout + assert layerupdate.errors == 1 + finally: + set_layerbranch(layerbranch.vcs_subdir, layerbranch.vcs_last_rev) + From patchwork Sat Nov 14 02:26:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 104027 X-Patchwork-Delegate: ticotimo@gmail.com Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.4398.1794623210130432502 for ; Fri, 13 Nov 2026 18:26:50 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=VtZyDzyF; spf=pass (domain: gmail.com, ip: 209.85.216.50, mailfrom: ticotimo@gmail.com) Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-3b0b3754f57so1268824a91.0 for ; Fri, 13 Nov 2026 18:26:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1794623209; x=1795228009; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=N2NlTLTGa4I6/VNdZ6lRUoV4eve3rX5drX/bFg/U1lw=; b=VtZyDzyFkmQFP7pP/jYHpXPJoAYO9Hb/1b1GaJFVNpplm/QqLeAyMc6ICQu1TA7VW0 5cu5Mfr0y/r2r1XMy88bJcTk2isac2besau67ngxLI4VcT06c5GRu4gry+wMASorJktt aAsfhniEATkVn7uc6tBbM8JCiN9KlbeRoaNxE5HbsSUty6LyYssKREb9IHmGUhz599oO gbAs/p0WWsHxqAgDolq/GYDnlfUtQkhdapgCuTmJc70AWUQI9t+IrasZXqQ71URYoEsE qfyO6nbQVtcIF1X1GvAS3EN6/b7eSR/6FPaw/fdm/+hCPBm5qV983amoOdvan6nK4M/A gphQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794623209; x=1795228009; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=N2NlTLTGa4I6/VNdZ6lRUoV4eve3rX5drX/bFg/U1lw=; b=k5spgBzWram+9N964g6UWmd5cii9xsuVJ+EZgckXwPRhGk/O0KaxVK2FeonsEz10al csObiYIUYRuQjIgP3n5nE3behJJUduo8JdpSmwf3wujJt2c7qHDX8ZkYnzj2H41ZgAFw 03ZXa2qhwhzmhOk04GKcLXlfghakcxZcltGN5hgeXBsEeYoCMPV43u5jJG5GLwSts19P 3hoMJbVrpLKgi/rPTFxknr8sonQEWPNliMhFRoH2HOZAr04srKbSVzrfRQlGTbfJl3tx cuauiZV87LTZNsZ3+ZXL0lmEWAC9hY4mgZeU4i460+FrpuepRe+KZ83sqAfMk6Vs33zX Wcmw== X-Gm-Message-State: APuU4Z9i0sq3qvtcJauchFlJG8x9igAiLDcJCAk9nDNCC+R2bvSioqk5 09DF81Sp8zCMhHPWboIfd6A+FiisNYk6GFkoYY5j6LTG/Q8asVYVwxJ+7vF4VnaO X-Gm-Gg: AYBFou1bwY/RwDK13LdGe73bWgWIaIm5AJPZbfGhpUGOZdRK+83IBTlEEpio3MuorJC +wgo2AUFul7WT3QZMy9H9notJM7bpR067oucRbtrp9rb5N1uPN9dSZjsf0WVbiookbs7acHUf19 HUFfwUc1uWLxi6tZEmsD+lwIQiI9r1+3Upc+uKquESqdsPvunxFDrRcX0lV2QyQ9m1GAwJ7aBXo 1RND5Ho9dARcPz6eUZBjjWly2VqTLbHhcHDzSxL81t7pFpWv6qPOdtzYsgS8LZuvAYKoHJLUDd5 JSFbrhvVoXJnGjCijW6zRMACO258aCemixudztwuKmX4RL2tTumIiGzbe3sm/oTWlcDsez6l4Fx KsAtP7QQFXPyETbhmczD9BhtxsO6SOgbHMuHaZhWqm1Jp5+9wqDNSRyjDGlMATrwpLxZjP8dRHv Wm0qCMpm+s74s2CSzbU99nRC0Nb+fFNGZGG2ENV2miECcjoybnEtfmpHZs7YWxDr5qrnsrrGu5g fWBanUQZTw6/AzAhaSSQix3JkC8ZhHcXjDJ2coBQUL+mV6WcSE9d6RQaB1TRQWvsvVbeH4Sjub5 em6gNDurLN80d0btBQvMB9h3cv802W9O+mu2zl87NOzrVSadw6Hbi607lpct/w== X-Received: by 2002:a17:90b:4d0d:b0:3ad:d4aa:8559 with SMTP id 98e67ed59e1d1-3b1b09f37d9mr3777275a91.17.1794623209438; Fri, 13 Nov 2026 18:26:49 -0800 (PST) Return-Path: Received: from localhost.localdomain ([69.9.132.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3b1aec622b6sm7433636a91.11.2026.11.13.18.26.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 13 Nov 2026 18:26:48 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto-patches@lists.yoctoproject.org Cc: Tim Orling Subject: [layerindex-web][PATCH 3/3] templates: warn on the layer page when the first parse failed Date: Fri, 13 Nov 2026 18:26:37 -0800 Message-ID: <20261114022637.60789-4-tim.orling@konsulko.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261114022637.60789-1-tim.orling@konsulko.com> References: <20261114022637.60789-1-tim.orling@konsulko.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Sat, 14 Nov 2026 02:26:55 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/5359 The page of a layer branch that has not been indexed always said that indexing should happen within a few hours, even when the update had already failed, e.g. because of a bad conf/layer.conf. Show a warning with a link to the failed update's log instead. [YOCTO #13489] AI-Generated: Claude Opus 5.5 Signed-off-by: Tim Orling --- templates/layerindex/detail.html | 8 ++++++ tests/test_first_parse_status.py | 49 ++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 tests/test_first_parse_status.py diff --git a/templates/layerindex/detail.html b/templates/layerindex/detail.html index 6c20f47..cc86ff3 100644 --- a/templates/layerindex/detail.html +++ b/templates/layerindex/detail.html @@ -193,10 +193,18 @@ {% if not layerbranch.vcs_last_commit %} + {% with lastupdate=updates.first %} + {% if lastupdate.errors %} +
+ Warning: The content of this layer on branch {{ url_branch }} could not be indexed: the last update, on {{ lastupdate.started }}, failed. See its log for the errors. +
+ {% else %}
Note: The content of this layer on branch {{ url_branch }} has not yet been indexed. Indexing should take place within a few hours.
{% endif %} + {% endwith %} + {% endif %}