From patchwork Sat Aug 15 13:46:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1536FC5DF74 for ; Sat, 15 Aug 2026 13:47:41 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9195.1786801649527364720 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=k3p90txv; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-202608151347264c54d623e700020780-3khgmh@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 202608151347264c54d623e700020780 for ; Sat, 15 Aug 2026 15:47:26 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=QHLo+BKxXu33FplVcx1AVuUamysT79lwxjYsH68bsdc=; b=k3p90txvjTzxECdYOaQQF9gODIssJcPj9mHw2UFEZ96Xkwm9tOHTh5UK2a79Y1yCsTZ8r5 JjZlT9FmNymOqptZSQX49mwFREOYtI3ZircZ5ZQBHyayRM9srfJ0+TeNpmJV0rVXhjIPI5Gn AlwMNlpKIZ1yneLWAIGS14OLbtVW/hz4i9QrmZJ7dONQd6xVjsY5SV9kKsTVkKlRea5BZ32A PYWCNPBloUWA4EqkLb0p1RragsGg5071rR4C1GC/pP81P73aS6Gjr7fmkOGuCqxNpxdudn9H wn54PC2lIknHLqqSnY3lQy9oHO3KFJ0rB/IdY2wl2qGYIMVqe6taFIZw==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 1/7] cooker: fix bitbake -b silently ignoring bbappends Date: Sat, 15 Aug 2026 15:46:52 +0200 Message-ID: <20260815134722.497586-2-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:41 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19953 From: Adrian Freihofer "bitbake -b " builds the recipe without applying any of its .bbappend files. buildFileInternal() resolves appends via self.collections[mc].get_file_appends(fn), but self.collections[mc] is only ever filled in by collect_bbfiles(), called from updateCache() - a path -b deliberately skips. matchFiles(), the one -b-path function that does call collect_bbfiles(), built a fresh CookerCollectFiles into a throwaway local instead of self.collections[mc], so the append list stayed empty (or, on a memory-resident server, stale from the last full parse - e.g. missing a devtool/externalsrc workspace .bbappend added since). Nothing warns that the built metadata differs from disk. Make matchFiles() refresh self.collections[mc] itself so the later append lookup for the same fn sees the same fresh collection. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/cooker.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 4b6ba3196..108551a60 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1322,8 +1322,10 @@ You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in si if bf.startswith("/") or bf.startswith("../"): bf = os.path.abspath(bf) - collections = {mc: CookerCollectFiles(self.bbfile_config_priorities, mc)} - filelist, masked, searchdirs = collections[mc].collect_bbfiles(self.databuilder.mcdata[mc], self.databuilder.mcdata[mc]) + # The only place the "bitbake -b" path fills in the bbappends which + # buildFileInternal() then reads back from self.collections[mc]. + self.collections[mc] = CookerCollectFiles(self.bbfile_config_priorities, mc) + filelist, masked, searchdirs = self.collections[mc].collect_bbfiles(self.databuilder.mcdata[mc], self.databuilder.mcdata[mc]) try: os.stat(bf) bf = os.path.abspath(bf) From patchwork Sat Aug 15 13:46:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95426 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 041D1C5DF75 for ; Sat, 15 Aug 2026 13:47:41 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9198.1786801649527888987 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=QpL3cbpL; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-20260815134726beef1a040d00020773-biozip@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20260815134726beef1a040d00020773 for ; Sat, 15 Aug 2026 15:47:27 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=eDAW/NRXcnVWIpAut/yBjXCarUXs9aJPm7pLPDIt1nI=; b=QpL3cbpL/Z3HVZOlKH5ytcu8N3o6TSlelE2gAULJNNxWB6jdG2puD0BgIiNmsGqBhNVEM+ AcuX4Xr4ZDJ9YRWHTPwukDi3edmTQ4mkyZt82MR1rOj4psIqIkjYBi1kDupme2pD4ziUPJOI MQ5Izd2LVcv4FKqvdOBtOHziMUXry2xhGfvK9Q6Uy3K1ZCXl4yWNM7LJPKWAgP5Dm6N/61O+ 8K1HrcbdrayXw+bYC1Vu6EKYE0gZw9XNts4tKWy+838g8XW/IB093qkw+VqSg3wG52eEaU+s h2qusHERWkCJ1Z0SE4LGgVz3Bslusr6AXFDYRnyh2oNjmPFXXyTpiwsQ==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 2/7] tests/cooker: add a bitbake -b bbappend test Date: Sat, 15 Aug 2026 15:46:53 +0200 Message-ID: <20260815134722.497586-3-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:41 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19949 From: Adrian Freihofer The buildfile ("bitbake -b") mode had no test coverage at all. Add BuildFileTest.test_buildfile_applies_bbappends(), which writes a recipe plus a matching .bbappend to a temporary directory, points EXTRA_BBFILES at both and runs a task recording a variable the bbappend overrides. It uses the parse-tests BBPATH because that bitbake.conf already globs *.bbappend. Without the preceding fix the recipe's own default is recorded, i.e. -b dropped the bbappend silently. If the previous commit is reverted, this test faila with: AssertionError: 'no-bbappend' != 'bbappend-applied' - no-bbappend + bbappend-applied AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/tests/cooker.py | 52 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/lib/bb/tests/cooker.py b/lib/bb/tests/cooker.py index 9e524ae34..76ec65540 100644 --- a/lib/bb/tests/cooker.py +++ b/lib/bb/tests/cooker.py @@ -8,6 +8,8 @@ import unittest import os +import subprocess +import tempfile import bb, bb.cooker import re import logging @@ -69,3 +71,53 @@ class CookerTest(unittest.TestCase): expected = [] self.assertEqual(log_handler.logdata, expected) + + +class BuildFileTest(unittest.TestCase): + """Tests for the buildfile ("bitbake -b") mode.""" + + # parse-tests BBPATH: minimal bitbake.conf whose BBFILES honours + # EXTRA_BBFILES and already includes *.bbappend + _parsetests = os.path.realpath(os.path.join(os.path.dirname(__file__), "parse-tests")) + + recipe = """\ +MARKER ??= "no-bbappend" +python do_marker() { + with open(d.expand("${TOPDIR}/marker.log"), "w") as f: + f.write(d.getVar("MARKER")) +} +addtask marker +""" + + bbappend = 'MARKER = "bbappend-applied"\n' + + def test_buildfile_applies_bbappends(self): + """bitbake -b must build the recipe with its bbappends applied. + + buildFileInternal() looks the appends up in self.collections[mc], which + on the -b path is only ever populated by matchFiles(). + """ + with tempfile.TemporaryDirectory(prefix="buildfilerecipes") as recipes, \ + tempfile.TemporaryDirectory(prefix="buildfiletest") as builddir: + recipe = os.path.join(recipes, "appendtest.bb") + with open(recipe, "w") as f: + f.write(self.recipe) + with open(os.path.join(recipes, "appendtest.bbappend"), "w") as f: + f.write(self.bbappend) + + env = os.environ.copy() + env["BBPATH"] = self._parsetests + env["BB_ENV_PASSTHROUGH_ADDITIONS"] = "TOPDIR EXTRA_BBFILES" + env["TOPDIR"] = builddir + env["EXTRA_BBFILES"] = "%s/*.bb %s/*.bbappend" % (recipes, recipes) + + cmd = ["bitbake", "-b", recipe, "-c", "marker"] + try: + subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT, + universal_newlines=True, cwd=builddir) + except subprocess.CalledProcessError as e: + self.fail("Command %s failed with %s" % (cmd, e.output)) + + with open(os.path.join(builddir, "marker.log")) as f: + self.assertEqual(f.read(), "bbappend-applied", + "bitbake -b did not apply the recipe's bbappend") From patchwork Sat Aug 15 13:46:54 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97272C5DF6E for ; Sat, 15 Aug 2026 13:47:40 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9199.1786801649528025338 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=XMuY+nbG; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-20260815134726403ac30869000207dc-txpyuo@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 20260815134726403ac30869000207dc for ; Sat, 15 Aug 2026 15:47:26 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=Z80GARX2coisDMT58FFgCiS2Mdk+Hz2R7++QHzW0Fzs=; b=XMuY+nbG9kJHuv7H8ANfch5MsT1X2jTjGNCPO9sy6POXZZxV0RhUWtGRvj8c4lWIL+L4d3 MdjqDim0/QnKpiWFTYay6uc7z5TPz9dHOJUAiK/RS+AysYXn+eV1Lai9YO/q/nuutM3yfeYS LX7Zs9F/t4heYMKQV6w+MxEH5am95HQEGV+H7Rmwv/+cxgzrIueq+0FQ/5LRRoGU7QaApUEh d/J3kDrIgc6slT94gsS+vt+crgIUAQJYzwhMD9z15OHt8TFcBqURSBZaXO6LstwAlaFkH49z WZPnD2AGlbXM9S1tuNZAt7RVqJjNjNRW6hClDhXmHzV2vn2kG6L8XgNg==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 3/7] command: fix setConfig coercing bool config values to truthy strings Date: Sat, 15 Aug 2026 15:46:54 +0200 Message-ID: <20260815134722.497586-4-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:40 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19954 From: Adrian Freihofer CommandsSync.setConfig() unconditionally stringified the value with str(params[1]) before assigning it to the cooker configuration attribute. This breaks boolean values, since str(True) and str(False) are both non-empty and therefore both truthy. That makes it impossible to turn a boolean option back off over the command interface: setting 'force' to False leaves configuration.force holding the truthy string "False", so it stays effectively enabled for the rest of the bitbake server session and spuriously invalidates tasks in later, unrelated builds sharing that session. Preserve the caller's original type instead of coercing to str. The only other caller (cookerdata.py) already passes a plain string, so this does not change behavior for it. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/command.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index 59a979ee9..b57c5d4a3 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -228,7 +228,7 @@ class CommandsSync: Set the value of variable in configuration """ varname = params[0] - value = str(params[1]) + value = params[1] setattr(command.cooker.configuration, varname, value) def enableDataTracking(self, command, params): From patchwork Sat Aug 15 13:46:55 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 752DEC5DF6A for ; Sat, 15 Aug 2026 13:47:39 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9197.1786801649527839236 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=MJ4AuCW2; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-20260815134726018326367c000207a1-5ncxqi@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 20260815134726018326367c000207a1 for ; Sat, 15 Aug 2026 15:47:26 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=OEv6tFJRl4U/kYGr7+7L3KMeFAhUjQDCluBqS8sTTU0=; b=MJ4AuCW2UpB41nZXU6DXhIZkwzNogafHlfGcma1sg8bMne6rotBfGbPiXmM4/zHYCjoacv OXztL5YR/VbXLAZR9vwPW9adtZV1E15wcQANd7/9eeNJNa15bvNajsA7AL3Qn8JiJY/CBnnV LycdChJtXZcIPcR/5a1hEZcOD9ZXFbjsWs/it/oRkAl658vARdwCF51NNV1iNahxBsagUdFj XeShj/vshSXoCLVGfBOaGI4Dn1Ku6B+8Z1MZlZpUamKsobzji4ZRFxgvgkeleuHLFj5Zz6Sb wi/6SJgnXdV3V+oEQ5+lwcqJVwYKvmdOCfUEpJRfRhb7v2QkymJBfVPQ==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 4/7] cooker: add a buildFile mode that runs a single task Date: Sat, 15 Aug 2026 15:46:55 +0200 Message-ID: <20260815134722.497586-5-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19955 From: Adrian Freihofer devtool ide-sdk lets the IDE build directly (e.g. via cmake or meson, outside of bitbake). do_install still needs bitbake: it needs pseudo, and it usually does more than the underlying build tool's own install step (e.g. `cmake --build --target install`) would - packaging-related fixups the recipe or its classes add on top. It must run as only that one already-prepared task against the source the developer just edited - not the whole recipe: predecessors, if they ran again, would rebuild/overwrite the very output the caller is about to inspect or has already staged. Add a taskonly argument to buildFileInternal(), plumbed through the buildFile command, that also clears the intra-recipe task parents, leaving the requested task as the runqueue's only entry. Default unchanged, so "bitbake -b" behaviour is unaffected. Note that for example: - buildFileInternal() cannot do this: it drops external dependencies but keeps intra-recipe task ordering ('addtask X after Y'), so requesting do_install on an unbuilt recipe still pulls in do_fetch, do_unpack, do_patch, do_prepare_recipe_sysroot, do_configure and do_compile. That's the right default for "bitbake -b", but not here. - --runonly can't express this either: mark_active() ignores its depth argument and recurses over depends, so the whole chain stays active regardless. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/command.py | 6 +++++- lib/bb/cooker.py | 8 +++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index b57c5d4a3..1b16884fa 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -622,9 +622,13 @@ class CommandsAsync: internal = params[2] else: internal = False + if len(params) > 3: + taskonly = params[3] + else: + taskonly = False if internal: - command.cooker.buildFileInternal(bfile, task, fireevents=False, quietlog=True) + command.cooker.buildFileInternal(bfile, task, fireevents=False, quietlog=True, taskonly=taskonly) else: command.cooker.buildFile(bfile, task) buildFile.needcache = False diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 108551a60..0321478d3 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1368,7 +1368,7 @@ You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in si self.buildFileInternal(buildfile, task) - def buildFileInternal(self, buildfile, task, fireevents=True, quietlog=False): + def buildFileInternal(self, buildfile, task, fireevents=True, quietlog=False, taskonly=False): """ Build the file matching regexp buildfile """ @@ -1418,6 +1418,12 @@ You can also remove the BB_HASHSERVE_UPSTREAM setting, but this may result in si self.recipecaches[mc].rundeps[fn] = defaultdict(list) self.recipecaches[mc].runrecs[fn] = defaultdict(list) + if taskonly: + # Drop the intra-recipe task ordering too ('addtask X after Y'), so + # that task is the only entry left in the runqueue. + task_deps = self.recipecaches[mc].task_deps[fn] + task_deps['parents'] = {t: [] for t in task_deps['tasks']} + bb.parse.siggen.setup_datacache(self.recipecaches) # Invalidate task for target if force mode active From patchwork Sat Aug 15 13:46:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6904EC5DF6D for ; Sat, 15 Aug 2026 13:47:39 +0000 (UTC) Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net [185.136.64.228]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9196.1786801649527771685 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=IwSyQLpN; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.228, mailfrom: fm-1329275-202608151347263f03302b9b00020758-waksgz@rts-flowmailer.siemens.com) Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 202608151347263f03302b9b00020758 for ; Sat, 15 Aug 2026 15:47:26 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=EJa+rFJHd+b8IRBuxWlquBJ1y7PKeOSmUlQyuNmQofs=; b=IwSyQLpNjiHLmjzGdFcTbCtklEizkLap/R30AyZUj0uPiF+32jfKmUxQyB2cMP8/RQq0ti xh7jZN4sYOUXkGMZUaE2BW4KUoaCOWH0jTppcUKZ/JfbQULgvGqiiy/j9TPkkt8rQMQ1R0wU pfPdweudez+jMdCqsy2MpDxo7uGxBsrqETfjG6plTQE59gxS/BcZ1W7Ct+HOLGUKz5Ja/kCu 6ZrW/fys3Br8cA7fVcYK/3tpJoUBrGTvXqduOP1QmnaJvjYxtIi3Pe8hQ6YRtFK25q/UWbAt jDbADFALeeQgT3AiHZpAG7rHZjPXI4HpI2jwixuVkRGob5pBxvNeL+DA==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 5/7] tinfoil: add a prepared task runner Date: Sat, 15 Aug 2026 15:46:56 +0200 Message-ID: <20260815134722.497586-6-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19950 From: Adrian Freihofer Add run_prepared_task() to execute a single recipe task, and nothing else. No dependencies are resolved and no other task of the recipe is run, so everything the task consumes must already be in place. It uses BitBake's normal worker path, retaining the standard dispatch for shell and Python task bodies as well as fakeroot setup. devtool ide-sdk needs to rerun an already prepared BitBake task after a developer changes its source. A normal target build would create a runqueue and resolve dependencies again, and even "bitbake -b" would re-run the task's intra-recipe predecessors. That is wrong for this workflow: the developer intentionally wants to rerun only the prepared task against content the IDE has just produced. Expose the single-task worker path so devtool can reuse it instead of maintaining a separate task executor and pseudo-session setup. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/tinfoil.py | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/lib/bb/tinfoil.py b/lib/bb/tinfoil.py index 634d7796f..42385c879 100644 --- a/lib/bb/tinfoil.py +++ b/lib/bb/tinfoil.py @@ -826,20 +826,39 @@ class Tinfoil: else: return None - def build_file(self, buildfile, task, internal=True): + def build_file(self, buildfile, task, internal=True, taskonly=False): """ Runs the specified task for just a single recipe (i.e. no dependencies). This is equivalent to bitbake -b, except with the default internal=True no warning about dependencies will be produced, normal info messages from the runqueue will be silenced and BuildInit, BuildStarted and BuildCompleted events will not be fired. + With taskonly=True the recipe's own task ordering is dropped as well, so + only the requested task runs. """ - return self.run_command('buildFile', buildfile, task, internal) + return self.run_command('buildFile', buildfile, task, internal, taskonly) @wait_for def build_file_sync(self, *args): self.build_file(*args) + def run_prepared_task(self, recipe, task): + """Run *task* for one parsed recipe, and nothing else. + + No dependencies are resolved and no other task of the recipe is run, + so everything the task consumes must already be in place. The task + runs through the normal BitBake worker path, including fakeroot setup + and dispatch of shell or Python task bodies. + + The task is forced to execute rather than being skipped as up to date. + Returns False if the task failed. + """ + self.run_command('setConfig', 'force', True) + try: + return self.build_file_sync(self.get_recipe_file(recipe), task, True, True) + finally: + self.run_command('setConfig', 'force', False) + def build_targets(self, targets, task=None, handle_events=True, extra_events=None, event_callback=None): """ Builds the specified targets. This is equivalent to a normal invocation From patchwork Sat Aug 15 13:46:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AA01C5AD5A for ; Sat, 15 Aug 2026 13:47:39 +0000 (UTC) Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net [185.136.64.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9123.1786801649528231987 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=RUIxe3MN; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.228, mailfrom: fm-1329275-202608151347267bc178dd8f00020721-qzwogp@rts-flowmailer.siemens.com) Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 202608151347267bc178dd8f00020721 for ; Sat, 15 Aug 2026 15:47:26 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=7bQjlaX2icqv10ehUAghyIxmLksAJuTFScKPL1lsWP8=; b=RUIxe3MNvG6JoZech3Xvsa2Et76aoQYhNnINRPcx/1HZds6dGbR7zMm+DxHN7GBCziDcqe 26CcwbqVveOgw9AVK2MFoFZvE8fBEAEibJkfeaZsJcmPHs0RP6SzbnSJ72G0Wga7B7XUrwN2 pmQkjUPhClhF+BKvChLGrDhN+5jFcA8AJFK50NHyJjMJr3pEoZQcs/CiYQKTW5Ht9g89VAg+ 3nilenQFx4hb7Nmxw83yhySJlLTQ5R7BXpnis7UMjSOwr8ndjsBkU+7yLhV4sw02zMLyDYMK W6YlWsbI9PI5IYHZXEsq3cyhDVvC4++D9m7T2Ft/Dz7zAIkI5SdHP6lQ==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 6/7] tests/cooker: add TinfoilTests for run_prepared_task Date: Sat, 15 Aug 2026 15:46:57 +0200 Message-ID: <20260815134722.497586-7-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19956 From: Adrian Freihofer Add TinfoilTests covering 'tinfoil: add a prepared task runner'. Each test spawns a subprocess to isolate tinfoil's server lifecycle. TestEquivHash is needed because the noop siggen's invalidate_task() removes the base stamp path instead of the task-specific one, making force=True a no-op otherwise. Lives in cooker.py rather than runqueue.py since it tests Tinfoil's Python API, not CLI-level runqueue behaviour. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/tests/cooker.py | 159 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 159 insertions(+) diff --git a/lib/bb/tests/cooker.py b/lib/bb/tests/cooker.py index 76ec65540..a08843e98 100644 --- a/lib/bb/tests/cooker.py +++ b/lib/bb/tests/cooker.py @@ -9,12 +9,171 @@ import unittest import os import subprocess +import sys import tempfile +import time import bb, bb.cooker import re import logging # Cooker tests + + +class TinfoilTests(unittest.TestCase): + """Tests for the Tinfoil API that require a running bitbake server.""" + + # Library directory containing bb.tinfoil + _bblib = os.path.realpath(os.path.join(os.path.dirname(__file__), '..', '..')) + # runqueue-tests BBPATH (provides the simple a1/b1/... test recipes) + _runqueuetests = os.path.realpath(os.path.join(os.path.dirname(__file__), 'runqueue-tests')) + + failing_recipe = """\ +python do_install() { + bb.fatal("deliberate failure") +} +addtask install +""" + + def _make_env(self, builddir, extra=None): + env = os.environ.copy() + env['PYTHONPATH'] = self._bblib + (':' + env['PYTHONPATH'] if 'PYTHONPATH' in env else '') + env['BBPATH'] = self._runqueuetests + env['BB_ENV_PASSTHROUGH_ADDITIONS'] = 'SSTATEVALID SLOWTASKS TOPDIR BB_HASHSERVE BB_SIGNATURE_HANDLER EXTRA_BBFILES' + env['SSTATEVALID'] = '' + env['SLOWTASKS'] = '' + env['TOPDIR'] = builddir + # TestEquivHash creates taint files so that force=True actually + # invalidates the task hash; the default noop siggen cannot do this. + env['BB_HASHSERVE'] = 'auto' + env['BB_SIGNATURE_HANDLER'] = 'TestEquivHash' + if extra: + env.update(extra) + return env + + def _run_script(self, builddir, script, extra=None): + """Run script in a subprocess to isolate tinfoil's server lifecycle.""" + proc = subprocess.run( + [sys.executable, '-c', script], + env=self._make_env(builddir, extra), + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True, + cwd=builddir, + ) + if proc.returncode: + self.fail('tinfoil script failed: %s' % proc.stdout) + return proc.stdout + + def _read_tasklog(self, builddir, cleanup=True): + tasklog = os.path.join(builddir, 'task.log') + tasks = [] + if os.path.exists(tasklog): + with open(tasklog) as f: + tasks = [line.rstrip() for line in f] + if cleanup: + os.remove(tasklog) + return tasks + + def _shutdown(self, builddir): + """Wait for the bitbake server and hashserv to release builddir. + + Must run before the caller's TemporaryDirectory is removed, so it + cannot be a tearDown(). + """ + deadline = time.monotonic() + 30 + while time.monotonic() < deadline: + if not any(os.path.exists(os.path.join(builddir, p)) + for p in ('hashserve.sock', 'bitbake.lock')): + return + time.sleep(0.5) + + def test_run_prepared_task(self): + """tinfoil.run_prepared_task() reruns one task without resolving deps. + + Uses do_install since that's the real devtool ide-sdk scenario: it + needs pseudo and so must run via bitbake, unlike do_compile which the + IDE invokes directly (e.g. via cmake/meson). + + Builds a1 completely so all stamps/hashes are valid, then calls + run_prepared_task('a1', 'install') through the Python API and verifies + that only do_install re-runs while its intra-recipe predecessors + (fetch, unpack, patch, prepare_recipe_sysroot, configure, compile) are + skipped. + """ + # The script runs inside a subprocess so that tinfoil's server + # lifecycle and environment modifications are isolated. + script = """ +import os, sys +import bb.tinfoil + +builddir = os.environ['TOPDIR'] +tasklog = os.path.join(builddir, 'task.log') + +with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(quiet=2) + # Full build so all stamps and hashes are valid. + tinfoil.build_targets(['a1']) + # Clear the log so only the run_prepared_task() entries are counted. + if os.path.exists(tasklog): + os.remove(tasklog) + # run_prepared_task() sets force=True (taint) and calls build_file_sync + # with the recipe file resolved via get_recipe_file(), bypassing the + # normal runqueue dependency resolver. + tinfoil.run_prepared_task('a1', 'install') +""" + with tempfile.TemporaryDirectory(prefix='tinfoiltest') as builddir: + try: + self._run_script(builddir, script) + + tasks = self._read_tasklog(builddir) + self.assertEqual(tasks, ['a1:install'], + 'run_prepared_task should rerun only install, got: %s' % tasks) + finally: + self._shutdown(builddir) + + def test_run_prepared_task_unbuilt(self): + """run_prepared_task() runs the task and nothing else. + + The recipe was never built, so if any dependency task were still in + the runqueue it would have to run here. + """ + script = """ +import bb.tinfoil + +with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(quiet=2) + assert tinfoil.run_prepared_task('a1', 'install') is True +""" + with tempfile.TemporaryDirectory(prefix='tinfoiltest') as builddir: + try: + self._run_script(builddir, script) + + tasks = self._read_tasklog(builddir) + self.assertEqual(tasks, ['a1:install'], + 'run_prepared_task should run no dependency task, got: %s' % tasks) + finally: + self._shutdown(builddir) + + def test_run_prepared_task_failure(self): + """A failing task makes run_prepared_task() return False, not raise.""" + script = """ +import bb.tinfoil + +with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(quiet=2) + assert tinfoil.run_prepared_task('failer', 'install') is False +""" + with tempfile.TemporaryDirectory(prefix='tinfoilrecipes') as recipes, \ + tempfile.TemporaryDirectory(prefix='tinfoiltest') as builddir: + with open(os.path.join(recipes, 'failer.bb'), 'w') as f: + f.write(self.failing_recipe) + try: + self._run_script(builddir, script, + {'EXTRA_BBFILES': '%s/*.bb' % recipes}) + finally: + self._shutdown(builddir) + + class CookerTest(unittest.TestCase): def setUp(self): # At least one variable needs to be set From patchwork Sat Aug 15 13:46:58 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 95427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC7D3C5DF72 for ; Sat, 15 Aug 2026 13:47:40 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9201.1786801649528139380 for ; Sat, 15 Aug 2026 06:47:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=PnAmZXYq; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-20260815134726af2a17cec7000207b7-zzxhyu@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 20260815134726af2a17cec7000207b7 for ; Sat, 15 Aug 2026 15:47:27 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=s2hwCxkhz+DiyjPA/d3nuKSAYwmJQWcsmjHBBte2dBI=; b=PnAmZXYq+E2JMMUQ0FtHXXhTIHuk575It8Mo3OgLdFTBp2qew1o1AEHyitzhjINBfCNeE0 3dSNRXSUCvO27+vAr6QUN6ey6mhW7y7wmnAFPHMmlAcaY3N24LyZ65cab8WTblkV2zcHOh+h nfzDnEU6onv1zr+kc69fjMKz0S2mdebybbkFF6GEhHZF2PwrSfGF0VnOQNF8JEsYL98TxDzr 2yEdhOKVZ4tGgHBjdfZYxZmJX895NtBRm/t72sbbUIWNGzgmN1OUJuIXRip8w6t9+C7ol/nD KbZGgs8qvoQHtPgzp0zlUDGC7pZ/jpY84PB8iygUebItCsAyfTDqn/TA==; From: AdrianF To: bitbake-devel@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 7/7] parse/ast: skip empty BBPATH segments in include_all Date: Sat, 15 Aug 2026 15:46:58 +0200 Message-ID: <20260815134722.497586-8-adrian.freihofer@siemens.com> In-Reply-To: <20260815134722.497586-1-adrian.freihofer@siemens.com> References: <20260815134722.497586-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 15 Aug 2026 13:47:40 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/19952 From: Adrian Freihofer BBPATH can end up with an empty ":"-split segment when different layer.conf files mix the "${LAYERDIR}:" (prepend) and ":${LAYERDIR}" (append) idioms, e.g. openembedded-core's own meta/conf/layer.conf uses "BBPATH .= \":${LAYERDIR}\"" while every other layer.conf in a typical poky setup uses "BBPATH =. \"${LAYERDIR}:\"". Combined, this produces a literal "::" in the final value. IncludeAllNode.eval() iterates every BBPATH segment and calls os.path.join(path, s) to build the candidate file to include. For an empty segment, os.path.join("", s) returns s unchanged, i.e. a relative path instead of an absolute one. include_single_file() then takes its relative-path branch, which does its own independent search across the whole BBPATH and marks every path it tries (found or not) as a dependency via mark_dependency(), as a side effect of resolving that one (bogus) relative candidate. If that side search happens to try the real target file before this loop's own iteration for its actual BBPATH entry runs, check_dependency() reports it as already seen and include_single_file() logs a spurious "Duplicate inclusion" warning for it, even though the file is only ever included once. This is how e.g. oe-core's "include_all conf/distro/include/maintainers.inc" in defaultsetup.conf ends up warning about itself on every parse. Skip empty segments so an empty BBPATH entry cannot trigger this false-positive dependency marking. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- lib/bb/parse/ast.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bb/parse/ast.py b/lib/bb/parse/ast.py index a372b3534..866ab8ed1 100644 --- a/lib/bb/parse/ast.py +++ b/lib/bb/parse/ast.py @@ -56,6 +56,10 @@ class IncludeAllNode(AstNode): logger.debug2("CONF %s:%s: including %s", self.filename, self.lineno, s) for path in data.getVar("BBPATH").split(":"): + # Skip empty segments (e.g. from a stray "::" if some layer.conf + # uses ".= \":${LAYERDIR}\"" instead of "=. \"${LAYERDIR}:\""). + if not path: + continue bb.parse.ConfHandler.include(self.filename, os.path.join(path, s), self.lineno, data, False) class ExportNode(AstNode):