From patchwork Sun Aug 30 14:28:28 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96816 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F9F4C624A4 for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9457.1788100183424561305 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=iaNlMKd0; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-1329275-20260830142940d062225f6a00020788-es__xi@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20260830142940d062225f6a00020788 for ; Sun, 30 Aug 2026 16:29:40 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=1MFyrhowzv77hBhM/3YCJUI/rso4OyTeawAo9wglhOw=; b=iaNlMKd03v1Z2Ts6DcrIS7m8zguKmSChUsksoIXcov2fvQwn32Re0Gpw/8mFpbJPAVkXvR o0bMzlin1aOIKAOxCbuNFZf3rliEADhbOuHxepx7XaOF1bwLtfcDHgiIm7ppDcglHcGysFYC GJCKIeuR3LjO4b+8/MdPYrfmeIoBMXZmfzJ6W0S6h6u68GpGraayXyF+x8RaI089jMTTb8lS WZ0gUSJaCFFDtHTCprI/rifcbnn14EC392acj8cIqaUXv4MzOh9W9DCBiiPwyriPhItxiVOJ oGouPzHIzWCEEShrv/cTOXXpM4TrqFhyHqCij+nT6Oo/jWfcM0YBNMjA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 01/24] runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction Date: Sun, 30 Aug 2026 16:28:28 +0200 Message-ID: <20260830142922.17241-2-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244672 From: Adrian Freihofer pseudo inverted its path-matching logic some time ago: when set, PSEUDO_INCLUDE_PATHS acts as an ownership-tracking allowlist, and anything outside it silently falls through to real chown()/chmod() instead of being recorded by pseudo. This script never set it, so an inherited restrictive value could leave the extracted rootfs with wrong ownership. Since this script only ever touches SDK_ROOTFS_DIR, setting PSEUDO_INCLUDE_PATHS to it is enough. Signed-off-by: Adrian Freihofer --- scripts/runqemu-extract-sdk | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/runqemu-extract-sdk b/scripts/runqemu-extract-sdk index 528ef76430..db9813e1dc 100755 --- a/scripts/runqemu-extract-sdk +++ b/scripts/runqemu-extract-sdk @@ -85,6 +85,8 @@ mkdir -p "$pseudo_state_dir" touch "$pseudo_state_dir/pseudo.pid" PSEUDO_LOCALSTATEDIR="$pseudo_state_dir" export PSEUDO_LOCALSTATEDIR +PSEUDO_INCLUDE_PATHS="$SDK_ROOTFS_DIR" +export PSEUDO_INCLUDE_PATHS echo "Extracting rootfs tarball using pseudo..." echo "$PSEUDO $PSEUDO_OPTS tar -C \"$SDK_ROOTFS_DIR\" $TAR_OPTS \"$ROOTFS_TARBALL\"" From patchwork Sun Aug 30 14:28:29 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1C7EC624A6 for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9458.1788100183424670277 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=Um24XbTy; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-1329275-20260830142940fad5b9324700020791-rdrn4f@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20260830142940fad5b9324700020791 for ; Sun, 30 Aug 2026 16:29:40 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=dRA0GrgzP21Y7AktCLuUt6URHG5IBVeRB4L3ox7Hw1Q=; b=Um24XbTyKVV8zbLFTizfgcfUP4ppC9jyfbIshMsfPMALnz8HOiObnOeHEOVLyX2rzBODYI Mu4KsR03LckSiBrEW3D9aRObWJVPDpi8AOqZXEKD4+zuJPzU15CZLDV0fUKjiM3eiKI1VEc9 WSnUtA3Ky+Xejop7K163VIN0YHyw7DAHuIYYNkDOVtldiJU8DqcqVJDQvlILaNIeiRiNYEmS LROB8dSaSVTR6QUaKD6bhvKKR1ncbRzlOio/0gyoIfu0bcCjAJHonfXTD0+Vrx4flWk+kRZh VoNpCXB3lxrFPXdGmIUNBBCNsK3IDUg0GKM/I9dQyMp9mlg+R/r73HVQ==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 02/24] devtool: ide-sdk: dedupe solib_search_path entries Date: Sun, 30 Aug 2026 16:28:29 +0200 Message-ID: <20260830142922.17241-3-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244671 From: Adrian Freihofer With usrmerge in DISTRO_FEATURES (the default), root_prefix becomes exec_prefix, so base_libdir and libdir resolve to the same path. This made most entries of solib_search_path() literal duplicates. Dedupe while preserving order. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_sdk.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index 4fdeeec24d..6f08940c9c 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -643,7 +643,8 @@ class RecipeModified: os.path.join(image.rootfs, base_libdir), os.path.join(image.rootfs, libdir) ] - return so_paths + # base_libdir == libdir when usrmerge is in DISTRO_FEATURES (the default), dedupe + return list(dict.fromkeys(so_paths)) def solib_search_path_str(self, image): """Return a : separated list of paths usable by GDB's set solib-search-path""" From patchwork Sun Aug 30 14:28:30 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96812 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A713CC61DFB for ; Sun, 30 Aug 2026 14:29:53 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9451.1788100183423090784 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=gS57/z4j; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-20260830142940a2dbff4d3600020776-gporln@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 20260830142940a2dbff4d3600020776 for ; Sun, 30 Aug 2026 16:29:40 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=O74ILPKjsMtLZF35PvGlc+SYG3LTRheACCWsCJMwFXk=; b=gS57/z4ja/Fh+//48ct1BmczLTMQlnY+zNSwvMcr3CESZ8srnN58D1rGIBQ1sr46x+Hto3 kCoOyqV6BbFlrJs1v18+AENSOIcQVAP4mLR2hWdJD/dK5ISQ7QmD5CBw6aop75TuHhPma5KZ kIN8USWGWrPjoSzKYzHg2Y7vYH7aHssVSwbff9jBKMl2MDXNxshfkSIQCz94VUgT3ftymlGg OJwOT8IiIAooKoBBKl1N1yweugbbpLh1OO2zSG2n+UOxawSVZouPQwFSRIFKL2iJJN4vnNlS lfm8bU46uNG5Ccd6m2rGQU3ZKTDS+Yqp0yF203xYnDZjwJRn6FCg7yKg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 03/24] devtool: ide-sdk: VSCode IntelliSense for rootfs-dbg sources Date: Sun, 30 Aug 2026 16:28:30 +0200 Message-ID: <20260830142922.17241-4-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244678 From: Adrian Freihofer Add includePath fallback entries to c_cpp_properties.json pointing at recipe-sysroot/usr/include and rootfs-dbg/usr/src/debug, so the C/C++ extension can resolve symbols in other recipes' sources whenever they are opened from rootfs-dbg, e.g. while stepping into them with the debugger. Also widen files.readonlyInclude from just the recipe's sysroots to the whole TMPDIR: more robust, since it also covers oe-workdir, rootfs, and other generated content reachable from the workspace. oe-logs and oe-workdir are kept visible in the VSCode explorer instead of being hidden via files.exclude, so build logs and temp scripts can still be browsed, while staying out of the file watcher/indexer and read-only. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 8 ++-- scripts/lib/devtool/ide_plugins/ide_code.py | 44 +++++++++++++++------ scripts/lib/devtool/ide_sdk.py | 2 + 3 files changed, 38 insertions(+), 16 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 7c989f9dc2..c0df13b718 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3945,11 +3945,11 @@ class DevtoolIdeSdkKernelTests(DevtoolIdeSdkTests): os.path.join(tempdir, 'Makefile'), 'makefilePath should point to the Makefile in the source tree') - # Verify kernel sources are set read-only + # Verify kernel sources (under TMPDIR) are set read-only + tmpdir = get_bb_var('TMPDIR', recipe_name) readonly_includes = settings_d.get('files.readonlyInclude', {}) - self.assertTrue( - any(k for k in readonly_includes if 'staging_kernel' in k.lower() or 'linux' in k.lower()), - 'Kernel staging dir should be set read-only in files.readonlyInclude: %s' % readonly_includes) + self.assertIn(os.path.realpath(tmpdir) + '/**', readonly_includes, + 'TMPDIR should be set read-only in files.readonlyInclude: %s' % readonly_includes) # Verify the cross-build environment is exported for the terminal self.assertIn('terminal.integrated.env.linux', settings_d, diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py index 1a8a79f623..5abf0492e8 100644 --- a/scripts/lib/devtool/ide_plugins/ide_code.py +++ b/scripts/lib/devtool/ide_plugins/ide_code.py @@ -205,9 +205,6 @@ class IdeVSCode(IdeBase): settings_dict["files.watcherExclude"].update(files_excludes_kernel) settings_dict["python.analysis.exclude"] += kernel_exclude_patterns - # protect the kernel sources - settings_dict["files.readonlyInclude"][modified_recipe.staging_kernel_dir + '/**'] = True - # Export the complete cross-build environment settings_dict["terminal.integrated.env.linux"] = modified_recipe.exported_vars @@ -232,12 +229,15 @@ class IdeVSCode(IdeBase): ] def vscode_settings(self, modified_recipe, image_recipe): - files_excludes = { + files_hide = { "**/.git/**": True, - "**/oe-logs/**": True, - "**/oe-workdir/**": True, "**/source-date-epoch/**": True } + files_watcher_exclude = dict(files_hide) + files_watcher_exclude.update({ + "**/oe-logs/**": True, + "**/oe-workdir/**": True, + }) python_exclude = [ "**/.git/**", "**/oe-logs/**", @@ -245,14 +245,16 @@ class IdeVSCode(IdeBase): "**/source-date-epoch/**" ] files_readonly = { - modified_recipe.recipe_sysroot + '/**': True, - modified_recipe.recipe_sysroot_native + '/**': True, + modified_recipe.tmpdir + '/**': True, + "**/oe-logs/**": True, + "**/oe-workdir/**": True, } if image_recipe.rootfs_dbg is not None: files_readonly[image_recipe.rootfs_dbg + '/**'] = True settings_dict = { - "files.watcherExclude": files_excludes, - "files.exclude": files_excludes, + "files.watcherExclude": files_watcher_exclude, + "files.exclude": files_hide, + "search.exclude": dict(files_watcher_exclude), "files.readonlyInclude": files_readonly, "python.analysis.exclude": python_exclude } @@ -287,7 +289,7 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), extensions_file, {"recommendations": recommendations}) - def vscode_c_cpp_properties(self, modified_recipe): + def vscode_c_cpp_properties(self, modified_recipe, image_recipe): properties_dict = { "name": modified_recipe.recipe_id_pretty, } @@ -322,6 +324,24 @@ class IdeVSCode(IdeBase): else: # no C/C++ build return + # configurationProvider/compileCommands only cover the recipe under + # development. Add includePath as a fallback so the C/C++ extension + # also resolves symbols in other recipes sources found in rootfs-dbg. + if image_recipe.rootfs_dbg is not None: + recipe_sysroot_include = os.path.join(modified_recipe.recipe_sysroot, "usr", "include") + # rootfs_dbg/usr/include is empty, target headers come from recipe-sysroot, + # consistent with the GDB sourceFileMap for "/usr/include". + rootfs_dbg_src_debug = os.path.join(image_recipe.rootfs_dbg, "usr", "src", "debug") + include_path = properties_dict.get("includePath", ["${workspaceFolder}/**"]) + for path in (recipe_sysroot_include, rootfs_dbg_src_debug + "/**"): + if path not in include_path: + include_path.append(path) + properties_dict["includePath"] = include_path + # That's the default, but make it easy to change if a big index is preferred. + properties_dict["browse"] = { + "limitSymbolsToIncludedHeaders": True + } + properties_dicts = { "configurations": [ properties_dict @@ -873,7 +893,7 @@ class IdeVSCode(IdeBase): def setup_modified_recipe(self, args, image_recipe, modified_recipe): self.vscode_settings(modified_recipe, image_recipe) self.vscode_extensions(modified_recipe) - self.vscode_c_cpp_properties(modified_recipe) + self.vscode_c_cpp_properties(modified_recipe, image_recipe) if args.target: if modified_recipe.toolchain == 'clang': self.initialize_cross_debug_configs( diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index 6f08940c9c..d7575ec523 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -457,6 +457,7 @@ class RecipeModified: self.staging_incdir = None self.strip_cmd = None self.target_arch = None + self.tmpdir = None self.toolchain = None self.topdir = None self.workdir = None @@ -545,6 +546,7 @@ class RecipeModified: recipe_d.getVar('STAGING_INCDIR')) self.strip_cmd = recipe_d.getVar('STRIP') self.target_arch = recipe_d.getVar('TARGET_ARCH') + self.tmpdir = os.path.realpath(recipe_d.getVar('TMPDIR')) self.toolchain = recipe_d.getVar('TOOLCHAIN') self.topdir = recipe_d.getVar('TOPDIR') self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR')) From patchwork Sun Aug 30 14:28:31 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A005C61DF0 for ; Sun, 30 Aug 2026 14:29:53 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9456.1788100183424035097 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=UN8+jhW3; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-202608301429405bcf82dbc50002079a-k3iqt_@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202608301429405bcf82dbc50002079a for ; Sun, 30 Aug 2026 16:29:40 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=GvtcMGv1+qUfYqMYpWxrxcwFlDgtWUWmABpeChZgvws=; b=UN8+jhW3Gys/cdxv6B929aCm4yP8KX3bzeNVh89sI6COPwxZZ9fhOyc8heWHNzT54FbJuj 57yE7uqeIIUEVX4fxd1NqUtVUvIEvjgGWcZyKNBYvGN5VaIJGHOI9TWin+dE4e2T6N/7UIlz Fi/dee3y2A12M7pYzlvBeyGIb+hukLqyXZZz5R3MlKGguHSAMzaEgcPlUUER97uNCVjCFexp NNgFN3tkHw6uFpjFSSR6ywcYzoaCdXBedQ1p81RzIni+78GKpv9mYmVWkKx16mCOFYJUydPL eneYdnoMhPqOwHwn6GnZJtYjT6bVImYQWALrus8+VG7/Ng4MOPGSejJA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 04/24] cpp-example: fix stuck breakpoints when attaching and daemonize properly Date: Sun, 30 Aug 2026 16:28:31 +0200 Message-ID: <20260830142922.17241-5-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244679 From: Adrian Freihofer Attaching gdbserver to the already-running SysV-init-started instance and then adding a breakpoint left it unbound (grey/hollow) forever, even though setting the same breakpoint before starting the debugger worked fine. Root cause: cpp-example.init started the daemon via `start-stop-daemon --background`, which double-forks to detach from the terminal. That leaves the final daemon process in a process group whose leader has already exited, i.e. its pgid no longer equals its own pid. gdbserver's remote-interrupt mechanism pauses a running inferior with `kill(-pid, SIGINT)` (signal the whole process group); with a mismatched pgid this fails silently (ESRCH), GDB's `-exec-interrupt` never actually stops the target, and MIEngine's pending-breakpoint bind times out and reports it as unbound. Fix by dropping the double-fork daemonization we inherited from `start-stop-daemon`/SysV-init cargo culting in favor of the classic, correct Unix daemon recipe: a single fork() + setsid(), which makes the process its own session AND process-group leader (pgid == pid) and keeps gdbserver's SIGINT-to-process-group interrupt working. As part of doing this properly: - cpp-example itself now daemonizes (--daemonize), writes its own pidfile (--pidfile), and drops root privileges (--uid/--gid) instead of depending on start-stop-daemon/su for that. - stdio is redirected to /dev/null after setsid() so the service actually releases the console instead of holding it open forever. - Since stdio is gone once daemonized, log through syslog instead so the service's messages aren't silently discarded. - All of this legacy SysV-init-only behavior lives in a new daemonize.cpp/.hpp, compiled in only when WITH_SYSTEMD is off; under systemd (Type=simple) none of it is needed since systemd already daemonizes, drops privileges, and tracks the pid itself. Signed-off-by: Adrian Freihofer --- .../recipes-test/cpp/cmake-example.inc | 1 + .../recipes-test/cpp/cpp-example.inc | 4 + .../recipes-test/cpp/files/CMakeLists.txt | 6 ++ .../recipes-test/cpp/files/cpp-example.cpp | 91 +++++++++++++++++-- .../recipes-test/cpp/files/cpp-example.init | 10 +- .../recipes-test/cpp/files/daemonize.cpp | 57 ++++++++++++ .../recipes-test/cpp/files/daemonize.hpp | 27 ++++++ .../recipes-test/cpp/files/meson.build | 9 +- .../recipes-test/cpp/files/meson.options | 2 + .../recipes-test/cpp/meson-example.inc | 1 + meta/lib/oeqa/selftest/cases/devtool.py | 8 +- 11 files changed, 198 insertions(+), 18 deletions(-) create mode 100644 meta-selftest/recipes-test/cpp/files/daemonize.cpp create mode 100644 meta-selftest/recipes-test/cpp/files/daemonize.hpp diff --git a/meta-selftest/recipes-test/cpp/cmake-example.inc b/meta-selftest/recipes-test/cpp/cmake-example.inc index eb023d389a..38d53af7b8 100644 --- a/meta-selftest/recipes-test/cpp/cmake-example.inc +++ b/meta-selftest/recipes-test/cpp/cmake-example.inc @@ -11,6 +11,7 @@ SRC_URI += "file://CMakeLists.txt" inherit cmake-qemu PACKAGECONFIG[failing_test] = "-DFAILING_TEST=ON" +PACKAGECONFIG[systemd] = "-DWITH_SYSTEMD=ON,-DWITH_SYSTEMD=OFF" # Support installing all recipe variants in parallel EXTRA_OECMAKE += "\ diff --git a/meta-selftest/recipes-test/cpp/cpp-example.inc b/meta-selftest/recipes-test/cpp/cpp-example.inc index 0070d17201..3934345f17 100644 --- a/meta-selftest/recipes-test/cpp/cpp-example.inc +++ b/meta-selftest/recipes-test/cpp/cpp-example.inc @@ -15,6 +15,8 @@ SRC_URI = "\ file://cpp-example.cpp \ file://cpp-example-lib.hpp \ file://cpp-example-lib.cpp \ + file://daemonize.cpp \ + file://daemonize.hpp \ file://test-cpp-example.cpp \ file://cpp-example.conf \ file://config.h.in \ @@ -33,6 +35,8 @@ SYSTEMD_SERVICE:${PN} = "${BPN}.service" INITSCRIPT_NAME = "${BPN}" INITSCRIPT_PARAMS = "defaults 99" +PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', '', d)}" + # Create cpp-example user and group USERADD_PACKAGES = "${PN}" GROUPADD_PARAM:${PN} = "--system ${EX_SERVICE_USER}" diff --git a/meta-selftest/recipes-test/cpp/files/CMakeLists.txt b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt index 8802839702..e063bdafb3 100644 --- a/meta-selftest/recipes-test/cpp/files/CMakeLists.txt +++ b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt @@ -13,6 +13,7 @@ project(cmake-example option(BUILD_SHARED_LIBS "Build using shared libraries" ON) option(FAILING_TEST "Compile a failing unit test to test the test infrastructure" OFF) +option(WITH_SYSTEMD "Target is managed by systemd: skip the legacy SysV daemonize/pidfile/privilege-drop code" OFF) set(BINARY_NAME "cmake-example" CACHE STRING "Name of the installed executable and library prefix") set(TEST_BINARY_NAME "test-cmake-example" CACHE STRING "Name of the installed test executable") @@ -54,6 +55,11 @@ install(TARGETS ${BINARY_NAME}-lib add_executable(${BINARY_NAME} cpp-example.cpp) target_include_directories(${BINARY_NAME} PRIVATE ${CMAKE_CURRENT_BINARY_DIR}) target_link_libraries(${BINARY_NAME} PRIVATE ${BINARY_NAME}-lib) +if (WITH_SYSTEMD) + target_compile_definitions(${BINARY_NAME} PRIVATE WITH_SYSTEMD) +else() + target_sources(${BINARY_NAME} PRIVATE daemonize.cpp daemonize.hpp) +endif(WITH_SYSTEMD) install(TARGETS ${BINARY_NAME} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example.cpp b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp index a376419c13..f18911554c 100644 --- a/meta-selftest/recipes-test/cpp/files/cpp-example.cpp +++ b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp @@ -6,44 +6,117 @@ #include "cpp-example-lib.hpp" +#include #include #include #include #include +#ifndef WITH_SYSTEMD +#include +#include + +#include "daemonize.hpp" +#endif + +namespace { + +#ifndef WITH_SYSTEMD +bool g_use_syslog = false; +#endif + +// Prints an informational message. Under systemd, stdout is already captured +// by the journal. Otherwise, use stdout until daemonized (stdio is then +// redirected to /dev/null), and syslog afterwards so messages aren't lost. +void log_info(const std::string& msg) +{ +#ifndef WITH_SYSTEMD + if (g_use_syslog) { + syslog(LOG_INFO, "%s", msg.c_str()); + return; + } +#endif + std::cout << msg << std::endl; +} + +} // namespace + int main(int argc, char* argv[]) { bool endless_mode = false; +#ifndef WITH_SYSTEMD + bool daemonize_mode = false; + std::string pidfile_path; + bool have_uid = false; + bool have_gid = false; + uid_t target_uid = 0; + gid_t target_gid = 0; +#endif // Parse command line arguments for (int i = 1; i < argc; i++) { - if (std::string(argv[i]) == "--endless") { + std::string arg = argv[i]; + if (arg == "--endless") { endless_mode = true; - } else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") { +#ifndef WITH_SYSTEMD + } else if (arg == "--daemonize") { + daemonize_mode = true; + } else if (arg == "--pidfile" && i + 1 < argc) { + pidfile_path = argv[++i]; + } else if (arg == "--uid" && i + 1 < argc) { + target_uid = static_cast(std::strtoul(argv[++i], nullptr, 10)); + have_uid = true; + } else if (arg == "--gid" && i + 1 < argc) { + target_gid = static_cast(std::strtoul(argv[++i], nullptr, 10)); + have_gid = true; +#endif + } else if (arg == "--help" || arg == "-h") { std::cout << "Usage: " << argv[0] << " [OPTIONS]" << std::endl; std::cout << "Options:" << std::endl; - std::cout << " --endless Run in endless loop mode (for service)" << std::endl; - std::cout << " --help, -h Show this help message" << std::endl; + std::cout << " --endless Run in endless loop mode (for service)" << std::endl; +#ifndef WITH_SYSTEMD + std::cout << " --daemonize Detach from the controlling terminal" << std::endl; + std::cout << " --pidfile Write the daemon's PID to " << std::endl; + std::cout << " --uid Drop root privileges to this user ID" << std::endl; + std::cout << " --gid Drop root privileges to this group ID" << std::endl; +#endif + std::cout << " --help, -h Show this help message" << std::endl; return 0; } } +#ifndef WITH_SYSTEMD + if (daemonize_mode) { + daemonize(); + openlog(argv[0], LOG_PID, LOG_DAEMON); + g_use_syslog = true; + } + + if (!pidfile_path.empty()) { + write_pidfile(pidfile_path); + } + + // Drop privileges after daemonizing/writing the pidfile (both may need + // root, e.g. to create files under /var/run), but before doing any work. + drop_privileges(have_gid, target_gid, have_uid, target_uid); +#endif + auto cpp_example = CppExample(); if (endless_mode) { - std::cout << "Starting cpp-example service in endless mode..." << std::endl; + log_info("Starting cpp-example service in endless mode..."); } else { - std::cout << "Running cpp-example once..." << std::endl; + log_info("Running cpp-example once..."); } - std::cout << "C++ example linking " << cpp_example.get_string() << std::endl; - std::cout << "Linking json-c version " << cpp_example.get_json_c_version() << std::endl; + log_info("C++ example linking " + cpp_example.get_string()); + log_info(std::string("Linking json-c version ") + cpp_example.get_json_c_version()); cpp_example.print_json(); do { // Read and print message from config file std::string config_message = cpp_example.read_config_message(); - std::cout << "Config file message: " << config_message << std::endl; + log_info("Config file message: " + config_message); if (endless_mode) { // Sleep for 1 second diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example.init b/meta-selftest/recipes-test/cpp/files/cpp-example.init index 30b8486eeb..6ef9aaac38 100644 --- a/meta-selftest/recipes-test/cpp/files/cpp-example.init +++ b/meta-selftest/recipes-test/cpp/files/cpp-example.init @@ -25,8 +25,7 @@ start() { fi echo -n "Starting $DAEMON: " - start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \ - --background --chuid $USER --exec $DAEMON_PATH -- $DAEMON_ARGS + $DAEMON_PATH --daemonize --pidfile "$PIDFILE" --uid "$(id -u "$USER")" --gid "$(id -g "$USER")" $DAEMON_ARGS RETVAL=$? if [ $RETVAL -eq 0 ]; then echo "OK" @@ -39,8 +38,11 @@ start() { stop() { echo -n "Stopping $DAEMON: " - start-stop-daemon --stop --quiet --pidfile $PIDFILE - RETVAL=$? + if [ -f "$PIDFILE" ] && kill "$(cat "$PIDFILE")" 2>/dev/null; then + RETVAL=0 + else + RETVAL=1 + fi if [ $RETVAL -eq 0 ]; then echo "OK" rm -f $PIDFILE $LOCK_FILE diff --git a/meta-selftest/recipes-test/cpp/files/daemonize.cpp b/meta-selftest/recipes-test/cpp/files/daemonize.cpp new file mode 100644 index 0000000000..7d91259bfa --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/daemonize.cpp @@ -0,0 +1,57 @@ +/* + * Copyright OpenEmbedded Contributors + * + * SPDX-License-Identifier: MIT + */ + +#include "daemonize.hpp" + +#include +#include +#include +#include +#include + +void daemonize() +{ + pid_t pid = fork(); + if (pid < 0) { + std::perror("fork"); + std::exit(1); + } + if (pid > 0) { + _exit(0); + } + setsid(); + + // setsid() only drops the controlling-terminal association; stdio still + // points at the console, so redirect it or an --endless service keeps + // the console open/busy with its output forever. + int null_fd = open("/dev/null", O_RDWR); + if (null_fd >= 0) { + dup2(null_fd, STDIN_FILENO); + dup2(null_fd, STDOUT_FILENO); + dup2(null_fd, STDERR_FILENO); + if (null_fd > STDERR_FILENO) { + close(null_fd); + } + } +} + +void write_pidfile(const std::string& path) +{ + std::ofstream pidfile(path, std::ios::trunc); + pidfile << getpid() << std::endl; +} + +void drop_privileges(bool have_gid, gid_t gid, bool have_uid, uid_t uid) +{ + if (have_gid && (setgroups(0, nullptr) != 0 || setgid(gid) != 0)) { + std::perror("setgid"); + std::exit(1); + } + if (have_uid && setuid(uid) != 0) { + std::perror("setuid"); + std::exit(1); + } +} diff --git a/meta-selftest/recipes-test/cpp/files/daemonize.hpp b/meta-selftest/recipes-test/cpp/files/daemonize.hpp new file mode 100644 index 0000000000..17d85d690a --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/daemonize.hpp @@ -0,0 +1,27 @@ +/* + * Copyright OpenEmbedded Contributors + * + * SPDX-License-Identifier: MIT + */ + +#pragma once + +// Legacy SysV-init daemonizing helpers. Not needed (and not built) when +// WITH_SYSTEMD is set, since systemd already daemonizes, drops privileges, +// and tracks the pid itself. + +#include +#include + +// Detaches from the controlling terminal. Uses a single fork (not the classic +// double-fork) so this process becomes its own session AND process-group +// leader (pgid == pid); a second fork would hand it off to a child with a +// stale pgid, which breaks gdbserver's SIGINT-to-process-group interrupt. +void daemonize(); + +// Writes the current process's pid to the given path. +void write_pidfile(const std::string& path); + +// Permanently drops from root to the given group/user. Must run gid before +// uid: once uid is dropped, the process no longer has permission to setgid. +void drop_privileges(bool have_gid, gid_t gid, bool have_uid, uid_t uid); diff --git a/meta-selftest/recipes-test/cpp/files/meson.build b/meta-selftest/recipes-test/cpp/files/meson.build index 3cb4669dfa..620c220dbd 100644 --- a/meta-selftest/recipes-test/cpp/files/meson.build +++ b/meta-selftest/recipes-test/cpp/files/meson.build @@ -30,6 +30,13 @@ configure_file(input : 'config.h.in', # Include the build directory for config.h inc_dir = include_directories('.') +example_sources = ['cpp-example.cpp'] +if get_option('WITH_SYSTEMD').enabled() + add_project_arguments('-DWITH_SYSTEMD', language: 'cpp') +else + example_sources += ['daemonize.cpp', 'daemonize.hpp'] +endif + exlib = shared_library(binary_name + 'lib', 'cpp-example-lib.cpp', 'cpp-example-lib.hpp', version: meson.project_version(), @@ -40,7 +47,7 @@ exlib = shared_library(binary_name + 'lib', ) executable(binary_name, - 'cpp-example.cpp', + example_sources, link_with : exlib, include_directories : inc_dir, install : true diff --git a/meta-selftest/recipes-test/cpp/files/meson.options b/meta-selftest/recipes-test/cpp/files/meson.options index 374e346197..a507518990 100644 --- a/meta-selftest/recipes-test/cpp/files/meson.options +++ b/meta-selftest/recipes-test/cpp/files/meson.options @@ -1,6 +1,8 @@ option('FAILING_TEST', type : 'feature', value : 'disabled', description : 'Compile a failing unit test to test the test infrastructure') +option('WITH_SYSTEMD', type : 'feature', value : 'disabled', + description : 'Target is managed by systemd: skip the legacy SysV daemonize/pidfile/privilege-drop code') option('BINARY_NAME', type : 'string', value : 'mesonex', description : 'Name of the installed executable') option('TEST_BINARY_NAME', type : 'string', value : 'test-mesonex', diff --git a/meta-selftest/recipes-test/cpp/meson-example.inc b/meta-selftest/recipes-test/cpp/meson-example.inc index 2937be27f8..eb60649636 100644 --- a/meta-selftest/recipes-test/cpp/meson-example.inc +++ b/meta-selftest/recipes-test/cpp/meson-example.inc @@ -16,6 +16,7 @@ SRC_URI += "\ inherit pkgconfig meson PACKAGECONFIG[failing_test] = "-DFAILING_TEST=enabled" +PACKAGECONFIG[systemd] = "-DWITH_SYSTEMD=enabled,-DWITH_SYSTEMD=disabled" # Support installing all recipes variants in parallel EXTRA_OEMESON += "\ diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index c0df13b718..87efa1ee80 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3123,7 +3123,7 @@ class DevtoolIdeSdkTests(DevtoolBase): # the first _gdb_cross_debugging_multi call above. self._gdb_cross_debugging_multi( qemu, recipe_name, example_exe, MAGIC_STRING_NEW, - exe_break_line=63 + LINE_SHIFT, exe_list_line=55 + LINE_SHIFT, + exe_break_line=136 + LINE_SHIFT, exe_list_line=128 + LINE_SHIFT, hpp_break_line=21 + LINE_SHIFT, lib_break_line=31 + LINE_SHIFT) def _verify_cmake_preset(self, tempdir): @@ -3239,7 +3239,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self.assertIn("GNU gdb", r.output) def _gdb_debug_cpp_example(self, magic_string, gdb_start_cmd="run", - exe_break_line=63, exe_list_line=55, hpp_break_line=21, + exe_break_line=136, exe_list_line=128, hpp_break_line=21, lib_break_line=31): """Get a series of gdb commands to debug the cpp-example-lib example""" gdb_batch_cmd = " -ex 'break main' -ex '%s'" % gdb_start_cmd @@ -3285,7 +3285,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): gdb_batch_cmd += " -ex 'continue'" return gdb_batch_cmd - def _gdb_debug_cpp_example_check(self, gdb_output, magic_string, exe_list_line=55, lib_break_line=31): + def _gdb_debug_cpp_example_check(self, gdb_output, magic_string, exe_list_line=128, lib_break_line=31): self.assertIn("Breakpoint 1, main", gdb_output) self.assertIn("$1 = 0", gdb_output) # test.string.compare equal self.assertIn("$2 = -3", gdb_output) # test.string.compare longer @@ -3313,7 +3313,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self.assertIn("exited normally", gdb_output) def _gdb_cross_debugging_multi(self, qemu, recipe_name, example_exe, magic_string, - exe_break_line=63, exe_list_line=55, hpp_break_line=21, + exe_break_line=136, exe_list_line=128, hpp_break_line=21, lib_break_line=31): """Verify gdb-cross is working From patchwork Sun Aug 30 14:28:32 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96822 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEC2DC624C2 for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9430.1788100183423104196 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=lm1/4NWu; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-1329275-20260830142940a1c4eced5600020715-nkeach@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20260830142940a1c4eced5600020715 for ; Sun, 30 Aug 2026 16:29:40 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=7/VaUD5YeLphG9cw5Smci7s/DApnGoNA/E+ahV2Me2Y=; b=lm1/4NWujuZAAL69pGdggb0naPlnr7nyQcDdx9iqJoQBOO4LWm8C/zV3IDgVbBqvxQWN1v JWgkd1c15ZcmggtBB3dFByiUlNosn8/fS56aeKfLzTJgKKeizQQ5Tgdk+Wu6iUthSWwrHNtv XAb8UZAH5iAqFQ57VIVP3NHJphLJ4rpM6CreuJyTRKdaCff6cA6Kyp5fdCfK0bZ8nhDh693c BIhxyKY9Sj5/MXrPbfutnbaM+T6YtCLWzNZFkm2ESsBNLjaT+ESufOx14/L9tyK7dosg6OZU 67y4sl6w4BeBRH2Wl24oV1fjyKV8aJKAc+Mf47W7lTke18WG+MqrY+ag==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 05/24] devtool: ide-sdk: auto-write image debug settings to bbappend Date: Sun, 30 Aug 2026 16:28:32 +0200 Message-ID: <20260830142922.17241-6-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244669 From: Adrian Freihofer Simplify devtool ide-sdk usage by automatically generating the image modifications required for debugging, instead of requiring manual local.conf edits. Generate the needed IMAGE_* and debug-server settings in the workspace image bbappend based on the selected recipes and debugger. Add only settings that are still missing, and remove the generated section again when it becomes unnecessary, so configuration stays minimal and up to date. This was not done before because rewriting image metadata after parse caused basehash inconsistency warnings. That is now handled by controlling the parse flow and allowing the required additional parse round(s) after bbappend updates. Build in two phases so debugger ports are known before the image build, and handle bbappend updates in a way that avoids basehash non-determinism warnings. This allows to simplify the documentation. We can remove the instructions for manually configuring IMAGE_GEN_DEBUGFS, IMAGE_FSTYPES_DEBUGFS, IMAGE_CLASSES as well as appends to IMAGE_INSTALL in local.conf. That just happens automatically now when running devtool ide-sdk. Support also bbappend files with version in the file name like for example: foo_1.2.bbappend or foo_git.bbappend. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_sdk.py | 251 +++++++++++++++++++++++++++------ 1 file changed, 206 insertions(+), 45 deletions(-) diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index d7575ec523..b62a0f612e 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -185,32 +185,79 @@ class RecipeImage: rootfs must be created as part of the SDK. """ - def __init__(self, name): - self.combine_dbg_image = False - self.gdbserver_missing = False + MARKER = '# devtool ide-sdk: image debug settings' + + def __init__(self, name, orig_bbappend_content=None): self.name = name self.rootfs = None self.__rootfs_dbg = None self.bootstrap_tasks = [self.name + ':do_build'] + # Debug settings already provided by the base configuration (e.g. + # local.conf, MACHINE, DISTRO, the recipe itself) plus any bbappend + # content other than devtool ide-sdk's own section (see + # strip_bbappend_sections()). Populated by initialize(). + self.base_image_gen_debugfs = False + self.base_image_fstypes_debugfs = '' + self.base_has_combined_dbg = False + self.base_image_install = set() + self._bbappend = None + # Content of the bbappend before strip_bbappend_sections() ran. + self._orig_bbappend_content = orig_bbappend_content + + @classmethod + def _strip_marker_section(cls, content): + """Remove devtool ide-sdk's own image debug settings section, if any""" + return re.sub( + r'^' + re.escape(cls.MARKER) + r'\n(?:[^\n]+\n)*', + '', content, flags=re.MULTILINE) + + @classmethod + def strip_bbappend_sections(cls, config, recipe_names): + """Remove devtool ide-sdk's own bbappend section from earlier runs + + Returns a {recipe name: content before stripping} dict for the recipes + that have a bbappend, to be passed on to the RecipeImage constructor. + """ + originals = {} + appends_dir = os.path.join(config.workspace_path, 'appends') + for name in recipe_names: + bbappend = os.path.join(appends_dir, name + '.bbappend') + if not os.path.exists(bbappend): + continue + with open(bbappend, 'r') as f: + content = f.read() + originals[name] = content + stripped = cls._strip_marker_section(content) + if stripped != content: + with open(bbappend, 'w') as f: + f.write(stripped) + return originals def initialize(self, config, tinfoil): + appends_dir = os.path.join(config.workspace_path, 'appends') + self._bbappend = os.path.join(appends_dir, self.name + '.bbappend') + + # strip_bbappend_sections() ran before the tinfoil session started, so + # this parse sees the same bbappend content as any other parse of this + # recipe until the tinfoil session ends. image_d = parse_recipe( config, tinfoil, self.name, appends=True, filter_workspace=False) if not image_d: raise DevtoolError( "Parsing image recipe %s failed" % self.name) - self.combine_dbg_image = bb.data.inherits_class( + self.base_image_gen_debugfs = image_d.getVar( + 'IMAGE_GEN_DEBUGFS') == '1' + self.base_image_fstypes_debugfs = image_d.getVar( + 'IMAGE_FSTYPES_DEBUGFS') or '' + self.base_has_combined_dbg = bb.data.inherits_class( 'image-combined-dbg', image_d) + self.base_image_install = set( + (image_d.getVar('IMAGE_INSTALL') or '').split()) workdir = image_d.getVar('WORKDIR') self.rootfs = os.path.join(workdir, 'rootfs') - if image_d.getVar('IMAGE_GEN_DEBUGFS') == "1": - self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg') - - package_install = image_d.getVar('PACKAGE_INSTALL').split() - self.gdbserver_missing = 'gdbserver' not in package_install - self.lldb_server_missing = 'lldb-server' not in package_install + self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg') @property def debug_support(self): @@ -222,6 +269,76 @@ class RecipeImage: return self.__rootfs_dbg return None + def update_image_bbappend(self, recipes_modified): + """Write debug settings for modified-mode recipes into the image bbappend. + + Writes IMAGE_GEN_DEBUGFS, IMAGE_FSTYPES_DEBUGFS, IMAGE_CLASSES for + image-combined-dbg, the appropriate debug server (gdbserver or + lldb-server), and IMAGE_INSTALL entries for each modified recipe + (including the ptest package when the recipe inherits ptest). + + initialize() already stripped this section from the bbappend on + disk before parsing, so it only needs to be added back here, if + still needed. Returns True if the resulting bbappend content + actually differs from what was on disk when initialize() ran, False + if it is left exactly as it was. + """ + wants_gdbserver = any( + r.wants_gdbserver and r.toolchain != 'clang' + for r in recipes_modified) + wants_lldb_server = any( + r.wants_gdbserver and r.toolchain == 'clang' + for r in recipes_modified) + + # Only add what the base configuration (e.g. local.conf) does not + # already provide, to avoid duplicate/conflicting settings. + lines = [] + if not self.base_image_gen_debugfs: + lines.append('IMAGE_GEN_DEBUGFS = "1"') + if self.base_image_fstypes_debugfs != '': + lines.append('IMAGE_FSTYPES_DEBUGFS = ""') + if not self.base_has_combined_dbg: + lines.append('IMAGE_CLASSES += "image-combined-dbg"') + if wants_gdbserver and 'gdbserver' not in self.base_image_install: + lines.append('IMAGE_INSTALL:append = " gdbserver"') + if wants_lldb_server and 'lldb-server' not in self.base_image_install: + lines.append('IMAGE_INSTALL:append = " lldb-server"') + for r in recipes_modified: + if r.name not in self.base_image_install: + lines.append('IMAGE_INSTALL:append = " %s"' % r.name) + if r.has_ptest and (r.name + '-ptest') not in self.base_image_install: + lines.append('IMAGE_INSTALL:append = " %s-ptest"' % r.name) + + original_content = self._orig_bbappend_content or '' + # strip_bbappend_sections() left this on disk, and it is what bitbake + # parsed. Any difference from it invalidates the parsed basehashes. + if os.path.exists(self._bbappend): + with open(self._bbappend, 'r') as f: + parsed_content = f.read() + else: + parsed_content = '' + + if not lines: + # The base configuration already provides everything needed. + if parsed_content != original_content: + logger.info( + "Removed image debug settings from %s: already provided by the base configuration", self._bbappend) + return False + + new_section = self.MARKER + '\n' + '\n'.join(lines) + '\n' + new_content = parsed_content + if new_content and not new_content.endswith('\n'): + new_content += '\n' + new_content += new_section + + appends_dir = os.path.dirname(self._bbappend) + os.makedirs(appends_dir, exist_ok=True) + with open(self._bbappend, 'w') as f: + f.write(new_content) + + logger.info("Updated image bbappend %s", self._bbappend) + return True + class RecipeMetaIdeSupport: """For the shared sysroots mode meta-ide-support is needed @@ -475,6 +592,8 @@ class RecipeModified: # Whether to warn when DEBUG_BUILD is not set. Kernel modules are built # by the kernel's build system and DEBUG_BUILD does not influence them. self.wants_debug_build = True + # Whether this recipe provides a ptest package + self.has_ptest = False # build_tool = cmake self.oecmake_generator = None self.cmake_cache_vars = None @@ -592,6 +711,8 @@ class RecipeModified: self.reverse_debug_prefix_map = self._init_reverse_debug_prefix_map( recipe_d.getVar('DEBUG_PREFIX_MAP')) + self.has_ptest = bb.data.inherits_class('ptest', recipe_d) + # Recipe ID is the identifier for IDE config sections self.recipe_id = self.bpn + "-" + self.package_arch self.recipe_id_pretty = self.bpn + ": " + self.package_arch @@ -1235,9 +1356,21 @@ def ide_setup(args, config, basepath, workspace): if recipe in ['meta-ide-support', 'build-sysroots']: raise DevtoolError("Invalid recipe: %s." % recipe) - # Collect information about tasks which need to be bitbaked + # Collect information about tasks which need to be bitbaked. + # In modified mode the image build is held back until after + # setup_modified_recipe() has assigned the debugger port numbers and + # update_image_bbappend() has written the complete bbappend. That way + # the image is built with a single, stable recipe hash so that no + # basehash-changed warnings are emitted. bootstrap_tasks = [] bootstrap_tasks_late = [] + image_bootstrap_tasks = [] + # Must happen before setup_tinfoil() so that every parse in this session + # sees the same bbappend content. Which of the recipes is the image is only + # known after parsing, so this covers all of them; for a recipe without an + # ide-sdk section it is a no-op. + orig_bbappend_contents = RecipeImage.strip_bbappend_sections( + config, args.recipenames) tinfoil = setup_tinfoil(config_only=False, basepath=basepath) try: # define mode depending on recipes which need to be processed @@ -1269,8 +1402,9 @@ def ide_setup(args, config, basepath, workspace): if not recipes_modified_names: appends_dir = os.path.join(config.workspace_path, 'appends') recipes_modified_names = sorted( - os.path.splitext(os.path.basename(path))[0] - for path in glob.glob(os.path.join(appends_dir, '*.bbappend'))) + bb.parse.vars_from_file(path, None)[0] + for path in glob.glob(os.path.join(appends_dir, '*.bbappend')) + if bb.parse.vars_from_file(path, None)[0] not in recipes_image_names) if recipes_modified_names: logger.info( "No modified recipes specified, using workspace bbappends from %s: %s", @@ -1302,9 +1436,17 @@ def ide_setup(args, config, basepath, workspace): recipes_images = [] for recipes_image_name in recipes_image_names: logger.info("Using image: %s" % recipes_image_name) - recipe_image = RecipeImage(recipes_image_name) + recipe_image = RecipeImage( + recipes_image_name, + orig_bbappend_contents.get(recipes_image_name)) recipe_image.initialize(config, tinfoil) - bootstrap_tasks += recipe_image.bootstrap_tasks + if args.mode == DevtoolIdeMode.modified: + # Keep the image build separate so that the complete bbappend + # can be written in one step before the image is built, + # avoiding sstate hash mismatches. + image_bootstrap_tasks += recipe_image.bootstrap_tasks + else: + bootstrap_tasks += recipe_image.bootstrap_tasks recipes_images.append(recipe_image) # Provide a Direct SDK with shared sysroots @@ -1357,37 +1499,22 @@ def ide_setup(args, config, basepath, workspace): finally: tinfoil.shutdown() + bb_cmd = 'bitbake ' + if args.bitbake_k: + bb_cmd += "-k " + if not args.skip_bitbake: - bb_cmd = 'bitbake ' - if args.bitbake_k: - bb_cmd += "-k " - bb_cmd_early = bb_cmd + ' '.join(bootstrap_tasks) - exec_build_env_command( - config.init_path, basepath, bb_cmd_early, watch=True) - if bootstrap_tasks_late: - bb_cmd_late = bb_cmd + ' '.join(bootstrap_tasks_late) + # Phase 1: build modified recipes and debug tools so that + # installed_binaries is populated and port numbers can be assigned. + # The image is built in phase 2, after the complete bbappend is written. + if bootstrap_tasks: exec_build_env_command( - config.init_path, basepath, bb_cmd_late, watch=True) - - wants_gdbserver = any( - r.wants_gdbserver and r.toolchain == 'gcc' - for r in recipes_modified) - wants_lldb_server = any( - r.wants_gdbserver and r.toolchain == 'clang' - for r in recipes_modified) - for recipe_image in recipes_images: - if wants_gdbserver and recipe_image.gdbserver_missing: - logger.warning( - "gdbserver not installed in image %s. Remote debugging will not be available" % recipe_image) - if wants_lldb_server and recipe_image.lldb_server_missing: - logger.warning( - "lldb-server not installed in image %s. " - "Remote debugging with LLDB (CodeLLDB) will not be available. " - "Add 'lldb-server' to IMAGE_INSTALL." % recipe_image) - - if (wants_gdbserver or wants_lldb_server) and recipe_image.combine_dbg_image is False: - logger.warning( - 'IMAGE_CLASSES += "image-combined-dbg" is missing for image %s. Remote debugging will not find debug symbols from rootfs-dbg.' % recipe_image) + config.init_path, basepath, + bb_cmd + ' '.join(bootstrap_tasks), watch=True) + if bootstrap_tasks_late: + exec_build_env_command( + config.init_path, basepath, + bb_cmd + ' '.join(bootstrap_tasks_late), watch=True) # Instantiate the active IDE plugin ide = ide_plugins[args.ide]() @@ -1408,6 +1535,39 @@ def ide_setup(args, config, basepath, workspace): 'You might want to add DEBUG_BUILD = "1" to %s. ' 'Note that devtool modify --debug-build can do this automatically.', recipe_modified.name, recipe_modified.bbappend) + + # Ports are now assigned. Write the complete image bbappend in a + # single step so that the image is built with exactly one recipe + # hash. This avoids the sstate basehash-changed warnings that + # arise when the bbappend is modified after the image has + # already been built. This also runs with --skip-bitbake, otherwise + # the section removed by strip_bbappend_sections() would be lost. + bbappend_changed = False + for ri in recipes_images: + if ri.update_image_bbappend(recipes_modified): + bbappend_changed = True + + if not args.skip_bitbake: + if image_bootstrap_tasks: + if bbappend_changed: + # The bbappend content just written differs from the one + # bitbake parsed during the tinfoil session above. With a + # memory resident server that session's basehashes are + # still cached, so reparsing would report "basehash value + # changed ... not deterministic" for every task. Tell + # bitbake the recipe intentionally changed by resetting the + # cooker: this clears its basehash history. + reparse_tinfoil = setup_tinfoil(config_only=True, basepath=basepath) + try: + reparse_tinfoil.run_command('resetCooker') + reparse_tinfoil.parse_recipes() + finally: + reparse_tinfoil.shutdown() + + # Phase 2: build the image + exec_build_env_command( + config.init_path, basepath, + bb_cmd + ' '.join(image_bootstrap_tasks), watch=True) else: raise DevtoolError("Must not end up here.") @@ -1483,7 +1643,8 @@ def register_commands(subparsers, context): parser_ide_sdk.add_argument( '-I', '--key', help='Specify ssh private key for connection to the target') parser_ide_sdk.add_argument( - '--skip-bitbake', help='Generate IDE configuration but skip calling bitbake to update the SDK', action='store_true') + '--skip-bitbake', help='Skip the bitbake builds which update the SDK. The recipes are still parsed, ' + 'the IDE configuration is generated from their metadata', action='store_true') parser_ide_sdk.add_argument( '-k', '--bitbake-k', help='Pass -k parameter to bitbake', action='store_true') parser_ide_sdk.add_argument( From patchwork Sun Aug 30 14:28:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96814 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C194C61DFD for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9453.1788100183423622171 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=U5FOK71D; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-2026083014294068bad33313000207a3-g5tgoh@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 2026083014294068bad33313000207a3 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=K1qrykSNcjEu7y502YB/cPZ2jFJx5l8WZmcl82y5JFc=; b=U5FOK71D/ECR9yS/iDbGoF+Hm5fUXLNrtU7MfRlO6JQf2ofq15NsLMqpbKuGriQCVGR1Xd cdWD/z6ZzbgPv+c1szi4EOL8CT+ogw8LOAFZFaGdftvAOD2vCWW/kht3OQIMQ7wnZI7linUx J9qTL1lRFqLvRk4CUowYKZVM/j3pmduGs39gM2hTKkYMIZdr1Jv+DLQDli88kX4u3s1UaN9+ QWlgClr3yZGNNhWTgtA2PWNJ6dWqBlIRMmHsujqJrDJESlGRAXsK7g10TTrVelaKvufzHyWA cy0sD0cr2e3yTwrXqBiDk8925Gn24AhWNeVJm6c6c+1COCYq1LKfcI/g==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 06/24] oe-selftest: devtool ide-sdk: adapt tests for auto-written image debug settings Date: Sun, 30 Aug 2026 16:28:33 +0200 Message-ID: <20260830142922.17241-7-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244676 From: Adrian Freihofer devtool ide-sdk now writes IMAGE_GEN_DEBUGFS, IMAGE_CLASSES += "image-combined-dbg" and the required IMAGE_INSTALL entries (gdbserver/lldb-server, per-recipe ptest packages) to the image workspace bbappend automatically. _write_bb_config() therefore no longer sets these in local.conf, so tests also exercise that ide-sdk does not need help from local.conf to enable image debug support. Tests that use runqemu now call devtool ide-sdk once before starting QEMU to build the image with the debug settings already applied, and again with --skip-bitbake -t root@ once the real target IP is known, to update the deploy scripts without triggering a redundant image rebuild. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 111 +++++++++++++----------- 1 file changed, 61 insertions(+), 50 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 87efa1ee80..1ecf4c24f5 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -2889,15 +2889,15 @@ class DevtoolIdeSdkTests(DevtoolBase): if self.logger.isEnabledFor(logging.DEBUG): self._cmd_logger = self.logger - def _write_bb_config(self, recipe_names, extra_packages=None): - """Helper to write the bitbake local.conf file""" - image_install = 'gdbserver ' + ' '.join([r + '-ptest' for r in recipe_names]) - if extra_packages: - image_install += ' ' + ' '.join(extra_packages) + def _write_bb_config(self): + """Helper to write the bitbake local.conf file. + + Image debug settings (IMAGE_GEN_DEBUGFS, IMAGE_CLASSES += image-combined-dbg, + IMAGE_INSTALL for gdbserver/lldb-server and the recipe under test) are written + automatically by devtool ide-sdk to the image workspace bbappend. + Only settings not managed by ide_sdk are configured here. + """ conf_lines = [ - 'IMAGE_CLASSES += "image-combined-dbg"', - 'IMAGE_GEN_DEBUGFS = "1"', - 'IMAGE_INSTALL:append = " %s"' % image_install, 'DISTRO_FEATURES:append = " ptest"', # Static UIDs/GIDs are required so that files installed via # "install -o ${BPN}" in do_install embed the same UID that gets @@ -3374,70 +3374,67 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): testimage = "oe-selftest-image" self._check_workspace() - self._write_bb_config(recipe_names) + self._write_bb_config() self._check_runqemu_prerequisites() # Verify deployment to Qemu (system mode) works - bitbake(testimage) + tempdir_cmake = self._devtool_ide_sdk_recipe("cmake-example", "CMakeLists.txt", None) + tempdir_meson = self._devtool_ide_sdk_recipe("meson-example", "meson.build", testimage) + runCmd('devtool ide-sdk cmake-example meson-example %s -c --ide=none' % testimage, + output_log=self._cmd_logger) + with runqemu(testimage, runqemuparams="nographic") as qemu: # cmake-example recipe recipe_name = "cmake-example" example_exe = "cmake-example" example_user_group = "cmake-example" conf_file = "/etc/cmake-example.conf" - build_file = "CMakeLists.txt" # Verify the cmake-example service is running on the target self._verify_service_running(qemu, example_exe) # Verify /etc/cmake-example.conf is owned by the cmake-example user self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group) - # Setup the recipe with devtool ide-sdk cmake-example ... - tempdir = self._devtool_ide_sdk_recipe( - recipe_name, build_file, testimage) - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=none' % ( + # Re-run ide-sdk with the actual QEMU IP; image is already built + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % ( recipe_name, testimage, qemu.ip) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) self._gdb_cross() - compile_cmd = self._verify_cmake_preset(tempdir) - self._devtool_ide_sdk_qemu(tempdir, qemu, recipe_name, example_exe, compile_cmd) + compile_cmd = self._verify_cmake_preset(tempdir_cmake) + self._devtool_ide_sdk_qemu(tempdir_cmake, qemu, recipe_name, example_exe, compile_cmd) # Verify the oe-scripts sym-link is valid self.assertEqual(self._workspace_scripts_dir( - recipe_name), self._sources_scripts_dir(tempdir)) + recipe_name), self._sources_scripts_dir(tempdir_cmake)) # Verify /etc/cmake-example.conf is still owned by the cmake-example user # after the install and deploy scripts updated the file self._verify_conf_file(qemu, conf_file, example_exe, example_exe) - # meson-example recipe recipe_name = "meson-example" example_exe = "mesonex" example_user_group = "meson-example" conf_file = "/etc/meson-example.conf" - build_file = "meson.build" # Verify the meson-example service is running on the target self._verify_service_running(qemu, example_exe) # Verify /etc/meson-example.conf is owned by the meson-example user self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group) - # Setup the recipe with devtool ide-sdk meson-example ... - tempdir = self._devtool_ide_sdk_recipe( - recipe_name, build_file, testimage) - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=none' % ( + # Re-run ide-sdk with the actual QEMU IP; image is already built + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % ( recipe_name, testimage, qemu.ip) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) self._gdb_cross() - compile_cmd = self._verify_meson_build(tempdir, recipe_name) - self._devtool_ide_sdk_qemu(tempdir, qemu, recipe_name, example_exe, compile_cmd) + compile_cmd = self._verify_meson_build(tempdir_meson, recipe_name) + self._devtool_ide_sdk_qemu(tempdir_meson, qemu, recipe_name, example_exe, compile_cmd) # Verify the oe-scripts sym-link is valid self.assertEqual(self._workspace_scripts_dir( - recipe_name), self._sources_scripts_dir(tempdir)) + recipe_name), self._sources_scripts_dir(tempdir_meson)) # Verify /etc/meson-example.conf is still owned by the meson-example user # after the install and deploy scripts updated the file @@ -3784,15 +3781,17 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): build_file = "CMakeLists.txt" self._check_workspace() - self._write_bb_config([recipe_name]) + self._write_bb_config() - # Verify deployment to Qemu (system mode) works + # Build image with debug settings before starting QEMU self._check_runqemu_prerequisites() - bitbake(testimage) + tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage), + output_log=self._cmd_logger) + with runqemu(testimage, runqemuparams="nographic") as qemu: - tempdir = self._devtool_ide_sdk_recipe( - recipe_name, build_file, testimage) - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=code' % ( + # Re-run with actual QEMU IP; image is already built + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % ( recipe_name, testimage, qemu.ip) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) self._verify_cmake_preset(tempdir) @@ -3814,15 +3813,18 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): testimage = "oe-selftest-image" self._check_workspace() - self._write_bb_config([recipe_name]) + self._write_bb_config() - # Verify deployment to Qemu (system mode) works + # Build image with debug settings before starting QEMU self._check_runqemu_prerequisites() - bitbake(testimage) + tempdir = self._devtool_ide_sdk_recipe( + recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage), + output_log=self._cmd_logger) + with runqemu(testimage, runqemuparams="nographic") as qemu: - tempdir = self._devtool_ide_sdk_recipe( - recipe_name, build_file, testimage) - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=code' % ( + # Re-run with actual QEMU IP; image is already built + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % ( recipe_name, testimage, qemu.ip) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) @@ -4507,13 +4509,17 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): testimage = "oe-selftest-image" self._check_workspace() - self._write_bb_config([recipe_name], extra_packages=['lldb-server']) + self._write_bb_config() + # Build image with debug settings (lldb-server for clang) before starting QEMU self._check_runqemu_prerequisites() - bitbake(testimage) + tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage), + output_log=self._cmd_logger) + with runqemu(testimage, runqemuparams="nographic") as qemu: - tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage) - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=code' % ( + # Re-run with actual QEMU IP; image is already built + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % ( recipe_name, testimage, qemu.ip) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) @@ -4566,7 +4572,7 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): testimage = "oe-selftest-image" self._check_workspace() - self._write_bb_config([recipe_name], extra_packages=['lldb-server']) + self._write_bb_config() # Build image with debug settings (lldb-server for clang) before starting QEMU self._check_runqemu_prerequisites() @@ -4678,12 +4684,17 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): testimage = 'oe-selftest-image' self._check_workspace() - self._write_bb_config([recipe_name], extra_packages=['lldb-server']) + self._write_bb_config() self._check_runqemu_prerequisites() - bitbake(testimage) + + # Build image with debug settings (lldb-server for clang) before starting QEMU + tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=none' % (recipe_name, testimage), + output_log=self._cmd_logger) + with runqemu(testimage, runqemuparams='nographic') as qemu: - tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage) - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=none' % ( + # Re-run with actual QEMU IP; image is already built + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % ( recipe_name, testimage, qemu.ip) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) @@ -4742,7 +4753,7 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): testimage = 'oe-selftest-image' self._check_workspace() - self._write_bb_config([recipe_name], extra_packages=['lldb-server']) + self._write_bb_config() self._check_runqemu_prerequisites() # Build image with debug settings (lldb-server for clang) before starting QEMU @@ -4803,7 +4814,7 @@ class DevtoolIdeSdkMiscTests(DevtoolIdeSdkTests): shared_recipe_name = "cmake-example" self._check_workspace() - self._write_bb_config([modified_recipe_name]) + self._write_bb_config() tempdir = self._devtool_ide_sdk_recipe( modified_recipe_name, modified_build_file, None) From patchwork Sun Aug 30 14:28:34 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68C84C624A3 for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9454.1788100183423738933 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=iXTwJIkV; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-2026083014294168724626d6000207e5-rla8vf@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 2026083014294168724626d6000207e5 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=qQsiGQuN9PWLAXM0tV2Si2q1EIjTUFuCISwIE2mvvFs=; b=iXTwJIkVe07MRtQwu4/uBfo3n0Rlz0ACXyxpZH7oirlSlTc6uzCtmDpEzZbkhOYKwBtlOx hlkZTwPZK9RKSv7UZPdDQcTHK0P7pRCUjIucnA5QQZTXEvShCW+Obop6EHexSAHp0yM822kY mBqokg1KlMKfbXWCLyT8/Y2g0i+y6BymiVMAhTkpdWifjH+AEa2ZXEv/aGzOrdRfkhZ9AbMP op3mlha9QnLSwhsOfUYX3ch9rdrbZ7rOWa8qWukq+NkCrZ0Ir0sDTM6VaBCMJC6UlJN3Luyn KYu8o2GzrUvUmIBmgS6MFmmWd6QlrqbWDFYFpkeq9ot7iyeq+AHZ+HfQ==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 07/24] oeqa: QemuTarget: set use_slirp when slirp is in runqemuparams Date: Sun, 30 Aug 2026 16:28:34 +0200 Message-ID: <20260830142922.17241-8-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244673 From: Adrian Freihofer QemuRunner needs use_slirp=True to parse the guest IP address from the hostfwd=tcp: entry in the QEMU command line rather than from the TAP-style ip= kernel argument. QemuRunner is constructed before the runqemuparams are known, so use_slirp defaults to False. Set self.runner.use_slirp = True in QemuTarget.start() when 'slirp' appears in runqemuparams, before calling the runner. This is the earliest point at which runqemuparams is available and is the natural place to propagate mode flags to the already-constructed runner. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/targetcontrol.py | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py index fa6b43bb35..1deea2e5ce 100644 --- a/meta/lib/oeqa/targetcontrol.py +++ b/meta/lib/oeqa/targetcontrol.py @@ -24,6 +24,7 @@ class BaseTarget(object, metaclass=ABCMeta): def __init__(self, d, logger): self.connection = None self.ip = None + self.port = None self.server_ip = None self.datetime = d.getVar('DATETIME') self.testdir = d.getVar("TEST_LOG_DIR") @@ -163,6 +164,8 @@ class QemuTarget(BaseTarget): super(QemuTarget, self).deploy() def start(self, params=None, ssh=True, extra_bootparams='', runqemuparams='', launch_cmd='', discard_writes=True): + if 'slirp' in runqemuparams.split(): + self.runner.use_slirp = True if launch_cmd: start = self.runner.launch(get_ip=ssh, launch_cmd=launch_cmd, qemuparams=params) else: @@ -171,8 +174,16 @@ class QemuTarget(BaseTarget): if start: if ssh: self.ip = self.runner.ip + self.port = None + if self.runner.use_slirp: + # In slirp mode the runner's ip is "host:port" + ip_port = self.runner.ip.split(':') + if len(ip_port) == 2: + self.ip, self.port = ip_port + else: + self.logger.error("Could not get host machine port to connect qemu with slirp, ssh will not be able to connect to qemu with slirp") self.server_ip = self.runner.server_ip - self.connection = SSHControl(ip=self.ip, logfile=self.sshlog) + self.connection = SSHControl(ip=self.ip, logfile=self.sshlog, port=self.port) else: self.stop() if os.path.exists(self.qemulog): @@ -192,13 +203,21 @@ class QemuTarget(BaseTarget): self.loggerhandler.close() self.connection = None self.ip = None + self.port = None self.server_ip = None def restart(self, params=None): if self.runner.restart(params): self.ip = self.runner.ip + self.port = None + if self.runner.use_slirp: + ip_port = self.runner.ip.split(':') + if len(ip_port) == 2: + self.ip, self.port = ip_port + else: + self.logger.error("Could not get host machine port to connect qemu with slirp, ssh will not be able to connect to qemu with slirp") self.server_ip = self.runner.server_ip - self.connection = SSHControl(ip=self.ip, logfile=self.sshlog) + self.connection = SSHControl(ip=self.ip, logfile=self.sshlog, port=self.port) else: raise RuntimeError("%s - FAILED to re-start qemu - check the task log and the boot log" % self.pn) From patchwork Sun Aug 30 14:28:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96824 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CD5DC624C8 for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9432.1788100183423521046 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=gg+maEF2; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-1329275-202608301429414a50b2e19d000207ed-ll1jy2@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 202608301429414a50b2e19d000207ed for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=0bzlD3QpycxDG+Xrm3mTE+g+QZ6QQ4yxG6qumr+cxyA=; b=gg+maEF2ypLGuUuDl4S4GREKFmUMspPAxX3ur81sKJTvWjJZ6J5H+5/1TnjgbauV9QTFyO OWTwAtAgnPWJGzv2JJSyLgiiv26ixLT++5dbTwJsI5J7PQQAYwuRwArUDZyuxCqtuF7pJzXk fnaA+DssYmXbNvOpPlFcVrkqXbaKwmjeKml7skIxguFdmLlLl5wRHgMWDPFhp20rzyAe5RKz M5Kyeu0U6elnYlwA6b+XlKidCw8mBHfdKACSXjVi+kxsy3w9snUUWA1vUrVYNtKsZ8zkKgF4 IpvFCbCtRhIss8Q0Iv86YzCo4gcfpuba53YTMEZUl0tlm63J67JLw6og==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 08/24] devtool: ide-sdk: support runqemu slirp Date: Sun, 30 Aug 2026 16:28:35 +0200 Message-ID: <20260830142922.17241-9-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244668 From: Adrian Freihofer ide-sdk needs to make sure the debugger server ports are reachable, so when runqemu uses slirp networking, add the required port forwards to QB_SLIRP_OPT in the image bbappend. Example: devtool ide-sdk oe-selftest-image -t root@localhost -P 2222 -c runqemu slirp serialstdio snapshot Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_plugins/__init__.py | 15 ++- scripts/lib/devtool/ide_plugins/ide_none.py | 6 +- scripts/lib/devtool/ide_sdk.py | 133 +++++++++++++++----- 3 files changed, 121 insertions(+), 33 deletions(-) diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py index 136ddd8914..a5069cae85 100644 --- a/scripts/lib/devtool/ide_plugins/__init__.py +++ b/scripts/lib/devtool/ide_plugins/__init__.py @@ -65,6 +65,10 @@ class DebuggerCrossConfig: DebuggerCrossConfig._port_next += 1 self.debug_server_port = self.debug_server_ports[self.default_mode] self.id_pretty = "%d_%s" % (self.debug_server_port, self.binary_pretty) + # Hook for subclasses needing additional fixed ports forwarded through + # slirp beyond the one-per-mode debug_server_ports (e.g. lldb-server's + # spawned gdbserver instances). + self.extra_ports = [] if self.id_pretty in DebuggerCrossConfig._configs: raise DevtoolError( @@ -269,6 +273,13 @@ class LldbServerConfig(DebuggerCrossConfig): default_mode=DebuggerServerModes.MULTI): super().__init__(image_recipe, modified_recipe, binary, default_mode) + # lldb-server platform spawns a separate gdb-remote-protocol + # "gdbserver" instance per debug session; without --gdbserver-port it + # picks a random port, which cannot be forwarded through slirp NAT. + # Pin it to a fixed, dedicated port that gets slirp-forwarded too. + self.gdbserver_port = DebuggerCrossConfig._port_next + DebuggerCrossConfig._port_next += 1 + self.extra_ports.append(self.gdbserver_port) def _lldb_server_tmp_dir(self, mode): return os.path.join('/tmp', 'lldb_server_%s' % self.id_pretty_mode(mode)) @@ -294,8 +305,8 @@ class LldbServerConfig(DebuggerCrossConfig): cmd = self._target_tcp_port_check_cmd() + " && exit 0; " cmd += "mkdir -p %s; " % tmp_dir cmd += "cd %s; " % tmp_dir - cmd += "%s platform --server --listen *:%s > %s 2>&1 & _lldb_server_pid=\\$!; " % ( - lldb_server, self.debug_server_port, log_file) + cmd += "%s platform --server --listen *:%s --gdbserver-port %s > %s 2>&1 & _lldb_server_pid=\\$!; " % ( + lldb_server, self.debug_server_port, self.gdbserver_port, log_file) cmd += "echo \\$_lldb_server_pid > %s; " % pid_file cmd += self._target_wait_for_tcp_port_cmd( "lldb_server_pid", log_file) diff --git a/scripts/lib/devtool/ide_plugins/ide_none.py b/scripts/lib/devtool/ide_plugins/ide_none.py index 959140cedb..e4b255f2fa 100644 --- a/scripts/lib/devtool/ide_plugins/ide_none.py +++ b/scripts/lib/devtool/ide_plugins/ide_none.py @@ -207,10 +207,12 @@ class LldbServerConfigNone(LldbServerConfig): lines = ['# This file is generated by devtool ide-sdk'] if server_mode == DebuggerServerModes.MULTI: lines.append('# On the remote target:') - lines.append('# lldb-server platform --server --listen *:%d' % self.debug_server_port) + lines.append('# lldb-server platform --server --listen *:%d --gdbserver-port %d' % ( + self.debug_server_port, self.gdbserver_port)) else: lines.append('# On the remote target:') - lines.append('# lldb-server platform --one-shot --server --listen *:%d' % self.debug_server_port) + lines.append('# lldb-server platform --one-shot --server --listen *:%d --gdbserver-port %d' % ( + self.debug_server_port, self.gdbserver_port)) lines.append('# On the build machine:') lines.append('# cd ' + self.modified_recipe.real_srctree) lines.append('# ' + self.debugger_cross.lldb + ' -s ' + self.lldbinit) diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index b62a0f612e..719648a3eb 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -23,7 +23,7 @@ import scriptutils import bb from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, DevtoolError, parse_recipe from devtool.standard import get_real_srctree -from devtool.ide_plugins import BuildTool +from devtool.ide_plugins import BuildTool, DebuggerCrossConfig from oe.kernel_module import kernel_module_os_env @@ -186,15 +186,17 @@ class RecipeImage: """ MARKER = '# devtool ide-sdk: image debug settings' + QB_SLIRP_MARKER = '# devtool ide-sdk: QB_SLIRP_OPT' def __init__(self, name, orig_bbappend_content=None): self.name = name self.rootfs = None self.__rootfs_dbg = None + self.qb_slirp_opt = '' self.bootstrap_tasks = [self.name + ':do_build'] # Debug settings already provided by the base configuration (e.g. # local.conf, MACHINE, DISTRO, the recipe itself) plus any bbappend - # content other than devtool ide-sdk's own section (see + # content other than devtool ide-sdk's own sections (see # strip_bbappend_sections()). Populated by initialize(). self.base_image_gen_debugfs = False self.base_image_fstypes_debugfs = '' @@ -204,11 +206,11 @@ class RecipeImage: # Content of the bbappend before strip_bbappend_sections() ran. self._orig_bbappend_content = orig_bbappend_content - @classmethod - def _strip_marker_section(cls, content): - """Remove devtool ide-sdk's own image debug settings section, if any""" + @staticmethod + def _strip_marker_section(content, marker): + """Remove one devtool ide-sdk marker section, if present""" return re.sub( - r'^' + re.escape(cls.MARKER) + r'\n(?:[^\n]+\n)*', + r'^' + re.escape(marker) + r'\n(?:[^\n]+\n)*', '', content, flags=re.MULTILINE) @classmethod @@ -227,7 +229,8 @@ class RecipeImage: with open(bbappend, 'r') as f: content = f.read() originals[name] = content - stripped = cls._strip_marker_section(content) + stripped = cls._strip_marker_section(content, cls.MARKER) + stripped = cls._strip_marker_section(stripped, cls.QB_SLIRP_MARKER) if stripped != content: with open(bbappend, 'w') as f: f.write(stripped) @@ -259,6 +262,8 @@ class RecipeImage: self.rootfs = os.path.join(workdir, 'rootfs') self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg') + self.qb_slirp_opt = image_d.getVar('QB_SLIRP_OPT') or '' + @property def debug_support(self): return bool(self.rootfs_dbg) @@ -279,9 +284,10 @@ class RecipeImage: initialize() already stripped this section from the bbappend on disk before parsing, so it only needs to be added back here, if - still needed. Returns True if the resulting bbappend content - actually differs from what was on disk when initialize() ran, False - if it is left exactly as it was. + still needed. Also updates QB_SLIRP_OPT with the debugger server + port forwards (see update_qb_slirp_opt()). Returns True if the + resulting bbappend content actually differs from what was on disk + when initialize() ran, False if it is left exactly as it was. """ wants_gdbserver = any( r.wants_gdbserver and r.toolchain != 'clang' @@ -319,24 +325,90 @@ class RecipeImage: parsed_content = '' if not lines: - # The base configuration already provides everything needed. - if parsed_content != original_content: + if self.MARKER in original_content: logger.info( "Removed image debug settings from %s: already provided by the base configuration", self._bbappend) + image_changed = False + else: + new_section = self.MARKER + '\n' + '\n'.join(lines) + '\n' + new_content = parsed_content + if new_content and not new_content.endswith('\n'): + new_content += '\n' + new_content += new_section + + appends_dir = os.path.dirname(self._bbappend) + os.makedirs(appends_dir, exist_ok=True) + with open(self._bbappend, 'w') as f: + f.write(new_content) + logger.info("Updated image bbappend %s", self._bbappend) + image_changed = True + + slirp_changed = self.update_qb_slirp_opt() + return image_changed or slirp_changed + + def update_qb_slirp_opt(self): + """Update QB_SLIRP_OPT in the image bbappend + + Support connecting to a debugger server running on the target device via + runqemu's slirp network: + - If the base value is non-empty (recipe/machine sets QB_SLIRP_OPT): + only missing port forwards are appended via QB_SLIRP_OPT:append. + - If the base value is empty (runqemu would use its own built-in default + of SSH 2222, telnet 2323, tftp): a full QB_SLIRP_OPT assignment is + written that mirrors that default plus the debugger ports, so that + runqemu reads the complete set from the .qemuboot.conf. + + Returns True if the bbappend content actually changed, False otherwise. + """ + ports = sorted({port for cfg in DebuggerCrossConfig._configs.values() + for port in list(cfg.debug_server_ports.values()) + cfg.extra_ports}) + if not ports: return False - new_section = self.MARKER + '\n' + '\n'.join(lines) + '\n' - new_content = parsed_content + # Determine which ports are already in the base value + already = {int(m.group(1)) + for m in re.finditer(r':(\d+)-:\d+', self.qb_slirp_opt)} + missing_ports = [p for p in ports if p not in already] + if not missing_ports: + logger.info("QB_SLIRP_OPT already contains all needed port forwards") + return False + + if self.qb_slirp_opt: + # Base value exists: :append only the missing port forwards + extra = ''.join( + ',hostfwd=tcp:127.0.0.1:%d-:%d' % (p, p) for p in missing_ports) + new_line = 'QB_SLIRP_OPT:append = "%s"' % extra + else: + # No base value: mirror runqemu's built-in default (SSH 2222, telnet + # 2323, tftp) and add the debugger ports. + all_hostfwds = ( + 'hostfwd=tcp:127.0.0.1:2222-:22,' + 'hostfwd=tcp:127.0.0.1:2323-:23' + ) + all_hostfwds += ''.join( + ',hostfwd=tcp:127.0.0.1:%d-:%d' % (p, p) for p in missing_ports) + new_line = 'QB_SLIRP_OPT = "-netdev user,id=net0,%s,tftp=${DEPLOY_DIR_IMAGE}"' % all_hostfwds + + if os.path.exists(self._bbappend): + with open(self._bbappend, 'r') as f: + content = f.read() + else: + content = '' + stripped_content = self._strip_marker_section(content, self.QB_SLIRP_MARKER) + new_content = stripped_content if new_content and not new_content.endswith('\n'): new_content += '\n' - new_content += new_section + new_content += self.QB_SLIRP_MARKER + '\n' + new_line + '\n' + + if new_content == content: + logger.debug("QB_SLIRP_OPT in %s is already up to date", self._bbappend) + return False appends_dir = os.path.dirname(self._bbappend) os.makedirs(appends_dir, exist_ok=True) with open(self._bbappend, 'w') as f: f.write(new_content) - - logger.info("Updated image bbappend %s", self._bbappend) + logger.info("Updated QB_SLIRP_OPT in %s: %s", self._bbappend, new_line) return True @@ -1359,9 +1431,9 @@ def ide_setup(args, config, basepath, workspace): # Collect information about tasks which need to be bitbaked. # In modified mode the image build is held back until after # setup_modified_recipe() has assigned the debugger port numbers and - # update_image_bbappend() has written the complete bbappend. That way - # the image is built with a single, stable recipe hash so that no - # basehash-changed warnings are emitted. + # update_image_bbappend() has written the complete bbappend (including + # QB_SLIRP_OPT). That way the image is built with a single, stable + # recipe hash so that no basehash-changed warnings are emitted. bootstrap_tasks = [] bootstrap_tasks_late = [] image_bootstrap_tasks = [] @@ -1442,8 +1514,8 @@ def ide_setup(args, config, basepath, workspace): recipe_image.initialize(config, tinfoil) if args.mode == DevtoolIdeMode.modified: # Keep the image build separate so that the complete bbappend - # can be written in one step before the image is built, - # avoiding sstate hash mismatches. + # (IMAGE_ vars + QB_SLIRP_OPT) can be written in one step + # before the image is built, avoiding sstate hash mismatches. image_bootstrap_tasks += recipe_image.bootstrap_tasks else: bootstrap_tasks += recipe_image.bootstrap_tasks @@ -1536,12 +1608,13 @@ def ide_setup(args, config, basepath, workspace): 'Note that devtool modify --debug-build can do this automatically.', recipe_modified.name, recipe_modified.bbappend) - # Ports are now assigned. Write the complete image bbappend in a - # single step so that the image is built with exactly one recipe - # hash. This avoids the sstate basehash-changed warnings that - # arise when the bbappend is modified after the image has - # already been built. This also runs with --skip-bitbake, otherwise - # the section removed by strip_bbappend_sections() would be lost. + # Ports are now assigned. Write the complete image bbappend -- + # IMAGE_ debug settings and QB_SLIRP_OPT -- in a single step so + # that the image is built with exactly one recipe hash. This + # avoids the sstate basehash-changed warnings that arise when + # the bbappend is modified after the image has already been + # built. This also runs with --skip-bitbake, otherwise the sections + # removed by strip_bbappend_sections() would be lost. bbappend_changed = False for ri in recipes_images: if ri.update_image_bbappend(recipes_modified): @@ -1564,7 +1637,9 @@ def ide_setup(args, config, basepath, workspace): finally: reparse_tinfoil.shutdown() - # Phase 2: build the image + # Phase 2: build the image. do_image -> do_write_qemuboot_conf + # picks up QB_SLIRP_OPT from the bbappend written above, so no + # separate write_qemuboot_conf step is needed. exec_build_env_command( config.init_path, basepath, bb_cmd + ' '.join(image_bootstrap_tasks), watch=True) From patchwork Sun Aug 30 14:28:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7DB2C624CE for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9429.1788100183423032694 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=DD+0Vm5C; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-1329275-2026083014294183556ab2fc00020779-jiznzj@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 2026083014294183556ab2fc00020779 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=zkgVD51Vz547vo1vKaTITG5ZTUX2Ui45Jsazf727SXA=; b=DD+0Vm5CveCjzA46uh0/XinLormlzEWDMua8F+lT4kUaDbYtWhxO5/HHeod2+l+9UYbKxg corgw3mvyA0Ki4Z6eenVrZsH0L5hXEdIgK4ZyuM4lZjmEgweBeolEPDRwGifHC17keV6mDlg xsR7Oe9icS9+fzFsMbBFZgUZ5QYbuXgOjS216r+sJBxhFVm4nH37Pc3hckPp/0N5phJxXqRr 1mAuNrcdOQVUPf4hhd6bO7q+J3uC2yLDzfgGTh56qsfFea99WfVuhxghAkNHSKAx/N5WZxCL iltTzN52n953cjCH9mtBZUTFZVmkVXIsa+ppBQpWEinV06dgCE4JtvZQ==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 09/24] devtool: ide-sdk: auto-disable ssh host key checking for loopback targets Date: Sun, 30 Aug 2026 16:28:36 +0200 Message-ID: <20260830142922.17241-10-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244663 From: Adrian Freihofer QEMU instances reached via slirp/hostfwd (e.g. root@localhost) get a new ephemeral ssh host key on every boot, so StrictHostKeyChecking would fail on the second and later runs unless --no-host-check is passed explicitly. Detect loopback targets (localhost, 127.0.0.1, ::1) and disable host key checking automatically, both in TargetDevice and in the generated do_install-through-bitbake script that re-parses the target args on the build host. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_sdk.py | 50 ++++++++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 12 deletions(-) diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index 719648a3eb..e45752aff1 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -45,22 +45,24 @@ class DevtoolIdeMode(Enum): shared = 'shared' +# Hosts a ssh target is considered to loop back to the local machine, e.g. a +# QEMU instance reached through slirp/hostfwd port forwarding (root@localhost) +# which has an ephemeral ssh host key that changes on every boot. +LOOPBACK_HOSTS = ('localhost', '127.0.0.1', '::1') + + +def target_host(target): + return target.split('@')[-1] + + +def is_loopback_target(target): + return target_host(target) in LOOPBACK_HOSTS + + class TargetDevice: """SSH remote login parameters""" def __init__(self, args): - self.extraoptions = [] - if args.no_host_check: - self.extraoptions += ['-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no'] - self.ssh_sshexec = 'ssh' - if args.ssh_exec: - self.ssh_sshexec = args.ssh_exec - self.ssh_port = [] - if args.port: - self.ssh_port = ['-p', args.port] - if args.key: - self.extraoptions += ['-i', args.key] - self.target = args.target target_sp = args.target.split('@') if len(target_sp) == 1: @@ -72,6 +74,25 @@ class TargetDevice: else: logger.error("Invalid target argument: %s" % args.target) + no_host_check = args.no_host_check + if not no_host_check and is_loopback_target(args.target): + logger.debug( + "Target %s is a loopback address, disabling ssh host key checking " + "(assuming a QEMU instance with an ephemeral host key)." % args.target) + no_host_check = True + + self.extraoptions = [] + if no_host_check: + self.extraoptions += ['-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no'] + self.ssh_sshexec = 'ssh' + if args.ssh_exec: + self.ssh_sshexec = args.ssh_exec + self.ssh_port = [] + if args.port: + self.ssh_port = ['-p', args.port] + if args.key: + self.extraoptions += ['-i', args.key] + class RecipeNative: """Base class for calling bitbake to provide a -native recipe""" @@ -1324,6 +1345,8 @@ class RecipeModified: 'no_preserve', 'port', 'show_status', 'ssh_exec', 'strip', 'target'] filtered_args_dict = {key: value for key, value in vars( args).items() if key in args_filter} + if is_loopback_target(filtered_args_dict['target']): + filtered_args_dict['no_host_check'] = True cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict)) cmd_lines.append('class Dict2Class(object):') cmd_lines.append(' def __init__(self, my_dict):') @@ -1340,6 +1363,9 @@ class RecipeModified: cmd_lines.append(' i += 2') cmd_lines.append(' else:') cmd_lines.append(' i += 1') + cmd_lines.append( + "if filtered_args.target.split('@')[-1] in %s:" % str(LOOPBACK_HOSTS)) + cmd_lines.append(' filtered_args.no_host_check = True') cmd_lines.append( 'setattr(filtered_args, "recipename", "%s")' % self.bpn) cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args)' % From patchwork Sun Aug 30 14:28:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18C22C624D1 for ; Sun, 30 Aug 2026 14:29:56 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9427.1788100183422661276 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=liLJ6jB3; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-20260830142941347738eca1000207b3-3wxdyx@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 20260830142941347738eca1000207b3 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=QQsUOd5iJnjH6IUfwvWymw8iYFDcfzGw6O+x4sggBig=; b=liLJ6jB3vSizeZlAJ+MH2cnIjsw7r6ruBWpBQATQpOq3Me6rnyQjO6HhmXQWQNwo//pfyo 895/bU0Y6IDKHx9a/KCcFJOomomhomnsca5wSVvWftCyj2+h+7bB/F/LeiZPpC6Km0hXJC3r 5N0s5ou/AAYRvfQ988xiZic2VVsJQjqZf79eT2z7K4nLV94LzXXdzK3cB1mbI4DyUR5nxYpb kEyaPysvGnUKjPSyM6oCInPGnf7i+4T/CI/dM5tZxWLD+qmVPrkDk7/pjUQ9G39uznebUyqy JxBiu9/Utt/JIFUZKKC2mIPvn/JbIz3oOtoWD5vq6OGrfeXx4uMC5hJg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 10/24] oe-selftest: devtool ide-sdk: add slirp networking test Date: Sun, 30 Aug 2026 16:28:37 +0200 Message-ID: <20260830142922.17241-11-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:56 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244661 From: Adrian Freihofer Add test_devtool_ide_sdk_none_qemu_slirp which verifies the complete devtool ide-sdk workflow using runqemu slirp networking instead of TAP devices. The test checks that: - update_qb_slirp_opt() writes QB_SLIRP_OPT (including the SSH hostfwd entry) to the image workspace bbappend before the build. - The image boots in runqemu slirp mode (no TAP device setup needed). - devtool deploy-target successfully reaches the target via the slirp SSH port forward (localhost:2222 by default). - The full edit/compile/deploy/verify cycle works over the slirp connection, just as it does in the TAP-based tests. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 50 +++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 1ecf4c24f5..c18b94ab5a 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3864,6 +3864,56 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): # Verify deployment and remote debugging works self._verify_launch_json_debugging(tempdir, qemu, example_exe) + @OETestTag("runqemu") + def test_devtool_ide_sdk_none_qemu_slirp(self): + """Verify devtool ide-sdk works with runqemu slirp networking. + + Slirp mode uses SSH port forwarding (default: localhost:2222 -> guest:22). + This test checks that: + - update_qb_slirp_opt() writes the QB_SLIRP_OPT host-forwarding + entries to the image workspace bbappend before the build. + - runqemu boots the image in slirp mode. + - devtool deploy-target reaches the target via localhost:2222. + """ + recipe_name = "cmake-example" + build_file = "CMakeLists.txt" + testimage = "oe-selftest-image" + + self._check_workspace() + self._write_bb_config() + + # devtool modify and build image; QB_SLIRP_OPT is written to the bbappend. + tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, None) + runCmd('devtool ide-sdk %s %s -c --ide=none' % (recipe_name, testimage), + output_log=self._cmd_logger) + + # Verify QB_SLIRP_OPT was written to the workspace bbappend. + appends_dir = os.path.join(self.workspacedir, 'appends') + bbappend = os.path.join(appends_dir, testimage + '.bbappend') + self.assertExists(bbappend, 'Image bbappend not created at %s' % bbappend) + with open(bbappend) as f: + bbappend_content = f.read() + self.assertIn('QB_SLIRP_OPT', bbappend_content, + 'QB_SLIRP_OPT not written to image bbappend') + self.assertIn('hostfwd=tcp:127.0.0.1:2222-:22', bbappend_content, + 'SSH slirp port forward missing from QB_SLIRP_OPT') + + with runqemu(testimage, runqemuparams="nographic slirp") as qemu: + slirp_host = qemu.ip + slirp_port = qemu.port or '2222' + + # Re-run ide-sdk with the actual slirp address; image is already built. + bitbake_sdk_cmd = ( + 'devtool ide-sdk %s %s -t root@%s -P %s --skip-bitbake --ide=none' + % (recipe_name, testimage, slirp_host, slirp_port)) + runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) + + self._gdb_cross() + compile_cmd = self._verify_cmake_preset(tempdir) + self._devtool_ide_sdk_qemu(tempdir, qemu, recipe_name, + recipe_name, compile_cmd) + + class DevtoolIdeSdkKernelTests(DevtoolIdeSdkTests): @OETestTag("runqemu") From patchwork Sun Aug 30 14:28:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FBCFC624D0 for ; Sun, 30 Aug 2026 14:29:56 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9426.1788100183422503403 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=ONUkclbg; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-20260830142941e9170a4733000207ab-44crek@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20260830142941e9170a4733000207ab for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=v6X42Lxb5QO5vgSoHff7oSw8EBlDah6O662nFsTJe8k=; b=ONUkclbg31kHIr6G5RpyqxstPOCf8VkCC5qylYk8jWtulz7g4Juaf3wRnlxnfNsUX+NpkC u1E5JzLiFIpuXrtBszO1mkHyVk4Y58BxA5G9iY73PgCbgH3W7+z8QqZ9+sKA5ci5Wx/l3eW1 7N9pGa4ZSmlGexfAVW2joRnMlM+Uo45D4s7GGZDhn86tz/B5fc/y+wm3CbTM6/SBam7qH1zV NcFTgKaoV3KR1Y3b3HjhWjtl9CJ65Wwe5FzmEsbErOq7WVpw7aX/c1cGZuc7ldlOnGyRyA9k YlnTT6NvQvyS+M/DMXrRxm+ow484gZrOF4qxnoDwhRA4LQrmDA1sbpVA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 11/24] devtool: deploy-target: add --package/--file-glob filters Date: Sun, 30 Aug 2026 16:28:38 +0200 Message-ID: <20260830142922.17241-12-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:56 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244660 From: Adrian Freihofer Allow deploying only a subset of a recipe's installed files instead of always sending the whole do_install output. This can be crucial when a recipe's PACKAGES include additional content (e.g. -doc, -staticdev, or optional extras) that does not even fit into the target device's storage/memory; it also speeds up deployment significantly by avoiding the transfer of files that are not actually needed for testing on the target. - --package PACKAGE filters the deployed files down to those listed in that package's FILES variable. - --file-glob GLOB filters by an arbitrary glob pattern on the installed path. --package and --file-glob combine as a union when both are given. Examples: devtool deploy-target mdadm root@target deploy everything (minus -dbg/-src/-staticdev, see above) devtool deploy-target mdadm root@target --package mdadm-doc deploy only the mdadm-doc package's files devtool deploy-target mdadm root@target --package mdadm,mdadm-doc deploy the main and -doc packages together devtool deploy-target mdadm root@target --file-glob '/usr/bin/*' deploy only files under /usr/bin devtool ide-sdk mdadm cmake-example core-image-minimal \ --package "mdadm:,-doc" --package "cmake-example:,-doc" scope each --package entry to its own recipe when several modified recipes are targeted at once Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/deploy.py | 158 ++++++++++++++++++++++++++++++++-- 1 file changed, 152 insertions(+), 6 deletions(-) diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index 35ec0b1b4d..cf481d6b63 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -17,6 +17,7 @@ import tempfile import bb.utils import argparse_oe import oe.types +import oe.package from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, DevtoolError @@ -137,6 +138,92 @@ def _prepare_remote_script(deploy, destdir='/', verbose=False, dryrun=False, und return '\n'.join(lines) +def parse_file_globs_arg(entries, recipename=None): + """Flatten --file-glob entries ("[RECIPE:]GLOB") into glob patterns for recipename. + + E.g. ["other:/usr/bin/*", "/etc/*"] with recipename="myrecipe" -> ["/etc/*"]. + """ + globs = [] + for entry in entries or []: + recipe, spec = entry.split(':', 1) if ':' in entry else (None, entry) + if recipe is not None and recipename is not None and recipe != recipename: + continue + if spec: + globs.append(spec) + return globs + +def _match_deploy_files(recipe_outdir, file_globs, recipename=None): + """Match file_globs (FILES-variable-style patterns) against recipe_outdir. + + Returns the matched files (relative to recipe_outdir), or None if no glob + applies to recipename. + """ + file_globs = parse_file_globs_arg(file_globs, recipename) + if not file_globs: + return None + cwd = os.getcwd() + os.chdir(recipe_outdir) + try: + matched, _ = oe.package.files_from_filevars(file_globs) + finally: + os.chdir(cwd) + return {os.path.normpath(f) for f in matched} + +def parse_packages_arg(entries, recipename=None): + """Flatten --package entries ("[RECIPE:]PKG[,PKG...]") into package names. + + E.g. with recipename="myrecipe": "myrecipe:,-doc,-ptest" -> + ["myrecipe", "myrecipe-doc", "myrecipe-ptest"]; "other:foo" -> []. + """ + packages = [] + for entry in entries or []: + recipe, spec = entry.split(':', 1) if ':' in entry else (None, entry) + if recipe is not None and recipename is not None and recipe != recipename: + continue + for item in spec.split(','): + if not item: + if recipe is not None: + packages.append(recipe) + elif item.startswith('-'): + if recipe is None: + raise DevtoolError('Package suffix "%s" requires a "RECIPE:" prefix, ' + 'e.g. "RECIPE:%s"' % (item, item)) + packages.append(recipe + item) + else: + packages.append(item) + return packages + +def is_default_excluded_package(pkg): + """True for packages left out of a deploy""" + return pkg.endswith(('-dbg', '-src', '-staticdev')) + +def _match_package_files(recipe_outdir, packages_files, packages, recipename=None): + """Resolve packages (see parse_packages_arg) into their files under recipe_outdir.""" + packages = parse_packages_arg(packages, recipename) + all_package_names = [pkg for pkg, _ in packages_files] + for pkg in packages: + if pkg not in all_package_names: + raise DevtoolError('Package "%s" is not one of the packages produced ' + 'by this recipe (PACKAGES: %s)' % (pkg, ' '.join(all_package_names))) + cwd = os.getcwd() + os.chdir(recipe_outdir) + try: + seen = set() + result = set() if packages else None + default_excluded = set() + for pkg, files_var in packages_files: + matched, _ = oe.package.files_from_filevars((files_var or '').split()) + matched = {os.path.normpath(f) for f in matched} - seen + seen |= matched + if pkg in packages: + if result is not None: + result |= matched + elif is_default_excluded_package(pkg): + default_excluded |= matched + finally: + os.chdir(cwd) + return result, default_excluded + def deploy(args, config, basepath, workspace): """Entry point for the devtool 'deploy' subcommand""" import oe.utils @@ -160,14 +247,15 @@ def deploy(args, config, basepath, workspace): max_process = oe.utils.get_bb_number_threads(rd) fakerootcmd = rd.getVar('FAKEROOTCMD') fakerootenv = rd.getVar('FAKEROOTENV') + packages_files = [(pkg, rd.getVar('FILES:' + pkg) or '') + for pkg in (rd.getVar('PACKAGES') or '').split()] finally: tinfoil.shutdown() - return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args) + return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files) -def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args): +def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None): import math - import oe.package try: host, destdir = args.target.split(':') @@ -208,11 +296,35 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce if ret != 0: raise DevtoolError('Failed to strip files for deployment') + allowed_files = None + default_excluded_files = set() + file_sets = [] + if file_globs: + deploy_files = _match_deploy_files(recipe_outdir, file_globs, getattr(args, 'recipename', None)) + if deploy_files is not None: + file_sets.append(deploy_files) + if packages_files: + package_files, default_excluded_files = _match_package_files(recipe_outdir, packages_files, getattr(args, 'package', None), + getattr(args, 'recipename', None)) + if package_files is not None: + file_sets.append(package_files) + if file_sets: + allowed_files = set().union(*file_sets) + filelist = [] + tar_relpaths = [] inodes = set({}) ftotalsize = 0 for root, _, files in os.walk(recipe_outdir): for fn in files: + relpath = os.path.normpath(os.path.join(os.path.relpath(root, recipe_outdir), fn)) + if allowed_files is not None: + if relpath not in allowed_files: + continue + elif relpath in default_excluded_files: + # No explicit --package/--file-glob filter was given: still leave + # out packages like -staticdev that aren't needed on a live target. + continue fstat = os.lstat(os.path.join(root, fn)) # Get the size in kiB (since we'll be comparing it to the output of du -k) # MUST use lstat() here not stat() or getfilesize() since we don't want to @@ -226,6 +338,12 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce # The path as it would appear on the target fpath = os.path.join(destdir, os.path.relpath(root, recipe_outdir), fn) filelist.append((fpath, fsize)) + tar_relpaths.append(relpath) + + if allowed_files is not None and not filelist: + raise DevtoolError('No files to deploy for %s - the --package/--file-glob ' + 'filter(s) did not match any of the files installed by this ' + 'recipe.' % args.recipename) if args.dry_run: print('Files to be deployed for %s on target %s:' % (args.recipename, args.target)) @@ -282,8 +400,25 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce finally: shutil.rmtree(tmpdir) - # Now run the script - ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, 'tar cf - . | %s %s %s %s \'sh %s %s %s %s\'' % (ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) + # Now run the script. When a package/glob filter narrowed down filelist, + # tar is given an explicit list of relative paths (-T) instead of packing + # the whole recipe_outdir tree. + tar_filelist_path = None + try: + if allowed_files is not None: + tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-') + with os.fdopen(tar_fd, 'w') as f: + for relpath in tar_relpaths: + # './' prefix matches what 'tar cf - .' itself would produce, which + # the remote script's manifest handling (sed "s!^./!$2!") relies on. + f.write('./' + relpath + '\n') + tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path) + else: + tar_cmd = 'tar cf - .' + ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, '%s | %s %s %s %s \'sh %s %s %s %s\'' % (tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) + finally: + if tar_filelist_path: + os.remove(tar_filelist_path) if ret != 0: raise DevtoolError('Deploy failed - rerun with -s to get a complete ' 'error message') @@ -362,7 +497,7 @@ def register_commands(subparsers, context): parser_deploy = subparsers.add_parser('deploy-target', help='Deploy recipe output files to live target machine', - description='Deploys a recipe\'s build output (i.e. the output of the do_install task) to a live target machine over ssh. By default, any existing files will be preserved instead of being overwritten and will be restored if you run devtool undeploy-target. Note: this only deploys the recipe itself and not any runtime dependencies, so it is assumed that those have been installed on the target beforehand.', + description='Deploys a recipe\'s build output (i.e. the output of the do_install task) to a live target machine over ssh. By default, any existing files will be preserved instead of being overwritten and will be restored if you run devtool undeploy-target. Note: this only deploys the recipe itself and not any runtime dependencies, so it is assumed that those have been installed on the target beforehand. Use --package/--file-glob to deploy only a subset of the recipe\'s installed files.', group='testbuild') parser_deploy.add_argument('recipename', help='Recipe to deploy') parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]') @@ -375,6 +510,17 @@ def register_commands(subparsers, context): parser_deploy.add_argument('-P', '--port', help='Specify port to use for connection to the target') parser_deploy.add_argument('-I', '--key', help='Specify ssh private key for connection to the target') + parser_deploy.add_argument('--package', action='append', metavar='PACKAGE', + help='Only deploy files belonging to PACKAGE, as defined by that ' + 'package\'s FILES variable in the recipe metadata. May be a ' + 'comma-separated list and/or specified multiple times to ' + 'include several packages. May be prefixed with "RECIPE:" (must ' + 'match RECIPENAME), e.g. "RECIPE:,-doc,-ptest" is short for ' + '"RECIPE,RECIPE-doc,RECIPE-ptest".') + parser_deploy.add_argument('--file-glob', action='append', dest='file_globs', metavar='GLOB', + help='Only deploy files whose installed path matches this glob ' + 'pattern (e.g. "/usr/bin/*" or "${bindir}/myprog"). May be ' + 'specified multiple times. Combined with --package if both are given.') strip_opts = parser_deploy.add_mutually_exclusive_group(required=False) strip_opts.add_argument('-S', '--strip', From patchwork Sun Aug 30 14:28:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BD06C624D2 for ; Sun, 30 Aug 2026 14:29:56 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9428.1788100183422803269 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=TlCTgZHI; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-202608301429410c588ba6c2000207df-0ik8hs@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202608301429410c588ba6c2000207df for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=ChyoX/D353n8feQacxKnUb8s8WWUDIx33caTY9IJJC0=; b=TlCTgZHI6qbQ0+aKyOL9a/hfIq/GKeWMg78EyRFaTJmC7HVsN4VjgeiccfVC5zKCPpjhSn UsvB1U83KWGiXOwhyK0hfrkoqFLRmJWYTRXeIYgGNKI87htWTD4XTDXCx51GztzFpsrqdg3J hvtVUj2kGUSsJ3llyWa5QSfieAd2DiavI5pdv6kYXguDMU0yH9B9iAeK8NFHv6pZHus4K1x5 UwKeoGAwHRLFubzWdX7mqRW1wit1I0QSR9Jh6E6YIhtHAXi4f+hrhiM+EPZKFvCH3Nyz9CH2 zFKOaeXpMhRSf9kg2sjbNlTCD+2eGbkV5xuMjilcgGgFBSUvKqgwPHLg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 12/24] oe-selftest: devtool deploy-target: test --package/--file-glob filters Date: Sun, 30 Aug 2026 16:28:39 +0200 Message-ID: <20260830142922.17241-13-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:56 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244659 From: Adrian Freihofer Extend test_devtool_deploy_target to validate --package/--file-glob filter behavior for both default deploy and --strip deploy. The test now covers package-only, glob-only, and combined filtering, plus recipe-scoped "RECIPE:" forms and shorthand expansion. Examples covered by the test: - devtool deploy-target -c mdadm root@ --package mdadm - devtool deploy-target -c mdadm root@ --package mdadm-doc - devtool deploy-target -c mdadm root@ --package mdadm,mdadm-doc - devtool deploy-target -c mdadm root@ --file-glob /usr/sbin/mdadm - devtool deploy-target -c mdadm root@ --package mdadm --file-glob /usr/sbin/mdadm - devtool deploy-target -c mdadm root@ --package mdadm:-doc - devtool deploy-target -c mdadm root@ --package mdadm:,-doc - devtool deploy-target -c mdadm root@ --file-glob mdadm:/usr/sbin/mdadm This also verifies that non-matching "other-recipe:" prefixes are ignored for the current recipe. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 135 +++++++++++++++++------- 1 file changed, 99 insertions(+), 36 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index c18b94ab5a..b3eb3c28fb 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -2038,8 +2038,14 @@ class DevtoolDeployTargetTests(DevtoolBase): # Definitions testrecipe = 'mdadm' testfile = '/sbin/mdadm' + # mdmon is installed by the same do_install, used to check --package/--file-glob filtering + otherfile = '/sbin/mdmon' if "usrmerge" in get_bb_var('DISTRO_FEATURES'): testfile = '/usr/sbin/mdadm' + otherfile = '/usr/sbin/mdmon' + # Use the mdadm-doc package for testing --package and --file-glob filtering + mandir = get_bb_var('mandir', testrecipe) + docfile = os.path.join(mandir, 'man8', '%s.8' % testrecipe) testimage = 'oe-selftest-image' testcommand = '/sbin/mdadm --help' # Build an image to run @@ -2066,48 +2072,105 @@ class DevtoolDeployTargetTests(DevtoolBase): self.assertIn(' %s' % testfile, result.output) # Boot the image with runqemu(testimage) as qemu: - # Now really test deploy-target - for extra_opt in ['', '--strip']: - deploy_cmd= 'devtool deploy-target -c %s root@%s %s' % (testrecipe, qemu.ip, extra_opt) - self.logger.debug(deploy_cmd) + def _deploy_and_check(extra_args, check_full_filelist, expected_files): + """Deploy with extra_args, verify the result, then undeploy and verify removal. + + check_full_filelist is only meaningful for an unfiltered deploy: it + compares every file installed by do_install against what actually + landed on the target. For a --package/--file-glob filtered deploy, + expected_files instead maps a handful of representative paths + (testfile, otherfile, docfile) to whether they should have been + deployed by this particular filter. + """ + deploy_cmd = 'devtool deploy-target -c %s root@%s %s' % (testrecipe, qemu.ip, extra_args) + if self.logger.isEnabledFor(logging.DEBUG): + deploy_cmd += ' -s' result = runCmd(deploy_cmd) - # Run a test command to see if it was installed properly - status, _ = qemu.run(testcommand) - self.assertEqual(status, 0) - # Check if it deployed all of the files with the right ownership/perms - # First look on the host - need to do this under pseudo to get the correct ownership/perms - bb_vars = get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD', 'PATH'], testrecipe) - installdir = bb_vars['D'] - fakerootenv = bb_vars['FAKEROOTENV'] - fakerootcmd = bb_vars['FAKEROOTCMD'] - path = bb_vars['PATH'] - result = runCmd('PATH="%s" %s %s find . -type f -exec ls -l {} \\;' % (path, fakerootenv, fakerootcmd), cwd=installdir) - filelist1 = self._process_ls_output(result.output) + self.logger.debug('%s\n%s' % (deploy_cmd, result.output)) + + if check_full_filelist: + # Run a test command to see if it was installed properly + status, _ = qemu.run(testcommand) + self.assertEqual(status, 0, '%s: %s was not deployed' % (extra_args, testfile)) + + # Check if it deployed all of the files with the right ownership/perms + # First look on the host - need to do this under pseudo to get the correct ownership/perms + bb_vars = get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD', 'PATH'], testrecipe) + installdir = bb_vars['D'] + fakerootenv = bb_vars['FAKEROOTENV'] + fakerootcmd = bb_vars['FAKEROOTCMD'] + path = bb_vars['PATH'] + result = runCmd('PATH="%s" %s %s find . -type f -exec ls -l {} \\;' % (path, fakerootenv, fakerootcmd), cwd=installdir) + filelist1 = self._process_ls_output(result.output) + + # Now look on the target + tempdir2 = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir2) + tmpfilelist = os.path.join(tempdir2, 'files.txt') + with open(tmpfilelist, 'w') as f: + for line in filelist1: + splitline = line.split() + f.write(splitline[-1] + '\n') + remotefilelist = '/tmp/%s' % os.path.basename(tmpfilelist) + status, _ = qemu.copy_to(tmpfilelist, remotefilelist) + self.assertEqual(status, 0) + status, output = qemu.run( + 'xargs ls -l < %s; status=$?; rm -f %s; exit $status' % ( + remotefilelist, remotefilelist)) + self.assertEqual(status, 0) + filelist2 = self._process_ls_output(output) + filelist1.sort(key=lambda item: item.split()[-1]) + filelist2.sort(key=lambda item: item.split()[-1]) + self.assertEqual(filelist1, filelist2) + else: + for path, expected in expected_files.items(): + status, _ = qemu.run('test -e %s' % path) + if expected: + self.assertEqual(status, 0, '%s: %s should have been deployed' % (extra_args, path)) + else: + self.assertNotEqual(status, 0, '%s: %s should not have been deployed' % (extra_args, path)) - # Now look on the target - tempdir2 = tempfile.mkdtemp(prefix='devtoolqa') - self.track_for_cleanup(tempdir2) - tmpfilelist = os.path.join(tempdir2, 'files.txt') - with open(tmpfilelist, 'w') as f: - for line in filelist1: - splitline = line.split() - f.write(splitline[-1] + '\n') - remotefilelist = '/tmp/%s' % os.path.basename(tmpfilelist) - status, _ = qemu.copy_to(tmpfilelist, remotefilelist) - self.assertEqual(status, 0) - status, output = qemu.run( - 'xargs ls -l < %s; status=$?; rm -f %s; exit $status' % ( - remotefilelist, remotefilelist)) - self.assertEqual(status, 0) - filelist2 = self._process_ls_output(output) - filelist1.sort(key=lambda item: item.split()[-1]) - filelist2.sort(key=lambda item: item.split()[-1]) - self.assertEqual(filelist1, filelist2) # Test undeploy-target - result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip)) + runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip)) status, _ = qemu.run(testcommand) self.assertNotEqual(status, 0, 'undeploy-target did not remove command as it should have') + filter_cases = [ + ('', {}), + ('--package %s' % testrecipe, + {testfile: True, otherfile: True, docfile: False}), + ('--package %s-doc' % testrecipe, + {testfile: False, otherfile: False, docfile: True}), + ('--package %s,%s-doc' % (testrecipe, testrecipe), + {testfile: True, otherfile: True, docfile: True}), + ('--file-glob %s' % testfile, + {testfile: True, otherfile: False, docfile: False}), + ('--package %s --file-glob %s' % (testrecipe, testfile), + {testfile: True, otherfile: True, docfile: False}), + # "RECIPE:" prefix (needed so a single 'devtool ide-sdk' invocation + # can scope --package/--file-glob entries to one of several recipes) + ('--package %s:%s' % (testrecipe, testrecipe), + {testfile: True, otherfile: True, docfile: False}), + # shorthand: "RECIPE:-suffix" expands to "RECIPE-suffix" + ('--package %s:-doc' % testrecipe, + {testfile: False, otherfile: False, docfile: True}), + # shorthand: empty item + "-suffix" combined in one entry + ('--package %s:,-doc' % testrecipe, + {testfile: True, otherfile: True, docfile: True}), + # a "RECIPE:" prefix that doesn't match this recipe is skipped, + # leaving no package filter applied (same as passing none at all) + ('--package other-recipe:%s' % testrecipe, + {testfile: True, otherfile: True, docfile: True}), + ('--file-glob %s:%s' % (testrecipe, testfile), + {testfile: True, otherfile: False, docfile: False}), + ('--file-glob other-recipe:%s' % testfile, + {testfile: True, otherfile: True, docfile: True}), + ] + for strip_opt in ['', '--strip']: + for filter_args, expected_files in filter_cases: + extra_args = ' '.join(a for a in (strip_opt, filter_args) if a) + _deploy_and_check(extra_args, check_full_filelist=not filter_args, expected_files=expected_files) + class DevtoolBuildImageTests(DevtoolBase): def test_devtool_build_image(self): From patchwork Sun Aug 30 14:28:40 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E72C9C624CF for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net [185.136.64.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9433.1788100183423761221 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=HC5OxRa/; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.228, mailfrom: fm-1329275-202608301429414d03c4984c000207e0-rdlwmw@rts-flowmailer.siemens.com) Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 202608301429414d03c4984c000207e0 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=p58kJqBHmnLDmZfr9d22ea23vPSxJFh6o15q2rGZ2oU=; b=HC5OxRa/fBQtL5lipHF42lL4P86bxRvAxhXhfREu9Kz3kK7p7EWO38l+E1C+0GQIveLuB3 9Rm6IT2TnzwAqRy0BWdBl0q6kiJy8+mHORcF590Psz7dWpe7qVudQbvuXQg1hreZh5JS8OV1 KtAbc4B6bQ+Buhr2rcvvBu8gI1p1hSpTrVjIJH8mSMEcVn5Nnwkco0XoVK8il/BPgdt38oW0 KKiGEffbBWmtk2s1RWgYZkie36OBWxuFc1pJKQPTYFWUi5bHMBKrnSGAFyk6ia5f0+OHzQ/S CRcTu+8MTVVp3TTo0LzoCxxMxydsB4HJaGyBxIjvAgJZ2LF1C4GU8UMg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 13/24] devtool: ide-sdk: forward --package/--file-glob deploy filters Date: Sun, 30 Aug 2026 16:28:40 +0200 Message-ID: <20260830142922.17241-14-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244662 From: Adrian Freihofer Wire deploy-target's --package and --file-glob filtering into devtool ide-sdk so IDE-driven deploys can limit what is copied to the target. This keeps ide-sdk's generated fast deploy script behavior aligned with devtool deploy-target. RecipeModified now snapshots PACKAGES and FILES:* during initialize(), and the generated deploy script passes those mappings to deploy_no_d(). Requested packages are validated early so unknown package names fail with a clear error. The generated script also accepts runtime -g/--file-glob overrides, so users can refine filters from tasks.json without regenerating the SDK. Examples: - devtool ide-sdk my-recipe --package my-recipe-doc - devtool ide-sdk my-recipe --file-glob '/usr/bin/*' - devtool ide-sdk my-recipe --package my-recipe:,-doc,-ptest \ --file-glob 'my-recipe:/usr/lib/*.so*' Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_plugins/__init__.py | 4 ++ scripts/lib/devtool/ide_plugins/ide_code.py | 14 ++++++- scripts/lib/devtool/ide_sdk.py | 41 ++++++++++++++++++++- 3 files changed, 56 insertions(+), 3 deletions(-) diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py index a5069cae85..0d0117344d 100644 --- a/scripts/lib/devtool/ide_plugins/__init__.py +++ b/scripts/lib/devtool/ide_plugins/__init__.py @@ -427,4 +427,8 @@ def get_devtool_deploy_opts(args): devtool_deploy_opts += ["-I", args.key] if args.strip is False: devtool_deploy_opts += ["--no-strip"] + for package in (args.package or []): + devtool_deploy_opts += ["--package", package] + for file_glob in (args.file_globs or []): + devtool_deploy_opts += ["--file-glob", file_glob] return devtool_deploy_opts diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py index 5abf0492e8..937c63a4a5 100644 --- a/scripts/lib/devtool/ide_plugins/ide_code.py +++ b/scripts/lib/devtool/ide_plugins/ide_code.py @@ -596,6 +596,11 @@ class IdeVSCode(IdeBase): def vscode_tasks_cpp(self, args, modified_recipe): run_install_deploy = modified_recipe.gen_install_deploy_script(args) install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty + deploy_args = ["--target", args.target] + if args.port: + deploy_args += ["--port", args.port] + for package in args.package or []: + deploy_args += ["--package", package] tasks_dict = { "version": "2.0.0", "tasks": [ @@ -603,7 +608,7 @@ class IdeVSCode(IdeBase): "label": install_task_name, "type": "shell", "command": run_install_deploy, - "args": ["--target", args.target] + (["--port", args.port] if args.port else []), + "args": deploy_args, "problemMatcher": [] } ] @@ -712,6 +717,11 @@ class IdeVSCode(IdeBase): reload_task_name = "reload module %s" % modified_recipe.recipe_id_pretty verify_task_name = "verify module %s" % modified_recipe.recipe_id_pretty run_install_deploy = modified_recipe.gen_install_deploy_script(args) + deploy_args = ["--target", args.target] + if args.port: + deploy_args += ["--port", args.port] + for package in args.package or []: + deploy_args += ["--package", package] tasks_dict = { "version": "2.0.0", "tasks": [ @@ -719,7 +729,7 @@ class IdeVSCode(IdeBase): "label": install_task_name, "type": "shell", "command": run_install_deploy, - "args": ["--target", args.target] + (["--port", args.port] if args.port else []), + "args": deploy_args, "problemMatcher": [] }, { diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index e45752aff1..ac4930af50 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -23,6 +23,7 @@ import scriptutils import bb from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, DevtoolError, parse_recipe from devtool.standard import get_real_srctree +from devtool.deploy import parse_packages_arg from devtool.ide_plugins import BuildTool, DebuggerCrossConfig from oe.kernel_module import kernel_module_os_env @@ -671,6 +672,8 @@ class RecipeModified: self.toolchain = None self.topdir = None self.workdir = None + # Maps each package name (from PACKAGES) to the glob patterns from its FILES variable + self.packages_files = {} # Service management self.systemd_services = {} self.init_scripts = {} @@ -745,6 +748,8 @@ class RecipeModified: self.package_arch = recipe_d.getVar('PACKAGE_ARCH') self.package_debug_split_style = recipe_d.getVar( 'PACKAGE_DEBUG_SPLIT_STYLE') + for package in (recipe_d.getVar('PACKAGES') or '').split(): + self.packages_files[package] = recipe_d.getVar('FILES:' + package) or '' self.path = recipe_d.getVar('PATH') self.pn = recipe_d.getVar('PN') self.recipe_sysroot = os.path.realpath( @@ -1326,6 +1331,17 @@ class RecipeModified: self._installed_binaries = dict(sorted(binaries.items())) return self._installed_binaries + def _validate_requested_packages(self, args): + """Raise if --package (once scoped to this recipe and expanded via + parse_packages_arg) references a package this recipe doesn't produce. + """ + packages = parse_packages_arg(getattr(args, 'package', None), self.bpn) + for package in packages: + if package not in self.packages_files: + raise DevtoolError('Package "%s" is not one of the packages produced ' + 'by the %s recipe (PACKAGES: %s)' % + (package, self.pn, ' '.join(self.packages_files.keys()))) + def gen_deploy_target_script(self, args): """Generate a script which does what devtool deploy-target does @@ -1333,6 +1349,7 @@ class RecipeModified: does not need to start a bitbake server. All information from tinfoil is hard-coded in the generated script. """ + self._validate_requested_packages(args) cmd_lines = ['#!%s' % str(sys.executable)] cmd_lines.append('import sys') cmd_lines.append('devtool_sys_path = %s' % str(sys.path)) @@ -1353,6 +1370,8 @@ class RecipeModified: cmd_lines.append(' for key in my_dict:') cmd_lines.append(' setattr(self, key, my_dict[key])') cmd_lines.append('filtered_args = Dict2Class(filtered_args_dict)') + cmd_lines.append('packages_files = %s' % repr(list(self.packages_files.items()))) + cmd_lines.append('file_globs = %s' % repr(list(getattr(args, 'file_globs', None) or []) or None)) cmd_lines.append('i = 1') cmd_lines.append('while i < len(sys.argv) - 1:') cmd_lines.append(' if sys.argv[i] in ("-t", "--target"):') @@ -1361,6 +1380,13 @@ class RecipeModified: cmd_lines.append(' elif sys.argv[i] in ("-P", "--port"):') cmd_lines.append(' setattr(filtered_args, "port", sys.argv[i + 1])') cmd_lines.append(' i += 2') + cmd_lines.append(' elif sys.argv[i] in ("-g", "--file-glob"):') + cmd_lines.append(' file_globs = (file_globs or []) + [sys.argv[i + 1]]') + cmd_lines.append(' i += 2') + cmd_lines.append(' elif sys.argv[i] in ("-p", "--package"):') + cmd_lines.append(' packages = getattr(filtered_args, "package", None) or []') + cmd_lines.append(' setattr(filtered_args, "package", packages + [sys.argv[i + 1]])') + cmd_lines.append(' i += 2') cmd_lines.append(' else:') cmd_lines.append(' i += 1') cmd_lines.append( @@ -1368,7 +1394,7 @@ class RecipeModified: cmd_lines.append(' filtered_args.no_host_check = True') cmd_lines.append( 'setattr(filtered_args, "recipename", "%s")' % self.bpn) - cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args)' % + cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args, file_globs=file_globs, packages_files=packages_files)' % (self.d, self.workdir, self.path, self.strip_cmd, self.libdir, self.base_libdir, self.max_process, self.fakerootcmd, self.fakerootenv)) @@ -1758,4 +1784,17 @@ def register_commands(subparsers, context): '-p', '--no-preserve', help='Do not preserve existing files', action='store_true') parser_ide_sdk.add_argument( '--no-check-space', help='Do not check for available space before deploying', action='store_true') + parser_ide_sdk.add_argument( + '--package', action='append', metavar='PACKAGE', + help='Only deploy files belonging to PACKAGE, as defined by that package\'s ' + 'FILES variable in the recipe metadata. May be a comma-separated list ' + 'and/or specified multiple times. May be prefixed with "RECIPE:" to target ' + 'one of several recipes at once, e.g. "RECIPE:,-doc,-ptest" is short for ' + '"RECIPE,RECIPE-doc,RECIPE-ptest".') + parser_ide_sdk.add_argument( + '--file-glob', action='append', dest='file_globs', metavar='GLOB', + help='Only deploy files whose installed path matches this glob pattern ' + '(e.g. "/usr/bin/*"). May be specified multiple times. Combined with ' + '--package if both are given. May be prefixed with "RECIPE:" to scope ' + 'the entry to one of the recipes being processed e.g. "RECIPE:/usr/bin/*".') parser_ide_sdk.set_defaults(func=ide_setup) From patchwork Sun Aug 30 14:28:41 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A09B5C624CC for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9431.1788100183423143317 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=aQuySH7d; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-1329275-20260830142941544ec2611c00020754-zl5s_x@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20260830142941544ec2611c00020754 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=uvvjk5mDWov4odCeb+80KHOhC+w3aYRuOrQdfDxLQFE=; b=aQuySH7dMHRPsSSWxG6FKZxm+5Kw1a8a7bcd5/j2q3W5SRVKU0gEzFezECq9aKfUrut0og pSjCkgdYu0x179ad4RjI8Q//NIoAPwbZcSCHuG/zQS4q34FilmR5NnxFnY9FBT8kfSMKTXb3 Mtg/kvomRPlxNwmkVsoM+oaoyLeOyDsT3AofkDEYgWZrIrIEUWSXVsmMhO8lT5Aa+F53FRAh c2Ify1UYM4RctI+rKAVyuR55bgbivn9CXoD4AnDxSXVLEM03k9VvzOZ4JSRne00CFuLZnC/6 9jDlyvIm82UysEH1e4SBNogdE1YmmjrpcGTI9AkPTYQBpO5PkdnzhS3g==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 14/24] oe-selftest: devtool ide-sdk: cover --package filters Date: Sun, 30 Aug 2026 16:28:41 +0200 Message-ID: <20260830142922.17241-15-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244664 From: Adrian Freihofer Extend ide-sdk selftests to pass recipe-scoped --package filters through both ide=none and ide=code flows. The ide=none multi-recipe test now exercises repeated filter forwarding: - --package cmake-example:,-ptest - --package meson-example:,-ptest The ide=code cmake and meson tests now also pass: - --package :,-ptest Example command covered by the test: - devtool ide-sdk cmake-example meson-example oe-selftest-image \ -c --ide=none --package cmake-example:,-ptest \ --package meson-example:,-ptest Example ide=code command covered by the test: - devtool ide-sdk cmake-example oe-selftest-image \ -c --ide=code --package cmake-example:,-ptest Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 55 ++++++++++++++++++------- 1 file changed, 39 insertions(+), 16 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index b3eb3c28fb..3e0eb7948f 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3039,7 +3039,7 @@ class DevtoolIdeSdkTests(DevtoolBase): recipe_id_pretty = recipe_name + ": " + package_arch return (recipe_id, recipe_id_pretty) - def _verify_install_script_code(self, tempdir, recipe_name): + def _verify_install_script_code(self, tempdir, recipe_name, package_args=None): """Verify the scripts referred by the tasks.json file are fine. This function does not depend on Qemu. Therefore it verifies the scripts @@ -3053,12 +3053,22 @@ class DevtoolIdeSdkTests(DevtoolBase): task_install = next( (task for task in tasks if task["label"] == "install && deploy-target %s" % recipe_id_pretty), None) self.assertIsNot(task_install, None) + for package_arg in package_args or []: + self.assertIn("--package", task_install["args"]) + self.assertIn(package_arg, task_install["args"]) # execute only the bb_run_do_install script since the deploy would require e.g. Qemu running. i_and_d_script = "install_and_deploy_" + recipe_id i_and_d_script_path = os.path.join( self._workspace_scripts_dir(recipe_name), i_and_d_script) self.assertExists(i_and_d_script_path) + deploy_script_path = os.path.join( + self._workspace_scripts_dir(recipe_name), 'deploy_target_' + recipe_id) + with open(deploy_script_path) as deploy_script: + deploy_script_content = deploy_script.read() + self.assertIn('packages_files = ', deploy_script_content) + self.assertNotIn("'package':", deploy_script_content) + def _devtool_ide_sdk_qemu(self, tempdir, qemu, recipe_name, example_exe, compile_cmd): """Verify deployment, execution and remote debugging in Qemu system work for one recipe. @@ -3434,6 +3444,12 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): def test_devtool_ide_sdk_none_qemu(self): """Start qemu-system and run tests for multiple recipes. ide=none is used.""" recipe_names = ["cmake-example", "meson-example"] + package_filters = [ + # Exercise multi-recipe package scoping via repeated --package: + # RECIPE:,-ptest expands to RECIPE and RECIPE-ptest. + "cmake-example:,-ptest", + "meson-example:,-ptest", + ] testimage = "oe-selftest-image" self._check_workspace() @@ -3443,7 +3459,8 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): # Verify deployment to Qemu (system mode) works tempdir_cmake = self._devtool_ide_sdk_recipe("cmake-example", "CMakeLists.txt", None) tempdir_meson = self._devtool_ide_sdk_recipe("meson-example", "meson.build", testimage) - runCmd('devtool ide-sdk cmake-example meson-example %s -c --ide=none' % testimage, + package_opts = ' '.join('--package %s' % p for p in package_filters) + runCmd('devtool ide-sdk cmake-example meson-example %s -c --ide=none %s' % (testimage, package_opts), output_log=self._cmd_logger) with runqemu(testimage, runqemuparams="nographic") as qemu: @@ -3458,9 +3475,11 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): # Verify /etc/cmake-example.conf is owned by the cmake-example user self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group) - # Re-run ide-sdk with the actual QEMU IP; image is already built - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % ( - recipe_name, testimage, qemu.ip) + # Re-run ide-sdk with only the image name. Modified recipes must + # be discovered from workspace appends; the image bbappend itself + # must not be mistaken for a modified recipe. + bitbake_sdk_cmd = 'devtool ide-sdk %s -t root@%s -c --skip-bitbake --ide=none %s' % ( + testimage, qemu.ip, package_opts) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) self._gdb_cross() @@ -3486,9 +3505,9 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): # Verify /etc/meson-example.conf is owned by the meson-example user self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group) - # Re-run ide-sdk with the actual QEMU IP; image is already built - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % ( - recipe_name, testimage, qemu.ip) + # Re-run ide-sdk with the actual QEMU IP for this recipe + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none %s' % ( + recipe_name, testimage, qemu.ip, package_opts) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) self._gdb_cross() @@ -3839,6 +3858,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): """Verify a cmake recipe works with ide=code mode""" recipe_name = "cmake-example" example_exe = "cmake-example" + package_opt = '--package %s:,-ptest' % recipe_name build_file = "CMakeLists.txt" testimage = "oe-selftest-image" build_file = "CMakeLists.txt" @@ -3849,16 +3869,17 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): # Build image with debug settings before starting QEMU self._check_runqemu_prerequisites() tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage) - runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage), + runCmd('devtool ide-sdk %s %s -c --ide=code %s' % (recipe_name, testimage, package_opt), output_log=self._cmd_logger) with runqemu(testimage, runqemuparams="nographic") as qemu: # Re-run with actual QEMU IP; image is already built - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % ( - recipe_name, testimage, qemu.ip) + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code %s' % ( + recipe_name, testimage, qemu.ip, package_opt) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) self._verify_cmake_preset(tempdir) - self._verify_install_script_code(tempdir, recipe_name) + self._verify_install_script_code( + tempdir, recipe_name, ['%s:,-ptest' % recipe_name]) self._gdb_cross() # Verify the launch.json file created is valid @@ -3872,6 +3893,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): """Verify a meson recipe works with ide=code mode""" recipe_name = "meson-example" example_exe = "mesonex" + package_opt = '--package %s:,-ptest' % recipe_name build_file = "meson.build" testimage = "oe-selftest-image" @@ -3882,13 +3904,13 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self._check_runqemu_prerequisites() tempdir = self._devtool_ide_sdk_recipe( recipe_name, build_file, testimage) - runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage), + runCmd('devtool ide-sdk %s %s -c --ide=code %s' % (recipe_name, testimage, package_opt), output_log=self._cmd_logger) with runqemu(testimage, runqemuparams="nographic") as qemu: # Re-run with actual QEMU IP; image is already built - bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % ( - recipe_name, testimage, qemu.ip) + bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code %s' % ( + recipe_name, testimage, qemu.ip, package_opt) runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger) with open(os.path.join(tempdir, '.vscode', 'settings.json')) as settings_j: @@ -3918,7 +3940,8 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): runCmd('%s test -C %s' % (meson_exe, meson_build_folder), cwd=tempdir, output_log=self._cmd_logger) - self._verify_install_script_code(tempdir, recipe_name) + self._verify_install_script_code( + tempdir, recipe_name, ['%s:,-ptest' % recipe_name]) self._gdb_cross() # Verify the launch.json file created is valid From patchwork Sun Aug 30 14:28:42 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEE87C61DF1 for ; Sun, 30 Aug 2026 14:29:53 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9452.1788100183423549590 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=KBdLNn2q; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-1329275-2026083014294149620e0f8000020791-ua86qh@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 2026083014294149620e0f8000020791 for ; Sun, 30 Aug 2026 16:29:41 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=zapn1ATaEPuLk0BjjMDJRIZsVJtdMIz1iekdlLb+m+U=; b=KBdLNn2q0Cbnx35EUfpOFnSTb0krRzN5nzLS6sxK2WsM3ZfrKf9xdNuwkts3bfNjD9jxjt cuJbEzIqWWocamxPwlnqQoTk/xV+7VCXz/pt4tjGjDmX/DB294amylH6lj4uqQrBG7L8UDtF je5R+JYZvfOA6xolPfIkKx/pJU3r+KyvMb37TsqCwODDU5cp1TztyjD2kfcpC/rj95gneMg0 WF48d3k0uJAzILDoEhIgDJ7L8A1IlaTjtvm91xL4U+BJaWjVZ4jKvC6yUIyoS39PxYvU9Pw/ aO/q8NIwhl2BwNXtM+vr4lF3+8qa+GzYGY0fIj/4FaloBFyulWjIk8Bw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 15/24] runqemu-extract-sdk: refactor in Python Date: Sun, 30 Aug 2026 16:28:42 +0200 Message-ID: <20260830142922.17241-16-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244677 From: Adrian Freihofer Move the pseudo-backed rootfs extraction logic into a reusable Python module while preserving the existing command interface. Install the module with qemu-helper so the wrapper also works from an SDK. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- .../qemu/nativesdk-qemu-helper_1.0.bb | 3 + scripts/lib/runqemu_utils.py | 124 ++++++++++++++++++ scripts/runqemu-extract-sdk | 103 +-------------- 3 files changed, 133 insertions(+), 97 deletions(-) create mode 100644 scripts/lib/runqemu_utils.py diff --git a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb index e1f343989a..16fc07fadf 100644 --- a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb +++ b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb @@ -9,6 +9,7 @@ RDEPENDS:${PN} = "nativesdk-qemu nativesdk-unfs3 nativesdk-pseudo \ LIC_FILES_CHKSUM = "file://${COREBASE}/scripts/runqemu;beginline=5;endline=10;md5=ac2b489a58739c7628a2604698db5e7f" SRC_URI = "file://${COREBASE}/scripts/runqemu \ + file://${COREBASE}/scripts/lib/runqemu_utils.py \ file://${COREBASE}/scripts/runqemu-addptable2image \ file://${COREBASE}/scripts/runqemu-gen-tapdevs \ file://${COREBASE}/scripts/runqemu-ifup \ @@ -30,4 +31,6 @@ do_install() { install -d ${D}${bindir} install -m 0755 ${S}${COREBASE}/scripts/oe-* ${D}${bindir}/ install -m 0755 ${S}${COREBASE}/scripts/runqemu* ${D}${bindir}/ + # The runqemu-* wrappers import this module from their own directory. + install -m 0644 ${S}${COREBASE}/scripts/lib/runqemu_utils.py ${D}${bindir}/ } diff --git a/scripts/lib/runqemu_utils.py b/scripts/lib/runqemu_utils.py new file mode 100644 index 0000000000..dff0ca8d3a --- /dev/null +++ b/scripts/lib/runqemu_utils.py @@ -0,0 +1,124 @@ +#!/usr/bin/env python3 +# +# Helpers shared by runqemu and NFS-rootfs preparation tools. +# +# SPDX-License-Identifier: GPL-2.0-only + +"""Extract and export rootfs tarballs for NFS booting.""" + +import os +import subprocess +import sys + + +class RunQemuRootfsError(Exception): + """Raised when an NFS rootfs cannot be prepared or exported.""" + + +def native_environment(): + """Return the qemu-helper-native environment set by oe-find-native-sysroot.""" + command = ''' + helper=$(command -v oe-find-native-sysroot) || exit 127 + . "$helper" qemu-helper-native >/dev/null + env -0 + printf 'PSEUDO=%s\\0OECORE_NATIVE_SYSROOT=%s\\0' "$PSEUDO" "$OECORE_NATIVE_SYSROOT" + ''' + result = subprocess.run(['bash', '-c', command], capture_output=True) + if result.returncode: + if result.returncode == 127: + raise RunQemuRootfsError( + 'Unable to find the oe-find-native-sysroot script.\n' + 'Did you forget to source your build system environment setup script?') + raise RunQemuRootfsError( + result.stderr.decode(errors='replace').strip() or + 'Unable to set up the qemu-helper-native sysroot') + + environment = {} + for entry in result.stdout.split(b'\0'): + if b'=' in entry: + key, value = entry.split(b'=', 1) + environment[key.decode()] = value.decode() + return environment + + +def _tar_options(rootfs_tarball): + if rootfs_tarball.endswith('.tar.xz'): + return ['--numeric-owner', '-xJf'] + if rootfs_tarball.endswith('.tar.bz2'): + return ['--numeric-owner', '-xjf'] + if rootfs_tarball.endswith('.tar.gz'): + return ['--numeric-owner', '-xzf'] + if rootfs_tarball.endswith('.tar.zst'): + return ['--numeric-owner', '--zstd', '-xf'] + if rootfs_tarball.endswith('.tar'): + return ['--numeric-owner', '-xf'] + raise RunQemuRootfsError( + 'Unable to determine sdk tarball format\n' + 'Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / .tar.zst') + + +def pseudo_state_dir(rootfs_dir): + """Return the pseudo database location associated with an extracted rootfs.""" + rootfs_dir = os.path.realpath(rootfs_dir) + return os.path.join(os.path.dirname(rootfs_dir), + os.path.basename(rootfs_dir) + '.pseudo_state') + + +def extract_sdk_rootfs(rootfs_tarball, rootfs_dir): + """Extract a rootfs tarball under pseudo and return its absolute directory.""" + if not os.path.exists(rootfs_tarball): + raise RunQemuRootfsError("sdk tarball '%s' does not exist" % rootfs_tarball) + + rootfs_tarball = os.path.realpath(rootfs_tarball) + rootfs_dir = os.path.realpath(rootfs_dir) + tar_options = _tar_options(rootfs_tarball) + state_dir = pseudo_state_dir(rootfs_dir) + debug_image = '-dbg' in os.path.basename(rootfs_tarball) + + if os.path.exists(state_dir) and not debug_image: + raise RunQemuRootfsError( + '%s already exists!\n' + 'Please delete the rootfs tree and pseudo directory manually\n' + 'if this is really what you want.' % state_dir) + + os.makedirs(rootfs_dir, exist_ok=True) + os.makedirs(state_dir, exist_ok=True) + open(os.path.join(state_dir, 'pseudo.pid'), 'a').close() + + environment = native_environment() + environment['PSEUDO_LOCALSTATEDIR'] = state_dir + environment['PSEUDO_INCLUDE_PATHS'] = rootfs_dir + pseudo = environment.get('PSEUDO') + native_sysroot = environment.get('OECORE_NATIVE_SYSROOT') + if not pseudo or not native_sysroot: + raise RunQemuRootfsError('qemu-helper-native did not provide pseudo') + + command = [pseudo, '-P', os.path.join(native_sysroot, 'usr'), 'tar', '-C', rootfs_dir] + command.extend(tar_options) + command.append(rootfs_tarball) + print('Extracting rootfs tarball using pseudo...') + print(' '.join(command)) + try: + subprocess.run(command, env=environment, check=True) + except subprocess.CalledProcessError as exc: + raise RunQemuRootfsError('Failed to extract rootfs tarball') from exc + + if len(os.listdir(rootfs_dir)) < 4: + print("Warning: I don't see many files in %s" % rootfs_dir) + print('Please double-check the extraction worked as intended') + else: + print('SDK image successfully extracted to %s' % rootfs_dir) + return rootfs_dir + + +def extract_sdk_main(argv=None): + argv = sys.argv[1:] if argv is None else argv + if len(argv) != 2: + print('Usage: %s ' % sys.argv[0]) + return 1 + try: + extract_sdk_rootfs(*argv) + except RunQemuRootfsError as exc: + print('Error: %s' % exc) + return 1 + return 0 diff --git a/scripts/runqemu-extract-sdk b/scripts/runqemu-extract-sdk index db9813e1dc..6cad2fdf90 100755 --- a/scripts/runqemu-extract-sdk +++ b/scripts/runqemu-extract-sdk @@ -1,104 +1,13 @@ -#!/bin/bash -# -# This utility extracts an SDK image tarball using pseudo, and stores -# the pseudo database in var/pseudo within the rootfs. If you want to -# boot QEMU using an nfsroot, you *must* use this script to create the -# rootfs to ensure it is done correctly with pseudo. -# -# Copyright (c) 2010 Intel Corp. +#!/usr/bin/env python3 # # SPDX-License-Identifier: GPL-2.0-only -# -function usage() { - echo "Usage: $0 " -} +import os +import sys -if [ $# -ne 2 ]; then - usage - exit 1 -fi +sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')) -SYSROOT_SETUP_SCRIPT=`which oe-find-native-sysroot 2> /dev/null` -if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then - echo "Error: Unable to find the oe-find-native-sysroot script" - echo "Did you forget to source your build system environment setup script?" - exit 1 -fi -. $SYSROOT_SETUP_SCRIPT qemu-helper-native -PSEUDO_OPTS="-P $OECORE_NATIVE_SYSROOT/usr" +from runqemu_utils import extract_sdk_main -ROOTFS_TARBALL=$1 -SDK_ROOTFS_DIR=$2 -if [ ! -e "$ROOTFS_TARBALL" ]; then - echo "Error: sdk tarball '$ROOTFS_TARBALL' does not exist" - usage - exit 1 -fi - -# Convert SDK_ROOTFS_DIR to a full pathname -if [[ ${SDK_ROOTFS_DIR:0:1} != "/" ]]; then - SDK_ROOTFS_DIR=$(readlink -f $(pwd)/$SDK_ROOTFS_DIR) -fi - -TAR_OPTS="" -if [[ "$ROOTFS_TARBALL" =~ tar\.xz$ ]]; then - TAR_OPTS="--numeric-owner -xJf" -fi -if [[ "$ROOTFS_TARBALL" =~ tar\.bz2$ ]]; then - TAR_OPTS="--numeric-owner -xjf" -fi -if [[ "$ROOTFS_TARBALL" =~ tar\.gz$ ]]; then - TAR_OPTS="--numeric-owner -xzf" -fi -if [[ "$ROOTFS_TARBALL" =~ tar\.zst$ ]]; then - TAR_OPTS="--numeric-owner --zstd -xf" -fi -if [[ "$ROOTFS_TARBALL" =~ \.tar$ ]]; then - TAR_OPTS="--numeric-owner -xf" -fi -if [ -z "$TAR_OPTS" ]; then - echo "Error: Unable to determine sdk tarball format" - echo "Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / tar.zst" - exit 1 -fi - -if [ ! -d "$SDK_ROOTFS_DIR" ]; then - echo "Creating directory $SDK_ROOTFS_DIR" - mkdir -p "$SDK_ROOTFS_DIR" -fi - -pseudo_state_dir="$SDK_ROOTFS_DIR/../$(basename "$SDK_ROOTFS_DIR").pseudo_state" -pseudo_state_dir="$(readlink -f $pseudo_state_dir)" - -debug_image="`echo $ROOTFS_TARBALL | grep '\-dbg\.rootfs\.tar'`" - -if [ -e "$pseudo_state_dir" -a -z "$debug_image" ]; then - echo "Error: $pseudo_state_dir already exists!" - echo "Please delete the rootfs tree and pseudo directory manually" - echo "if this is really what you want." - exit 1 -fi - -mkdir -p "$pseudo_state_dir" -touch "$pseudo_state_dir/pseudo.pid" -PSEUDO_LOCALSTATEDIR="$pseudo_state_dir" -export PSEUDO_LOCALSTATEDIR -PSEUDO_INCLUDE_PATHS="$SDK_ROOTFS_DIR" -export PSEUDO_INCLUDE_PATHS - -echo "Extracting rootfs tarball using pseudo..." -echo "$PSEUDO $PSEUDO_OPTS tar -C \"$SDK_ROOTFS_DIR\" $TAR_OPTS \"$ROOTFS_TARBALL\"" -$PSEUDO $PSEUDO_OPTS tar -C "$SDK_ROOTFS_DIR" $TAR_OPTS "$ROOTFS_TARBALL" - -DIRCHECK=`ls -l "$SDK_ROOTFS_DIR" | wc -l` -if [ "$DIRCHECK" -lt 5 ]; then - echo "Warning: I don't see many files in $SDK_ROOTFS_DIR" - echo "Please double-check the extraction worked as intended" - exit 0 -fi - -echo "SDK image successfully extracted to $SDK_ROOTFS_DIR" - -exit 0 +sys.exit(extract_sdk_main()) From patchwork Sun Aug 30 14:28:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A9CFC624C9 for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9434.1788100183548899591 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=ggcC8Rls; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-1329275-20260830142941d13b8271b200020785-j22xqp@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20260830142941d13b8271b200020785 for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=TjpKwxplJ65szEK4WRHX/4jOzQfhjUMVS3QU3vc4uxY=; b=ggcC8RlsSj+NzB9gCBqN+HRmWCVYXMpgBcc7J8lKAYmUKogkyuEnfCmFS53Ju0slfPyuUp og5rjj4GnAIiEtIjlNuKJVzH34Yr5DFrZFrF4sScUJc4Cet36ArhBVD6kiK9WWDHOilqSeI8 SACTDMpFU4D471CDnJc7gRYyDmb+bk4CYI+ClQ3GvBq/kpbrlzRX86JGiSjgshxAPQsWNDZ2 0Ls+n9WoJ6yogHj1kXurbWTjah7xg5FtZCiuiloAI+Jj4MEh40o0FCgItVRRw/tzobbHVm/J ltZZNSW3PcOurvOa1xPBw3vS5OH/AWuAgl6A6G7nuR8+7WB2pSU/FbXw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 16/24] runqemu-export-rootfs: refactor in Python Date: Sun, 30 Aug 2026 16:28:43 +0200 Message-ID: <20260830142922.17241-17-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244666 From: Adrian Freihofer Move the userspace NFS export lifecycle into the reusable runqemu helper module while preserving the existing command interface. AI-Generated: Uses GitHub Copilot Signed-off-by: Adrian Freihofer --- scripts/lib/runqemu_utils.py | 122 ++++++++++++++++++++++++++++++++ scripts/runqemu-export-rootfs | 126 ++-------------------------------- 2 files changed, 128 insertions(+), 120 deletions(-) diff --git a/scripts/lib/runqemu_utils.py b/scripts/lib/runqemu_utils.py index dff0ca8d3a..c36e30d76b 100644 --- a/scripts/lib/runqemu_utils.py +++ b/scripts/lib/runqemu_utils.py @@ -7,6 +7,7 @@ """Extract and export rootfs tarballs for NFS booting.""" import os +import signal import subprocess import sys @@ -122,3 +123,124 @@ def extract_sdk_main(argv=None): print('Error: %s' % exc) return 1 return 0 + + +_NFS_ACTIONS = ('start', 'stop', 'restart') + + +def _nfs_paths(instance): + state_dir = os.path.join(os.path.expanduser('~'), '.runqemu-sdk') + os.makedirs(state_dir, exist_ok=True) + return { + 'exports': os.path.join(state_dir, 'exports%s' % instance), + 'nfs_pid': os.path.join(state_dir, 'nfs%s.pid' % instance), + } + + +def _nfs_ports(instance): + nfs_port = int(os.environ.get('NFSD_PORT', 3049 + 2 * instance)) + mount_port = int(os.environ.get('MOUNTD_PORT', 3048 + 2 * instance)) + return nfs_port, mount_port + + +def _export_usage(): + return 'Usage: %s {%s} ' % ( + sys.argv[0], '|'.join(_NFS_ACTIONS)) + + +def export_rootfs(action, rootfs_dir): + """Start, stop, or restart the userspace NFS server for *rootfs_dir*.""" + if action not in _NFS_ACTIONS: + raise RunQemuRootfsError("Unknown command '%s'" % action) + if not os.path.isdir(rootfs_dir): + raise RunQemuRootfsError("'%s' does not exist" % rootfs_dir) + + rootfs_dir = os.path.realpath(rootfs_dir) + state_dir = pseudo_state_dir(rootfs_dir) + if not os.path.isdir(state_dir): + raise RunQemuRootfsError( + '%s does not exist.\n' + 'Did you create the export directory using runqemu-extract-sdk?' % state_dir) + + if action == 'restart': + export_rootfs('stop', rootfs_dir) + return export_rootfs('start', rootfs_dir) + + instance = int(os.environ.get('NFS_INSTANCE', '0')) + paths = _nfs_paths(instance) + if action == 'stop': + if os.path.exists(paths['nfs_pid']): + print('Stopping rpc.nfsd') + with open(paths['nfs_pid']) as pid_file: + pid = pid_file.read().strip() + try: + os.kill(int(pid), signal.SIGTERM) + except (ValueError, ProcessLookupError): + # A stale PID file must not stop the cleanup below. + print('rpc.nfsd is not running') + os.unlink(paths['nfs_pid']) + else: + print('No PID file, not stopping rpc.nfsd') + if os.path.exists(paths['exports']): + print('Removing exports file') + os.unlink(paths['exports']) + return + + environment = native_environment() + native_sysroot = environment.get('OECORE_NATIVE_SYSROOT') + pseudo = environment.get('PSEUDO') + if not native_sysroot or not pseudo: + raise RunQemuRootfsError('qemu-helper-native did not provide pseudo') + + unfsd = os.path.join(native_sysroot, 'usr', 'bin', 'unfsd') + if not os.path.exists(unfsd): + raise RunQemuRootfsError( + 'Unable to find unfsd binary in %s/usr/bin/\n' + "This shouldn't happen - something is missing from your toolchain installation" + % native_sysroot) + + nfs_port, mount_port = _nfs_ports(instance) + environment['PSEUDO_LOCALSTATEDIR'] = state_dir + with open(paths['exports'], 'w') as exports_file: + exports_file.write('%s (rw,no_root_squash,no_all_squash,insecure)\n' % rootfs_dir) + + command = [pseudo, '-P', os.path.join(native_sysroot, 'usr'), unfsd, + '-p', '-i', paths['nfs_pid'], '-e', paths['exports'], + '-n', str(nfs_port), '-m', str(mount_port)] + print('Creating exports file...') + print('Starting User Mode nfsd') + print(' %s' % ' '.join(command)) + try: + subprocess.run(command, env=environment, check=True) + except subprocess.CalledProcessError as exc: + raise RunQemuRootfsError('Error starting nfsd') from exc + + if not os.path.exists(paths['nfs_pid']): + raise RunQemuRootfsError('rpc.nfsd did not start correctly') + with open(paths['nfs_pid']) as pid_file: + try: + os.kill(int(pid_file.read()), 0) + except OSError as exc: + raise RunQemuRootfsError('rpc.nfsd did not start correctly') from exc + + print('') + print('On your target please remember to add the following options for NFS') + print('nfsroot=IP_ADDRESS:%s,nfsvers=3,port=%s,udp,mountport=%s' % + (rootfs_dir, nfs_port, mount_port)) + + +def export_rootfs_main(argv=None): + argv = sys.argv[1:] if argv is None else argv + if len(argv) != 2: + print(_export_usage()) + return 1 + if argv[0] not in _NFS_ACTIONS: + print("Unknown command '%s'" % argv[0]) + print(_export_usage()) + return 1 + try: + export_rootfs(*argv) + except RunQemuRootfsError as exc: + print('Error: %s' % exc) + return 1 + return 0 diff --git a/scripts/runqemu-export-rootfs b/scripts/runqemu-export-rootfs index 6a8acd0d5a..dde364d37e 100755 --- a/scripts/runqemu-export-rootfs +++ b/scripts/runqemu-export-rootfs @@ -1,127 +1,13 @@ -#!/bin/bash -# -# Copyright (c) 2005-2009 Wind River Systems, Inc. +#!/usr/bin/env python3 # # SPDX-License-Identifier: GPL-2.0-only -# -usage() { - echo "Usage: $0 {start|stop|restart} " -} +import os +import sys -if [ $# != 2 ]; then - usage - exit 1 -fi +sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')) -if [[ "$1" != "start" && "$1" != "stop" && "$1" != "restart" ]]; then - echo "Unknown command '$1'" - usage - exit 1 -fi +from runqemu_utils import export_rootfs_main -if [ ! -d "$2" ]; then - echo "Error: '$2' does not exist" - usage - exit 1 -fi -# Ensure the nfs-export-dir is an absolute path -NFS_EXPORT_DIR=$(cd "$2" && pwd) -SYSROOT_SETUP_SCRIPT=`which oe-find-native-sysroot 2> /dev/null` -if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then - echo "Error: Unable to find the oe-find-native-sysroot script" - echo "Did you forget to source your build environment setup script?" - exit 1 -fi -. $SYSROOT_SETUP_SCRIPT qemu-helper-native - -if [ ! -e "$OECORE_NATIVE_SYSROOT/usr/bin/unfsd" ]; then - echo "Error: Unable to find unfsd binary in $OECORE_NATIVE_SYSROOT/usr/bin/" - - echo "This shouldn't happen - something is missing from your toolchain installation" - exit 1 -fi - -if [ ! -d ~/.runqemu-sdk ]; then - mkdir -p ~/.runqemu-sdk -fi - -NFS_INSTANCE=${NFS_INSTANCE:=0} -EXPORTS=~/.runqemu-sdk/exports$NFS_INSTANCE -RMTAB=~/.runqemu-sdk/rmtab$NFS_INSTANCE -NFSPID=~/.runqemu-sdk/nfs$NFS_INSTANCE.pid -MOUNTPID=~/.runqemu-sdk/mount$NFS_INSTANCE.pid - -PSEUDO_OPTS="-P $OECORE_NATIVE_SYSROOT/usr" -PSEUDO_LOCALSTATEDIR="$NFS_EXPORT_DIR/../$(basename $NFS_EXPORT_DIR).pseudo_state" -export PSEUDO_LOCALSTATEDIR - -if [ ! -d "$PSEUDO_LOCALSTATEDIR" ]; then - echo "Error: $PSEUDO_LOCALSTATEDIR does not exist." - echo "Did you create the export directory using runqemu-extract-sdk?" - exit 1 -fi - -# NFS server port number -NFSD_PORT=${NFSD_PORT:=$[ 3049 + 2 * $NFS_INSTANCE ]} -# mountd port number -MOUNTD_PORT=${MOUNTD_PORT:=$[ 3048 + 2 * $NFS_INSTANCE ]} - -## For debugging you would additionally add -## --debug all -UNFSD_OPTS="-p -i $NFSPID -e $EXPORTS -n $NFSD_PORT -m $MOUNTD_PORT" - -# See how we were called. -case "$1" in - start) - echo "Creating exports file..." - echo "$NFS_EXPORT_DIR (rw,no_root_squash,no_all_squash,insecure)" > $EXPORTS - - echo "Starting User Mode nfsd" - echo " $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/bin/unfsd $UNFSD_OPTS" - $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/bin/unfsd $UNFSD_OPTS - if [ ! $? = 0 ]; then - echo "Error starting nfsd" - exit 1 - fi - # Check to make sure everything started ok. - if [ ! -f $NFSPID ]; then - echo "rpc.nfsd did not start correctly" - exit 1 - fi - ps -fp `cat $NFSPID` > /dev/null 2> /dev/null - if [ ! $? = 0 ]; then - echo "rpc.nfsd did not start correctly" - exit 1 - fi - echo " " - echo "On your target please remember to add the following options for NFS" - echo "nfsroot=IP_ADDRESS:$NFS_EXPORT_DIR,nfsvers=3,port=$NFSD_PORT,udp,mountport=$MOUNTD_PORT" - ;; - stop) - if [ -f "$NFSPID" ]; then - echo "Stopping rpc.nfsd" - kill `cat $NFSPID` - rm -f $NFSPID - else - echo "No PID file, not stopping rpc.nfsd" - fi - if [ -f "$EXPORTS" ]; then - echo "Removing exports file" - rm -f $EXPORTS - fi - ;; - restart) - $0 stop $NFS_EXPORT_DIR - $0 start $NFS_EXPORT_DIR - if [ ! $? = 0 ]; then - exit 1 - fi - ;; - *) - echo "$0 {start|stop|restart} " - ;; -esac - -exit 0 +sys.exit(export_rootfs_main()) From patchwork Sun Aug 30 14:28:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33C0CC61DFF for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-65-225.siemens.flowmailer.net (mta-65-225.siemens.flowmailer.net [185.136.65.225]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9459.1788100184012431295 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=lXEReJIh; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.225, mailfrom: fm-1329275-20260830142942ccc3a96926000207dc-0l8f_n@rts-flowmailer.siemens.com) Received: by mta-65-225.siemens.flowmailer.net with ESMTPSA id 20260830142942ccc3a96926000207dc for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=eqp6S81Ngk5WrYFs3TiLjt8JsRRgPRMX+jdcAMIZtc8=; b=lXEReJIhWn6pPfgV/Tc/wXjgVg/hvBgAVWXsiPzPIg5/106JMW5jyR5XUlkX+IXDVMi6wf 00BDqMxja1kKA+Ia5heHNMQpIDPSO1lQ9njsf0XUIark6JXKgu3L6MrY4jrZekQAUNIzFFY4 Yo3kQG2jEYCZAymiWQn/E2bDxVxdAPSs5FYgKJcWghjsiUELIVnj/abSO98BRJ5d8GhjUkuV EKMPAewt9iRT1vDgiLZpYELFcbn1GbbZjUjULSnSkFIAlDiSKuURAXBPpbGsT09GtMeGI9Rn +47iWonIHzhAeMY9cs0Fu9vrxokaGq0zEm0eyFiSkj8cGwE3UBnE9VOA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 17/24] devtool: deploy: split ssh deployment into a separate function Date: Sun, 30 Aug 2026 16:28:44 +0200 Message-ID: <20260830142922.17241-18-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244674 From: Adrian Freihofer This is a refactoring preparation for adding a new local deployment method that will be used for NFS rootfs deployments. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/deploy.py | 168 ++++++++++++++++++---------------- 1 file changed, 87 insertions(+), 81 deletions(-) diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index cf481d6b63..a4fd83d305 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -254,6 +254,91 @@ def deploy(args, config, basepath, workspace): return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files) +def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files, + fakerootcmd, fakerootenv, path, recipe_outdir): + """Copy files to target_dir over ssh/scp (user@hostname[:destdir]).""" + extraoptions = '' + if args.no_host_check: + extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' + if not args.show_status: + extraoptions += ' -q' + + scp_sshexec = '' + ssh_sshexec = 'ssh' + if args.ssh_exec: + scp_sshexec = "-S %s" % args.ssh_exec + ssh_sshexec = args.ssh_exec + scp_port = '' + ssh_port = '' + if args.port: + scp_port = "-P %s" % args.port + ssh_port = "-p %s" % args.port + + if args.key: + extraoptions += ' -i %s' % args.key + + # In order to delete previously deployed files and have the manifest file on + # the target, we write out a shell script and then copy it to the target + # so we can then run it (piping tar output to it). + # (We cannot use scp here, because it doesn't preserve symlinks.) + tmpdir = tempfile.mkdtemp(prefix='devtool') + try: + tmpscript = '/tmp/devtool_deploy.sh' + tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list') + shellscript = _prepare_remote_script(deploy=True, + destdir=destdir, + verbose=args.show_status, + nopreserve=args.no_preserve, + nocheckspace=args.no_check_space) + # Write out the script to a file + with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f: + f.write(shellscript) + # Write out the file list + with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f: + f.write('%d\n' % ftotalsize) + for fpath, fsize in filelist: + f.write('%s %d\n' % (fpath, fsize)) + # Copy them to the target + ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True) + if ret != 0: + raise DevtoolError('Failed to copy script to %s - rerun with -s to ' + 'get a complete error message' % args.target) + finally: + shutil.rmtree(tmpdir) + + # Now run the script. When a package/glob filter narrowed down filelist, + # tar is given an explicit list of relative paths (-T) instead of packing + # the whole recipe_outdir tree. + tar_filelist_path = None + try: + if allowed_files is not None: + tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-') + with os.fdopen(tar_fd, 'w') as f: + for relpath in tar_relpaths: + # './' prefix matches what 'tar cf - .' itself would produce, which + # the remote script's manifest handling (sed "s!^./!$2!") relies on. + f.write('./' + relpath + '\n') + tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path) + else: + tar_cmd = 'tar cf - .' + ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, '%s | %s %s %s %s \'sh %s %s %s %s\'' % (tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) + finally: + if tar_filelist_path: + os.remove(tar_filelist_path) + if ret != 0: + raise DevtoolError('Deploy failed - rerun with -s to get a complete ' + 'error message') + + logger.info('Successfully deployed %s' % recipe_outdir) + + files_list = [] + for root, _, files in os.walk(recipe_outdir): + for filename in files: + filename = os.path.relpath(os.path.join(root, filename), recipe_outdir) + files_list.append(os.path.join(destdir, filename)) + + return 0 + def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None): import math @@ -351,87 +436,8 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce print(' %s' % item) return 0 - extraoptions = '' - if args.no_host_check: - extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' - if not args.show_status: - extraoptions += ' -q' - - scp_sshexec = '' - ssh_sshexec = 'ssh' - if args.ssh_exec: - scp_sshexec = "-S %s" % args.ssh_exec - ssh_sshexec = args.ssh_exec - scp_port = '' - ssh_port = '' - if args.port: - scp_port = "-P %s" % args.port - ssh_port = "-p %s" % args.port - - if args.key: - extraoptions += ' -i %s' % args.key - - # In order to delete previously deployed files and have the manifest file on - # the target, we write out a shell script and then copy it to the target - # so we can then run it (piping tar output to it). - # (We cannot use scp here, because it doesn't preserve symlinks.) - tmpdir = tempfile.mkdtemp(prefix='devtool') - try: - tmpscript = '/tmp/devtool_deploy.sh' - tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list') - shellscript = _prepare_remote_script(deploy=True, - destdir=destdir, - verbose=args.show_status, - nopreserve=args.no_preserve, - nocheckspace=args.no_check_space) - # Write out the script to a file - with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f: - f.write(shellscript) - # Write out the file list - with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f: - f.write('%d\n' % ftotalsize) - for fpath, fsize in filelist: - f.write('%s %d\n' % (fpath, fsize)) - # Copy them to the target - ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True) - if ret != 0: - raise DevtoolError('Failed to copy script to %s - rerun with -s to ' - 'get a complete error message' % args.target) - finally: - shutil.rmtree(tmpdir) - - # Now run the script. When a package/glob filter narrowed down filelist, - # tar is given an explicit list of relative paths (-T) instead of packing - # the whole recipe_outdir tree. - tar_filelist_path = None - try: - if allowed_files is not None: - tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-') - with os.fdopen(tar_fd, 'w') as f: - for relpath in tar_relpaths: - # './' prefix matches what 'tar cf - .' itself would produce, which - # the remote script's manifest handling (sed "s!^./!$2!") relies on. - f.write('./' + relpath + '\n') - tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path) - else: - tar_cmd = 'tar cf - .' - ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, '%s | %s %s %s %s \'sh %s %s %s %s\'' % (tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) - finally: - if tar_filelist_path: - os.remove(tar_filelist_path) - if ret != 0: - raise DevtoolError('Deploy failed - rerun with -s to get a complete ' - 'error message') - - logger.info('Successfully deployed %s' % recipe_outdir) - - files_list = [] - for root, _, files in os.walk(recipe_outdir): - for filename in files: - filename = os.path.relpath(os.path.join(root, filename), recipe_outdir) - files_list.append(os.path.join(destdir, filename)) - - return 0 + return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, + allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir) def undeploy(args, config, basepath, workspace): """Entry point for the devtool 'undeploy' subcommand""" From patchwork Sun Aug 30 14:28:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A874C61DD9 for ; Sun, 30 Aug 2026 14:29:52 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9427.1788100183422661276 for ; Sun, 30 Aug 2026 07:29:45 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=efLB+vi5; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-202608301429426e79cd1381000207f9-svua4o@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202608301429426e79cd1381000207f9 for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=cjV+f23McuxXialyiT814EX6HwKpUdJOhcJDq1JQiv4=; b=efLB+vi5wFXy+9niHrvXXfqxK6Ju0Jw7AIVn7ObwuAsaMCJUr0tgVBj7fZRfD7eR9BVvfg WfoxfoEYTpAVczvZQQRwpIeYz+ZK7oNoBoQ2VnznSg3HccsPUQOl+4vUZAk1T7es2RhOoy7L c+Jr7SRDJNvfI1OG5vLJmo9oecNLy9v7Gi0Vj5BxBFrx1oIgr2RiKTECm34s7cX4ukXB6ppC VQn9WJYu31rEPKpyAqjoDsMnvrGIjKX8njkvozwVTLWaM/LbOU7OxO2iMgj34v5t/j/pkazI DfnscT7bRuNA1VqRMBw3+9JWFFanVLF+/ylM/jtMDNXSMB81E+c/Yi7w==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 18/24] devtool: deploy-target: allow deploying directly into a local rootfs Date: Sun, 30 Aug 2026 16:28:45 +0200 Message-ID: <20260830142922.17241-19-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:52 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244682 From: Adrian Freihofer Accept an absolute path as the deploy-target "target" argument to copy files directly into a local pseudo-managed rootfs directory (e.g. one extracted by 'devtool ide-sdk --nfs' for NFS booting), instead of always requiring an ssh connection. Deploying over ssh to the target would also work for an NFS-mounted rootfs, but that means dealing with the target's IP address and sshd port, which is way more cumbersome than just passing a local path. Copying the files in directly is also faster, since it avoids the network/ssh round trip entirely. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/deploy.py | 113 +++++++++++++++++++++++++++++++--- 1 file changed, 106 insertions(+), 7 deletions(-) diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index a4fd83d305..88c6e383a0 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -20,6 +20,7 @@ import oe.types import oe.package from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, DevtoolError +from runqemu_utils import native_environment, pseudo_state_dir logger = logging.getLogger('devtool') @@ -254,6 +255,85 @@ def deploy(args, config, basepath, workspace): return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files) +def _deploy_local(args, target_dir, filelist, ftotalsize, tar_relpaths, allowed_files, + fakerootcmd, fakerootenv, path, recipe_outdir): + """Copy files directly into target_dir instead of over ssh/scp. + + target_dir is expected to be a pseudo-managed rootfs previously extracted + on the host, e.g. by runqemu-extract-sdk or 'devtool ide-sdk --nfs' for a + target device or QEMU instance that NFS-mounts it. + """ + if not os.path.isdir(target_dir): + raise DevtoolError('Target directory %s does not exist' % target_dir) + state_dir = pseudo_state_dir(target_dir) + if not os.path.isdir(state_dir): + raise DevtoolError( + '%s does not exist - %s does not look like a pseudo-managed rootfs ' + '(e.g. one extracted by devtool ide-sdk --nfs).' % (state_dir, target_dir)) + + environment = native_environment() + pseudo = environment.get('PSEUDO') + native_sysroot = environment.get('OECORE_NATIVE_SYSROOT') + if not pseudo or not native_sysroot: + raise DevtoolError('qemu-helper-native did not provide pseudo') + + if not args.no_check_space: + freespace = shutil.disk_usage(target_dir).free // 1024 + if ftotalsize > freespace: + raise DevtoolError('Deploy failed - insufficient space on target ' + '(available %d, needed %d)' % (freespace, ftotalsize)) + + # destdir is target_dir itself: the script writes directly into the + # local pseudo-managed rootfs directory, not into the real filesystem root. + destdir = target_dir + shellscript = _prepare_remote_script(deploy=True, + destdir=destdir, + verbose=args.show_status, + nopreserve=args.no_preserve, + nocheckspace=True) + + tmpdir = tempfile.mkdtemp(prefix='devtool') + tar_filelist_path = None + try: + script_path = os.path.join(tmpdir, 'devtool_deploy.sh') + with open(script_path, 'w') as f: + f.write(shellscript) + filelist_path = os.path.join(tmpdir, 'devtool_deploy.list') + with open(filelist_path, 'w') as f: + f.write('%d\n' % ftotalsize) + for fpath, fsize in filelist: + f.write('%s %d\n' % (fpath, fsize)) + + if allowed_files is not None: + tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-') + with os.fdopen(tar_fd, 'w') as f: + for relpath in tar_relpaths: + f.write('./' + relpath + '\n') + tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path) + else: + tar_cmd = 'tar cf - .' + + # The extraction side needs its own pseudo session (with different database) than the recipe's own. + extract_cmd = 'PSEUDO_LOCALSTATEDIR=%s PSEUDO_INCLUDE_PATHS=%s %s -P %s sh %s %s %s %s' % ( + shlex.quote(state_dir), shlex.quote(destdir), shlex.quote(pseudo), + shlex.quote(os.path.join(native_sysroot, 'usr')), + shlex.quote(script_path), shlex.quote(args.recipename), + shlex.quote(destdir), shlex.quote(filelist_path)) + ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, + '%s | %s' % (tar_cmd, extract_cmd), + cwd=recipe_outdir, shell=True) + finally: + if tar_filelist_path: + os.remove(tar_filelist_path) + shutil.rmtree(tmpdir) + + if ret != 0: + raise DevtoolError('Deploy failed - rerun with -s to get a complete ' + 'error message') + + logger.info('Successfully deployed %s to %s' % (recipe_outdir, target_dir)) + return 0 + def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir): """Copy files to target_dir over ssh/scp (user@hostname[:destdir]).""" @@ -342,12 +422,16 @@ def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None): import math - try: - host, destdir = args.target.split(':') - except ValueError: - destdir = '/' + if os.path.isabs(args.target): + # A local pseudo-managed rootfs directory (e.g. NFS-exported) + destdir = os.path.realpath(args.target) else: - args.target = host + try: + host, destdir = args.target.split(':') + except ValueError: + destdir = '/' + else: + args.target = host if not destdir.endswith('/'): destdir += '/' @@ -436,6 +520,12 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce print(' %s' % item) return 0 + if os.path.isabs(args.target): + # A local directory (e.g. an NFS-exported rootfs) rather than a + # user@host ssh target: copy the files in directly, no network needed. + return _deploy_local(args, os.path.realpath(args.target), filelist, ftotalsize, tar_relpaths, + allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir) + return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir) @@ -503,10 +593,19 @@ def register_commands(subparsers, context): parser_deploy = subparsers.add_parser('deploy-target', help='Deploy recipe output files to live target machine', - description='Deploys a recipe\'s build output (i.e. the output of the do_install task) to a live target machine over ssh. By default, any existing files will be preserved instead of being overwritten and will be restored if you run devtool undeploy-target. Note: this only deploys the recipe itself and not any runtime dependencies, so it is assumed that those have been installed on the target beforehand. Use --package/--file-glob to deploy only a subset of the recipe\'s installed files.', + description='Deploys a recipe\'s build output (i.e. the output of ' + 'the do_install task) to a live target machine over ssh, ' + 'or directly into a local pseudo-managed rootfs directory ' + '(e.g. one extracted for NFS booting). Existing files are ' + 'preserved by default and restored by devtool ' + 'undeploy-target. Only the recipe itself is deployed, not ' + 'its runtime dependencies. Use --package/--file-glob to ' + 'deploy only a subset of the recipe\'s installed files.', group='testbuild') parser_deploy.add_argument('recipename', help='Recipe to deploy') - parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]') + parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]. ' + 'Alternatively, an absolute path to a local pseudo-managed rootfs directory ' + '(e.g. one extracted by devtool ide-sdk --nfs) to copy the files into directly, without ssh.') parser_deploy.add_argument('-c', '--no-host-check', help='Disable ssh host key checking', action='store_true') parser_deploy.add_argument('-s', '--show-status', help='Show progress/status output', action='store_true') parser_deploy.add_argument('-n', '--dry-run', help='List files to be deployed only', action='store_true') From patchwork Sun Aug 30 14:28:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33956C624C6 for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9435.1788100183850000808 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=EpUndzpD; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-20260830142942267301f6080002071a-bf_krj@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20260830142942267301f6080002071a for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=mdB2D9oOIhE1+3lfi9RCmJPeu6zpwig5nfpZvKdceLM=; b=EpUndzpDudJi0lORkitnnHhemdT5GWXqSvddDplF+VUl4TXTBQ+UNTxgkTnTuXG/SazQnm YDavM3tHlxQL7NuTdTBxv2j1uZ/qH4E8/JxAX7NrLqCMfQuEDXp4jW4hX93kz62dKz5fIB2G Xa/rGelCiNtVrxfOiKkESsT+qbe00rYpL4Kjv0N4TBkJW/9ZTTAecrbbeE6Vc26i2edwj9CT hgw+QcqeF9lW4M/YBt5xnn0R5wdpJuP6digY5DePxxn7C7yBX3cYa3KjQTF+K8Oadlxht0Sl 3uZpg5P5yWudSUuCiOcX6GQToeIzOKZ58wYLNfbs6I7q1O8XkL0+INgA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 19/24] oe-selftest: devtool deploy-target: test deploying into a local rootfs path Date: Sun, 30 Aug 2026 16:28:46 +0200 Message-ID: <20260830142922.17241-20-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244667 From: Adrian Freihofer Extract an image's rootfs tarball with runqemu-extract-sdk, boot it via NFS with runqemu, then deploy a devtool-modified recipe directly into that directory with plain 'devtool deploy-target ' (no ssh) while the target has it mounted live, and confirm the target immediately sees the change. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 73 +++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 3e0eb7948f..35f2aa2e6c 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -2171,6 +2171,79 @@ class DevtoolDeployTargetTests(DevtoolBase): extra_args = ' '.join(a for a in (strip_opt, filter_args) if a) _deploy_and_check(extra_args, check_full_filelist=not filter_args, expected_files=expected_files) + @OETestTag("runqemu") + def test_devtool_deploy_target_path(self): + """Verify 'devtool deploy-target ' deploys straight into + a local pseudo-managed rootfs directory (no ssh), and that a target + booting that same directory via NFS immediately sees the change. + """ + self._check_runqemu_prerequisites() + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + testrecipe = 'mdadm' + testfile = '/sbin/mdadm' + if "usrmerge" in get_bb_var('DISTRO_FEATURES'): + testfile = '/usr/sbin/mdadm' + testcommand = '/sbin/mdadm --help' + testimage = 'oe-selftest-image' + + # A tar rootfs is needed both to extract a local copy of it (below) + # and for runqemu to NFS-boot straight from that extracted directory. + self.append_config('IMAGE_FSTYPES:append = " tar"\n') + bitbake("%s qemu-native qemu-helper-native" % testimage) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], testimage) + deploy_dir_image = bb_vars['DEPLOY_DIR_IMAGE'] + image_link_name = bb_vars['IMAGE_LINK_NAME'] + self.add_command_to_tearDown('bitbake -c clean %s' % testimage) + self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, testimage)) + + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + runCmd('devtool modify %s -x %s' % (testrecipe, tempdir)) + runCmd('devtool build %s' % testrecipe) + + # Extract a local pseudo-managed rootfs the same way + rootfs_tarball = os.path.join(deploy_dir_image, image_link_name + '.tar') + self.assertExists(rootfs_tarball) + extractdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(extractdir) + nfs_rootfs = os.path.join(extractdir, 'rootfs') + runCmd('runqemu-extract-sdk %s %s' % (rootfs_tarball, nfs_rootfs)) + self.assertExists(nfs_rootfs) + self.assertExists(nfs_rootfs + '.pseudo_state') + + # oe-selftest-image does not install mdadm by default, so the target must not see it yet. + self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + qemuboot = os.path.join(deploy_dir_image, image_link_name + '.qemuboot.conf') + self.assertExists(qemuboot) + launch_cmd = 'runqemu %s %s nographic' % (shlex.quote(qemuboot), shlex.quote(nfs_rootfs)) + with runqemu(testimage, launch_cmd=launch_cmd) as qemu: + status, output = qemu.run("awk '$2 == \"/\" {print $3}' /proc/mounts") + self.assertEqual(status, 0) + self.assertEqual(output.strip(), 'nfs') + + status, _ = qemu.run(testcommand) + self.assertNotEqual(status, 0, '%s should not be deployed yet' % testfile) + + # Deploy directly into the local rootfs path (no ssh) while the target has it NFS-mounted live + deploy_cmd = 'devtool deploy-target %s %s' % (testrecipe, nfs_rootfs) + if self.logger.isEnabledFor(logging.DEBUG): + deploy_cmd += ' -s' + result = runCmd(deploy_cmd) + self.assertEqual(result.status, 0) + self.assertExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + status, _ = qemu.run(testcommand) + self.assertEqual(status, 0, '%s was not deployed' % testfile) + + # Deploying again while the target still has this directory NFS-mounted live must still succeed. + result = runCmd(deploy_cmd) + self.assertEqual(result.status, 0) + + class DevtoolBuildImageTests(DevtoolBase): def test_devtool_build_image(self): From patchwork Sun Aug 30 14:28:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96810 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A996BC61DE8 for ; Sun, 30 Aug 2026 14:29:52 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9426.1788100183422503403 for ; Sun, 30 Aug 2026 07:29:45 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=iZmH/ofx; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-202608301429421db69207520002074e-uusizo@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 202608301429421db69207520002074e for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=wjXYdBKkBti3fvtRYwV2IoFngp/gZU0c9Ucxai+hSvM=; b=iZmH/ofxLZZ5rWa1fXcNdY7wyJZa7e3Dwf9WNrvaaAHl2sP03Jld6bRETHnEOVVBuIyZ74 nwyFGOVqkLpW71wlXLdBfROonz2Dhg0UE+/wmNpES3b1kS8WTIGAnuC0siuhs3UjfXzZLFLh LIHUwprjOKKKfKcuRcblNgZ5WEwWhBpr16wkQtGo1Fn6fdOEm86SN6qowtYnuRdrfB97lNfz BDUE4Q0P+EADmZxf3AUkQcb78Zw0f3bxELGb6jR3AmB17GFG8V8An7H7zhmDBBVv1sfm8K2X UReOaEPJfrmDjY2PAtn7WWo4FTCI1Tfwmsyfz1WJYGPDQ5UN1znUnejw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 20/24] oe-selftest: tinfoil: test prepared task runner Date: Sun, 30 Aug 2026 16:28:47 +0200 Message-ID: <20260830142922.17241-21-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:52 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244681 From: Adrian Freihofer Add shell and Python marker tasks to pseudo-pyc-test, with and without fakeroot. Build the tasks normally to prepare their dependencies, then run them again through Tinfoil's run_prepared_task() API. This verifies that a prepared task can be rerun through BitBake's worker path without resolving its dependencies, including shell and Python task execution within and outside a pseudo session as needed by devtool ide-sdk. Exercise Tinfoil.run_prepared_task() with shell and Python tasks, both with and without fakeroot. Build the exact task targets normally to prepare their prerequisites, remove their markers, then rerun them through Tinfoil. This verifies the no-dependency rerun uses BitBake's worker path for each task type and pseudo context. Signed-off-by: Adrian Freihofer --- .../pseudo-pyc-test/pseudo-pyc-test.bb | 45 ++++++++++++++ meta/lib/oeqa/selftest/cases/tinfoil.py | 60 +++++++++++++++++++ 2 files changed, 105 insertions(+) diff --git a/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb b/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb index 12dc91a8f3..f6b31b63cd 100644 --- a/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb +++ b/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb @@ -13,3 +13,48 @@ python do_install() { import pseudo_pyc_test2 print(pseudo_pyc_test2.STRING) } + +TINFOIL_TEST_MARKER_DIR = "${TMPDIR}/tinfoil-prepared-task" +TINFOIL_TEST_MARKER_VALUE ?= "" + +write_tinfoil_shell_marker() { + mkdir -p ${TINFOIL_TEST_MARKER_DIR} + printf '%s%s' "$1" "${TINFOIL_TEST_MARKER_VALUE}" > ${TINFOIL_TEST_MARKER_DIR}/"$1" +} + +do_tinfoil_dep() { + write_tinfoil_shell_marker dep +} +addtask tinfoil_dep + +do_tinfoil_shell() { + write_tinfoil_shell_marker shell +} +addtask tinfoil_shell after do_tinfoil_dep + +do_tinfoil_shell_fakeroot() { + write_tinfoil_shell_marker shell-fakeroot +} +do_tinfoil_shell_fakeroot[fakeroot] = "1" +do_tinfoil_shell_fakeroot[depends] += "virtual/fakeroot-native:do_populate_sysroot" +addtask tinfoil_shell_fakeroot after do_tinfoil_dep + +def write_tinfoil_marker(d, name): + import os + + marker_dir = d.getVar('TINFOIL_TEST_MARKER_DIR') + os.makedirs(marker_dir, exist_ok=True) + with open(os.path.join(marker_dir, name), 'w') as marker: + marker.write(name + d.getVar('TINFOIL_TEST_MARKER_VALUE')) + +python do_tinfoil_python() { + write_tinfoil_marker(d, 'python') +} +addtask tinfoil_python after do_tinfoil_dep + +python do_tinfoil_python_fakeroot() { + write_tinfoil_marker(d, 'python-fakeroot') +} +do_tinfoil_python_fakeroot[fakeroot] = "1" +do_tinfoil_python_fakeroot[depends] += "virtual/fakeroot-native:do_populate_sysroot" +addtask tinfoil_python_fakeroot after do_tinfoil_dep diff --git a/meta/lib/oeqa/selftest/cases/tinfoil.py b/meta/lib/oeqa/selftest/cases/tinfoil.py index 21c8686b2a..d8009eddfe 100644 --- a/meta/lib/oeqa/selftest/cases/tinfoil.py +++ b/meta/lib/oeqa/selftest/cases/tinfoil.py @@ -6,11 +6,14 @@ import os import re +import shutil import time import logging +import uuid import bb.tinfoil from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake class TinfoilTests(OESelftestTestCase): """ Basic tests for the tinfoil API """ @@ -77,6 +80,63 @@ class TinfoilTests(OESelftestTestCase): localdata.setVar('PN', 'hello') self.assertEqual('hello', localdata.getVar('BPN')) + def test_run_prepared_task(self): + """Verify run_prepared_task() runs only the requested task, not its + dependency. + + pseudo-pyc-test's do_tinfoil_* tasks depend on do_tinfoil_dep; both + write a marker file containing the task name plus the current value + of TINFOIL_TEST_MARKER_VALUE. A normal bitbake build is expected to + update both the dependency's and the requested tasks' markers to a + new value. Calling run_prepared_task() with another new value must + update only the requested tasks' markers, leaving do_tinfoil_dep's + marker at the value written by the earlier bitbake build. + """ + marker_dir = os.path.join( + self.builddir, 'tmp', 'tinfoil-prepared-task') + shutil.rmtree(marker_dir, ignore_errors=True) + self.track_for_cleanup(marker_dir) + + tasks = ('shell', 'shell_fakeroot', 'python', 'python_fakeroot') + + # Use a fresh, random value each run so the tasks' signatures change + # and bitbake can't skip them as "up to date" from a previous run. + value1 = uuid.uuid4().hex + bitbake(' '.join('pseudo-pyc-test:do_tinfoil_%s' % task + for task in tasks), + postconfig='TINFOIL_TEST_MARKER_VALUE = "%s"\n' % value1) + + # do_tinfoil_dep is a dependency of all the tasks above, so a normal + # bitbake build is expected to have run it too, with the same value. + with open(os.path.join(marker_dir, 'dep')) as marker: + self.assertEqual(marker.read(), 'dep' + value1) + for task in tasks: + marker_name = task.replace('_', '-') + with open(os.path.join(marker_dir, marker_name)) as marker: + self.assertEqual(marker.read(), marker_name + value1) + + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(config_only=False, quiet=2) + # A distinct value per call proves each run_prepared_task() + # invocation re-reads the variable rather than reusing a value + # cached from an earlier call in this loop. + values2 = {task: uuid.uuid4().hex for task in tasks} + for task in tasks: + tinfoil.run_command('setVariable', 'TINFOIL_TEST_MARKER_VALUE', values2[task]) + tinfoil.run_prepared_task( + 'pseudo-pyc-test', 'do_tinfoil_%s' % task) + + # The requested tasks must have re-run and picked up their own value + for task in tasks: + marker_name = task.replace('_', '-') + with open(os.path.join(marker_dir, marker_name)) as marker: + self.assertEqual(marker.read(), marker_name + values2[task]) + + # ...but run_prepared_task() must not have run do_tinfoil_dep, so its + # marker should still hold the value written by the earlier bitbake build + with open(os.path.join(marker_dir, 'dep')) as marker: + self.assertEqual(marker.read(), 'dep' + value1) + # The config_data API to parse_recipe_file is used by: # layerindex-web layerindex/update_layer.py def test_parse_recipe_custom_data(self): From patchwork Sun Aug 30 14:28:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 884B4C624CD for ; Sun, 30 Aug 2026 14:29:55 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9437.1788100184254935137 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=IVh7KIkE; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-20260830142942f8f3e6afa8000207e4-xbb5l6@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 20260830142942f8f3e6afa8000207e4 for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=kTdZ8cCfp5h5iAAVS7oinrBDwFaRgChydc2Y0MbV/Zk=; b=IVh7KIkEj93jp9IUcW0PN4HMkPJ31YkPa740ogPM67E3BMZbOScAwdBSZRLggaHShR7UTy QVFNb4IsdcYtxjNIl0rk2KBc29KzuT0XuDNAzNYVwTxLtuVr88qiyipocOsBc5tyuh6l00AP GecoDf0MRBEu6D91T1q8DVB+cpucmuZt9sDf2lCPfTxccXBqetlbAjf4PCMxzUssotQyuiUJ 4td3R0OEOfws1il+lpgYpsM58Qye0SZZ4Xi6kHMeiCPMjgb+/JsX55rncR0TK2QVgxloP350 lqOz3ostL/CWRH8NZZsKjrlSgaHsofS9cGb5W/5yqUeRcnLDikgXcpIw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 21/24] devtool: ide-sdk: run do_install through BitBake Date: Sun, 30 Aug 2026 16:28:48 +0200 Message-ID: <20260830142922.17241-22-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244665 From: Adrian Freihofer Generate a Python install-task runner for the IDE SDK environment. It prepares Tinfoil normally, then invokes run_prepared_task() to rerun do_install through BitBake's worker path before deploying the result. The IDE owns configure, compile, and unit-test steps in the modified source tree. At deployment time, resolving task dependencies could rerun those steps, discard the IDE's build results, and add unnecessary delay. The prepared-task runner therefore executes only do_install after BitBake has prepared its prerequisites. This supports shell and Python task bodies and lets BitBake establish the fakeroot environment when the task requires pseudo. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_sdk.py | 42 +++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index ac4930af50..ee3c50460a 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -1400,26 +1400,32 @@ class RecipeModified: self.fakerootcmd, self.fakerootenv)) return self.write_script(cmd_lines, 'deploy_target') + def gen_install_task_script(self): + """Generate a script which runs do_install through BitBake.""" + cmd_lines = ['#!%s' % sys.executable] + if self.cmd_compile: + cmd_lines += ['import subprocess', + 'subprocess.run(%s, cwd=%r, shell=True, check=True)' % + (repr(self.cmd_compile), self.real_srctree)] + cmd_lines += ['import os', + 'import sys', + 'sys.path.insert(0, %r)' % os.path.realpath( + os.path.join(self.bitbakepath, '..', 'lib')), + 'import bb.tinfoil', + 'os.chdir(%r)' % self.topdir, + 'tinfoil = bb.tinfoil.Tinfoil()', + 'try:', + ' tinfoil.prepare(config_only=False, quiet=2)', + ' tinfoil.run_prepared_task(%r, "do_install")' % self.pn, + 'finally:', + ' tinfoil.shutdown()'] + return self.write_script(cmd_lines, 'bb_run_do_install') + def gen_install_deploy_script(self, args): """Generate a script which does install and deploy""" - cmd_lines = ['#!/bin/sh'] - - # Save the original command-line args before 'set' overwrites $@ - cmd_lines.append('_args="$@"') - # . oe-init-build-env $BUILDDIR $BITBAKEDIR - # Using 'set' to pass the build directory to oe-init-build-env in sh syntax - cmd_lines.append('cd "%s" || { echo "cd %s failed"; exit 1; }' % ( - self.oe_init_dir, self.oe_init_dir)) - cmd_lines.append('set %s %s' % (self.topdir, self.bitbakepath.rstrip('/bin'))) - cmd_lines.append('. "%s" || { echo ". %s %s failed"; exit 1; }' % ( - self.oe_init_build_env, self.oe_init_build_env, self.topdir)) - - # bitbake -c install - cmd_lines.append( - 'bitbake %s -c install --force || { echo "bitbake %s -c install --force failed"; exit 1; }' % (self.bpn, self.bpn)) - - # Self contained devtool deploy-target - use saved args, not $@ (overwritten by 'set') - cmd_lines.append(self.gen_deploy_target_script(args) + ' $_args') + cmd_lines = ['#!/bin/sh -e'] + cmd_lines.append(self.gen_install_task_script()) + cmd_lines.append(self.gen_deploy_target_script(args) + ' "$@"') return self.write_script(cmd_lines, 'install_and_deploy') From patchwork Sun Aug 30 14:28:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 317E0C61DEB for ; Sun, 30 Aug 2026 14:29:53 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.9460.1788100184322595397 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=WjUgnIaU; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-20260830142942d539595e1f0002071f-dmxmqi@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20260830142942d539595e1f0002071f for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=c6H5zgyZn5heEV9AENr8J0pQu4Hx8RK9NjPOajNazuo=; b=WjUgnIaUS2NQ79DU8YvMNVJ7c2IcynV+xHBo052SdbBsKMNBtQffi7bfynykH/c7FJa6M6 GKaPs60ppHt5XvnQ+7JdCLXy0kRaFi1fnIoThHJH5hh2ulb47Z/pPkZDEIqf4zSMYhhF7csA jcvhF+ZYK1h5uogqDxTgK5LBM/kdu+Ube+JRp6Qd2KgUrL/h1XC5ZHTIFXUdbzuiy8h1SJoP 1xKiFVGF2sKZzX3XyKBGYknx7jNDkubxjvPzNabz9uO/jVjqWhUo4tChQnrOgzqDZ77ytHxR fFfTfNFbbu5PiqLC41J1cH3JjFrAyJPC/zMmv4bi7KcO7QrERcHXq+Fg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 22/24] oe-selftest: devtool ide-sdk: test install task Date: Sun, 30 Aug 2026 16:28:49 +0200 Message-ID: <20260830142922.17241-23-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244680 From: Adrian Freihofer Verify that the ide-sdk setup generates the do_install task script and that the script runs successfully. This covers the prepared-task runner used by the IDE deployment workflow without requiring Qemu. Replace the ad-hoc dependsOn handling in _verify_launch_config() with a new helper that mirrors VS Code's task dependency resolution and execution order. This ensures that the selftest runs the same sequence of tasks as VS Code would, including compile and install/deploy prerequisites, before starting the debug server. This also aligns the gdb and lldb selftest paths to avoid behavior drift and improve test reliability. --- meta/lib/oeqa/selftest/cases/devtool.py | 86 ++++++++++++++++++++----- 1 file changed, 70 insertions(+), 16 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 35f2aa2e6c..c05cf7bc82 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3116,7 +3116,7 @@ class DevtoolIdeSdkTests(DevtoolBase): """Verify the scripts referred by the tasks.json file are fine. This function does not depend on Qemu. Therefore it verifies the scripts - exists and the delete step works as expected. But it does not try to + exists and the install step works as expected. But it does not try to deploy to Qemu. """ recipe_id, recipe_id_pretty = self._get_recipe_ids(recipe_name) @@ -3134,6 +3134,11 @@ class DevtoolIdeSdkTests(DevtoolBase): i_and_d_script_path = os.path.join( self._workspace_scripts_dir(recipe_name), i_and_d_script) self.assertExists(i_and_d_script_path) + install_script = 'bb_run_do_install_' + recipe_id + install_script_path = os.path.join( + self._workspace_scripts_dir(recipe_name), install_script) + self.assertExists(install_script_path) + runCmd(install_script_path, cwd=tempdir, output_log=self._cmd_logger) deploy_script_path = os.path.join( self._workspace_scripts_dir(recipe_name), 'deploy_target_' + recipe_id) @@ -3368,6 +3373,60 @@ class DevtoolIdeSdkTests(DevtoolBase): self.assertEqual(actual_group, group, msg="%s not owned by group %s: got %s" % (conf_file, group, actual_group)) + def _run_vscode_task_with_dependencies(self, tempdir, task_label, run_main_task=True): + """Run a VS Code shell task and its dependsOn chain. + + The helper mirrors VS Code task dependency ordering for selftests, so + preLaunchTask prerequisites (e.g. compile and do_install/deploy tasks) + run before the debug server task. + """ + tasks_path = os.path.join(tempdir, '.vscode', 'tasks.json') + self.assertExists(tasks_path, 'tasks.json not found at %s' % tasks_path) + + with open(tasks_path) as tasks_j: + tasks_d = json.load(tasks_j) + + task_by_label = { + task.get('label'): task + for task in tasks_d.get('tasks', []) + if task.get('label') + } + self.assertIn(task_label, task_by_label, + "Task '%s' not found in tasks.json" % task_label) + + ran = set() + + def run_task(label, execute_task): + if label in ran: + return + + task = task_by_label[label] + deps = task.get('dependsOn', []) + if isinstance(deps, str): + deps = [deps] + + for dep in deps: + self.assertIn(dep, task_by_label, + "DependsOn task '%s' not found in tasks.json" % dep) + run_task(dep, True) + + if execute_task: + command = task.get('command') + if command: + args = task.get('args', []) + if not isinstance(args, list): + args = [args] + task_cmd = [command] + [str(arg) for arg in args] + + task_cwd = task.get('options', {}).get('cwd', tempdir) + if isinstance(task_cwd, str): + task_cwd = task_cwd.replace('${workspaceFolder}', tempdir) + runCmd(task_cmd, cwd=task_cwd, output_log=self._cmd_logger) + + ran.add(label) + + run_task(task_label, run_main_task) + class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): @@ -3853,21 +3912,10 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): (task for task in tasks if task["label"] == prelaunch_task_name), None) self.assertIsNotNone(prelaunch_task, f"PreLaunchTask '{prelaunch_task_name}' not found in tasks.json") - # Find the dependsOn task if exists (install and deploy-target) - if "dependsOn" in prelaunch_task: - depends_task_names = prelaunch_task["dependsOn"] - for depends_task_name in depends_task_names: - depends_task = next( - (task for task in tasks if task["label"] == depends_task_name), None) - self.assertIsNotNone(depends_task, f"DependsOn task '{depends_task_name}' not found in tasks.json") - # For simplicity, we assume the dependsOn task is a prerequisite and does not affect the main command - self.logger.debug(f"PreLaunchTask '{prelaunch_task_name}' depends on '{depends_task_name}'") - - # Extract command details from dependsOn task - depends_task_command = depends_task["command"] - depends_task_args = depends_task.get("args", []) - self.logger.debug(f"Would execute dependsOn task: {depends_task_command} {' '.join(depends_task_args)}") - runCmd(f"{depends_task_command} {' '.join(depends_task_args)}", output_log=self._cmd_logger) + # Execute preLaunchTask prerequisites exactly like VS Code dependsOn + # handling, but keep starting the debug server in this method. + self._run_vscode_task_with_dependencies( + tempdir, prelaunch_task_name, run_main_task=False) # Verify task structure and extract command details self.assertEqual(prelaunch_task["type"], "shell", f"Task '{prelaunch_task_name}' should be shell type") @@ -4635,6 +4683,12 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): self.assertIsNotNone(prelaunch_task, "preLaunchTask '%s' not found in tasks.json" % prelaunch_task_name) + # Run compile/install/deploy prerequisites exactly as VS Code would, + # but keep starting the debug server itself in this method so we can + # retain explicit readiness handling and diagnostics. + self._run_vscode_task_with_dependencies( + tempdir, prelaunch_task_name, run_main_task=False) + # Extract the SSH command and start lldb-server on the target task_command = prelaunch_task["command"] task_args = prelaunch_task["args"] From patchwork Sun Aug 30 14:28:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96808 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EA56C61DE1 for ; Sun, 30 Aug 2026 14:29:52 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9428.1788100183422803269 for ; Sun, 30 Aug 2026 07:29:45 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=cOodslCw; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-2026083014294229b937ac9b000207b8-f8fevb@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 2026083014294229b937ac9b000207b8 for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=B8M3e2ftQKbU/9etaO/tIPHceGHCINHXs6ltOBNq8IU=; b=cOodslCw3uu0bMPE6CIq6O3narqZjOcd2DQlL5hTONmAE1ei7e1iLWS9ZBTpCrV8aRxiP7 5dQELPR1TMo0JnJOApGJHf87ltfM+Cvj6zCWcr5Md+o5OObTeJpmm21uBMjxa7YtG5eZkOYH 2+nDqz58oFRQK7qkbUdUlzUtkD/ZVvim74J9mPXKTrfO1fPzl3bSgXVFh8kcDQelEQ4MVO6Z SWDJ2PivskY7jVElxMww+VK9Iud1r5l9CV0XbtRGzoHM5xuzwzwUpwTO74A2JtB9IUceqO16 /m7tq/4XIC38t6E4ewygW62i8rofiHRPS+bDfSKxWtQDo0ObvCHQZrdw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 23/24] devtool: ide-sdk: support NFS rootfs Date: Sun, 30 Aug 2026 16:28:50 +0200 Message-ID: <20260830142922.17241-24-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:52 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244683 From: Adrian Freihofer Add --nfs=rootfs and --nfs=rootfs-dbg to build the matching tar image, extract it below the workspace image directory, and use it as the deploy-target/debug destination instead of ssh. Configure remote debuggers to search the extracted workspace rootfs-dbg when --nfs=rootfs-dbg is selected, matching the filesystem exported to the target. devtool deploy-target and the generated deploy_target/install_and_deploy scripts learn to copy files directly into a local pseudo-managed rootfs directory (deploy_no_d()/_deploy_local()) when the target is an absolute path, instead of requiring ssh. The pseudo binary needed for this is resolved from qemu-helper-native's STAGING_DIR_NATIVE at devtool ide-sdk generation time, when the build environment is known to be sourced, and baked into the generated script so it keeps working when later re-run standalone, e.g. as a VS Code task, without oe-find-native-sysroot on PATH. tasks.json generation (vscode_tasks_cpp/vscode_tasks_kernel_module) now resolves the effective deploy target via resolve_deploy_target(), so the --nfs local directory is used consistently instead of the ssh target when both apply. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/deploy.py | 22 ++- scripts/lib/devtool/ide_plugins/__init__.py | 22 ++- scripts/lib/devtool/ide_plugins/ide_code.py | 39 +++-- scripts/lib/devtool/ide_plugins/ide_none.py | 2 +- scripts/lib/devtool/ide_sdk.py | 153 ++++++++++++++++++-- 5 files changed, 203 insertions(+), 35 deletions(-) diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index 88c6e383a0..020174502a 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -256,7 +256,7 @@ def deploy(args, config, basepath, workspace): return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files) def _deploy_local(args, target_dir, filelist, ftotalsize, tar_relpaths, allowed_files, - fakerootcmd, fakerootenv, path, recipe_outdir): + fakerootcmd, fakerootenv, path, recipe_outdir, qemu_helper_native_sysroot=None): """Copy files directly into target_dir instead of over ssh/scp. target_dir is expected to be a pseudo-managed rootfs previously extracted @@ -271,10 +271,17 @@ def _deploy_local(args, target_dir, filelist, ftotalsize, tar_relpaths, allowed_ '%s does not exist - %s does not look like a pseudo-managed rootfs ' '(e.g. one extracted by devtool ide-sdk --nfs).' % (state_dir, target_dir)) - environment = native_environment() - pseudo = environment.get('PSEUDO') - native_sysroot = environment.get('OECORE_NATIVE_SYSROOT') - if not pseudo or not native_sysroot: + if qemu_helper_native_sysroot: + # Already resolved at script-generation time (when the build + # environment was sourced), so this can run standalone, e.g. from a + # devtool ide-sdk generated script/VS Code task. + native_sysroot = qemu_helper_native_sysroot + pseudo = os.path.join(native_sysroot, 'usr', 'bin', 'pseudo') + else: + environment = native_environment() + pseudo = environment.get('PSEUDO') + native_sysroot = environment.get('OECORE_NATIVE_SYSROOT') + if not pseudo or not native_sysroot or not os.path.exists(pseudo): raise DevtoolError('qemu-helper-native did not provide pseudo') if not args.no_check_space: @@ -419,7 +426,7 @@ def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files return 0 -def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None): +def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None, qemu_helper_native_sysroot=None): import math if os.path.isabs(args.target): @@ -524,7 +531,8 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce # A local directory (e.g. an NFS-exported rootfs) rather than a # user@host ssh target: copy the files in directly, no network needed. return _deploy_local(args, os.path.realpath(args.target), filelist, ftotalsize, tar_relpaths, - allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir) + allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir, + qemu_helper_native_sysroot=qemu_helper_native_sysroot) return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir) diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py index 0d0117344d..fd66060682 100644 --- a/scripts/lib/devtool/ide_plugins/__init__.py +++ b/scripts/lib/devtool/ide_plugins/__init__.py @@ -406,11 +406,27 @@ class IdeBase: raise err -def get_devtool_deploy_opts(args): +def resolve_deploy_target(args, target_override=None): + """Resolve the effective devtool deploy-target destination. + + target_override, e.g. the local NFS rootfs directory when --nfs was used + (see devtool.deploy for how a directory target is handled without ssh), + always takes precedence over the ssh target from -t/--target. Returns + None if neither is set. + """ + if target_override: + return target_override + if args.target: + return args.target + return None + + +def get_devtool_deploy_opts(args, target_override=None): """Filter args for devtool deploy-target args""" - if not args.target: + target = resolve_deploy_target(args, target_override) + if not target: return None - devtool_deploy_opts = [args.target] + devtool_deploy_opts = [target] if args.no_host_check: devtool_deploy_opts += ["-c"] if args.show_status: diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py index 937c63a4a5..88aa3ff805 100644 --- a/scripts/lib/devtool/ide_plugins/ide_code.py +++ b/scripts/lib/devtool/ide_plugins/ide_code.py @@ -9,7 +9,8 @@ import json import logging import os import shutil -from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, DebuggerServerModes, LldbServerConfig, get_devtool_deploy_opts +from devtool import DevtoolError +from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, DebuggerServerModes, LldbServerConfig, get_devtool_deploy_opts, resolve_deploy_target logger = logging.getLogger('devtool') @@ -593,10 +594,15 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), launch_file, launch_dict) - def vscode_tasks_cpp(self, args, modified_recipe): - run_install_deploy = modified_recipe.gen_install_deploy_script(args) + def vscode_tasks_cpp(self, args, image_recipe, modified_recipe): + run_install_deploy = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir) install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty - deploy_args = ["--target", args.target] + deploy_target = resolve_deploy_target(args, image_recipe.nfs_deploy_dir) + if not deploy_target: + raise DevtoolError('No deploy target available for %s' % modified_recipe.recipe_id_pretty) + # Same target the script already has baked in as its default, so this + # is a harmless no-op override, but keeps the task self-contained. + deploy_args = ["--target", deploy_target] if args.port: deploy_args += ["--port", args.port] for package in args.package or []: @@ -693,7 +699,7 @@ class IdeVSCode(IdeBase): args += [target_device.target, remote_cmd] return args - def vscode_tasks_kernel_module(self, args, modified_recipe): + def vscode_tasks_kernel_module(self, args, image_recipe, modified_recipe): """Generate tasks.json for kernel module recipes. Three tasks are generated and chained in sequence: @@ -716,8 +722,13 @@ class IdeVSCode(IdeBase): install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty reload_task_name = "reload module %s" % modified_recipe.recipe_id_pretty verify_task_name = "verify module %s" % modified_recipe.recipe_id_pretty - run_install_deploy = modified_recipe.gen_install_deploy_script(args) - deploy_args = ["--target", args.target] + run_install_deploy = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir) + deploy_target = resolve_deploy_target(args, image_recipe.nfs_deploy_dir) + if not deploy_target: + raise DevtoolError('No deploy target available for %s' % modified_recipe.recipe_id_pretty) + # Same target the script already has baked in as its default, so this + # is a harmless no-op override, but keeps the task self-contained. + deploy_args = ["--target", deploy_target] if args.port: deploy_args += ["--port", args.port] for package in args.package or []: @@ -759,7 +770,7 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), tasks_file, tasks_dict) - def vscode_tasks_fallback(self, args, modified_recipe): + def vscode_tasks_fallback(self, args, image_recipe, modified_recipe): oe_init_dir = modified_recipe.oe_init_dir oe_init = ". %s %s > /dev/null && " % (modified_recipe.oe_init_build_env, modified_recipe.topdir) dt_build = "devtool build " @@ -772,7 +783,7 @@ class IdeVSCode(IdeBase): dt_deploy_label = dt_deploy + modified_recipe.recipe_id_pretty dt_deploy_cmd = dt_deploy + modified_recipe.bpn dt_build_deploy_label = "devtool build & deploy-target %s" % modified_recipe.recipe_id_pretty - deploy_opts = ' '.join(get_devtool_deploy_opts(args)) + deploy_opts = ' '.join(get_devtool_deploy_opts(args, image_recipe.nfs_deploy_dir)) tasks_dict = { "version": "2.0.0", "tasks": [ @@ -892,13 +903,13 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), tasks_file, tasks_dict) - def vscode_tasks(self, args, modified_recipe): + def vscode_tasks(self, args, image_recipe, modified_recipe): if modified_recipe.build_tool.is_c_cpp: - self.vscode_tasks_cpp(args, modified_recipe) + self.vscode_tasks_cpp(args, image_recipe, modified_recipe) elif modified_recipe.build_tool == BuildTool.KERNEL_MODULE: - self.vscode_tasks_kernel_module(args, modified_recipe) + self.vscode_tasks_kernel_module(args, image_recipe, modified_recipe) else: - self.vscode_tasks_fallback(args, modified_recipe) + self.vscode_tasks_fallback(args, image_recipe, modified_recipe) def setup_modified_recipe(self, args, image_recipe, modified_recipe): self.vscode_settings(modified_recipe, image_recipe) @@ -912,7 +923,7 @@ class IdeVSCode(IdeBase): self.initialize_cross_debug_configs( image_recipe, modified_recipe, GdbCrossConfigVSCode) self.vscode_launch(args, modified_recipe) - self.vscode_tasks(args, modified_recipe) + self.vscode_tasks(args, image_recipe, modified_recipe) def register_ide_plugin(ide_plugins): diff --git a/scripts/lib/devtool/ide_plugins/ide_none.py b/scripts/lib/devtool/ide_plugins/ide_none.py index e4b255f2fa..559940fda2 100644 --- a/scripts/lib/devtool/ide_plugins/ide_none.py +++ b/scripts/lib/devtool/ide_plugins/ide_none.py @@ -312,7 +312,7 @@ class IdeNone(IdeBase): - Generate a gdbinit file per executable - Generate the oe-scripts sym-link """ - script_path = modified_recipe.gen_install_deploy_script(args) + script_path = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir) logger.info("Created: %s" % script_path) if modified_recipe.toolchain == 'clang': diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index ee3c50460a..8090117959 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -26,6 +26,7 @@ from devtool.standard import get_real_srctree from devtool.deploy import parse_packages_arg from devtool.ide_plugins import BuildTool, DebuggerCrossConfig from oe.kernel_module import kernel_module_os_env +from runqemu_utils import RunQemuRootfsError, extract_sdk_rootfs, pseudo_state_dir logger = logging.getLogger('devtool') @@ -212,8 +213,14 @@ class RecipeImage: def __init__(self, name, orig_bbappend_content=None): self.name = name - self.rootfs = None + self.pn = None + self.__rootfs = None self.__rootfs_dbg = None + self.__nfs_rootfs = None + self.__nfs_rootfs_dbg = None + self.__nfs_deploy_dir = None + self.deploy_dir_image = None + self.image_link_name = None self.qb_slirp_opt = '' self.bootstrap_tasks = [self.name + ':do_build'] # Debug settings already provided by the base configuration (e.g. @@ -221,6 +228,7 @@ class RecipeImage: # content other than devtool ide-sdk's own sections (see # strip_bbappend_sections()). Populated by initialize(). self.base_image_gen_debugfs = False + self.base_image_fstypes = set() self.base_image_fstypes_debugfs = '' self.base_has_combined_dbg = False self.base_image_install = set() @@ -271,8 +279,11 @@ class RecipeImage: raise DevtoolError( "Parsing image recipe %s failed" % self.name) + self.pn = image_d.getVar('PN') self.base_image_gen_debugfs = image_d.getVar( 'IMAGE_GEN_DEBUGFS') == '1' + self.base_image_fstypes = set( + (image_d.getVar('IMAGE_FSTYPES') or '').split()) self.base_image_fstypes_debugfs = image_d.getVar( 'IMAGE_FSTYPES_DEBUGFS') or '' self.base_has_combined_dbg = bb.data.inherits_class( @@ -281,22 +292,75 @@ class RecipeImage: (image_d.getVar('IMAGE_INSTALL') or '').split()) workdir = image_d.getVar('WORKDIR') - self.rootfs = os.path.join(workdir, 'rootfs') + self.__rootfs = os.path.join(workdir, 'rootfs') self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg') + self.deploy_dir_image = image_d.getVar('DEPLOY_DIR_IMAGE') + self.image_link_name = image_d.getVar('IMAGE_LINK_NAME') self.qb_slirp_opt = image_d.getVar('QB_SLIRP_OPT') or '' @property def debug_support(self): return bool(self.rootfs_dbg) + @property + def rootfs(self): + """Prefer the live NFS-exported rootfs (if --nfs=rootfs is used) over the + static WORKDIR/rootfs left over from the image build, so solib_search_path() + finds files as devtool deploy-target actually updates them.""" + if self.__nfs_rootfs: + return self.__nfs_rootfs + return self.__rootfs + @property def rootfs_dbg(self): + if self.__nfs_rootfs_dbg: + return self.__nfs_rootfs_dbg if self.__rootfs_dbg and os.path.isdir(self.__rootfs_dbg): return self.__rootfs_dbg return None - def update_image_bbappend(self, recipes_modified): + def set_nfs_rootfs(self, workspace_path, nfs): + """Select the workspace NFS rootfs for generated debugger paths and deploys.""" + if not nfs: + return + self.__nfs_deploy_dir = self.nfs_rootfs_dir(workspace_path, nfs) + if nfs == 'rootfs-dbg': + self.__nfs_rootfs_dbg = self.__nfs_deploy_dir + elif nfs == 'rootfs': + self.__nfs_rootfs = self.__nfs_deploy_dir + + @property + def nfs_deploy_dir(self): + """Directory to deploy modified-recipe files into instead of over ssh, if --nfs was used.""" + return self.__nfs_deploy_dir + + def nfs_rootfs_dir(self, workspace_path, nfs): + """Return the workspace directory for the selected NFS rootfs.""" + return os.path.join(workspace_path, 'nfs-exports', self.pn, nfs) + + def nfs_runqemu_helper(self, workspace_path, nfs): + """Create a helper that boots the selected rootfs through runqemu.""" + export_dir = os.path.join(workspace_path, 'nfs-exports', self.pn) + rootfs_dir = self.nfs_rootfs_dir(workspace_path, nfs) + qemuboot = os.path.join( + self.deploy_dir_image, self.image_link_name + '.qemuboot.conf') + if not os.path.exists(qemuboot): + logger.info( + 'No qemuboot configuration was generated for %s; ' + 'not creating a runqemu helper.', self.name) + return None + + helper = os.path.join(export_dir, 'runqemu-' + nfs) + with open(helper, 'w') as helper_file: + helper_file.write('#!/bin/sh\n') + helper_file.write( + 'exec runqemu %s %s "$@"\n' % ( + shlex.quote(qemuboot), shlex.quote(rootfs_dir))) + os.chmod(helper, os.stat(helper).st_mode | stat.S_IEXEC) + return helper + + def update_image_bbappend(self, recipes_modified, nfs=None): """Write debug settings for modified-mode recipes into the image bbappend. Writes IMAGE_GEN_DEBUGFS, IMAGE_FSTYPES_DEBUGFS, IMAGE_CLASSES for @@ -323,7 +387,17 @@ class RecipeImage: lines = [] if not self.base_image_gen_debugfs: lines.append('IMAGE_GEN_DEBUGFS = "1"') - if self.base_image_fstypes_debugfs != '': + if nfs == 'rootfs': + if 'tar' not in self.base_image_fstypes: + lines.append('IMAGE_FSTYPES:append = " tar"') + elif nfs == 'rootfs-dbg': + if self.base_image_fstypes_debugfs: + if 'tar' not in self.base_image_fstypes_debugfs.split(): + lines.append('IMAGE_FSTYPES_DEBUGFS:append = " tar"') + else: + lines.append('IMAGE_FSTYPES_DEBUGFS = "tar"') + elif self.base_image_fstypes_debugfs != '': + # Without --nfs no debug filesystem image is needed at all. lines.append('IMAGE_FSTYPES_DEBUGFS = ""') if not self.base_has_combined_dbg: lines.append('IMAGE_CLASSES += "image-combined-dbg"') @@ -368,6 +442,33 @@ class RecipeImage: slirp_changed = self.update_qb_slirp_opt() return image_changed or slirp_changed + def extract_nfs_rootfs(self, workspace_path, nfs, target): + """Refresh the selected rootfs under the devtool workspace.""" + suffix = '-dbg' if nfs == 'rootfs-dbg' else '' + rootfs_tarball = os.path.join( + self.deploy_dir_image, self.image_link_name + suffix + '.tar') + rootfs_dir = self.nfs_rootfs_dir(workspace_path, nfs) + state_dir = pseudo_state_dir(rootfs_dir) + + for stale_dir in (rootfs_dir, state_dir): + if os.path.exists(stale_dir): + shutil.rmtree(stale_dir) + + try: + extract_sdk_rootfs(rootfs_tarball, rootfs_dir) + except RunQemuRootfsError as exc: + raise DevtoolError('Unable to prepare NFS rootfs: %s' % exc) from exc + + logger.info('NFS rootfs extracted to %s', rootfs_dir) + helper = self.nfs_runqemu_helper(workspace_path, nfs) + if helper: + opts = 'slirp' if is_loopback_target(target) else '' + logger.info( + 'With the build environment sourced, start QEMU with NFS rootfs:\n' + ' %s %s\n' + 'Pass any additional runqemu options to this helper.', + helper, opts) + def update_qb_slirp_opt(self): """Update QB_SLIRP_OPT in the image bbappend @@ -655,6 +756,11 @@ class RecipeModified: self.reverse_debug_prefix_map = {} self.fakerootcmd = None self.fakerootenv = None + # STAGING_DIR_NATIVE of qemu-helper-native, needed by deploy_no_d() to + # locate the pseudo binary for a local (e.g. NFS-exported) deploy + # target without depending on oe-find-native-sysroot / a sourced + # build environment at script run time. + self.qemu_helper_native_sysroot = None self.libdir = None self.max_process = None self.package_arch = None @@ -742,6 +848,12 @@ class RecipeModified: self.debug_build = recipe_d.getVar('DEBUG_BUILD') self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD') self.fakerootenv = recipe_d.getVar('FAKEROOTENV') + qemu_helper_native_d = parse_recipe( + config, tinfoil, 'qemu-helper-native', appends=True, filter_workspace=False) + if not qemu_helper_native_d: + raise DevtoolError("Parsing qemu-helper-native recipe failed") + self.qemu_helper_native_sysroot = os.path.realpath( + qemu_helper_native_d.getVar('STAGING_DIR_NATIVE')) self.libdir = recipe_d.getVar('libdir') self.max_process = int(recipe_d.getVar( "BB_NUMBER_THREADS") or os.cpu_count() or 1) @@ -1342,12 +1454,17 @@ class RecipeModified: 'by the %s recipe (PACKAGES: %s)' % (package, self.pn, ' '.join(self.packages_files.keys()))) - def gen_deploy_target_script(self, args): + def gen_deploy_target_script(self, args, deploy_target=None): """Generate a script which does what devtool deploy-target does This script is much quicker than devtool target-deploy. Because it does not need to start a bitbake server. All information from tinfoil is hard-coded in the generated script. + + deploy_target overrides args.target as the baked-in default, e.g. with + the local NFS rootfs directory when --nfs was used (see devtool.deploy + for how a directory target is handled without ssh). A runtime -t/--target + can still override this default, same as without --nfs. """ self._validate_requested_packages(args) cmd_lines = ['#!%s' % str(sys.executable)] @@ -1362,6 +1479,8 @@ class RecipeModified: 'no_preserve', 'port', 'show_status', 'ssh_exec', 'strip', 'target'] filtered_args_dict = {key: value for key, value in vars( args).items() if key in args_filter} + if deploy_target: + filtered_args_dict['target'] = deploy_target if is_loopback_target(filtered_args_dict['target']): filtered_args_dict['no_host_check'] = True cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict)) @@ -1394,10 +1513,11 @@ class RecipeModified: cmd_lines.append(' filtered_args.no_host_check = True') cmd_lines.append( 'setattr(filtered_args, "recipename", "%s")' % self.bpn) - cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args, file_globs=file_globs, packages_files=packages_files)' % + cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args, file_globs=file_globs, packages_files=packages_files, qemu_helper_native_sysroot="%s")' % (self.d, self.workdir, self.path, self.strip_cmd, self.libdir, self.base_libdir, self.max_process, - self.fakerootcmd, self.fakerootenv)) + self.fakerootcmd, self.fakerootenv, + self.qemu_helper_native_sysroot)) return self.write_script(cmd_lines, 'deploy_target') def gen_install_task_script(self): @@ -1421,11 +1541,11 @@ class RecipeModified: ' tinfoil.shutdown()'] return self.write_script(cmd_lines, 'bb_run_do_install') - def gen_install_deploy_script(self, args): + def gen_install_deploy_script(self, args, deploy_target=None): """Generate a script which does install and deploy""" cmd_lines = ['#!/bin/sh -e'] cmd_lines.append(self.gen_install_task_script()) - cmd_lines.append(self.gen_deploy_target_script(args) + ' "$@"') + cmd_lines.append(self.gen_deploy_target_script(args, deploy_target) + ' "$@"') return self.write_script(cmd_lines, 'install_and_deploy') @@ -1528,6 +1648,9 @@ def ide_setup(args, config, basepath, workspace): logger.error("In shared sysroots mode modified recipes %s cannot be handled." % str( recipes_modified_names)) invalid_params = True + if args.nfs: + logger.error("--nfs is only supported in modified mode.") + invalid_params = True if args.mode == DevtoolIdeMode.modified: if not recipes_modified_names: appends_dir = os.path.join(config.workspace_path, 'appends') @@ -1570,6 +1693,7 @@ def ide_setup(args, config, basepath, workspace): recipes_image_name, orig_bbappend_contents.get(recipes_image_name)) recipe_image.initialize(config, tinfoil) + recipe_image.set_nfs_rootfs(config.workspace_path, args.nfs) if args.mode == DevtoolIdeMode.modified: # Keep the image build separate so that the complete bbappend # (IMAGE_ vars + QB_SLIRP_OPT) can be written in one step @@ -1675,7 +1799,7 @@ def ide_setup(args, config, basepath, workspace): # removed by strip_bbappend_sections() would be lost. bbappend_changed = False for ri in recipes_images: - if ri.update_image_bbappend(recipes_modified): + if ri.update_image_bbappend(recipes_modified, args.nfs): bbappend_changed = True if not args.skip_bitbake: @@ -1701,6 +1825,10 @@ def ide_setup(args, config, basepath, workspace): exec_build_env_command( config.init_path, basepath, bb_cmd + ' '.join(image_bootstrap_tasks), watch=True) + + if args.nfs and not args.skip_bitbake: + for ri in recipes_images: + ri.extract_nfs_rootfs(config.workspace_path, args.nfs, args.target) else: raise DevtoolError("Must not end up here.") @@ -1775,6 +1903,11 @@ def register_commands(subparsers, context): '-P', '--port', help='Specify ssh port to use for connection to the target') parser_ide_sdk.add_argument( '-I', '--key', help='Specify ssh private key for connection to the target') + parser_ide_sdk.add_argument( + '--nfs', choices=('rootfs', 'rootfs-dbg'), + help='Build and extract the selected image rootfs below ' + '/nfs-exports// ' + 'for NFS booting.') parser_ide_sdk.add_argument( '--skip-bitbake', help='Skip the bitbake builds which update the SDK. The recipes are still parsed, ' 'the IDE configuration is generated from their metadata', action='store_true') From patchwork Sun Aug 30 14:28:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 96818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFCE5C624A5 for ; Sun, 30 Aug 2026 14:29:54 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.9438.1788100184573511779 for ; Sun, 30 Aug 2026 07:29:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm2 header.b=f8ZUErSI; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-1329275-20260830142942cbb72ab62f0002070a-niuyju@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20260830142942cbb72ab62f0002070a for ; Sun, 30 Aug 2026 16:29:42 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm2; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=7g3PlO/+cF3lKflvvDNgM7bMu8xwLpQ1yvZAdh5u1Ts=; b=f8ZUErSIvmcA9Fsee3C4JZ97fYmesZ/E/Q5CPjpioyhc1+KoR/d94cBXj/LTOm6eXdgeRF 7qi439TJyJM2wyZ4gMtTIm3pjCzFMOlR5QzmCd9+ipQVap4MiSTjYbCmJKhB4mYA00vIPcvk gATfXxAImf/e1o7y2Qi2BO95hLFy7bSL54xMX/6Madw5QAz0q49Nt9oXM3/tI7oEh91HVprB KuKbr2rybNz0L76Nj73/gI50mDrFwx2XgTwVQdCpdOs91vNUuWSV/Jeoye9PgSQkHf/UQ1w5 8J4aSmXgLFoujyi0p2IzY3bhx40yhuQQMpmsnr1ZelD9YihRwtrm30Lg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 24/24] oe-selftest: devtool ide-sdk: test NFS debug rootfs Date: Sun, 30 Aug 2026 16:28:51 +0200 Message-ID: <20260830142922.17241-25-adrian.freihofer@siemens.com> In-Reply-To: <20260830142922.17241-1-adrian.freihofer@siemens.com> References: <20260830142922.17241-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 30 Aug 2026 14:29:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/244670 From: Adrian Freihofer Exercise --nfs=rootfs-dbg with a real image build and verify the extracted rootfs, pseudo state, and generated VS Code debug paths. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 118 ++++++++++++++++++++++++ 1 file changed, 118 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index c05cf7bc82..8d68809552 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3539,8 +3539,18 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): r = runCmd(gdbserver_script, output_log=self._cmd_logger) self.assertEqual(r.status, 0) + # The start script already waited for gdbserver's port to come up + # before returning, so the pid file it wrote must exist by now. Still + # retry the read a few times: on an NFS-root target (--nfs=rootfs-dbg) + # this check runs over a fresh SSH connection right after the one that + # wrote the file, and a brief visibility lag has been observed there. pid_file = '/tmp/gdbserver_1234_usr-bin-%s_multi/gdbserver.pid' % example_exe status, output = qemu.run('cat %s' % pid_file) + for _ in range(10): + if status == 0: + break + time.sleep(1) + status, output = qemu.run('cat %s' % pid_file) self.assertEqual(status, 0) gdbserver_pid = output.strip() self.assertRegex(gdbserver_pid, r'^\d+$') @@ -3974,6 +3984,114 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self.assertEqual(r.status, 0) debug_check_func(r.output, DevtoolIdeSdkTests.MAGIC_STRING_ORIG) + def test_devtool_ide_sdk_code_nfs_debug_rootfs(self): + """Verify ide-sdk extracts an NFS debug rootfs for VS Code debugging.""" + recipe_name = "cmake-example" + build_file = "CMakeLists.txt" + testimage = "oe-selftest-image" + + self._check_workspace() + self._write_bb_config() + tempdir = self._devtool_ide_sdk_recipe( + recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=code --nfs=rootfs-dbg' % + (recipe_name, testimage), output_log=self._cmd_logger) + + nfs_rootfs = os.path.join( + self.workspacedir, 'nfs-exports', testimage, 'rootfs-dbg') + self.assertExists(nfs_rootfs) + self.assertExists(nfs_rootfs + '.pseudo_state') + self.assertExists(os.path.join(nfs_rootfs, 'usr', 'bin', recipe_name)) + runqemu_helper = os.path.join( + self.workspacedir, 'nfs-exports', testimage, + 'runqemu-rootfs-dbg') + self.assertExists(runqemu_helper) + self.assertTrue(os.access(runqemu_helper, os.X_OK)) + with open(runqemu_helper) as helper_file: + helper = helper_file.read() + self.assertIn('exec runqemu ', helper) + self.assertIn(nfs_rootfs, helper) + + bbappend = os.path.join( + self.workspacedir, 'appends', testimage + '.bbappend') + with open(bbappend) as append_file: + self.assertRegex( + append_file.read(), + r'IMAGE_FSTYPES_DEBUGFS(?::append)? = " ?tar"') + + with open(os.path.join(tempdir, '.vscode', 'launch.json')) as launch_file: + launch_configurations = json.load(launch_file)['configurations'] + for configuration in launch_configurations: + self.assertIn( + nfs_rootfs, configuration['additionalSOLibSearchPath']) + self.assertEqual( + os.path.join(nfs_rootfs, 'usr', 'src', 'debug'), + configuration['sourceFileMap']['/usr/src/debug']) + + def _test_devtool_ide_sdk_nfs_debug_rootfs_qemu(self, slirp=False): + """Boot the extracted NFS debug rootfs and exercise remote GDB.""" + recipe_name = "cmake-example" + build_file = "CMakeLists.txt" + testimage = "oe-selftest-image" + + self._check_workspace() + self._write_bb_config() + if not slirp: + self._check_runqemu_prerequisites() + + tempdir = self._devtool_ide_sdk_recipe( + recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=none --nfs=rootfs-dbg' % + (recipe_name, testimage), + output_log=self._cmd_logger) + + nfs_rootfs = os.path.join( + self.workspacedir, 'nfs-exports', testimage, 'rootfs-dbg') + runqemu_helper = os.path.join( + self.workspacedir, 'nfs-exports', testimage, + 'runqemu-rootfs-dbg') + self.assertExists(nfs_rootfs) + self.assertExists(runqemu_helper) + + launch_cmd = '%s nographic' % shlex.quote(runqemu_helper) + runqemuparams = '' + if slirp: + launch_cmd += ' slirp' + # QemuTarget uses this only to select its localhost SSH endpoint; + # launch_cmd supplies the actual runqemu option. + runqemuparams = 'slirp' + + with runqemu(testimage, runqemuparams=runqemuparams, + launch_cmd=launch_cmd) as qemu: + status, output = qemu.run( + "awk '$2 == \"/\" {print $3}' /proc/mounts") + self.assertEqual(status, 0) + self.assertEqual(output.strip(), 'nfs') + + # Re-run with the real QEMU target address, instead of 192.168.7.2 IP + if slirp: + target_options = '-t root@%s -P %s ' % (qemu.ip, qemu.port or '2222') + else: + target_options = '-t root@%s -c ' % qemu.ip + runCmd('devtool ide-sdk %s %s %s--skip-bitbake --ide=none --nfs=rootfs-dbg' % + (recipe_name, testimage, target_options), + output_log=self._cmd_logger) + + self._gdb_cross() + compile_cmd = self._verify_cmake_preset(tempdir) + self._devtool_ide_sdk_qemu( + tempdir, qemu, recipe_name, recipe_name, compile_cmd) + + @OETestTag("runqemu") + def test_devtool_ide_sdk_none_nfs_qemu(self): + """Verify remote GDB debugging through an NFS-root QEMU target.""" + self._test_devtool_ide_sdk_nfs_debug_rootfs_qemu() + + @OETestTag("runqemu") + def test_devtool_ide_sdk_none_nfs_qemu_slirp(self): + """Verify remote GDB debugging through an NFS-root slirp target.""" + self._test_devtool_ide_sdk_nfs_debug_rootfs_qemu(slirp=True) + @OETestTag("runqemu") def test_devtool_ide_sdk_code_cmake(self): """Verify a cmake recipe works with ide=code mode"""