From patchwork Sun Oct 11 19:59:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A70CA9ED9 for ; Sun, 11 Oct 2026 20:00:00 +0000 (UTC) Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net [185.136.64.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.32876.1791748795226786089 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=BiHk1eFx; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.228, mailfrom: fm-1329275-202610111959527ace5e544f0002075b-jqigld@rts-flowmailer.siemens.com) Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 202610111959527ace5e544f0002075b for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=plPsOewJsL9wXgwr5lspsRAjmxjotxSyHk0tMj/Z0Lg=; b=BiHk1eFxtvCgYlIAF2T1m/QDxj+rqu4zGvSWo3YOx1TOtqlZH1l56hxhqNiEB+o7AJtmwv TkemYQ/FRPl6CsijRtMeftM88SkB6RDH14BTDAQ/BtxY1znRnLeMAfrAAHyk3ZrG+QNZOfjv 0Rztr2dojvJjblQJJwjUWCRiFobu1DAf/qxwTwQaGmiib1BUQr//WJOREa6vPcArEwEnccbD 8AoAdVodO0G+DafeuIyYZYWFLtIopIFuB9zXeiHUiCgEzQ1hNNdP1D2KEwgZO6NCs1MSb49I KS5srdXgDQhwCZL6skW/Yl/YjjmnnhSXRkFw5sltHqpWyO1Xoxj9k9JQ==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 01/11] runqemu: move all imports to the top of the file Date: Sun, 11 Oct 2026 21:59:06 +0200 Message-ID: <20261011195949.2118607-2-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 20:00:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247587 From: Adrian Freihofer Local imports in functions are inconsistent with the rest of the module. All of them are stdlib modules, so there is no reason to defer loading them. Signed-off-by: Adrian Freihofer --- scripts/runqemu | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/scripts/runqemu b/scripts/runqemu index 2c126e7f54..dee7710fb3 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -19,6 +19,10 @@ import glob import configparser import signal import time +import shlex +import socket +import traceback +from contextlib import closing class RunQemuError(Exception): """Custom exception to raise on known errors.""" @@ -258,9 +262,6 @@ class BaseConfig(object): def check_free_port(self, host, port, lockdir): """ Check whether the port is free or not """ - import socket - from contextlib import closing - lockfile = os.path.join(lockdir, str(port) + '.lock') if self.acquire_portlock(lockfile): with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock: @@ -1605,7 +1606,6 @@ to your build configuration. self.setup_vga() def start_qemu(self): - import shlex if self.kernel: kernel_opts = "-kernel %s" % (self.kernel) if self.get('QB_KERNEL_CMDLINE') == "none": @@ -1805,7 +1805,6 @@ def main(): logger.error(err) return 1 except Exception as err: - import traceback traceback.print_exc() return 1 finally: From patchwork Sun Oct 11 19:59:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CB5ECA9ED7 for ; Sun, 11 Oct 2026 20:00:00 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.32872.1791748795180893257 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=lFkmG5cp; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-1329275-20261011195952895737381400020742-vtdmfx@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20261011195952895737381400020742 for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=qxc+n+/RGDu0jxEpQHCY7yw8hBE6dQy+wS6EzkavbS8=; b=lFkmG5cpJHVQg4kGE4HcrHlJf68jxrmLOnjr3opuDMlh1DVFYaFQJyFgmNH2mqR6LL0jno bZdt4DfnuHsc3L+ByP1V00SnASQhuRZBYhcCt1HMLhYMl8WUafCWyYgVkq9iUMZennte5oH8 rdBkz7p2Y2hqdrOY/ijg7Dpuaoka9LmF6+E+/ssVZfa9ae2Y+5KEbBdgVRTm1Yntwic/5zHc Jgu1sKi/s65HhESdkxv/mKinyyAinsOJ0DJgqSArd8erRUEkFrBtZzhN43TIhh6cHWvM9Gg3 5H4isIBjrPB2id4rbtlwM7/6TXQzAH+jWzssJandmFIRviiruJEKu5ww==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 02/11] runqemu: don't drop a held lock when acquire_portlock() fails Date: Sun, 11 Oct 2026 21:59:07 +0200 Message-ID: <20261011195949.2118607-3-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 20:00:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247590 From: Adrian Freihofer acquire_portlock() stored the new file object in self.portlocks before calling flock(). If the process already held that lock, the entry was replaced, which closed the old file object and with it released the lock. The flock() on the new file then succeeded, so a lock held by this very process was reported as free. Only record the descriptor once flock() succeeded and close just the new one on failure. The current callers never ask for the same lock file twice, so this is latent, but it makes acquire_portlock() reliable for callers that do. Signed-off-by: Adrian Freihofer --- scripts/runqemu | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/scripts/runqemu b/scripts/runqemu index dee7710fb3..5f1af740c9 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -277,17 +277,17 @@ class BaseConfig(object): def acquire_portlock(self, lockfile): logger.debug("Acquiring lockfile %s..." % lockfile) + portlock_descriptor = None try: portlock_descriptor = open(lockfile, 'w') - self.portlocks.update({lockfile: portlock_descriptor}) - fcntl.flock(self.portlocks[lockfile], fcntl.LOCK_EX|fcntl.LOCK_NB) + fcntl.flock(portlock_descriptor, fcntl.LOCK_EX|fcntl.LOCK_NB) except Exception as e: msg = "Acquiring lockfile %s failed: %s" % (lockfile, e) logger.info(msg) - if lockfile in self.portlocks.keys() and self.portlocks[lockfile]: - self.portlocks[lockfile].close() - del self.portlocks[lockfile] + if portlock_descriptor: + portlock_descriptor.close() return False + self.portlocks[lockfile] = portlock_descriptor return True def release_portlock(self, lockfile=None): From patchwork Sun Oct 11 19:59:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B836CA9ED8 for ; Sun, 11 Oct 2026 20:00:00 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.32779.1791748795179815389 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=UW4uwx8M; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-1329275-20261011195952a7b10dad7200020728-g2w1jd@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20261011195952a7b10dad7200020728 for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=74L/jaV7VvqffBY4Zn4nO45hJB3XP1YCW+Cgii61EHo=; b=UW4uwx8Ml+CS7j5fWy2iEXbuIDuVMU70M8eCCrUACEeb23PythX8tSUwDHkLJlZidMyUVC H2oUNdnYYnblW7ryZL6b+H1Fkmbj09Pa79udzLxXuFZHRxHyEG5xEwoLi4H0l45IOfv2OCcA xtI0fXiKj/ycwRelEAyDnq3okgb0UYskl/i/i5irRWdjrnMlrt6SBcOThM15H4/bWcKXbHGG foYo2+5eqHvDT8i3dX5jlpWCbv7OatqXjj0yzZE/reGgcl3kAT6zXD7Sn/Gg93plip4XkFzY JYeKbUICZAz7pbrKHfTvQrOiTlAtalVbbT/uaH9DIaYuEWVqFF++7aHQ==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 03/11] runqemu: rename the port lock helpers to generic lock helpers Date: Sun, 11 Oct 2026 21:59:08 +0200 Message-ID: <20261011195949.2118607-4-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 20:00:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247582 From: Adrian Freihofer acquire_portlock()/release_portlock()/self.portlocks implement a plain "hold a flock on a numbered lockfile, pass the fd to qemu, close it on cleanup" scheme that is not specific to ports. Drop "port" from the names so the same helpers can be used to allocate other per-instance resources without the names becoming misleading. No functional change. Signed-off-by: Adrian Freihofer --- scripts/runqemu | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/scripts/runqemu b/scripts/runqemu index 5f1af740c9..6544c3884b 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -198,7 +198,7 @@ class BaseConfig(object): self.tcpserial_portnum = '' self.taplock = '' self.taplock_descriptor = None - self.portlocks = {} + self.locks = {} self.bitbake_e = '' self.snapshot = False self.wictypes = ('wic.zst', 'wic', 'wic.vmdk', 'wic.qcow2', 'wic.vdi', "wic.vhd", "wic.vhdx") @@ -263,11 +263,11 @@ class BaseConfig(object): def check_free_port(self, host, port, lockdir): """ Check whether the port is free or not """ lockfile = os.path.join(lockdir, str(port) + '.lock') - if self.acquire_portlock(lockfile): + if self.acquire_lock(lockfile): with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock: if sock.connect_ex((host, port)) == 0: # Port is open, so not free - self.release_portlock(lockfile) + self.release_lock(lockfile) return False else: # Port is not open, so free @@ -275,33 +275,33 @@ class BaseConfig(object): else: return False - def acquire_portlock(self, lockfile): + def acquire_lock(self, lockfile): logger.debug("Acquiring lockfile %s..." % lockfile) - portlock_descriptor = None + lock_descriptor = None try: - portlock_descriptor = open(lockfile, 'w') - fcntl.flock(portlock_descriptor, fcntl.LOCK_EX|fcntl.LOCK_NB) + lock_descriptor = open(lockfile, 'w') + fcntl.flock(lock_descriptor, fcntl.LOCK_EX|fcntl.LOCK_NB) except Exception as e: msg = "Acquiring lockfile %s failed: %s" % (lockfile, e) logger.info(msg) - if portlock_descriptor: - portlock_descriptor.close() + if lock_descriptor: + lock_descriptor.close() return False - self.portlocks[lockfile] = portlock_descriptor + self.locks[lockfile] = lock_descriptor return True - def release_portlock(self, lockfile=None): + def release_lock(self, lockfile=None): if lockfile != None: logger.debug("Releasing lockfile '%s'" % lockfile) # We pass the fd to the qemu process and if we unlock here, it would unlock for # that too. Therefore don't unlock, just close - # fcntl.flock(self.portlocks[lockfile], fcntl.LOCK_UN) - self.portlocks[lockfile].close() + # fcntl.flock(self.locks[lockfile], fcntl.LOCK_UN) + self.locks[lockfile].close() # Removing the file is a potential race, don't do that either # os.remove(lockfile) - del self.portlocks[lockfile] - elif len(self.portlocks): - for lockfile, descriptor in self.portlocks.items(): + del self.locks[lockfile] + elif len(self.locks): + for lockfile, descriptor in self.locks.items(): logger.debug("Releasing lockfile '%s'" % lockfile) # We pass the fd to the qemu process and if we unlock here, it would unlock for # that too. Therefore don't unlock, just close @@ -309,7 +309,7 @@ class BaseConfig(object): descriptor.close() # Removing the file is a potential race, don't do that either # os.remove(lockfile) - self.portlocks = {} + self.locks = {} def get(self, key): if key in self.d: @@ -1632,8 +1632,8 @@ to your build configuration. pass_fds = [] if self.taplock_descriptor: pass_fds = [self.taplock_descriptor.fileno()] - if len(self.portlocks): - for descriptor in self.portlocks.values(): + if len(self.locks): + for descriptor in self.locks.values(): pass_fds.append(descriptor.fileno()) process = subprocess.Popen(cmds, stderr=subprocess.PIPE, pass_fds=pass_fds, env=self.qemu_environ) self.qemuprocess = process @@ -1682,7 +1682,7 @@ to your build configuration. cmd = ("runqemu-export-rootfs", "stop", self.rootfs) logger.debug('Running %s' % str(cmd)) subprocess.check_call(cmd) - self.release_portlock() + self.release_lock() if self.saved_stty: subprocess.check_call(("stty", self.saved_stty)) From patchwork Sun Oct 11 19:59:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11B74CA9ED6 for ; Sun, 11 Oct 2026 20:00:00 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.32778.1791748795179627637 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=tYIrmX9U; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-2026101119595289ae00d61d000207a1-dabd3n@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 2026101119595289ae00d61d000207a1 for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=rRKrzjFMUmV9KWePOqOPAAEvsmvZ1cXOzUyNlUOllRg=; b=tYIrmX9Urc9nSXV4CSS1+MeRdWyZ0BmTfVLwO+317wcq9zQIO9hkJpwOjsZZl/Wzjl/hzE 0ls8ojdTGMZVAWAH+aLu3Y9G7f/BzwYNXavPmTaFags79SGNx700VpWG1EGrmc9uknsVolAE PHiPfNXz/w6JR5D1lWYDzXCv1KMuul1xGjpj+lGeGkpYGD2F4A7Dm74EC5iCmR5rJ/5LqUVY TDRTm7OoElhVr6BJPJz/kDWSIBLVeoxI/yBsBBjOKxYb+bamyGHs0rOErINGhGUFZF9JKl/l OjzdKBpCAUVi3M8T2j+ChjPAPku9D3X/9zMR4YQxo5IAzPTsaLdJqNIA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 04/11] runqemu: allocate the slirp MAC address via lock files Date: Sun, 11 Oct 2026 21:59:09 +0200 Message-ID: <20261011195949.2118607-5-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 20:00:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247581 From: Adrian Freihofer The last octet of the slirp MAC address was derived from how many times a host port had to be incremented to find a free one. That is unreliable in both directions: two concurrently starting instances that shift their ports by the same amount end up with the same MAC address, while an instance that finds all its wished-for ports free always picks the same one. It also does not scale with the number of forwarded ports, because every single port that has to be moved bumps the octet once more. A configuration asking for many ports, as devtool ide-sdk does for its debug servers, therefore runs through the single byte quickly, and past 0xff the formatted address stops being a valid MAC address at all, e.g. "52:54:00:12:35:124" once the counter passes 255. Allocate the octet from its own lock file directory instead, reusing the scheme already used for the tap and port locks, which makes it unique among the qemu instances running on the host and independent of how many ports are forwarded and whether any of them had to be moved. Signed-off-by: Adrian Freihofer --- scripts/runqemu | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/scripts/runqemu b/scripts/runqemu index 6544c3884b..ca1f74e9d7 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -1133,6 +1133,17 @@ to your build configuration. pass return + def alloc_mac_slot(self, lockdir): + """MAC address arbitration for slirp addresses: lock the first free + numbered lockfile (1-254) to get an octet unique among concurrently + running qemu instances on this host.""" + self.make_lock_dir(lockdir) + for slot in range(1, 255): + lockfile = os.path.join(lockdir, str(slot) + '.lock') + if self.acquire_lock(lockfile): + return slot + raise RunQemuError("Too many concurrent qemu slirp instances on this host, can't allocate a unique MAC address") + def setup_slirp(self): """Setup user networking""" @@ -1148,7 +1159,6 @@ to your build configuration. # Figure out the port ports = re.findall('hostfwd=[^-]*:([0-9]+)-[^,-]*', qb_slirp_opt) ports = [int(i) for i in ports] - mac = 2 lockdir = "/tmp/qemu-port-locks" self.make_lock_dir(lockdir) @@ -1158,15 +1168,13 @@ to your build configuration. p_new = p while not self.check_free_port('localhost', p_new, lockdir): p_new += 1 - mac += 1 while p_new in ports: p_new += 1 - mac += 1 if p != p_new: ports.append(p_new) qb_slirp_opt = re.sub(':%s-' % p, ':%s-' % p_new, qb_slirp_opt) logger.info("Port forward changed: %s -> %s" % (p, p_new)) - mac = "%s%02x" % (self.mac_slirp, mac) + mac = "%s%02x" % (self.mac_slirp, self.alloc_mac_slot("/tmp/qemu-mac-locks")) self.set('NETWORK_CMD', '%s %s' % (self.network_device.replace('@MAC@', mac), qb_slirp_opt)) # Print out port foward hostfwd = re.findall('(hostfwd=[^,]*)', qb_slirp_opt) From patchwork Sun Oct 11 19:59:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100366 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFCDCCA9ED1 for ; Sun, 11 Oct 2026 19:59:58 +0000 (UTC) Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net [185.136.64.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.32877.1791748795274036701 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=R9KxDjF0; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.227, mailfrom: fm-1329275-2026101119595228eba491cf000207c1-fkta_l@rts-flowmailer.siemens.com) Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 2026101119595228eba491cf000207c1 for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=7ZX13zzIFwP6HrfrLKc4BkLaOWqzaZC6wKBfFyasrxU=; b=R9KxDjF00vjLV8weL4fr9SHJEWxS0SBA47NPQw9J1SZwwSuqZB/Jzeqx/bYOK8bKksLoPi 6RsqZ/y7UAhAqluLF3BUNo9MJ49O53+ylwoMBc4pzAUjGVwBn1YdV4rvofMRf5jVV4Q/2eYm wCwHIXvrcqD+CWZ2/xM+KG1SsWjVMtOt0+4luAnG4pbp2p/BF+7OixMhkZ0P1+iT31Krz0Oj LbpNFbfBESsupbCvYjDCTJ7bWLwKiOmYBfzLJn3td1lSszDIAaCrzehCPnTZFSucTAOtkXc9 NMPhLsFOY/DKzQbuS7p64BhogtAjaKOTnE7sDDhdU4NWmyw5483LaQVQ==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 05/11] qemurunner: query the slirp port mapping from the running qemu Date: Sun, 11 Oct 2026 21:59:10 +0200 Message-ID: <20261011195949.2118607-6-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 19:59:58 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247589 From: Adrian Freihofer This series changes how runqemu allocates slirp host ports: instead of taking a flock per port and probing it before QEMU starts, QEMU binds each port itself and runqemu retries with the next port if the bind fails. This works for any number of forwarded ports, such as the debug server ports added by devtool ide-sdk or any other forwarded ports added to QB_SLIRP_OPT. The effective host ports are then only known to the running QEMU. As a first step, make QemuRunner read the forwards from QEMU instead of scraping its command line, which only resolved SSH and breaks once the forwards are no longer passed there. Query info usernet through QMP and expose mappings keyed by protocol and guest port. Preserve the SSH bind address, provide TCP-default lookup and TCP/UDP allocation through QemuTarget, and require an explicit successful QMP reply before reporting a forward. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/targetcontrol.py | 10 +++ meta/lib/oeqa/utils/qemurunner.py | 143 ++++++++++++++++++++++++------ 2 files changed, 124 insertions(+), 29 deletions(-) diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py index 1deea2e5ce..600ea3efcf 100644 --- a/meta/lib/oeqa/targetcontrol.py +++ b/meta/lib/oeqa/targetcontrol.py @@ -224,6 +224,16 @@ class QemuTarget(BaseTarget): def run_serial(self, command, timeout=60): return self.runner.run_serial(command, timeout=timeout) + def get_host_port(self, target_port, protocol='tcp'): + return self.runner.get_host_port(target_port, protocol=protocol) + + @property + def slirp_port_mappings(self): + return self.runner.slirp_port_mappings + + def add_hostfwd(self, target_port, host_port=None, protocol='tcp'): + return self.runner.add_hostfwd(target_port, host_port, protocol=protocol) + class SimpleRemoteTarget(BaseTarget): diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py index 5979b52650..5d7374446e 100644 --- a/meta/lib/oeqa/utils/qemurunner.py +++ b/meta/lib/oeqa/utils/qemurunner.py @@ -400,38 +400,44 @@ class QemuRunner: self.qemupid, time.strftime("%D %H:%M:%S"))) cmdline = '' if get_ip: - with open('/proc/%s/cmdline' % self.qemupid) as p: - cmdline = p.read() - # It is needed to sanitize the data received - # because is possible to have control characters - cmdline = re_control_char.sub(' ', cmdline) - try: - if self.use_slirp: - tcp_ports = cmdline.split("hostfwd=tcp:")[1] - ip, tcp_ports = tcp_ports.split(":")[:2] - host_port = tcp_ports[:tcp_ports.find('-')] - self.ip = "%s:%s" % (ip, host_port) - else: + if self.use_slirp: + # The running qemu is the authoritative source for the + # effective hostfwd rules: the host port a rule ends up on is + # not necessarily the one that was wished for. + try: + host, port = self._get_slirp_host_endpoint(22) + self.ip = "%s:%s" % (host, port) + except RuntimeError as err: + self.logger.error("Couldn't resolve the ssh host port for slirp qemu: %s" % err) + self.stop() + return False + else: + with open('/proc/%s/cmdline' % self.qemupid) as p: + cmdline = p.read() + # It is needed to sanitize the data received + # because is possible to have control characters + cmdline = re_control_char.sub(' ', cmdline) + try: ips = re.findall(r"((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) self.ip = ips[0] self.server_ip = ips[1] - self.logger.debug("qemu cmdline used:\n{}".format(cmdline)) - except (IndexError, ValueError): - # Try to get network configuration from runqemu output - match = re.match(r'.*Network configuration: (?:ip=)*([0-9.]+)::([0-9.]+):([0-9.]+).*', - out, re.MULTILINE | re.DOTALL) - if match: - self.ip, self.server_ip, self.netmask = match.groups() - # network configuration is required as we couldn't get it - # from the runqemu command line, so qemu doesn't run kernel - # and guest networking is not configured - netconf = True - else: - self.logger.error("Couldn't get ip from qemu command line and runqemu output! " - "Here is the qemu command line used:\n%s\n" - "and output from runqemu:\n%s" % (cmdline, out)) - self.stop() - return False + self.logger.debug("qemu cmdline used:\n{}".format(cmdline)) + except (IndexError, ValueError): + # Try to get network configuration from runqemu output + match = re.match(r'.*Network configuration: (?:ip=)*([0-9.]+)::([0-9.]+):([0-9.]+).*', + out, re.MULTILINE | re.DOTALL) + if match: + self.ip, self.server_ip, self.netmask = match.groups() + # network configuration is required as we couldn't get it + # from the runqemu command line, so qemu doesn't run kernel + # and guest networking is not configured + netconf = True + else: + self.logger.error("Couldn't get ip from qemu command line and runqemu output! " + "Here is the qemu command line used:\n%s\n" + "and output from runqemu:\n%s" % (cmdline, out)) + self.stop() + return False self.logger.debug("Target IP: %s" % self.ip) self.logger.debug("Server IP: %s" % self.server_ip) @@ -649,6 +655,85 @@ class QemuRunner: return True return False + @property + def slirp_port_mappings(self): + """Slirp hostfwd rules as {(protocol, target_port): host_port} read from Qemu's monitor.""" + return {key: endpoint[1] + for key, endpoint in self._slirp_hostfwd_mappings.items()} + + @property + def _slirp_hostfwd_mappings(self): + """Return {(protocol, guest_port): (host_address, host_port)} from QEMU.""" + result = self.run_monitor('human-monitor-command', + {'command-line': 'info usernet'}) + output = (result or {}).get('return', '') if isinstance(result, dict) else '' + mappings = {} + # Protocol[State] FD Source Address Port Dest. Address Port RecvQ SendQ + # TCP[HOST_FORWARD] 10 127.0.0.1 2222 10.0.2.15 22 0 0 + for line in (output or '').splitlines(): + fields = line.split() + if len(fields) < 6: + continue + protocol, separator, state = fields[0].partition('[') + if not protocol or not separator or state != 'HOST_FORWARD]': + continue + try: + mappings[(protocol.lower(), int(fields[5]))] = (fields[2], int(fields[3])) + except ValueError: + continue + return mappings + + def _get_slirp_host_endpoint(self, target_port, protocol='tcp'): + protocol = protocol.lower() + endpoint = self._slirp_hostfwd_mappings.get((protocol, target_port)) + if endpoint is None: + return '127.0.0.1', self.add_hostfwd(target_port, protocol=protocol) + host, port = endpoint + if host in ('*', '0.0.0.0'): + host = '127.0.0.1' + return host, port + + def add_hostfwd(self, target_port, host_port=None, protocol='tcp'): + """Add a TCP or UDP slirp hostfwd rule and return the host port. + + Without host_port a free one is picked automatically. Returns None if + an explicitly requested host_port is already in use. + """ + protocol = protocol.lower() + socket_type = {'tcp': socket.SOCK_STREAM, 'udp': socket.SOCK_DGRAM}.get(protocol) + if socket_type is None: + raise ValueError("Unsupported slirp host forwarding protocol: %s" % protocol) + candidates = [host_port] if host_port else [] + if not host_port: + for _ in range(10): + with socket.socket(socket.AF_INET, socket_type) as s: + s.bind(('127.0.0.1', 0)) + candidates.append(s.getsockname()[1]) + for candidate in candidates: + result = self.run_monitor('human-monitor-command', { + 'command-line': 'hostfwd_add %s:127.0.0.1:%d-:%d' % (protocol, candidate, target_port)}) + output = result.get('return') if isinstance(result, dict) else None + # hostfwd_add prints nothing on success; on failure it prints a message like + # "Could not set up host forwarding rule ...". Check for empty output to determine success. + if isinstance(result, dict) and 'error' not in result and isinstance(output, str) and output.strip() == '': + self.logger.debug("Added %s hostfwd for target port %d on host port %d" + % (protocol, target_port, candidate)) + return candidate + self.logger.debug("hostfwd_add for %s host port %d failed: %s" % (protocol, candidate, result)) + if host_port: + return None + raise RuntimeError( + "Failed to add a slirp %s hostfwd rule for target port %d" % (protocol, target_port)) + + def get_host_port(self, target_port, protocol='tcp'): + """Resolve a slirp protocol and target port to the host port qemu + forwards it on, adding a rule for it if there is none yet.""" + protocol = protocol.lower() + host_port = self.slirp_port_mappings.get((protocol, target_port)) + if host_port is None: + host_port = self.add_hostfwd(target_port, protocol=protocol) + return host_port + def run_monitor(self, command, args=None, timeout=60): if hasattr(self, 'qmp') and self.qmp: self.qmp.settimeout(timeout) From patchwork Sun Oct 11 19:59:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100368 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA2A7CA9ED3 for ; Sun, 11 Oct 2026 19:59:59 +0000 (UTC) Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net [185.136.65.227]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.32874.1791748795181070821 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=OmsanzWf; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.227, mailfrom: fm-1329275-20261011195952d772e228b8000207d6-zdyvsn@rts-flowmailer.siemens.com) Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20261011195952d772e228b8000207d6 for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=urfVPRJpvto8SGYKkWZhmUQlSlQVwrVQbbZ2MwhoDU8=; b=OmsanzWfXzWkvi/qAf3EM/OF0ByVTJ9pEDrordSyBood721Gjsc8vUbMttLKh0ErTda1Vh 5UWS5bkJd6Z6LhjVQMnrrvx5cLzeGkMCTH0angzuYtYq6PiuiouVVvYCf0MsbhaGohs2g7RB 2Iod7C2IRcUrFuw7xb2tLSrabgzcC7ByCRLfJffbQauZHbQu99jHfEpEz82coclqCsP6UWn7 LZwmsUrILlY3JbVPeGorX8lRPm6OMX7ELFMYPQD8LEWusFhwaYVo484jpRqfI71gEFdZTMWQ ecfK8zNm1Mh6bOpCd4UfcxRuStoXp4r1dldoW1WljVu1OmHYghtXQMKw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 06/11] runqemu: add slirp hostfwd ports via QMP after qemu started Date: Sun, 11 Oct 2026 21:59:11 +0200 Message-ID: <20261011195949.2118607-7-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 19:59:59 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247591 From: Adrian Freihofer This is the second step of letting QEMU bind the slirp host ports and retrying on failure, instead of taking a flock per port. So far runqemu locked and probed each hostfwd port before starting QEMU. The lock only coordinates runqemu instances, so any other process can still bind the port before QEMU does, and QEMU then fails to start. Now runqemu strips the hostfwd rules from the QEMU command line and adds them through a private QMP monitor once QEMU runs. If QEMU cannot bind a host port, runqemu tries the next one. TCP and UDP are tracked separately, so both can use the same host port. QemuRunner already reads the effective forwards from QEMU and briefly waits for the SSH forward before adding its own. The QMP exchange is bounded and validated so a broken monitor cannot hang startup, and QEMU's stderr is reported if it exits early. Coordination with older runqemu instances via the legacy port locks is added separately later in this series, so it can be reverted once older branches have the backports or reach EOL. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/utils/qemurunner.py | 5 ++ scripts/runqemu | 127 +++++++++++++++++++++++++----- 2 files changed, 112 insertions(+), 20 deletions(-) diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py index 5d7374446e..81434dc117 100644 --- a/meta/lib/oeqa/utils/qemurunner.py +++ b/meta/lib/oeqa/utils/qemurunner.py @@ -404,6 +404,11 @@ class QemuRunner: # The running qemu is the authoritative source for the # effective hostfwd rules: the host port a rule ends up on is # not necessarily the one that was wished for. + # runqemu adds its forwards via QMP shortly after qemu starts. + endtime = time.time() + 10 + while (('tcp', 22) not in self.slirp_port_mappings + and self.is_alive() and time.time() < endtime): + time.sleep(0.1) try: host, port = self._get_slirp_host_endpoint(22) self.ip = "%s:%s" % (host, port) diff --git a/scripts/runqemu b/scripts/runqemu index ca1f74e9d7..c569fc7c84 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -19,6 +19,8 @@ import glob import configparser import signal import time +import json +import tempfile import shlex import socket import traceback @@ -221,6 +223,8 @@ class BaseConfig(object): # slirp qemus are running. self.mac_tap = "52:54:00:12:34:" self.mac_slirp = "52:54:00:12:35:" + self.slirp_hostfwd_allowlist = [] + self.internal_qmp_path = None # pid of the actual qemu process self.qemu_environ = os.environ.copy() self.qemuprocess = None @@ -1156,30 +1160,103 @@ to your build configuration. hostfwd = ",hostfwd=tcp:127.0.0.1:2222-:22,hostfwd=tcp:127.0.0.1:2323-:23" qb_slirp_opt_default = "-netdev user,id=net0%s,tftp=%s" % (hostfwd, self.get('DEPLOY_DIR_IMAGE')) qb_slirp_opt = self.get('QB_SLIRP_OPT') or qb_slirp_opt_default - # Figure out the port - ports = re.findall('hostfwd=[^-]*:([0-9]+)-[^,-]*', qb_slirp_opt) - ports = [int(i) for i in ports] - lockdir = "/tmp/qemu-port-locks" - self.make_lock_dir(lockdir) + # Strip hostfwd rules from argv and add them via QMP after startup, + # making QEMU's host port binding authoritative. + self.slirp_hostfwd_allowlist = [ + (proto, hostip, int(hostport), guestip, int(targetport)) for + proto, hostip, hostport, guestip, targetport in + re.findall(r'hostfwd=([a-z]+):([^:]*):([0-9]+)-([^:,]*):([0-9]+)', qb_slirp_opt) + ] + qb_slirp_opt = re.sub(r',hostfwd=[a-z]+:[^:]*:[0-9]+-[^:,]*:[0-9]+', '', qb_slirp_opt) - # Find a free port to avoid conflicts - for p in ports[:]: - p_new = p - while not self.check_free_port('localhost', p_new, lockdir): - p_new += 1 - while p_new in ports: - p_new += 1 - if p != p_new: - ports.append(p_new) - qb_slirp_opt = re.sub(':%s-' % p, ':%s-' % p_new, qb_slirp_opt) - logger.info("Port forward changed: %s -> %s" % (p, p_new)) mac = "%s%02x" % (self.mac_slirp, self.alloc_mac_slot("/tmp/qemu-mac-locks")) self.set('NETWORK_CMD', '%s %s' % (self.network_device.replace('@MAC@', mac), qb_slirp_opt)) - # Print out port foward - hostfwd = re.findall('(hostfwd=[^,]*)', qb_slirp_opt) - if hostfwd: - logger.info('Port forward: %s' % ' '.join(hostfwd)) + + SLIRP_QMP_TIMEOUT = 10 + + def add_slirp_hostfwds(self): + """Add the slirp hostfwd allowlist to the running qemu via QMP""" + # qemu creates/binds this socket itself soon after it starts, so it + # may not be ready to accept a connection yet right after Popen(). + sock = None + for _ in range(100): + if self.qemuprocess and self.qemuprocess.poll() is not None: + raise RunQemuError("Failed to run qemu: %s" % self.qemuprocess.stderr.read().decode()) + try: + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + sock.settimeout(self.SLIRP_QMP_TIMEOUT) + sock.connect(self.internal_qmp_path) + break + except (FileNotFoundError, ConnectionRefusedError): + sock.close() + sock = None + time.sleep(0.1) + except socket.timeout as error: + sock.close() + raise RunQemuError("Timed out connecting to internal QMP socket %s" % self.internal_qmp_path) from error + except OSError: + sock.close() + raise + else: + raise RunQemuError("Timed out waiting for internal QMP socket %s" % self.internal_qmp_path) + sockf = sock.makefile('rw') + + def read_response(): + # QMP may interleave asynchronous event messages with command + # responses; skip those and return the first actual reply. + while True: + line = sockf.readline() + if not line: + raise RunQemuError("Internal QMP connection closed unexpectedly") + msg = json.loads(line) + if 'event' not in msg: + return msg + + try: + greeting = read_response() + if 'QMP' not in greeting: + raise RunQemuError("Invalid internal QMP greeting: %s" % greeting) + sockf.write(json.dumps({"execute": "qmp_capabilities"}) + '\n') + sockf.flush() + response = read_response() + if 'error' in response or response.get('return') != {}: + raise RunQemuError("Internal QMP negotiation failed: %s" % response) + + used_ports = set() + hostfwd_added = [] + for proto, hostip, hostport, guestip, targetport in self.slirp_hostfwd_allowlist: + candidate = hostport + for _ in range(100): + while (proto, candidate) in used_ports: + candidate += 1 + cmdline = 'hostfwd_add %s:%s:%d-%s:%d' % (proto, hostip, candidate, guestip, targetport) + sockf.write(json.dumps({"execute": "human-monitor-command", + "arguments": {"command-line": cmdline}}) + '\n') + sockf.flush() + response = read_response() + # hostfwd_add prints nothing on success; on failure it + # prints a message like "Could not set up host forwarding + # rule ..." which does NOT contain the word "error", so + # check for empty output rather than absence of that word. + result = response.get('return') + if 'error' not in response and isinstance(result, str) and result.strip() == '': + used_ports.add((proto, candidate)) + hostfwd_added.append('hostfwd=%s:%s:%d-%s:%d' % (proto, hostip, candidate, guestip, targetport)) + if candidate != hostport: + logger.info("Port forward changed: %s -> %s" % (hostport, candidate)) + break + candidate += 1 + else: + raise RunQemuError("Could not find a free host port for hostfwd target %d after 100 attempts" % targetport) + except socket.timeout as error: + raise RunQemuError("Timed out communicating with internal QMP socket %s" % self.internal_qmp_path) from error + finally: + sockf.close() + sock.close() + + if hostfwd_added: + logger.info('Port forward: %s' % ' '.join(hostfwd_added)) def setup_tap(self): """Setup tap""" @@ -1460,6 +1537,14 @@ to your build configuration. def setup_qmp(self): if self.qmp: self.qemu_opt += " -qmp %s,server,nowait" % self.qmp + if self.slirp_hostfwd_allowlist: + # A private monitor, in addition to any user-requested self.qmp, + # used internally to add the hostfwd allowlist once qemu is up + # (see add_slirp_hostfwds()). + qmp_dir = tempfile.mkdtemp(prefix='qemu-slirp-qmp-') + self.cleanup_files.append(qmp_dir) + self.internal_qmp_path = os.path.join(qmp_dir, 'monitor.sock') + self.qemu_opt += " -qmp unix:%s,server,nowait" % self.internal_qmp_path def setup_vga(self): if self.nographic == True: @@ -1645,6 +1730,8 @@ to your build configuration. pass_fds.append(descriptor.fileno()) process = subprocess.Popen(cmds, stderr=subprocess.PIPE, pass_fds=pass_fds, env=self.qemu_environ) self.qemuprocess = process + if self.slirp_hostfwd_allowlist: + self.add_slirp_hostfwds() retcode = process.wait() if retcode: if retcode == -signal.SIGTERM: From patchwork Sun Oct 11 19:59:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 766F4CA9EDB for ; Sun, 11 Oct 2026 20:00:00 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.32878.1791748795283185554 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=OQN0HG0z; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-202610111959528ef0daf9670002073c-skt_l0@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202610111959528ef0daf9670002073c for ; Sun, 11 Oct 2026 21:59:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=JHRW9ZrbXfzaP4wBrmNscPqF/Kd7PxyPgPoVn0aiReY=; b=OQN0HG0zL1JZkaigGLHYvZzZN92qzNTNi29G59y0YhszPVOKJrWrIznYixfoziZgQRpDQc dHjs6mCKs116MYhiZ+whzjDBNIICkPwJ3cYDJ7bpMdXO9G9ou1Vdo1pa27tCuEFbXDU6G8hF WH1UBPwxGxDqUG3UHXWO3bcBaTdO1QSIA4xXG4TCR+ueTmWzpp4sHhWMvBpMBmCU3dX6L169 O8XMgHJfCZ02aJgs9pxSn7NEDTYGqCvbdb2TOPdQrs57EYMCudLO/RuXGC0gt6k0XMD3h/xL k57mOnS7td8BjK6maoVjDtXTUivc8miD7sfxxeKW0wI2C1KuYRNtE92g==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 07/11] devtool: ide-sdk: use the effective slirp host ports in debugger configs Date: Sun, 11 Oct 2026 21:59:12 +0200 Message-ID: <20261011195949.2118607-8-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 20:00:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247586 From: Adrian Freihofer The generated debugger configurations only support forwards where the host port equals the target port. If the requested host port is in use, runqemu forwards the target port from the next free host port instead, e.g. target port 1234 from host port 1235, and the debugger connects to the wrong port. Look up the host port of each debug server's TCP target port in the QB_SLIRP_OPT forwards, including those already in the bbappend, and keep known mappings when adding missing forwards. To debug a QEMU whose ports were moved, write its effective mappings back into the bbappend and rerun devtool ide-sdk with --skip-bitbake. This mostly matters on autobuilders running several QEMU instances at once. For ide=none, the connect and run commands move from the gdbinit and lldbinit files into the generated start scripts, which hold the resolved port. The init files keep only symbol and source setup, and their help comments point to the scripts. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide_plugins/__init__.py | 12 +++++ scripts/lib/devtool/ide_plugins/ide_code.py | 4 +- scripts/lib/devtool/ide_plugins/ide_none.py | 43 ++++++++++-------- scripts/lib/devtool/ide_sdk.py | 49 ++++++++++++++++----- 4 files changed, 78 insertions(+), 30 deletions(-) diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py index 7841bc37b9..1e1e32b74b 100644 --- a/scripts/lib/devtool/ide_plugins/__init__.py +++ b/scripts/lib/devtool/ide_plugins/__init__.py @@ -163,6 +163,18 @@ class DebuggerCrossConfig: """Called after construction to generate any required config files.""" pass + def resolve_host_port(self, mode=None): + """Return the host port this debug server's target port is reachable on.""" + target_port = self.port(mode) + if not is_loopback_target(self.debugger_cross.target_device.target): + return target_port + return self.image_recipe.slirp_port_mappings.get(('tcp', target_port), target_port) + + def _resolve_host_port_shell_lines(self, var_name, mode=None): + """Shell line assigning $ to resolve_host_port(), for use in + a generated ide=none wrapper script.""" + return ['%s=%d' % (var_name, self.resolve_host_port(mode))] + # Abstract — subclasses must implement def _target_start_cmd(self, mode): raise NotImplementedError diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py index 081dc08657..89013139ea 100644 --- a/scripts/lib/devtool/ide_plugins/ide_code.py +++ b/scripts/lib/devtool/ide_plugins/ide_code.py @@ -423,7 +423,7 @@ class IdeVSCode(IdeBase): "MIMode": "gdb", "preLaunchTask": cross_debug_config.id_pretty_mode(server_mode), "miDebuggerPath": modified_recipe.debugger_cross.gdb, - "miDebuggerServerAddress": "%s:%d" % (modified_recipe.debugger_cross.host, cross_debug_config.port(server_mode)) + "miDebuggerServerAddress": "%s:%d" % (modified_recipe.debugger_cross.host, cross_debug_config.resolve_host_port(server_mode)) } if is_attach: @@ -530,7 +530,7 @@ class IdeVSCode(IdeBase): init_commands = [ "platform select remote-linux", - "platform connect connect://%s:%d" % (debugger_cross.host, lldb_config.port(server_mode)), + "platform connect connect://%s:%d" % (debugger_cross.host, lldb_config.resolve_host_port(server_mode)), # Clear the default step-avoid-regexp so std:: and other library # namespaces are not silently skipped on step-in. (default is "std::" in LLDB 15+) "settings set target.process.thread.step-avoid-regexp \"\"", diff --git a/scripts/lib/devtool/ide_plugins/ide_none.py b/scripts/lib/devtool/ide_plugins/ide_none.py index eba576bfdb..d5fed0809e 100644 --- a/scripts/lib/devtool/ide_plugins/ide_none.py +++ b/scripts/lib/devtool/ide_plugins/ide_none.py @@ -60,17 +60,14 @@ class GdbCrossConfigNone(GdbCrossConfig): gdbinit_lines = ['# This file is generated by devtool ide-sdk'] if server_mode == DebuggerServerModes.MULTI: target_help = '# gdbserver --multi :%d' % self.debug_server_port - remote_cmd = 'target extended-remote' else: target_help = '# gdbserver :%d %s' % ( self.debug_server_port, self.binary) - remote_cmd = 'target remote' gdbinit_lines.append('# On the remote target:') gdbinit_lines.append(target_help) gdbinit_lines.append('# On the build machine:') gdbinit_lines.append('# cd ' + self.modified_recipe.real_srctree) - gdbinit_lines.append( - '# ' + self.debugger_cross.gdb + ' -ix ' + self.gdbinit) + gdbinit_lines.append('# ' + self.gdb_script) gdbinit_lines.append('set sysroot ' + self.modified_recipe.d) if self.image_recipe.rootfs_dbg: @@ -114,18 +111,25 @@ class GdbCrossConfigNone(GdbCrossConfig): gdbinit_lines.append("end" + os.linesep) gdbinit_lines.append( - '%s %s:%d' % (remote_cmd, self.debugger_cross.host, self.debug_server_port)) - gdbinit_lines.append('set remote exec-file ' + self.binary.binary_path) - gdbinit_lines.append('run ' + self.binary.binary_path) + 'set remote exec-file ' + self.binary.binary_path) GdbCrossConfigNone.write_file(self.gdbinit, gdbinit_lines) - def _gen_gdb_start_script(self): + def _gen_gdb_start_script(self, server_mode=None): """Generate a script starting GDB with the corresponding gdbinit configuration.""" + if server_mode is None: + server_mode = self.default_mode + remote_cmd = ('target extended-remote' + if server_mode == DebuggerServerModes.MULTI + else 'target remote') cmd_lines = ['#!/bin/sh'] cmd_lines.append('cd ' + self.modified_recipe.real_srctree) - cmd_lines.append(self.debugger_cross.gdb + ' -ix ' + - self.gdbinit + ' "$@"') + cmd_lines += self._resolve_host_port_shell_lines('GDB_PORT', server_mode) + gdb_cmd = self.debugger_cross.gdb + ' -ix ' + self.gdbinit + gdb_cmd += " -ex '%s %s:'\"$GDB_PORT\"" % (remote_cmd, self.debugger_cross.host) + gdb_cmd += " -ex 'run %s'" % self.binary.binary_path + gdb_cmd += ' "$@"' + cmd_lines.append(gdb_cmd) GdbCrossConfigNone.write_file(self.gdb_script, cmd_lines, True) def initialize(self): @@ -210,10 +214,8 @@ class LldbServerConfigNone(LldbServerConfig): self.port(server_mode), self.gdbserver_ports[server_mode])) lines.append('# On the build machine:') lines.append('# cd ' + self.modified_recipe.real_srctree) - lines.append('# ' + self.debugger_cross.lldb + ' -s ' + self.lldbinit) + lines.append('# ' + self.lldb_script) lines.append('platform select remote-linux') - lines.append('platform connect connect://%s:%d' % ( - self.debugger_cross.host, self.port(server_mode))) lines.append('settings set target.process.thread.step-avoid-regexp ""') if self.image_recipe.rootfs_dbg: @@ -248,16 +250,21 @@ class LldbServerConfigNone(LldbServerConfig): 'Cannot setup debug symbols configuration for LLDB. ' 'IMAGE_GEN_DEBUGFS is not enabled.') - lines.append('target create --remote-file %s %s' % ( - self.binary.binary_path, self.binary.binary_host_path)) - LldbServerConfigNone.write_file(self.lldbinit, lines) - def _gen_lldb_start_script(self): + def _gen_lldb_start_script(self, server_mode=None): """Generate a script starting lldb with the lldbinit configuration.""" + if server_mode is None: + server_mode = self.default_mode cmd_lines = ['#!/bin/sh'] cmd_lines.append('cd ' + self.modified_recipe.real_srctree) - cmd_lines.append(self.debugger_cross.lldb + ' -s ' + self.lldbinit + ' "$@"') + cmd_lines += self._resolve_host_port_shell_lines('LLDB_PORT', server_mode) + lldb_cmd = self.debugger_cross.lldb + ' -s ' + self.lldbinit + lldb_cmd += " -o 'platform connect connect://%s:'\"$LLDB_PORT\"" % self.debugger_cross.host + lldb_cmd += " -o 'target create --remote-file %s %s'" % ( + self.binary.binary_path, self.binary.binary_host_path) + lldb_cmd += ' "$@"' + cmd_lines.append(lldb_cmd) LldbServerConfigNone.write_file(self.lldb_script, cmd_lines, True) def initialize(self): diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index 9faf041957..65f6e02675 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -306,6 +306,24 @@ class RecipeImage: def debug_support(self): return bool(self.rootfs_dbg) + @property + def slirp_port_mappings(self): + """QB_SLIRP_OPT's hostfwd rules as {(protocol, target_port): host_port}. + + QB_SLIRP_OPT is only a wishlist: runqemu moves a forward to another + host port whenever qemu cannot bind the wished-for one, so host + port == target port cannot be assumed. A caller that knows the + effective mapping of a running qemu (see QemuRunner.slirp_port_mappings) + can write it back into the bbappend and re-run devtool ide-sdk + --skip-bitbake to have the debugger configs regenerated against the + ports that really work. + """ + mappings = {} + for content in (self.qb_slirp_opt, self._orig_bbappend_content or ''): + for m in re.finditer(r'hostfwd=([a-z]+):[^:]*:(\d+)-[^:,]*:(\d+)', content): + mappings[(m.group(1), int(m.group(3)))] = int(m.group(2)) + return mappings + @property def rootfs(self): """Prefer the live NFS-exported rootfs (if --nfs=rootfs is used) over the @@ -548,28 +566,39 @@ class RecipeImage: if not ports: return False - # Determine which ports are already in the base value - already = {int(m.group(1)) - for m in re.finditer(r':(\d+)-:\d+', self.qb_slirp_opt)} - missing_ports = [p for p in ports if p not in already] + # Determine which ports are already in the base value. Match the + # target port, not the host port: once the effective mapping is + # written back the two differ, and a forward already exists for a + # target port regardless of which host port it ended up on. + already = {(m.group(1), int(m.group(2))) + for m in re.finditer(r'hostfwd=([a-z]+):[^:]*:\d+-[^:,]*:(\d+)', self.qb_slirp_opt)} + missing_ports = [p for p in ports if ('tcp', p) not in already] if not missing_ports: logger.info("QB_SLIRP_OPT already contains all needed port forwards") return False + # Keep any already known effective mapping instead of wishing for + # host port == target port again, so re-running does not discard a + # mapping written back from a running qemu (see slirp_port_mappings). + mappings = self.slirp_port_mappings + + def hostfwd(target_port): + return ',hostfwd=tcp:127.0.0.1:%d-:%d' % ( + mappings.get(('tcp', target_port), target_port), target_port) + if self.qb_slirp_opt: # Base value exists: :append only the missing port forwards - extra = ''.join( - ',hostfwd=tcp:127.0.0.1:%d-:%d' % (p, p) for p in missing_ports) + extra = ''.join(hostfwd(p) for p in missing_ports) new_line = 'QB_SLIRP_OPT:append = "%s"' % extra else: # No base value: mirror runqemu's built-in default (SSH 2222, telnet # 2323, tftp) and add the debugger ports. all_hostfwds = ( - 'hostfwd=tcp:127.0.0.1:2222-:22,' - 'hostfwd=tcp:127.0.0.1:2323-:23' + 'hostfwd=tcp:127.0.0.1:%d-:22,' + 'hostfwd=tcp:127.0.0.1:%d-:23' + % (mappings.get(('tcp', 22), 2222), mappings.get(('tcp', 23), 2323)) ) - all_hostfwds += ''.join( - ',hostfwd=tcp:127.0.0.1:%d-:%d' % (p, p) for p in missing_ports) + all_hostfwds += ''.join(hostfwd(p) for p in missing_ports) new_line = 'QB_SLIRP_OPT = "-netdev user,id=net0,%s,tftp=${DEPLOY_DIR_IMAGE}"' % all_hostfwds if os.path.exists(self._bbappend): From patchwork Sun Oct 11 19:59:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF376CA9ED2 for ; Sun, 11 Oct 2026 19:59:58 +0000 (UTC) Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net [185.136.64.228]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.32781.1791748795226074486 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=tVRUPiSz; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.228, mailfrom: fm-1329275-20261011195952168f7d1eb40002070d-vtryde@rts-flowmailer.siemens.com) Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20261011195952168f7d1eb40002070d for ; Sun, 11 Oct 2026 21:59:53 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=1d4HjYt7MP8wF7neRuwv9J8KvolG/6RXWXA6DkrvTPs=; b=tVRUPiSzhg2PJkro8HmhyRkEzjNb3MdwwN0OtIf7CsJJV9rb7YO5ABGUuEymb/E+NtMyMy p4COLN4EUDxsHhJ+I2I5liNoDbrakmeGt6gXhNOdMhw1adF2qoypsjSPaJ5QBGqt1Wrnpbqp GAM2R+oojHI2o+JMZ0a0YsmSeVdpsE/jaS7ioPJHILTJHiYQ8kmH+bBmv2Brzv1rXvdqFWrK tN5PO/Iruq/Mu3uJvBZHe52r13KBlU1ohpPhfzYW/mSyIaSuISFKZkJgfZ59ZXQIaOkz92uw Lq66MFrjaHVrtCx2hycZgn3fFKLyctq31oIAgf/zZi34Irp0mKIyLwDg==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 08/11] oe-selftest: devtool ide-sdk: allocate slirp debug server ports via qemu Date: Sun, 11 Oct 2026 21:59:13 +0200 Message-ID: <20261011195949.2118607-9-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 19:59:58 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247583 From: Adrian Freihofer The ide-sdk tests assumed every slirp port forwarding gets its requested host port. With several QEMU instances running in parallel, runqemu may move a port forwarding to another host port. After booting QEMU, read the effective forwards from QemuRunner, write them back into QB_SLIRP_OPT and regenerate the debugger configurations with devtool ide-sdk --skip-bitbake. Each test still reserves a block of debug server ports, so parallel tests do not request the same ports. This only coordinates the tests; QEMU decides which host port is actually free. Warn if a forward was moved anyway. This should not happen, and points to another process holding a port from the reserved block which is probably unexpected. The gdbserver spawned by lldb-server is reached by the lldb client on its target port number, so it needs a forward with equal host and target port. Add that forward explicitly if it is missing. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 91 +++++++++++++++++++++---- 1 file changed, 78 insertions(+), 13 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 265f2147a4..6d3be66188 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3082,10 +3082,6 @@ class DevtoolIdeSdkTests(DevtoolBase): MAGIC_STRING_ORIG = "Magic: 123456789" SLIRP_PORT_BLOCK_SIZE = 32 - # runqemu locks /tmp/qemu-port-locks itself while picking the slirp host - # ports, so the reservation below needs a lock directory of its own: - # holding runqemu's locks would make it consider the reserved ports taken - # and move the forwards away again. SLIRP_PORT_LOCK_DIR = '/tmp/oe-selftest-ide-sdk-port-locks' def setUp(self): @@ -3544,12 +3540,12 @@ class DevtoolIdeSdkTests(DevtoolBase): def _reserve_slirp_port_block(self): """Reserve a block of host ports for this test's debug servers and return its first port. - runqemu moves a QB_SLIRP_OPT forward to the next host port whenever the - wished-for one is already taken, and lldb-server's spawned gdbserver - has no way to tell its client about a remapped host port, so the wishes - have to be granted 1:1. Every parallel oe-selftest worker otherwise - starts from the same default port, so claim a block that is free right - now and hold it for the rest of the test. + A QB_SLIRP_OPT port forward only stays 1:1 while the wished-for host + port is free, and lldb-server's spawned gdbserver has no way to tell + its client about a remapped host port, so it needs host == target. + Since every parallel oe-selftest worker otherwise starts from the same + default port, claim a block that is free right now and hold it for the + rest of the test. """ os.makedirs(self.SLIRP_PORT_LOCK_DIR, exist_ok=True) for start in range(1234, 20000, self.SLIRP_PORT_BLOCK_SIZE): @@ -3585,6 +3581,45 @@ class DevtoolIdeSdkTests(DevtoolBase): self.fail('Could not reserve %d consecutive free host ports' % self.SLIRP_PORT_BLOCK_SIZE) + def _apply_slirp_port_mappings(self, testimage, qemu): + """Rewrite the image bbappend's QB_SLIRP_OPT with qemu's effective port mappings. + + QB_SLIRP_OPT is only a wishlist: runqemu moves a forward to another + host port whenever qemu cannot bind the wished-for one (e.g. because + a parallel oe-selftest worker's qemu holds it). Writing the mappings + the running qemu reports back into QB_SLIRP_OPT is what lets the + subsequent `devtool ide-sdk --skip-bitbake` run generate debugger + configs pointing at the ports that actually work. + """ + mappings = qemu.slirp_port_mappings + self.assertTrue(mappings, 'Running qemu reported no slirp port mappings') + + bbappend = os.path.join(self.workspacedir, 'appends', testimage + '.bbappend') + with open(bbappend) as f: + content = f.read() + + def remap(m): + protocol = m.group(2) + wished_port, target_port = int(m.group(3)), int(m.group(4)) + self.assertIn((protocol, target_port), mappings, + 'qemu is not forwarding wished-for %s target port %d' % (protocol, target_port)) + host_port = mappings[(protocol, target_port)] + if host_port != wished_port: + # The reserved block (see _reserve_slirp_port_block) should + # make this unreachable; if it still happens, something else + # grabbed a port from the block between the reservation and + # qemu's own bind. + self.logger.warning( + 'Slirp %s host port for target %d moved from the reserved ' + '%d to %d: the reserved port block was not exclusive ' + 'after all.', protocol, target_port, wished_port, host_port) + return '%s%d-:%d' % (m.group(1), host_port, target_port) + + new_content = re.sub(r'(hostfwd=([a-z]+):[^:]*:)(\d+)-:(\d+)', remap, content) + with open(bbappend, 'w') as f: + f.write(new_content) + return mappings + def _verify_nfs_debug_rootfs(self, testimage, nfs): """Verify the NFS debug rootfs was extracted and its runqemu launch helper generated.""" nfs_rootfs = os.path.join(self.workspacedir, 'nfs-exports', testimage, nfs) @@ -3680,9 +3715,8 @@ class DevtoolIdeSdkTests(DevtoolBase): self._meson_recipe_name, "meson.build", testimage) package_opts = self._ide_sdk_package_opts() nfs_opts = ' --nfs=%s' % nfs_export if nfs else '' - # runqemu only grants a QB_SLIRP_OPT wish while the host port is free, - # and lldb-server's spawned gdbserver needs host == target, so give the - # debug servers a block no parallel worker uses (see + # A reserved block keeps the QB_SLIRP_OPT wishlist 1:1, which + # lldb-server's spawned gdbserver depends on (see # _reserve_slirp_port_block); both invocations must agree on it. port_opts = ' -G %d' % self._reserve_slirp_port_block() if slirp else '' runCmd('devtool ide-sdk %s -c --ide=code --ide=none %s%s%s' % ( @@ -3723,6 +3757,10 @@ class DevtoolIdeSdkTests(DevtoolBase): if slirp: self.assertIsNotNone(qemu.port, 'No SSH port for the slirp target') target_options = '-t root@%s -P %s' % (qemu.ip, qemu.port) + # The wished-for ports in QB_SLIRP_OPT are only known to have + # been honoured now that qemu is up, so feed its effective + # mappings back in before regenerating the debugger configs. + self._apply_slirp_port_mappings(testimage, qemu) else: target_options = '-t root@%s -c' % qemu.ip @@ -4763,6 +4801,19 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): self.assertTrue(task, f"Configuration '{config_name}' preLaunchTask should not be empty") + def _ensure_1to1_hostfwd(self, qemu, lldb_server_cmd): + """Ensure the --gdbserver-port in lldb_server_cmd is forwarded host==target.""" + match = re.search(r'--gdbserver-port (\d+)', lldb_server_cmd) + self.assertIsNotNone(match, + "lldb-server start command should specify --gdbserver-port") + port = int(match.group(1)) + if not qemu.slirp_port_mappings: + return + if qemu.slirp_port_mappings.get(('tcp', port)) == port: + return + self.assertEqual(qemu.add_hostfwd(port, port), port, + "Could not forward lldb-server gdbserver port %d 1:1" % port) + def _find_source_break_line(self, tempdir, source_file, marker): """Find the 1-based line number of `marker` in tempdir/source_file @@ -5016,6 +5067,14 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): connect_cmd = next((c for c in init_commands if "platform connect" in c), None) self.assertIsNotNone(connect_cmd, "initCommands should contain a platform connect command") + # Once a debug session is started, the lldb client automatically + # reconnects to lldb-server's spawned gdbserver sub-instance using the + # raw --gdbserver-port target port number, with no hook to tell it to + # use a different (remapped) host port -- unlike the platform connect + # port above, which is resolved from the effective mapping. So this + # port needs an explicit host==target slirp forward. + self._ensure_1to1_hostfwd(qemu, ssh_cmd[-1]) + lldb_binary = self._lldb_native_binary() self.logger.debug("Starting lldb-server via SSH: %s", " ".join(ssh_cmd)) @@ -5093,6 +5152,12 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): self.assertExists(lldb_server_script) self.assertExists(lldb_script) + # See the comment in _lldb_server_debugging_once() about why this + # port specifically needs an explicit host==target slirp forward. + with open(lldb_server_script) as f: + lldb_server_script_content = f.read() + self._ensure_1to1_hostfwd(qemu, lldb_server_script_content) + # Start lldb-server (script exits once the port is ready) r = runCmd(lldb_server_script, output_log=self._cmd_logger) self.assertEqual(r.status, 0) From patchwork Sun Oct 11 19:59:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B08E6CA9EBD for ; Sun, 11 Oct 2026 19:59:58 +0000 (UTC) Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net [185.136.64.225]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.32780.1791748795180028612 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=ko9/yN2F; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.225, mailfrom: fm-1329275-202610111959531a87668086000207a0-r_3eii@rts-flowmailer.siemens.com) Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 202610111959531a87668086000207a0 for ; Sun, 11 Oct 2026 21:59:53 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=jiywu+vFjiRrMIH+anyzNSMTD5oM4c2RHYXNEsrTynI=; b=ko9/yN2FMOLohA/I958O6C6/oQhIqFC4fJmSl06au6ntD7dzVyfijUfu3Ka1WhSxrISWEv zU5+G2Rb54hs/uHYYV4Qe1vQG6MkjZTqoQb3tOMv5ecau4j5vOkPubShXimNfqT4dDYZ9vYR PwQpN9OskOR/b1C89bNNwWVZniAhqf8dWhUROWZKhwmkOhSndjngJDLQEQV3HRpoHaEoklZI AdqHwIDI9NVjV2pXnylW/P0ImjsW57mv9Vu1dzY9I7Lx+rqOWyaDzwBO+XlrweOIAxZIs37N 8nhnKRRd103bamhxE0O+PVKav3xDdkiSZSAUjX4ljGYU3ZFJjbSPScyw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 09/11] runqemu: make port lock directory patchable Date: Sun, 11 Oct 2026 21:59:14 +0200 Message-ID: <20261011195949.2118607-10-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 19:59:58 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247585 From: Adrian Freihofer Use a class setting for the shared port-lock directory instead of a literal path in NFS allocation. Keep the production default unchanged and let subsequent slirp locking use the same setting. Signed-off-by: Adrian Freihofer --- scripts/runqemu | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/runqemu b/scripts/runqemu index c569fc7c84..0e0c77076c 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -136,6 +136,8 @@ def get_first_file(globs): return '' class BaseConfig(object): + PORT_LOCK_DIR = "/tmp/qemu-port-locks" + def __init__(self): # The self.d saved vars from self.set(), part of them are from qemuboot.conf self.d = {'QB_KERNEL_ROOT': '/dev/vda'} @@ -1059,7 +1061,7 @@ to your build configuration. self.nfs_server = '192.168.7.@GATEWAY@' nfsd_port = 3048 + self.nfs_instance - lockdir = "/tmp/qemu-port-locks" + lockdir = self.PORT_LOCK_DIR self.make_lock_dir(lockdir) while not self.check_free_port('localhost', nfsd_port, lockdir): self.nfs_instance += 1 From patchwork Sun Oct 11 19:59:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2130DCA9ED5 for ; Sun, 11 Oct 2026 20:00:00 +0000 (UTC) Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.32873.1791748795181030790 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=QuX0AJbO; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.226, mailfrom: fm-1329275-202610111959531a2fddd9f30002073e-lnthqb@rts-flowmailer.siemens.com) Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 202610111959531a2fddd9f30002073e for ; Sun, 11 Oct 2026 21:59:53 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=RU+/5Lk/ATMRHlDcCL5Zo/YlJ/OjtO2OPfkLrcFjHFM=; b=QuX0AJbOTZrctuqaIi6AzODImlM+P2+hcB54cbLUJTG2Kpo5o61r8Zmecuocw7Dl+qnWKP Mn7q6nfYdXs+cn/H0q+Ky8EDmvOPM6a89mK1ry96pcdruLQLiBBqUsmmRRZqY207Y13AsMQV Hguab3BPAqcAC6/69pD5oBf/Rd11MpKSKRoEbaon/pN57BC9tUb5y92npUxYqu33+OjL4Pd6 7v3s4SzXivgX0VK2AYY4qW2L61pucn0ymzGcLlL3N7gsXd/y3DICFDA44/aczQVhIgrgQA5U k2GOirqiofVJpu6yb0JTeDLliNIHFtBGG/k1sJVNhTqI0L1iRXfKz+vw==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 10/11] oe-selftest: runqemu: cover slirp forwarding via QMP Date: Sun, 11 Oct 2026 21:59:15 +0200 Message-ID: <20261011195949.2118607-11-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 20:00:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247592 From: Adrian Freihofer Test runqemu and QemuRunner slirp forwarding against a native QEMU started with -machine none, so no image is needed and the tests run in seconds. Cover moved forwards, TCP/UDP sharing a host port, bind addresses, failing and silent monitors, MAC slot allocation and the devtool port block reservation. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/runqemu.py | 233 ++++++++++++++++++++++++ 1 file changed, 233 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py index bfa76812ae..455a5f2be2 100644 --- a/meta/lib/oeqa/selftest/cases/runqemu.py +++ b/meta/lib/oeqa/selftest/cases/runqemu.py @@ -7,11 +7,244 @@ import os import re import time +import fcntl +import glob +import json +import runpy +import shutil +import socket +import stat +import subprocess +import sys +import tempfile +from concurrent.futures import ThreadPoolExecutor +from types import SimpleNamespace +from unittest.mock import patch import oe.types from oeqa.core.decorator import OETestTag from oeqa.core.decorator.data import skipIfNotArch, skipIfNotMachine from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake, runqemu, get_bb_var, get_bb_vars +from oeqa.utils.qemurunner import QemuRunner + + +@OETestTag("runqemu") +class RunqemuSlirpTests(OESelftestTestCase): + """Exercise slirp forwarding without booting a guest.""" + + @classmethod + def setUpClass(cls): + super().setUpClass() + bitbake('qemu-helper-native -c addto_recipe_sysroot') + bindir = get_bb_var('STAGING_BINDIR_NATIVE', 'qemu-helper-native') + cls.qemu_binary = sorted(glob.glob(os.path.join(bindir, 'qemu-system-*')))[0] + sys.path.append(os.path.join(os.path.dirname(os.path.dirname(bindir)), 'usr/lib/qemu-python')) + from qmp.legacy import QEMUMonitorProtocol + cls.monitor_protocol = QEMUMonitorProtocol + runqemu_module = runpy.run_path(shutil.which('runqemu')) + cls.base_config = runqemu_module['BaseConfig'] + cls.runqemu_error = runqemu_module['RunQemuError'] + + def setUpLocal(self): + directory = tempfile.TemporaryDirectory(prefix='slirp-selftest-') + self.addCleanup(directory.cleanup) + self.config = self.base_config() + self.config.internal_qmp_path = os.path.join(directory.name, 'internal.sock') + self.config.slirp_hostfwd_allowlist = [] + self.addCleanup(self.config.release_lock) + self.runner = QemuRunner('', '', '', directory.name, directory.name, + None, 30, directory.name, False, self.logger, + use_slirp=True, workdir=directory.name) + self.addCleanup(self.runner.stop) + monitor_path = os.path.join(directory.name, 'monitor.sock') + self.process = subprocess.Popen([ + self.qemu_binary, '-machine', 'none', '-nodefaults', '-display', 'none', + '-S', '-netdev', 'user,id=net0', + '-qmp', 'unix:%s,server=on,wait=off' % self.config.internal_qmp_path, + '-qmp', 'unix:%s,server=on,wait=off' % monitor_path, + ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) + self.addCleanup(self._stop_qemu) + self.config.add_slirp_hostfwds() + self.runner.qmp = self.monitor_protocol(monitor_path) + self.runner.qmp.settimeout(5) + self.assertIn('QMP', self.runner.qmp.connect()) + + def _stop_qemu(self): + if self.process.poll() is None: + self.process.terminate() + self.process.communicate(timeout=10) + + def _monitor(self, command, arguments=None, timeout=60): + return self.runner.qmp.cmd_raw(command, arguments) + + def _socket(self, protocol='tcp'): + probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM if protocol == 'tcp' else socket.SOCK_DGRAM) + self.addCleanup(probe.close) + probe.bind(('127.0.0.1', 0)) + if protocol == 'tcp': + probe.listen() + return probe + + def _free_port(self): + probe = self._socket() + port = probe.getsockname()[1] + probe.close() + return port + + def _startup_forward(self, port, protocol='tcp', guest_port=1234, host='127.0.0.1'): + self.config.slirp_hostfwd_allowlist = [(protocol, host, port, '', guest_port)] + self.config.add_slirp_hostfwds() + return self.runner.get_host_port(guest_port, protocol=protocol) + + def test_startup_rejected_candidate(self): + """Move a startup forward away from a port QEMU cannot bind.""" + busy = self._socket() + port = busy.getsockname()[1] + actual = self._startup_forward(port) + self.assertNotEqual(actual, port) + + def test_protocols_share_owned_reservations(self): + """Allow TCP and UDP forwards to share a host port.""" + for guest_port, owner in ((1234, 'runqemu'), (1235, 'qemurunner'), (1236, 'runqemu-dynamic')): + with self.subTest(owner=owner): + port = self._free_port() + if owner == 'runqemu': + self.config.slirp_hostfwd_allowlist = [ + (protocol, '127.0.0.1', port, '', guest_port) + for protocol in ('tcp', 'udp')] + self.config.add_slirp_hostfwds() + elif owner == 'runqemu-dynamic': + self.assertEqual(self._startup_forward(port, guest_port=guest_port), port) + else: + self.assertEqual(self.runner.add_hostfwd(guest_port, port), port) + self.assertEqual(self.runner.get_host_port(guest_port), port) + if owner != 'runqemu': + self.assertEqual(self.runner.add_hostfwd(guest_port, port, protocol='udp'), port) + self.assertEqual(self.runner.get_host_port(guest_port, protocol='UDP'), port) + + def test_dynamic_rejected_candidates(self): + """Reject dynamic forwards when binding or monitor commands fail.""" + for protocol in ('tcp', 'udp'): + with self.subTest(protocol=protocol): + busy = self._socket(protocol) + port = busy.getsockname()[1] + self.assertIsNone(self.runner.add_hostfwd(1234, port, protocol=protocol)) + port = self._free_port() + for failure in (None, {}, {'error': {}}, {'return': '', 'error': {}}, + {'return': 'Could not set up host forwarding rule'}, {'return': 0}): + with self.subTest(response=failure): + def reply(command, arguments=None): + if arguments and arguments['command-line'].startswith('hostfwd_add'): + return failure + return self._monitor(command, arguments) + with patch.object(self.runner, 'run_monitor', side_effect=reply): + self.assertIsNone(self.runner.add_hostfwd(1234, port)) + + def test_internal_monitor_timeouts(self): + self.config.SLIRP_QMP_TIMEOUT = 0.1 + for phase in ('greeting', 'capabilities', 'hostfwd'): + with self.subTest(phase=phase), tempfile.TemporaryDirectory() as directory: + port = self._free_port() + self.config.internal_qmp_path = os.path.join(directory, 'silent.sock') + self.config.slirp_hostfwd_allowlist = [('tcp', '127.0.0.1', port, '', 1234)] + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: + listener.settimeout(5) + listener.bind(self.config.internal_qmp_path) + listener.listen() + with ThreadPoolExecutor(max_workers=1) as executor: + future = executor.submit(self.config.add_slirp_hostfwds) + with listener.accept()[0] as peer: + peer.settimeout(2) + with peer.makefile('rw') as stream: + if phase != 'greeting': + stream.write(json.dumps({'QMP': {}}) + '\n') + stream.flush() + self.assertEqual(json.loads(stream.readline())['execute'], 'qmp_capabilities') + if phase == 'hostfwd': + stream.write(json.dumps({'return': {}}) + '\n') + stream.flush() + self.assertEqual(json.loads(stream.readline())['execute'], 'human-monitor-command') + with self.assertRaisesRegex(self.runqemu_error, 'Timed out communicating'): + future.result(timeout=2) + + def test_private_internal_monitor(self): + """Add a private forwarding monitor without replacing the requested monitor. + + The internal socket's directory must be accessible only to its owner + and registered for cleanup. Both QMP sockets must appear in QEMU's + command line. + """ + self.config.slirp_hostfwd_allowlist = [('tcp', '127.0.0.1', 2222, '', 22)] + self.config.qmp = 'unix:requested-monitor.sock' + self.config.setup_qmp() + directory = os.path.dirname(self.config.internal_qmp_path) + self.addCleanup(shutil.rmtree, directory) + self.assertEqual(stat.S_IMODE(os.stat(directory).st_mode), 0o700) + self.assertIn(directory, self.config.cleanup_files) + self.assertIn('-qmp unix:requested-monitor.sock,server,nowait', self.config.qemu_opt) + self.assertIn('-qmp unix:%s,server,nowait' % self.config.internal_qmp_path, self.config.qemu_opt) + + def test_mac_slot_reuse(self): + """Allocate distinct MAC slots and reuse a slot after its lock is released. + + A third configuration must take the first configuration's freed slot + while the second configuration continues to hold its own slot. + """ + with tempfile.TemporaryDirectory() as directory: + configs = [self.base_config() for index in range(3)] + try: + self.assertEqual(configs[0].alloc_mac_slot(directory), 1) + self.assertEqual(configs[1].alloc_mac_slot(directory), 2) + configs[0].release_lock() + self.assertEqual(configs[2].alloc_mac_slot(directory), 1) + finally: + for config in configs: + config.release_lock() + + def test_mac_slot_exhaustion(self): + """Report exhaustion only after trying all 254 usable MAC slot locks.""" + with tempfile.TemporaryDirectory() as directory: + with patch.object(self.config, 'acquire_lock', return_value=False) as acquire: + with self.assertRaisesRegex(self.runqemu_error, 'Too many concurrent'): + self.config.alloc_mac_slot(directory) + self.assertEqual(acquire.call_count, 254) + self.assertEqual([os.path.basename(call.args[0]) for call in acquire.call_args_list], + ['%d.lock' % slot for slot in range(1, 255)]) + + def test_bind_addresses(self): + """Preserve explicit bind addresses and resolve wildcard binds to localhost.""" + for guest_port, host, expected in ((1234, '127.0.0.2', '127.0.0.2'), (1235, '0.0.0.0', '127.0.0.1')): + with self.subTest(host=host): + actual = self._startup_forward(self._free_port(), guest_port=guest_port, host=host) + endpoint = self.runner._get_slirp_host_endpoint(guest_port) + self.assertEqual(endpoint, (expected, actual)) + with socket.create_connection(endpoint, timeout=5): + pass + + def test_unsupported_protocol(self): + """Reject unsupported forwarding protocols.""" + with self.assertRaises(ValueError): + self.runner.add_hostfwd(1234, protocol='sctp') + + def test_explicit_forward_with_non_slirp_primary(self): + """Manage slirp forwards independently of the primary networking mode.""" + self.runner.use_slirp = False + for protocol in ('tcp', 'udp'): + with self.subTest(protocol=protocol): + port = self._free_port() + self.assertEqual(self.runner.add_hostfwd(1234, port, protocol=protocol), port) + self.assertEqual(self.runner.slirp_port_mappings[(protocol, 1234)], port) + self.assertEqual(self.runner.get_host_port(1234, protocol=protocol), port) + port = self.runner.get_host_port(1236) + self.assertEqual(self.runner.slirp_port_mappings[('tcp', 1236)], port) + with socket.create_connection(('127.0.0.1', port), timeout=5): + pass + response = self.runner.run_monitor('netdev_del', {'id': 'net0'}) + self.assertEqual(response, {'return': {}}) + port = self._free_port() + self.assertIsNone(self.runner.add_hostfwd(1235, port)) + with self.assertRaisesRegex(RuntimeError, 'Failed to add a slirp tcp hostfwd'): + self.runner.get_host_port(1235) @OETestTag("runqemu") From patchwork Sun Oct 11 19:59:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AdrianF X-Patchwork-Id: 100364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D26BECA9ECF for ; Sun, 11 Oct 2026 19:59:58 +0000 (UTC) Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.32782.1791748795282806123 for ; Sun, 11 Oct 2026 12:59:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=adrian.freihofer@siemens.com header.s=fm1 header.b=Jk5dfU8D; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.64.226, mailfrom: fm-1329275-202610111959536a4c6384a6000207ba-rasvlc@rts-flowmailer.siemens.com) Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202610111959536a4c6384a6000207ba for ; Sun, 11 Oct 2026 21:59:53 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adrian.freihofer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=viQaNTlz9w20AkzgtcuQRL5nsTl8tmXlArXBmHHkunI=; b=Jk5dfU8D+e4ij/wJXZGsmLyfD+7L6QUu7WbLl2UZzirfIOy1KFA0kQMZtHFMX3CNxvbofs /LAcuMXBPAUh/PaNKaIKjLIrLEyltl/bHzYBr3gOiEuzoQQrn/0F+PJUr3HACBk0DJWILVCi xS6SPtBNL3qgFJrxaJSIrVktJmpsIY56KN6RAIjf6u7WFIXbLFMSX8eI+Tnzm7Va1pYEi+RG r86hynhJd2keo9eeUx8riYhVa0aA7QLtQGnYarwQiZiQxHDrRxrEn1EF7iWGEtiCCAXwMJ7S hzSNk9FZL+VQFJmCJono7IjGdLMbrautJNIwgwYZGnserdWMIVWOMMNA==; From: AdrianF To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 11/11] runqemu: retain port locks for older autobuilder branches Date: Sun, 11 Oct 2026 21:59:16 +0200 Message-ID: <20261011195949.2118607-12-adrian.freihofer@siemens.com> In-Reply-To: <20261011195949.2118607-1-adrian.freihofer@siemens.com> References: <20261011195949.2118607-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1329275:519-21489:flowmailer List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 11 Oct 2026 19:59:58 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/247584 From: Adrian Freihofer runqemu up to wrynose and blacksail takes a flock per slirp host port before starting QEMU. On a shared autobuilder, a newer runqemu could bind such a port before the older QEMU does, so that QEMU fails. Make runqemu and QemuRunner skip locked ports and lock each port QEMU binds. Pass the locks to QEMU so they are held as long as QEMU runs, even if runqemu exits. Extend the selftests accordingly. Revert this once those branches (wrynose and blacksail) are EOL or have the QMP allocation backported. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 2 +- meta/lib/oeqa/selftest/cases/runqemu.py | 188 +++++++++++++++++++----- meta/lib/oeqa/utils/qemurunner.py | 68 +++++++-- scripts/runqemu | 43 ++++-- 4 files changed, 240 insertions(+), 61 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 6d3be66188..5a5ec2a385 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -3545,7 +3545,7 @@ class DevtoolIdeSdkTests(DevtoolBase): its client about a remapped host port, so it needs host == target. Since every parallel oe-selftest worker otherwise starts from the same default port, claim a block that is free right now and hold it for the - rest of the test. + rest of the test. Separate locks keep runqemu's legacy port locks free. """ os.makedirs(self.SLIRP_PORT_LOCK_DIR, exist_ok=True) for start in range(1234, 20000, self.SLIRP_PORT_BLOCK_SIZE): diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py index 455a5f2be2..3ef7d35ebd 100644 --- a/meta/lib/oeqa/selftest/cases/runqemu.py +++ b/meta/lib/oeqa/selftest/cases/runqemu.py @@ -30,10 +30,17 @@ from oeqa.utils.qemurunner import QemuRunner @OETestTag("runqemu") class RunqemuSlirpTests(OESelftestTestCase): - """Exercise slirp forwarding without booting a guest.""" + """Exercise forwarding and legacy arbitration without booting a guest.""" @classmethod def setUpClass(cls): + """Locate the native QEMU and load the QMP client and runqemu classes. + + Builds qemu-helper-native's sysroot once for the whole class, picks its + qemu-system binary, and imports the QMP client shipped with it. runqemu + is loaded as a module so its BaseConfig and RunQemuError can be used + directly. + """ super().setUpClass() bitbake('qemu-helper-native -c addto_recipe_sysroot') bindir = get_bb_var('STAGING_BINDIR_NATIVE', 'qemu-helper-native') @@ -46,15 +53,31 @@ class RunqemuSlirpTests(OESelftestTestCase): cls.runqemu_error = runqemu_module['RunQemuError'] def setUpLocal(self): + """Start a bare QEMU with a slirp netdev and connect runqemu and the runner to it. + + QEMU runs paused with no machine, so no guest boots. It exposes two QMP + sockets: an internal one used by runqemu's BaseConfig and a separate + monitor one used by the QemuRunner. Both see the same netdev, so + forwards added through either are visible to the other. Everything is + cleaned up after each test. Only the port-lock directory is mocked; + flock and QEMU's lock descriptor transfer remain real. + """ directory = tempfile.TemporaryDirectory(prefix='slirp-selftest-') self.addCleanup(directory.cleanup) + self.port_lock_dir = os.path.join(directory.name, 'port-locks') self.config = self.base_config() + config_lockdir = patch.object(self.config, 'PORT_LOCK_DIR', self.port_lock_dir) + config_lockdir.start() + self.addCleanup(config_lockdir.stop) self.config.internal_qmp_path = os.path.join(directory.name, 'internal.sock') self.config.slirp_hostfwd_allowlist = [] self.addCleanup(self.config.release_lock) self.runner = QemuRunner('', '', '', directory.name, directory.name, None, 30, directory.name, False, self.logger, use_slirp=True, workdir=directory.name) + runner_lockdir = patch.object(self.runner, 'PORT_LOCK_DIR', self.port_lock_dir) + runner_lockdir.start() + self.addCleanup(runner_lockdir.stop) self.addCleanup(self.runner.stop) monitor_path = os.path.join(directory.name, 'monitor.sock') self.process = subprocess.Popen([ @@ -85,66 +108,156 @@ class RunqemuSlirpTests(OESelftestTestCase): probe.listen() return probe - def _free_port(self): - probe = self._socket() - port = probe.getsockname()[1] - probe.close() - return port + def _occupied_port(self, protocol='tcp'): + """Return a port held by this test, so QEMU must reject it as a proposal.""" + return self._socket(protocol).getsockname()[1] + + # TODO: legacy-slirp-locks + def _lock(self, port): + lock = open(os.path.join(self.port_lock_dir, '%d.lock' % port), 'w') + self.addCleanup(lock.close) + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + return lock + + def _assert_locked(self, port): + with open(os.path.join(self.port_lock_dir, '%d.lock' % port), 'w') as lock: + with self.assertRaises(BlockingIOError): + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) def _startup_forward(self, port, protocol='tcp', guest_port=1234, host='127.0.0.1'): self.config.slirp_hostfwd_allowlist = [(protocol, host, port, '', guest_port)] self.config.add_slirp_hostfwds() return self.runner.get_host_port(guest_port, protocol=protocol) + def test_startup_legacy_reservation(self): + """Skip a startup port locked by an older runqemu and lock the replacement. + + Older releases (wrynose, blacksail) reserve ports via flock on + /tmp/qemu-port-locks/.lock. runqemu must not take a port whose + lock is held, must lock the port it picks instead, and the forward on + that port must accept connections. + + The port is also occupied, so QEMU would reject it; the lock refusal + is checked on runqemu's own lock attempt instead. + """ + port = self._occupied_port() + self._lock(port) + lockfile = os.path.join(self.port_lock_dir, '%d.lock' % port) + results = {} + acquire_lock = self.config.acquire_lock + + def spy(path): + results.setdefault(path, acquire_lock(path)) + return results[path] + with patch.object(self.config, 'acquire_lock', side_effect=spy): + actual = self._startup_forward(port) + self.assertIs(results[lockfile], False) + self.assertNotEqual(actual, port) + self._assert_locked(actual) + with socket.create_connection(('127.0.0.1', actual), timeout=5): + pass + def test_startup_rejected_candidate(self): - """Move a startup forward away from a port QEMU cannot bind.""" + """Release a rejected startup port's lock and lock the replacement. + + The requested port is bound by another process, so its lock is free + but QEMU rejects hostfwd_add. runqemu must release the lock it took + for the rejected port (it must be lockable again afterwards), then + retry on the next port and keep that port's lock held. + """ busy = self._socket() port = busy.getsockname()[1] actual = self._startup_forward(port) self.assertNotEqual(actual, port) + self._lock(port) + self._assert_locked(actual) - def test_protocols_share_owned_reservations(self): - """Allow TCP and UDP forwards to share a host port.""" - for guest_port, owner in ((1234, 'runqemu'), (1235, 'qemurunner'), (1236, 'runqemu-dynamic')): - with self.subTest(owner=owner): - port = self._free_port() - if owner == 'runqemu': - self.config.slirp_hostfwd_allowlist = [ - (protocol, '127.0.0.1', port, '', guest_port) - for protocol in ('tcp', 'udp')] - self.config.add_slirp_hostfwds() - elif owner == 'runqemu-dynamic': - self.assertEqual(self._startup_forward(port, guest_port=guest_port), port) - else: - self.assertEqual(self.runner.add_hostfwd(guest_port, port), port) - self.assertEqual(self.runner.get_host_port(guest_port), port) - if owner != 'runqemu': - self.assertEqual(self.runner.add_hostfwd(guest_port, port, protocol='udp'), port) - self.assertEqual(self.runner.get_host_port(guest_port, protocol='UDP'), port) + def test_dynamic_legacy_reservation(self): + """Reject explicit legacy-reserved ports and skip them during allocation.""" + port = self._occupied_port() + self._lock(port) + self.assertIsNone(self.runner.add_hostfwd(1234, port)) + with patch('oeqa.utils.qemurunner.socket.socket') as probe, \ + patch.object(self.runner, 'run_monitor') as monitor: + probe.return_value.getsockname.return_value = ('127.0.0.1', port) + with self.assertRaises(RuntimeError): + self.runner.add_hostfwd(1234, protocol='udp') + for call in monitor.call_args_list: + self.assertNotIn('hostfwd_add', call.args[1]['command-line']) def test_dynamic_rejected_candidates(self): - """Reject dynamic forwards when binding or monitor commands fail.""" + """Release dynamic port locks when binding or monitor commands fail.""" for protocol in ('tcp', 'udp'): with self.subTest(protocol=protocol): busy = self._socket(protocol) port = busy.getsockname()[1] self.assertIsNone(self.runner.add_hostfwd(1234, port, protocol=protocol)) - port = self._free_port() + self._lock(port) + port = self._occupied_port() for failure in (None, {}, {'error': {}}, {'return': '', 'error': {}}, {'return': 'Could not set up host forwarding rule'}, {'return': 0}): with self.subTest(response=failure): def reply(command, arguments=None): if arguments and arguments['command-line'].startswith('hostfwd_add'): + self._assert_locked(port) return failure return self._monitor(command, arguments) with patch.object(self.runner, 'run_monitor', side_effect=reply): self.assertIsNone(self.runner.add_hostfwd(1234, port)) + self._lock(port).close() + with patch.object(self.runner, 'run_monitor', side_effect=[{'return': ''}, RuntimeError('monitor disconnected')]): + with self.assertRaises(RuntimeError): + self.runner.add_hostfwd(1234, port) + self._lock(port) + + def test_lock_cleanup(self): + """Release startup and dynamic port locks after QEMU and its clients stop.""" + proposed = self._occupied_port() + startup_port = self._startup_forward(proposed) + self.assertNotEqual(startup_port, proposed) + dynamic_port = self.runner.add_hostfwd(4321, protocol='udp') + self._assert_locked(startup_port) + self._assert_locked(dynamic_port) + self._stop_qemu() + self.runner.stop() + self.config.release_lock() + self._lock(startup_port) + self._lock(dynamic_port) + + def test_locks_survive_client_exit(self): + """Keep port locks in QEMU after runqemu and the runner close their copies. + + Closing the clients' lock descriptors and monitor connection must not + release either forward's lock or UDP socket while QEMU is alive. + Both locks must become available once QEMU exits. + """ + proposed = self._occupied_port('udp') + startup_port = self._startup_forward(proposed, protocol='udp') + self.assertNotEqual(startup_port, proposed) + dynamic_port = self.runner.add_hostfwd(4321, protocol='udp') + # TODO: legacy-slirp-locks: simulate client death. + self.config.release_lock() + for lock in self.runner._slirp_port_locks.values(): + lock.close() + self.runner._slirp_port_locks.clear() + self.runner.qmp.close() + self.runner.qmp = None + self.assertIsNone(self.process.poll()) + for port in (startup_port, dynamic_port): + self._assert_locked(port) + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as probe: + with self.assertRaises(OSError): + probe.bind(('127.0.0.1', port)) + self._stop_qemu() + self._lock(startup_port) + self._lock(dynamic_port) def test_internal_monitor_timeouts(self): - self.config.SLIRP_QMP_TIMEOUT = 0.1 + """Report stalled QMP exchanges and release the rejected forward's lock.""" + self.config.SLIRP_QMP_TIMEOUT = 1 for phase in ('greeting', 'capabilities', 'hostfwd'): with self.subTest(phase=phase), tempfile.TemporaryDirectory() as directory: - port = self._free_port() + port = self._occupied_port() self.config.internal_qmp_path = os.path.join(directory, 'silent.sock') self.config.slirp_hostfwd_allowlist = [('tcp', '127.0.0.1', port, '', 1234)] with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: @@ -154,7 +267,7 @@ class RunqemuSlirpTests(OESelftestTestCase): with ThreadPoolExecutor(max_workers=1) as executor: future = executor.submit(self.config.add_slirp_hostfwds) with listener.accept()[0] as peer: - peer.settimeout(2) + peer.settimeout(5) with peer.makefile('rw') as stream: if phase != 'greeting': stream.write(json.dumps({'QMP': {}}) + '\n') @@ -165,7 +278,9 @@ class RunqemuSlirpTests(OESelftestTestCase): stream.flush() self.assertEqual(json.loads(stream.readline())['execute'], 'human-monitor-command') with self.assertRaisesRegex(self.runqemu_error, 'Timed out communicating'): - future.result(timeout=2) + future.result(timeout=5) + if phase == 'hostfwd': + self._lock(port) def test_private_internal_monitor(self): """Add a private forwarding monitor without replacing the requested monitor. @@ -215,24 +330,24 @@ class RunqemuSlirpTests(OESelftestTestCase): """Preserve explicit bind addresses and resolve wildcard binds to localhost.""" for guest_port, host, expected in ((1234, '127.0.0.2', '127.0.0.2'), (1235, '0.0.0.0', '127.0.0.1')): with self.subTest(host=host): - actual = self._startup_forward(self._free_port(), guest_port=guest_port, host=host) + actual = self._startup_forward(self._occupied_port(), guest_port=guest_port, host=host) endpoint = self.runner._get_slirp_host_endpoint(guest_port) self.assertEqual(endpoint, (expected, actual)) with socket.create_connection(endpoint, timeout=5): pass def test_unsupported_protocol(self): - """Reject unsupported forwarding protocols.""" + """Reject unsupported forwarding protocols without acquiring port locks.""" with self.assertRaises(ValueError): self.runner.add_hostfwd(1234, protocol='sctp') + self.assertFalse(self.runner._slirp_port_locks) def test_explicit_forward_with_non_slirp_primary(self): """Manage slirp forwards independently of the primary networking mode.""" self.runner.use_slirp = False for protocol in ('tcp', 'udp'): with self.subTest(protocol=protocol): - port = self._free_port() - self.assertEqual(self.runner.add_hostfwd(1234, port, protocol=protocol), port) + port = self.runner.add_hostfwd(1234, protocol=protocol) self.assertEqual(self.runner.slirp_port_mappings[(protocol, 1234)], port) self.assertEqual(self.runner.get_host_port(1234, protocol=protocol), port) port = self.runner.get_host_port(1236) @@ -241,8 +356,9 @@ class RunqemuSlirpTests(OESelftestTestCase): pass response = self.runner.run_monitor('netdev_del', {'id': 'net0'}) self.assertEqual(response, {'return': {}}) - port = self._free_port() + port = self._occupied_port() self.assertIsNone(self.runner.add_hostfwd(1235, port)) + self._lock(port) with self.assertRaisesRegex(RuntimeError, 'Failed to add a slirp tcp hostfwd'): self.runner.get_host_port(1235) diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py index 81434dc117..edacb73330 100644 --- a/meta/lib/oeqa/utils/qemurunner.py +++ b/meta/lib/oeqa/utils/qemurunner.py @@ -16,6 +16,7 @@ import re import socket import select import errno +import fcntl import string import threading import codecs @@ -43,6 +44,7 @@ def getOutput(o): return "" class QemuRunner: + PORT_LOCK_DIR = '/tmp/qemu-port-locks' def __init__(self, machine, rootfs, display, tmpdir, deploy_dir_image, logfile, boottime, dump_dir, use_kvm, logger, use_slirp=False, serial_ports=2, boot_patterns = defaultdict(str), use_ovmf=False, workdir=None, tmpfsdir=None, native_sysroot=None): @@ -78,6 +80,8 @@ class QemuRunner: self.tmpfsdir = tmpfsdir self.native_sysroot = native_sysroot self.kernel_messages_disabled = False + # TODO: legacy-slirp-locks + self._slirp_port_locks = {} self.runqemutime = 300 if not workdir: @@ -590,6 +594,10 @@ class QemuRunner: self.runqemu.stdout.close() self.runqemu_exited = True + # TODO: legacy-slirp-locks + for lock in self._slirp_port_locks.values(): + lock.close() + self._slirp_port_locks.clear() if hasattr(self, 'qmp') and self.qmp: self.qmp.close() self.qmp = None @@ -710,21 +718,53 @@ class QemuRunner: raise ValueError("Unsupported slirp host forwarding protocol: %s" % protocol) candidates = [host_port] if host_port else [] if not host_port: - for _ in range(10): - with socket.socket(socket.AF_INET, socket_type) as s: - s.bind(('127.0.0.1', 0)) - candidates.append(s.getsockname()[1]) + # Hold every probe open so the candidates are distinct. + probes = [] + try: + for _ in range(10): + probes.append(socket.socket(socket.AF_INET, socket_type)) + probes[-1].bind(('127.0.0.1', 0)) + candidates.append(probes[-1].getsockname()[1]) + finally: + for probe in probes: + probe.close() + forwarded_ports = set(self.slirp_port_mappings.values()) for candidate in candidates: - result = self.run_monitor('human-monitor-command', { - 'command-line': 'hostfwd_add %s:127.0.0.1:%d-:%d' % (protocol, candidate, target_port)}) - output = result.get('return') if isinstance(result, dict) else None - # hostfwd_add prints nothing on success; on failure it prints a message like - # "Could not set up host forwarding rule ...". Check for empty output to determine success. - if isinstance(result, dict) and 'error' not in result and isinstance(output, str) and output.strip() == '': - self.logger.debug("Added %s hostfwd for target port %d on host port %d" - % (protocol, target_port, candidate)) - return candidate - self.logger.debug("hostfwd_add for %s host port %d failed: %s" % (protocol, candidate, result)) + lock = None + if candidate not in self._slirp_port_locks and candidate not in forwarded_ports: + # TODO: legacy-slirp-locks: wrynose and blacksail still lock ports. + lockdir = self.PORT_LOCK_DIR + os.makedirs(lockdir, exist_ok=True) + lock = open(os.path.join(lockdir, '%d.lock' % candidate), 'w') + try: + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError as error: + lock.close() + if error.errno not in (errno.EACCES, errno.EAGAIN): + raise + continue + accepted = False + try: + result = self.run_monitor('human-monitor-command', { + 'command-line': 'hostfwd_add %s:127.0.0.1:%d-:%d' % (protocol, candidate, target_port)}) + output = result.get('return') if isinstance(result, dict) else None + accepted = isinstance(result, dict) and 'error' not in result and isinstance(output, str) and output.strip() == '' + if accepted: + if lock: + self._slirp_port_locks[candidate] = lock + # TODO: legacy-slirp-locks: QEMU keeps the lock if we die. + self.qmp.send_fd_scm(lock.fileno()) + response = self.run_monitor('getfd', { + 'fdname': 'legacy-slirp-lock-%d' % candidate}) + if not isinstance(response, dict) or 'error' in response or response.get('return') != {}: + raise RuntimeError("QEMU could not retain port lock: %s" % response) + self.logger.debug("Added %s hostfwd for target port %d on host port %d" + % (protocol, target_port, candidate)) + return candidate + self.logger.debug("hostfwd_add for %s host port %d failed: %s" % (protocol, candidate, result)) + finally: + if lock and not accepted: + lock.close() if host_port: return None raise RuntimeError( diff --git a/scripts/runqemu b/scripts/runqemu index 0e0c77076c..5f7f72aba9 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -21,6 +21,7 @@ import signal import time import json import tempfile +import array import shlex import socket import traceback @@ -1225,6 +1226,9 @@ to your build configuration. if 'error' in response or response.get('return') != {}: raise RunQemuError("Internal QMP negotiation failed: %s" % response) + # TODO: legacy-slirp-locks: wrynose and blacksail still lock ports. + lockdir = self.PORT_LOCK_DIR + self.make_lock_dir(lockdir) used_ports = set() hostfwd_added = [] for proto, hostip, hostport, guestip, targetport in self.slirp_hostfwd_allowlist: @@ -1232,17 +1236,36 @@ to your build configuration. for _ in range(100): while (proto, candidate) in used_ports: candidate += 1 + lockfile = os.path.join(lockdir, '%d.lock' % candidate) + new_lock = lockfile not in self.locks + if new_lock and not self.acquire_lock(lockfile): + candidate += 1 + continue cmdline = 'hostfwd_add %s:%s:%d-%s:%d' % (proto, hostip, candidate, guestip, targetport) - sockf.write(json.dumps({"execute": "human-monitor-command", - "arguments": {"command-line": cmdline}}) + '\n') - sockf.flush() - response = read_response() - # hostfwd_add prints nothing on success; on failure it - # prints a message like "Could not set up host forwarding - # rule ..." which does NOT contain the word "error", so - # check for empty output rather than absence of that word. - result = response.get('return') - if 'error' not in response and isinstance(result, str) and result.strip() == '': + accepted = False + try: + sockf.write(json.dumps({"execute": "human-monitor-command", + "arguments": {"command-line": cmdline}}) + '\n') + sockf.flush() + response = read_response() + # hostfwd_add returns empty output on success, and + # diagnostic text or a QMP error on failure. + result = response.get('return') + accepted = 'error' not in response and isinstance(result, str) and result.strip() == '' + finally: + if new_lock and not accepted: + self.release_lock(lockfile) + if accepted: + # TODO: legacy-slirp-locks: QEMU keeps the lock if runqemu dies. + if new_lock: + sock.sendmsg([b' '], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, + array.array('i', [self.locks[lockfile].fileno()]))]) + sockf.write(json.dumps({"execute": "getfd", "arguments": { + "fdname": "legacy-slirp-lock-%d" % candidate}}) + '\n') + sockf.flush() + response = read_response() + if 'error' in response or response.get('return') != {}: + raise RunQemuError("QEMU could not retain port lock: %s" % response) used_ports.add((proto, candidate)) hostfwd_added.append('hostfwd=%s:%s:%d-%s:%d' % (proto, hostip, candidate, guestip, targetport)) if candidate != hostport: