From patchwork Wed Dec 21 14:15:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1C10C4332F for ; Wed, 21 Dec 2022 14:15:53 +0000 (UTC) Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by mx.groups.io with SMTP id smtpd.web11.19439.1671632146372771116 for ; Wed, 21 Dec 2022 06:15:46 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=ektT0hel; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.49, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f49.google.com with SMTP id m14so15066097wrh.7 for ; Wed, 21 Dec 2022 06:15:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=rjUR8wW045Dmbgzbvu2jqdBO9AgPEXWTs4vUQPlmhSM=; b=ektT0helOLdNMDCdjlwm507oqAfShS9m7xQpQ2PBd4jSKHqMU8sltPUPtOJiwx52CE 65uwgOH3KRvEPV5ZLJVHtFZFapkd0R+eEXXhvduAOGf7rgqtUND7ZPITJ5n8TtL/eEJs Bg66phaiqIsJXImQAyVgtYz2BXdbjbdZNlzdI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rjUR8wW045Dmbgzbvu2jqdBO9AgPEXWTs4vUQPlmhSM=; b=NIIvsbEcrU6FtOr/DCZKXwwygTeL9jvhFRsLwGyFN92y2+1yx2vcXPCwqIhmwqdJce NtbmBaSGI5XuStwumTooJXJm7QVo3I3x4OfE23kNtBNbkMmKda5lWmdbPBRdZUq5kXWJ DfWs2rf1scxIIM31HrJsRM4zmVu1gMfpxDU9KPdeMZRCVKFHoeDjgqTHMR9LMk/XZtg9 85xL/uRcmI8Yy2rUP4xiCh1r5VsNkGhP+Zs1zjH9r2LDx9Cptfpo32LhbQiI0BKUwVvB YtNhuAefL1zX7RkRJTDsz7UmcgTZZ/oB1JlovKOjxgUMYtW54HaEDTxTo4GnL//q2ZC9 djdA== X-Gm-Message-State: AFqh2kpJFwOQDJFREtmdVVyFPyotbPQuLBi+9fCHzQ3Wdukz9AjrsyN8 c1KUp7EhwWgJU5RN3GLmhjZRKIBG0lz42gni X-Google-Smtp-Source: AMrXdXsdrZTU/2POaO/dzRYHZ6VSDdFGuMa9VJi7ZW2LWIBL3wBF9QGHB2dFJycsPKi0yBHznHqLyQ== X-Received: by 2002:a5d:4cd2:0:b0:242:2445:4a42 with SMTP id c18-20020a5d4cd2000000b0024224454a42mr1195957wrt.9.1671632144458; Wed, 21 Dec 2022 06:15:44 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:43 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 1/7] knotty: Ping the server/cooker periodically Date: Wed, 21 Dec 2022 14:15:37 +0000 Message-Id: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14218 We've seeing failures where the UI hangs if the server disappears. Ping the cooker/server if we've not had any events in the last minute so we can check if it is still alive. Signed-off-by: Richard Purdie --- lib/bb/ui/knotty.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py index 61cf0a37f4..ab1a367be0 100644 --- a/lib/bb/ui/knotty.py +++ b/lib/bb/ui/knotty.py @@ -625,7 +625,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): printintervaldelta = 10 * 60 # 10 minutes printinterval = printintervaldelta - lastprint = time.time() + pinginterval = 1 * 60 # 1 minutes + lastevent = lastprint = time.time() termfilter = tf(main, helper, console_handlers, params.options.quiet) atexit.register(termfilter.finish) @@ -637,6 +638,14 @@ def main(server, eventHandler, params, tf = TerminalFilter): printinterval += printintervaldelta event = eventHandler.waitEvent(0) if event is None: + if (lastevent + pinginterval) <= time.time(): + ret, error = server.runCommand(["ping"]) + if error or not ret: + termfilter.clearFooter() + print("No reply after pinging server (%s, %s), exiting." % (str(error), str(ret))) + return_value = 3 + main.shutdown = 2 + lastevent = time.time() if main.shutdown > 1: break if not parseprogress: @@ -644,6 +653,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): event = eventHandler.waitEvent(0.25) if event is None: continue + lastevent = time.time() helper.eventHandler(event) if isinstance(event, bb.runqueue.runQueueExitWait): if not main.shutdown: From patchwork Wed Dec 21 14:15:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EEB1C3DA7D for ; Wed, 21 Dec 2022 14:15:54 +0000 (UTC) Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by mx.groups.io with SMTP id smtpd.web11.19441.1671632146930597754 for ; Wed, 21 Dec 2022 06:15:47 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=HPNOYUzM; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.43, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f43.google.com with SMTP id y8so2060812wrl.13 for ; Wed, 21 Dec 2022 06:15:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=pN4lANml8++f8XUrrE/hEDZrzvwj32RGT8aPCf00bUM=; b=HPNOYUzMt/JJ6Z3OdFt3TUWS9TB93VEzs8U1ATl+LkU8eP6yuENSoRmVS4JtSxQNVb Ruh1FLHCRocmR72PDVnzg57zScAhBfZ+JKsdFAvC21oaFHCH+2G+gal/4RhQFXmeI+IV bbkEM6zKDHBgp9iaTi5DaG4pXS0kvOYZK2NYg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pN4lANml8++f8XUrrE/hEDZrzvwj32RGT8aPCf00bUM=; b=0uWft732xMEKe6t5+8hl9OP/V733z3PdYz3ETcZG7eYVhCrupj/O5fUrEGNLa27qRQ moyJEprpcknj61HIWIIezynaqR9I4B713UeJVJ1Pe3jcSlDh79N9X7FBp8CaHB3VjSL8 GkGPUUBdWJMRSWsZjDzKHoaD6iYZoJGduaWZPOb7+g1GPvermqE0v5YaQnIryrcXwTBV smwJsG3iQ5r1aOn7i427STGjyvCuGu9o7LbOA/6r0b+JECUSiQpe8NV9WiRoThIbrm64 t6w3HPNoCnuSW+TY8/XbAFeq76fa+sTt8TsmnsI0Sod69me89jLORrKPhKkSCPR1x3Tv IuHw== X-Gm-Message-State: AFqh2ko8MkdU4BLhBuoRCZ7s1ThSQ4WXKjzL9d1eXJcXAmN1jPtF6Ls1 0fh2ap49uGNgmwJFd+E1AqwuARt+9nUltF95 X-Google-Smtp-Source: AMrXdXv2EbPGUika40zXVe2+cJm8P7O+pgc3ldAkei36oW4b/BbTUdRzEmiuRN59RHusmtwGoG0B1Q== X-Received: by 2002:a5d:5408:0:b0:242:51ae:e9ca with SMTP id g8-20020a5d5408000000b0024251aee9camr1241974wrv.35.1671632145018; Wed, 21 Dec 2022 06:15:45 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:44 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 2/7] event: Add enable/disable heartbeat code Date: Wed, 21 Dec 2022 14:15:38 +0000 Message-Id: <20221221141543.497904-2-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> References: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14219 Currently heartbeat events are always generated by the server whilst it is active. Change this so they only appear when builds are running, which is when most code would expect to be executed. This removes a number of races around changes in the datastore which can happen outside of builds. Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 4 ++++ lib/bb/event.py | 9 +++++++++ lib/bb/server/process.py | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 48c3002ce3..815610ff82 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1467,6 +1467,7 @@ class BBCooker: buildname = self.databuilder.mcdata[mc].getVar("BUILDNAME") if fireevents: bb.event.fire(bb.event.BuildStarted(buildname, [item]), self.databuilder.mcdata[mc]) + bb.event.enable_heartbeat() # Execute the runqueue runlist = [[mc, item, task, fn]] @@ -1500,6 +1501,7 @@ class BBCooker: if not retval: if fireevents: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, item, failures, interrupted), self.databuilder.mcdata[mc]) + bb.event.disable_heartbeat() self.command.finishAsyncCommand(msg) # We trashed self.recipecaches above self.parsecache_valid = False @@ -1545,6 +1547,7 @@ class BBCooker: for mc in self.multiconfigs: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, targets, failures, interrupted), self.databuilder.mcdata[mc]) finally: + bb.event.disable_heartbeat() self.command.finishAsyncCommand(msg) return False if retval is True: @@ -1578,6 +1581,7 @@ class BBCooker: for mc in self.multiconfigs: bb.event.fire(bb.event.BuildStarted(buildname, ntargets), self.databuilder.mcdata[mc]) + bb.event.enable_heartbeat() rq = bb.runqueue.RunQueue(self, self.data, self.recipecaches, taskdata, runlist) if 'universe' in targets: diff --git a/lib/bb/event.py b/lib/bb/event.py index 97668601a1..21e9a1b025 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -69,6 +69,7 @@ _eventfilter = None _uiready = False _thread_lock = threading.Lock() _thread_lock_enabled = False +_heartbeat_enabled = False if hasattr(__builtins__, '__setitem__'): builtins = __builtins__ @@ -83,6 +84,14 @@ def disable_threadlock(): global _thread_lock_enabled _thread_lock_enabled = False +def enable_heartbeat(): + global _heartbeat_enabled + _heartbeat_enabled = True + +def disable_heartbeat(): + global _heartbeat_enabled + _heartbeat_enabled = False + def execute_handler(name, handler, event, d): event.data = d addedd = False diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 586d46af88..91eb6e0ad9 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -382,7 +382,7 @@ class ProcessServer(): # Create new heartbeat event? now = time.time() - if now >= self.next_heartbeat: + if bb.event._heartbeat_enabled and now >= self.next_heartbeat: # We might have missed heartbeats. Just trigger once in # that case and continue after the usual delay. self.next_heartbeat += self.heartbeat_seconds @@ -396,7 +396,7 @@ class ProcessServer(): if not isinstance(exc, bb.BBHandledException): logger.exception('Running heartbeat function') self.quit = True - if nextsleep and now + nextsleep > self.next_heartbeat: + if nextsleep and bb.event._heartbeat_enabled and now + nextsleep > self.next_heartbeat: # Shorten timeout so that we we wake up in time for # the heartbeat. nextsleep = self.next_heartbeat - now From patchwork Wed Dec 21 14:15:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 116DEC10F1B for ; Wed, 21 Dec 2022 14:15:54 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web10.19680.1671632147316626987 for ; Wed, 21 Dec 2022 06:15:47 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=TdmZ3eX8; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.47, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f47.google.com with SMTP id y8so2060834wrl.13 for ; Wed, 21 Dec 2022 06:15:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=dHHlILX9B2Dcb3I3IxVcze1ZIeSnGaT/zwJmYdZTp9I=; b=TdmZ3eX84LYdLV7YOOMOKwBvJjAlzqLCN0aTGRlIszfi5lWdxIaG77CisISrBZR8/d 9Yc4XGARvEM27DHrYNF5s+ho8sjxgV4HxhRw6i31OaAc2DVayKAFxzaASFz1d+pEDjPO no9VLfHXTwJiEOBJ4+5X+7G148BABX8shs8kU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dHHlILX9B2Dcb3I3IxVcze1ZIeSnGaT/zwJmYdZTp9I=; b=JJcL/RFo2GFY+8HAY52rqAirfqyA7nVGGtD2RVJowwxCY/E8P9B+zKMAyGWFJ1Bf6T 0aOEMd4sqZElIFwjKVv4alK9y5NWC08fyQbH++6mJIUVW0qAy/cpy06OEJ9roU7CXaN7 MzwdJEUuQaOVEsMklDHE1OSKrDZf5z8mtgpnL2VZHygPjruGMCN4HfoqskBLvrWty45X Ts687dqrGMDpNWKvov0QkqwuODbpfATXX1bUUwPUVyNuSAxUCEpHP48+JPHMTSXKwWsq Qnj0GQiNDJ4AcgFtpCmj/k0nowpcv7Felw8kKxxO3qViB+ol7VHyjy8a5XPcL4yJQdS0 46Lg== X-Gm-Message-State: AFqh2koCk95YGvlgGVQmBeILyHQcl60ND8wxLRNfpmD5/maH/wEZtgbE BAsn29OnSHbJ/ZhBNNN05/VVOqx7jc97a2z2 X-Google-Smtp-Source: AMrXdXs+pNjy8xJ7wDwVjmI5/vDwHdNwP27W0du/dC9X/JhPXDwR6WdwWfc+r4+ben9QElv3opL9nA== X-Received: by 2002:a5d:50c9:0:b0:236:770a:665a with SMTP id f9-20020a5d50c9000000b00236770a665amr1051436wrt.66.1671632145641; Wed, 21 Dec 2022 06:15:45 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:45 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 3/7] event: builtins fix for 'd' deletion Date: Wed, 21 Dec 2022 14:15:39 +0000 Message-Id: <20221221141543.497904-3-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> References: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14220 I've been seeing event handlers where 'd' seems to disappear half way through event handler execution. This is problematic when multiple threads are active since this code assumes single threading. The easiest fix is to change the handler function calls to contain d as a parameter as we do elsewhere for other functions. This will break any non-text handlers but I was only able to spot one of those in runqueue. It will also break handlers than call functions that assume 'd' is in the global namespace but those failures should be obvious and we can fix those to pass d around. This solution avoids manipulating builtins which was always a horrible thing to do anyway and solves the issue without needing locking, thankfully. Signed-off-by: Richard Purdie --- lib/bb/event.py | 18 ++++-------------- lib/bb/runqueue.py | 2 +- lib/bb/tests/color.py | 2 +- lib/bb/tests/event.py | 21 +++++++++++---------- 4 files changed, 17 insertions(+), 26 deletions(-) diff --git a/lib/bb/event.py b/lib/bb/event.py index 21e9a1b025..db90724444 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -71,11 +71,6 @@ _thread_lock = threading.Lock() _thread_lock_enabled = False _heartbeat_enabled = False -if hasattr(__builtins__, '__setitem__'): - builtins = __builtins__ -else: - builtins = __builtins__.__dict__ - def enable_threadlock(): global _thread_lock_enabled _thread_lock_enabled = True @@ -94,12 +89,8 @@ def disable_heartbeat(): def execute_handler(name, handler, event, d): event.data = d - addedd = False - if 'd' not in builtins: - builtins['d'] = d - addedd = True try: - ret = handler(event) + ret = handler(event, d) except (bb.parse.SkipRecipe, bb.BBHandledException): raise except Exception: @@ -113,8 +104,7 @@ def execute_handler(name, handler, event, d): raise finally: del event.data - if addedd: - del builtins['d'] + def fire_class_handlers(event, d): if isinstance(event, logging.LogRecord): @@ -262,12 +252,12 @@ def register(name, handler, mask=None, filename=None, lineno=None, data=None): if handler is not None: # handle string containing python code if isinstance(handler, str): - tmp = "def %s(e):\n%s" % (name, handler) + tmp = "def %s(e, d):\n%s" % (name, handler) try: code = bb.methodpool.compile_cache(tmp) if not code: if filename is None: - filename = "%s(e)" % name + filename = "%s(e, d)" % name code = compile(tmp, filename, "exec", ast.PyCF_ONLY_AST) if lineno is not None: ast.increment_lineno(code, lineno-1) diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py index b9dd830b31..ce711b6252 100644 --- a/lib/bb/runqueue.py +++ b/lib/bb/runqueue.py @@ -1511,7 +1511,7 @@ class RunQueue: if not self.dm_event_handler_registered: res = bb.event.register(self.dm_event_handler_name, - lambda x: self.dm.check(self) if self.state in [runQueueRunning, runQueueCleanUp] else False, + lambda x, y: self.dm.check(self) if self.state in [runQueueRunning, runQueueCleanUp] else False, ('bb.event.HeartbeatEvent',), data=self.cfgData) self.dm_event_handler_registered = True diff --git a/lib/bb/tests/color.py b/lib/bb/tests/color.py index 88dd278006..bb70cb393d 100644 --- a/lib/bb/tests/color.py +++ b/lib/bb/tests/color.py @@ -20,7 +20,7 @@ class ProgressWatcher: def __init__(self): self._reports = [] - def handle_event(self, event): + def handle_event(self, event, d): self._reports.append((event.progress, event.rate)) def reports(self): diff --git a/lib/bb/tests/event.py b/lib/bb/tests/event.py index 9ca7e9bc8e..4de4cced5e 100644 --- a/lib/bb/tests/event.py +++ b/lib/bb/tests/event.py @@ -157,7 +157,7 @@ class EventHandlingTest(unittest.TestCase): self._test_process.event_handler, event, None) - self._test_process.event_handler.assert_called_once_with(event) + self._test_process.event_handler.assert_called_once_with(event, None) def test_fire_class_handlers(self): """ Test fire_class_handlers method """ @@ -175,10 +175,10 @@ class EventHandlingTest(unittest.TestCase): bb.event.fire_class_handlers(event1, None) bb.event.fire_class_handlers(event2, None) bb.event.fire_class_handlers(event2, None) - expected_event_handler1 = [call(event1)] - expected_event_handler2 = [call(event1), - call(event2), - call(event2)] + expected_event_handler1 = [call(event1, None)] + expected_event_handler2 = [call(event1, None), + call(event2, None), + call(event2, None)] self.assertEqual(self._test_process.event_handler1.call_args_list, expected_event_handler1) self.assertEqual(self._test_process.event_handler2.call_args_list, @@ -205,7 +205,7 @@ class EventHandlingTest(unittest.TestCase): bb.event.fire_class_handlers(event2, None) bb.event.fire_class_handlers(event2, None) expected_event_handler1 = [] - expected_event_handler2 = [call(event1)] + expected_event_handler2 = [call(event1, None)] self.assertEqual(self._test_process.event_handler1.call_args_list, expected_event_handler1) self.assertEqual(self._test_process.event_handler2.call_args_list, @@ -223,7 +223,7 @@ class EventHandlingTest(unittest.TestCase): self.assertEqual(result, bb.event.Registered) bb.event.fire_class_handlers(event1, None) bb.event.fire_class_handlers(event2, None) - expected = [call(event1), call(event2)] + expected = [call(event1, None), call(event2, None)] self.assertEqual(self._test_process.event_handler1.call_args_list, expected) @@ -237,7 +237,7 @@ class EventHandlingTest(unittest.TestCase): self.assertEqual(result, bb.event.Registered) bb.event.fire_class_handlers(event1, None) bb.event.fire_class_handlers(event2, None) - expected = [call(event1), call(event2), call(event1)] + expected = [call(event1, None), call(event2, None), call(event1, None)] self.assertEqual(self._test_process.event_handler1.call_args_list, expected) @@ -251,7 +251,7 @@ class EventHandlingTest(unittest.TestCase): self.assertEqual(result, bb.event.Registered) bb.event.fire_class_handlers(event1, None) bb.event.fire_class_handlers(event2, None) - expected = [call(event1), call(event2), call(event1), call(event2)] + expected = [call(event1,None), call(event2, None), call(event1, None), call(event2, None)] self.assertEqual(self._test_process.event_handler1.call_args_list, expected) @@ -359,9 +359,10 @@ class EventHandlingTest(unittest.TestCase): event1 = bb.event.ConfigParsed() bb.event.fire(event1, None) - expected = [call(event1)] + expected = [call(event1, None)] self.assertEqual(self._test_process.event_handler1.call_args_list, expected) + expected = [call(event1)] self.assertEqual(self._test_ui1.event.send.call_args_list, expected) From patchwork Wed Dec 21 14:15:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 140D8C3DA7C for ; Wed, 21 Dec 2022 14:15:54 +0000 (UTC) Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by mx.groups.io with SMTP id smtpd.web11.19442.1671632148136336177 for ; Wed, 21 Dec 2022 06:15:48 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=hmlz2tNw; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.48, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f48.google.com with SMTP id h10so15071805wrx.3 for ; Wed, 21 Dec 2022 06:15:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=758nh+IjW+PDtVxDuFX7FJkJ6XmbtKXSkAdsuA5dm5M=; b=hmlz2tNwjzTJ1AHd5B0EGA/jQLoVOgez7gW2CPAuMf40qI1Q2PxHrB2dSj6v4PWhbP xUXvlGa1GKG5Aiui4fxJf53tvLloag2Q/7mr36Rg4ARnhNf42ACre6eUXQa8OabAItwR xcTsZZqqybC5cj/TntZ6GYB2V8aqNVQ26WIYE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=758nh+IjW+PDtVxDuFX7FJkJ6XmbtKXSkAdsuA5dm5M=; b=EmbqfT9pVP2tk0VbYUkonsk0JSD5X1z9mbG8Q8CvE8NKepf9xqyuv72hRv0TW2ceEO 0qA31Ks2DLqH3gK4GGbRHp7ECVGMZKrg9pMOBKlcxcsUf0HFdn0iMg7Xd32aNX4ILu9Z TGX69Kiy+4Zpvkz93jxtsX0ucYwOg1trQdB2KKeUzvEeHQE/o+eQd5sRP8E0cZ1Dhair QYd0Jy1otwn7powLzmZ+aNIvGLxRY8t344XLL3dXXkgFfFrIArOKyX0JwDdgJQHvG4W5 T1C3+EhuLdBT4tNfnXe/O0Dl8i8L9xkrnvqbg9dls34e9XI4uQ0MxA+D+varPlbPs+id hrTA== X-Gm-Message-State: AFqh2kqYnrWtx1yU6nhJvm2DPPgXyyhqE5gjVygsv5eswPJQNIxZULXo KIF3WO9eYreGOEhDDn1Ygl1r1vpQrgizZwS3 X-Google-Smtp-Source: AMrXdXu2MlammNyWkF5XRnhag3OIxWoAP4L1/Lz/xozLRxR1io+1bOQinnOhvs+9arsAlcDK9yZUPw== X-Received: by 2002:a5d:43c7:0:b0:236:50d4:c0a6 with SMTP id v7-20020a5d43c7000000b0023650d4c0a6mr1114135wrr.47.1671632146284; Wed, 21 Dec 2022 06:15:46 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:45 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 4/7] server/process: Run idle commands in a separate idle thread Date: Wed, 21 Dec 2022 14:15:40 +0000 Message-Id: <20221221141543.497904-4-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> References: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14221 When bitbake is off running heavier "idle" commands, it doesn't service it's command socket which means stopping/interrupting it is hard. It also means we can't "ping" from the UI to know if it is still alive. For those reasons, split idle command execution into it's own thread. The commands are generally already self containted so this is easier than expected. We do have to be careful to only handle inotify poll() from a single thread at a time. It also means we always have to use a thread lock when sending events since both the idle thread and the command thread may generate log messages (and hence events). The patch does depend on a couple of previous fixes to the builtins locking in event.py and the heartbeat enable/disable changes. Signed-off-by: Richard Purdie --- lib/bb/command.py | 4 +-- lib/bb/cooker.py | 19 +++++++++---- lib/bb/server/process.py | 60 +++++++++++++++++++++++++--------------- 3 files changed, 54 insertions(+), 29 deletions(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index 0208e30ec4..20a8b86653 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -84,7 +84,7 @@ class Command: if not hasattr(command_method, 'readonly') or not getattr(command_method, 'readonly'): return None, "Not able to execute not readonly commands in readonly mode" try: - self.cooker.process_inotify_updates() + self.cooker.process_inotify_updates_apply() if getattr(command_method, 'needconfig', True): self.cooker.updateCacheSync() result = command_method(self, commandline) @@ -109,7 +109,7 @@ class Command: def runAsyncCommand(self): try: - self.cooker.process_inotify_updates() + self.cooker.process_inotify_updates_apply() if self.cooker.state in (bb.cooker.state.error, bb.cooker.state.shutdown, bb.cooker.state.forceshutdown): # updateCache will trigger a shutdown of the parser # and then raise BBHandledException triggering an exit diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 815610ff82..1daa587ac5 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -220,6 +220,8 @@ class BBCooker: bb.debug(1, "BBCooker startup complete %s" % time.time()) sys.stdout.flush() + self.inotify_threadlock = threading.Lock() + def init_configdata(self): if not hasattr(self, "data"): self.initConfigurationData() @@ -248,11 +250,18 @@ class BBCooker: self.notifier = pyinotify.Notifier(self.watcher, self.notifications) def process_inotify_updates(self): - for n in [self.confignotifier, self.notifier]: - if n and n.check_events(timeout=0): - # read notified events and enqueue them - n.read_events() - n.process_events() + with self.inotify_threadlock: + for n in [self.confignotifier, self.notifier]: + if n and n.check_events(timeout=0): + # read notified events and enqueue them + n.read_events() + + def process_inotify_updates_apply(self): + with self.inotify_threadlock: + for n in [self.confignotifier, self.notifier]: + if n and n.check_events(timeout=0): + n.read_events() + n.process_events() def config_notifications(self, event): if event.maskname == "IN_Q_OVERFLOW": diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 91eb6e0ad9..6f43330fae 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -88,6 +88,7 @@ class ProcessServer(): self.maxuiwait = 30 self.xmlrpc = False + self.idle = None self._idlefuns = {} self.bitbake_lock = lock @@ -148,6 +149,7 @@ class ProcessServer(): self.cooker.pre_serve() bb.utils.set_process_name("Cooker") + bb.event.enable_threadlock() ready = [] newconnections = [] @@ -278,6 +280,9 @@ class ProcessServer(): ready = self.idle_commands(.1, fds) + if self.idle: + self.idle.join() + serverlog("Exiting (socket: %s)" % os.path.exists(self.sockname)) # Remove the socket file so we don't get any more connections to avoid races # The build directory could have been renamed so if the file isn't the one we created @@ -352,33 +357,44 @@ class ProcessServer(): msg.append(":\n%s" % procs) serverlog("".join(msg)) + def idle_thread(self): + while not self.quit: + nextsleep = 0.1 + fds = [] + for function, data in list(self._idlefuns.items()): + try: + retval = function(self, data, False) + if retval is False: + del self._idlefuns[function] + nextsleep = None + elif retval is True: + nextsleep = None + elif isinstance(retval, float) and nextsleep: + if (retval < nextsleep): + nextsleep = retval + elif nextsleep is None: + continue + else: + fds = fds + retval + except SystemExit: + raise + except Exception as exc: + if not isinstance(exc, bb.BBHandledException): + logger.exception('Running idle function') + del self._idlefuns[function] + self.quit = True + + if nextsleep is not None: + select.select(fds,[],[],nextsleep)[0] + def idle_commands(self, delay, fds=None): nextsleep = delay if not fds: fds = [] - for function, data in list(self._idlefuns.items()): - try: - retval = function(self, data, False) - if retval is False: - del self._idlefuns[function] - nextsleep = None - elif retval is True: - nextsleep = None - elif isinstance(retval, float) and nextsleep: - if (retval < nextsleep): - nextsleep = retval - elif nextsleep is None: - continue - else: - fds = fds + retval - except SystemExit: - raise - except Exception as exc: - if not isinstance(exc, bb.BBHandledException): - logger.exception('Running idle function') - del self._idlefuns[function] - self.quit = True + if not self.idle: + self.idle = threading.Thread(target=self.idle_thread) + self.idle.start() # Create new heartbeat event? now = time.time() From patchwork Wed Dec 21 14:15:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C13AC41535 for ; Wed, 21 Dec 2022 14:15:54 +0000 (UTC) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by mx.groups.io with SMTP id smtpd.web10.19681.1671632148567937802 for ; Wed, 21 Dec 2022 06:15:48 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=LnGzkqeL; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.42, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f42.google.com with SMTP id a17so13913352wrt.11 for ; Wed, 21 Dec 2022 06:15:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=poTDGlrbkqrHd+BGneklbkqOdwueYFkr4t0JdGPki4E=; b=LnGzkqeL0FRL3bc02n1L5pX+ayBHyg3HOPQy9JC4rorK4Zl6CpQt7i//tqG55W/iQO fC0IPTFt3XClTpYGToO0jgtVLP4hqJEtt0yKF5UidN3dzoIM5N9tAKRfFj6KvpCn6z4e bFtCQHGq0pYQhEjdYl1GEf1IxjKqfkIitw8fM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=poTDGlrbkqrHd+BGneklbkqOdwueYFkr4t0JdGPki4E=; b=gnBbhf2OiRAsBH89a1lTx4dQI6O5KQhf23eQ9PUtp349Ncw1ugDR4N/HAKm/Ji0cgI y71vXUXBTkHxQiMcgRvLyain0Hq1cD+d4ifNB1BUnxE+1ZgkmaqlWMGggcQDMKyG4cPl IK9bc9wgORns7PSVvdTlQ7yvpc7m9szkZunJhC6MpqWd8qimIMrJNRiN9ZSW7UfgJxr6 eVpB+Ana3LGCv8IFxovFA0uVBRfxrCmkzeqjNNkgCJeo1wpKyWGAbvYdxEBSl3yCIqLA LhDNI+kVszWIfFfjB6AtDPbyQLY9MYSy8GFE/5CJZv+fhvZK9XJkpuI/RwSO5n5wl3hk Qa7Q== X-Gm-Message-State: AFqh2kpQIALx4WBT+rkx6WiPelCoEXK5yQNHIkrW+D36520BbeLubEEs NB4QRxkeH7LZTYEUbVW9zlvPq5nIKvPAvLZe X-Google-Smtp-Source: AMrXdXvE9vimjMp6v473+sG1w9euDzHOGYo7QjYfkBVFtJBLlS1n/EZ0PGt348/esuc5L47/LCpzpQ== X-Received: by 2002:a5d:5a85:0:b0:264:ae8b:7dcd with SMTP id bp5-20020a5d5a85000000b00264ae8b7dcdmr1318105wrb.24.1671632146844; Wed, 21 Dec 2022 06:15:46 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:46 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 5/7] server/process: Avoid dangling async commands Date: Wed, 21 Dec 2022 14:15:41 +0000 Message-Id: <20221221141543.497904-5-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> References: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14222 There is a race where the client can have disconnected but the async command hasn't finished running and isn't cleared up by the idle handler. If a client connects and finds this state, it will fail as the server appears busy and no new async command can execute. The old idle handler code makes this unlikely but could explain some race issues. The new code with idle handling in a separate thread makes it much more likely. Signed-off-by: Richard Purdie --- lib/bb/command.py | 3 +++ lib/bb/server/process.py | 1 + 2 files changed, 4 insertions(+) diff --git a/lib/bb/command.py b/lib/bb/command.py index 20a8b86653..1224eee413 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -152,6 +152,9 @@ class Command: bb.event.fire(CommandExit(code), self.cooker.data) else: bb.event.fire(CommandCompleted(), self.cooker.data) + self.finishAsyncDisconnect() + + def finishAsyncDisconnect(self): self.currentAsyncCommand = None self.cooker.finishcommand() diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 6f43330fae..c9984539e8 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -177,6 +177,7 @@ class ProcessServer(): self.command_channel = False del self.event_writer self.lastui = time.time() + self.cooker.command.finishAsyncDisconnect() self.cooker.clientComplete() self.haveui = False ready = select.select(fds,[],[],0)[0] From patchwork Wed Dec 21 14:15:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17074 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03CC8C3DA7B for ; Wed, 21 Dec 2022 14:15:54 +0000 (UTC) Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by mx.groups.io with SMTP id smtpd.web10.19682.1671632149060965338 for ; Wed, 21 Dec 2022 06:15:49 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=gY5qrw4B; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.44, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f44.google.com with SMTP id h16so15030739wrz.12 for ; Wed, 21 Dec 2022 06:15:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=39OlN7u74S++Nw0/Z5+rVwMlQgmgdTPicMxTDzI/Y/g=; b=gY5qrw4B87phdwu/36Wtd3Mt7beBEMoGBYYziC0gIpgBWZrJ7UmrSXCNTOCBHu6lmD mAWS5of0o/M4A7s5tP2d3CypdfZiRfLp4TKBP1c0EUKS9BN2CODO2VmbOeUHzhMcRyCE KcS/LrJPQ6xibNGopwJgIybZJMMDvrguYiFP4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=39OlN7u74S++Nw0/Z5+rVwMlQgmgdTPicMxTDzI/Y/g=; b=ZSXzDfJi9M/JAT+Sb5rrfDNJHIBEsLMcWIhyMuH4bcqTksXCtu3wGtobzjoMSbxeSq KUk4sRIqEBeQRnt8WgcaYTLJ9OXt3ob6ev0ZfLCIZsGHN6xsnuZD+jG3myPuwRhMmVhv 8mA2+H1m96FO9cfx+jeOxUu87Miuolw2Nr2DL1/psxsr57mbR6u2Qbvggai2uRE5SKiZ kO8wPrHzx5mkrJA9lQgoZ1kj4Bbz7Fs4sKEhRqCAUcjTJu/mQF0bYn8vU8PAi/smkLnt /tOtw6cFIRPGk5mD/gzkSkmqMNQKP0fdETFry5u/40S2O7dqinISk1Z3HD823hB+5Aej mnVw== X-Gm-Message-State: AFqh2kqkWuMvXlkUV3WjzVxbt+v7fG697i1PZ1xreeIG/pCG2mZj502W IwI9RtqATOpidNnWRV7Mck+2YNrWuY95cLYU X-Google-Smtp-Source: AMrXdXv7dPTGE62McYLs8efN1K6IU+U0XOeFLqFfjkGeKk9uukgZGuO26TmIIwCHkO4Lhx0upGvL4A== X-Received: by 2002:a5d:5d11:0:b0:26c:aa9d:93f1 with SMTP id ch17-20020a5d5d11000000b0026caa9d93f1mr1123852wrb.25.1671632147419; Wed, 21 Dec 2022 06:15:47 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:47 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 6/7] process: Improve client disconnect/idle sync Date: Wed, 21 Dec 2022 14:15:42 +0000 Message-Id: <20221221141543.497904-6-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> References: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14223 When clients are disconnecting, we need to ensure things are in sync with the idle thread. This patch is probably a little paranoid but it doesn't really hurt to ensure things are properly in sync and nothing unexpected will execute. A simple counter to show activity seems to easiest way to synchronise. Signed-off-by: Richard Purdie --- lib/bb/server/process.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index c9984539e8..12dfb6ea19 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -88,6 +88,7 @@ class ProcessServer(): self.maxuiwait = 30 self.xmlrpc = False + self.idle_count = 1 self.idle = None self._idlefuns = {} @@ -178,6 +179,13 @@ class ProcessServer(): del self.event_writer self.lastui = time.time() self.cooker.command.finishAsyncDisconnect() + + # Wait for the idle loop to have executed (5s max) + count = self.idle_count + start = time.time() + while count == self.idle_count and time.time() < (start + 5): + continue + self.cooker.clientComplete() self.haveui = False ready = select.select(fds,[],[],0)[0] @@ -360,6 +368,11 @@ class ProcessServer(): def idle_thread(self): while not self.quit: + # Show activity + if self.idle_count > 100000000: + self.idle_count = 1 + self.idle_count = self.idle_count + 1 + nextsleep = 0.1 fds = [] for function, data in list(self._idlefuns.items()): From patchwork Wed Dec 21 14:15:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01DB9C4167B for ; Wed, 21 Dec 2022 14:15:54 +0000 (UTC) Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.web10.19683.1671632149630816611 for ; Wed, 21 Dec 2022 06:15:49 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=KlnGJVFg; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.45, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f45.google.com with SMTP id f18so15087055wrj.5 for ; Wed, 21 Dec 2022 06:15:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Nvlhv/xI3RCqmomtv1v37EcS/G6BUEAe5DWKoRCliPo=; b=KlnGJVFgtk5zhv8jQLQSHxVjgf7JVPdIzD0l78fyw/x1rLQui+/m0QCCCbhdEfJzlI LoS/KzwB/IVkBXPzwxuO3DZiD4sGZO5y0+oFxhAaatJVP+JyImfK1jDvL0qoTSjSmkws tgd6A+drqUn9QboxSmeaxQB2MP6z/jVnN0g/A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nvlhv/xI3RCqmomtv1v37EcS/G6BUEAe5DWKoRCliPo=; b=B0Es5MtOpDulCuAQX9igpXOrCgzG3NnVaN1z3VSizyPwdR9LzVi5NpQpgCDc4wgZAs aWuk4+IFPo/vTFRtI2aLvU6CykoLZZoOVW3BTs5YQE9AIjQJFzYxTqnIhUQTiMacc4H2 MdrYy969MOEG7i9OZHHIpgol2mbuA7nOjMBJdQpbfrhi8T92LjUmkx0UYYScj6YiUdW5 PDfPIJrkcCMIobrPdVMXbn0/+p7rFMEcSVMCXHu4nXawPjKNpi6xKRXqQufyehVWppwH iYb9pKfUNIUEVwXRPtfEyDyQm91V1TQc6z5Hu+Tj3O+WmfChEhT1aGhxNTSLiEMbLbtX PyYg== X-Gm-Message-State: AFqh2kr+XiW79uWAu5rzAum0DPadpuYrnVSkybULBN47eSu/cIWHCiZl 95MZzVpF2PMlwkv9l3MNWrl88MIN193KaRi1 X-Google-Smtp-Source: AMrXdXsDg/aOa/bNlxEWMzTCdIeZMxEUfnpKdUSHcAhkGYoG3DEfb7pqfxF89i3OHu0Eu6Q9NkDs0A== X-Received: by 2002:a5d:5511:0:b0:269:e5d7:e11f with SMTP id b17-20020a5d5511000000b00269e5d7e11fmr1443019wrv.38.1671632147999; Wed, 21 Dec 2022 06:15:47 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e749:b020:2cdb:af31]) by smtp.gmail.com with ESMTPSA id k12-20020adff28c000000b0022e57e66824sm17839412wro.99.2022.12.21.06.15.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Dec 2022 06:15:47 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 7/7] knotty: Avoid looping with tracebacks Date: Wed, 21 Dec 2022 14:15:43 +0000 Message-Id: <20221221141543.497904-7-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> References: <20221221141543.497904-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 Dec 2022 14:15:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14224 If there are events queued and there is an exception in the main loop of the UI code, it will print tracebacks on the console indefinitely. Avoid that by improving the loop exit conditions. Signed-off-by: Richard Purdie --- lib/bb/ui/knotty.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py index ab1a367be0..c398f591e8 100644 --- a/lib/bb/ui/knotty.py +++ b/lib/bb/ui/knotty.py @@ -631,7 +631,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): termfilter = tf(main, helper, console_handlers, params.options.quiet) atexit.register(termfilter.finish) - while True: + while main.shutdown < 2: try: if (lastprint + printinterval) <= time.time(): termfilter.keepAlive(printinterval) @@ -646,8 +646,6 @@ def main(server, eventHandler, params, tf = TerminalFilter): return_value = 3 main.shutdown = 2 lastevent = time.time() - if main.shutdown > 1: - break if not parseprogress: termfilter.updateFooter() event = eventHandler.waitEvent(0.25)