From patchwork Thu Dec 29 17:07:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41CA6C4332F for ; Thu, 29 Dec 2022 17:07:33 +0000 (UTC) Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by mx.groups.io with SMTP id smtpd.web11.209992.1672333652428022454 for ; Thu, 29 Dec 2022 09:07:32 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=T+VQfoC0; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.49, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f49.google.com with SMTP id w1so5778313wrt.8 for ; Thu, 29 Dec 2022 09:07:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Touq52SotAOn0qhLlACHQaacjfMd05+VLjI7h38+LSw=; b=T+VQfoC0qLRGqH3/RCakbopZDQamLpgL4uaBoi8OyxPOXYhf5/dq6O4N/JgM8Ex4d8 dQXX+tQcu93espZLew/wHqG+4h5y5rUd9f+/u4Tz2c36bp35fWw6FvsNk9CFrWwJezPr +UjjS9bFdGJIlGlCpwAN9TVAQG1J1vLtEPvbM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Touq52SotAOn0qhLlACHQaacjfMd05+VLjI7h38+LSw=; b=YBf0EC55vGObRhBNeYLY8ezFrjMPrXgnEp9wVpcJSnGE3Zw5ru1vg+bewVps01Hdx9 Fa1GNkLFbMwEsRoPgHDn1CmXPWXrame6uLYJgpLs5PawEL9WXrH6i0sVKywKGqhpOGfa 3x9pvH+MigzV3c04ULkW60xCqGWmfIiBu7xBBnZXmA+aiit4TngKlz6T+KnA3ku2AqiN RdG5rmR1wA7TF+VTJ26xoIFmhCAHif89OZ+hI55X2kRZ+FEb1hFwzBHz8nikzVKCnZyW XrPwMatphYFeI8Ii2qgRWw+hahHXhqUP5PWzgJ5qaWge/R1o+LQZKJJqBInYQ4JmgwMS 3xkA== X-Gm-Message-State: AFqh2koPx8jFG+nHcxI4uvmIgbOicwPbr30pHfUgAFO59bajngIoQf6/ N25fnPior4A864KN17yVlSqm/ahqrE194ZDR X-Google-Smtp-Source: AMrXdXu8vfDYGw7GoZIwB6y5OB6vKXNyB5kpS3zLTrsAWJYvFH9V9Xt1tHvQQCJJW+Dui8ueCh4rbg== X-Received: by 2002:a5d:6e8f:0:b0:276:473f:7120 with SMTP id k15-20020a5d6e8f000000b00276473f7120mr14714563wrz.26.1672333650792; Thu, 29 Dec 2022 09:07:30 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:30 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 01/15] command: Tweak finishAsyncCommand ordering to avoid races Date: Thu, 29 Dec 2022 17:07:14 +0000 Message-Id: <20221229170728.880367-2-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14244 Signed-off-by: Richard Purdie --- lib/bb/command.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index 0208e30ec4..cbac07f516 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -152,8 +152,8 @@ class Command: bb.event.fire(CommandExit(code), self.cooker.data) else: bb.event.fire(CommandCompleted(), self.cooker.data) - self.currentAsyncCommand = None self.cooker.finishcommand() + self.currentAsyncCommand = None def reset(self): if self.remotedatastores: From patchwork Thu Dec 29 17:07:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F6E0C4332F for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.209994.1672333653060041532 for ; Thu, 29 Dec 2022 09:07:33 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=KDnEfr8C; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.41, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f41.google.com with SMTP id j17so12539020wrr.7 for ; Thu, 29 Dec 2022 09:07:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=rTIt7iC+xqzzIzNB9KBjbdM1LBnvFop5Z3l0vl0rOLo=; b=KDnEfr8Clo1+yNWHC0HF3HSVCEqz0s1ptOCywhr8iEnpeW3XolRpx50k0FPDugUJai UJWWOrb/PYxH2SJbVbWwpVhd59afDXMRszGWJzavYWRmcbCUYjeMSrIJMKZ4hd8mZ+6L KLCLM9Mrk545P3ltulSGxYL6APnHYJErNMH+U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rTIt7iC+xqzzIzNB9KBjbdM1LBnvFop5Z3l0vl0rOLo=; b=PxU7bBCXwcccnrTJ1hTJbxNvaEa38YVQgzFVBgq7HitJH3wvyssjAS5atoHOFr5emJ LG4WhzUj/5i5TDb85iaYbBM7n/Evzn3HUynYcGOGRcoDDbYUmIxzyktKMQeLO5KcOovv 3h/mVNvSo2R2q4SfMW8jI3TC0liR9ntqnL1LfmRjTke2qUB/4FTp1ZzgucZSN1bDOQ+n EclULQG0URNHI3kql5PAeCDHZoU/QdBzy2GP+hCM38gGtuqBq1EL869rU2JnnLlKpUA+ llaNAYGrsadvbgrwwkoPGWIi96O7gplHduPLdvP8h8fqecDZl0wLWOABoTwT9oGmMds9 bSNg== X-Gm-Message-State: AFqh2kqj9iiM8owzEmVTI+pAkfORX2Z12uzsv9CLV7IFw6EZyETdRD0g F/LAmdwOAVhjHc2DFt1Si8jVRqWRzHpps3zt X-Google-Smtp-Source: AMrXdXsgI8MNlIceY8jz3knx7q4shD9smeRAT7O8RO8bKzh3VCdkw3z0YdJnHj/QcAO0rUFcn8w+tQ== X-Received: by 2002:adf:eb02:0:b0:265:faec:7086 with SMTP id s2-20020adfeb02000000b00265faec7086mr19287207wrn.9.1672333651329; Thu, 29 Dec 2022 09:07:31 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:31 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 02/15] cooker: Ensure commands clean up any parser processes Date: Thu, 29 Dec 2022 17:07:15 +0000 Message-Id: <20221229170728.880367-3-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14245 When finishing a command, we need to ensure any parsing processes that may have been started are cleaned up before we reset the cooker state. Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 4e49b0e90b..df31a413d7 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1768,6 +1768,9 @@ class BBCooker: self.parser.final_cleanup() def finishcommand(self): + if hasattr(self.parser, 'shutdown'): + self.parser.shutdown(clean=False) + self.parser.final_cleanup() self.state = state.initial def reset(self): From patchwork Thu Dec 29 17:07:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6345BC54EBE for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by mx.groups.io with SMTP id smtpd.web10.208595.1672333653668637530 for ; Thu, 29 Dec 2022 09:07:33 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=f6ANp/rE; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.42, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f42.google.com with SMTP id y16so17910366wrm.2 for ; Thu, 29 Dec 2022 09:07:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=rjUR8wW045Dmbgzbvu2jqdBO9AgPEXWTs4vUQPlmhSM=; b=f6ANp/rEi7kjAaHx6AG3G4sHtD1LTN7NKZwtJvpTbkDkhmFieYtEN7pbaufcd3vnnG 3fSSQHGkIQyM+HsJQhFfZRxND6g+ieO2+aUANNDy3qH0bjtnfhsF/0LmVGZ9Nf+PnZ8I tjP7QiQEwfgGAb1tIcRteLNS1dI3VJUcJ/rp0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rjUR8wW045Dmbgzbvu2jqdBO9AgPEXWTs4vUQPlmhSM=; b=xZpe2+UT8X0TF9qmmYEzY64q+g4G1KIWhl0dlgZry0eJ+cVTBaxZ4P5lVkj7S0a57z M6dmwXH5WTni4sjM6Jc7UgnWSgrNdi5x+iyXZvbn/vNwyk9TUdexuJYUDLFZCO28V5wJ t4/UE0gAZ1qMyjh6EHv+/UVBmRgSeJWX4N8ZIRAGYAUNCCrMKMgjRVUZoYCgbFsvEVv1 zDR0r11Bx/jVNfQ/fy68b6L2CmD3Z0EdYl0rVMTP6wYllmvhYK6g63aoPeCiSAgpulvo trxkrHRzlNf/+S+h3ByTujGHj3nxLcHFpXajMkMfCpRfrd4efV6f4ha1vQ1EZc07O05d uOtQ== X-Gm-Message-State: AFqh2kqgyxsfaBfvZG9kJL6fuLJC9/XOm/AoB93gYgIy2XgPwz54U0Uw S0OVf9nnYNRJ10V4YPOlm02CZCKHpN6TVEh/ X-Google-Smtp-Source: AMrXdXvR54038f6lC2XX6/G2XX72PLOzQr0LRB6FWcPAU4zA3GyL7uNXNV5ZFCZ9tJh+IlABhhUNIw== X-Received: by 2002:a5d:6a43:0:b0:26a:298:f067 with SMTP id t3-20020a5d6a43000000b0026a0298f067mr18651677wrw.43.1672333652000; Thu, 29 Dec 2022 09:07:32 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:31 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 03/15] knotty: Ping the server/cooker periodically Date: Thu, 29 Dec 2022 17:07:16 +0000 Message-Id: <20221229170728.880367-4-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14246 We've seeing failures where the UI hangs if the server disappears. Ping the cooker/server if we've not had any events in the last minute so we can check if it is still alive. Signed-off-by: Richard Purdie --- lib/bb/ui/knotty.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py index 61cf0a37f4..ab1a367be0 100644 --- a/lib/bb/ui/knotty.py +++ b/lib/bb/ui/knotty.py @@ -625,7 +625,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): printintervaldelta = 10 * 60 # 10 minutes printinterval = printintervaldelta - lastprint = time.time() + pinginterval = 1 * 60 # 1 minutes + lastevent = lastprint = time.time() termfilter = tf(main, helper, console_handlers, params.options.quiet) atexit.register(termfilter.finish) @@ -637,6 +638,14 @@ def main(server, eventHandler, params, tf = TerminalFilter): printinterval += printintervaldelta event = eventHandler.waitEvent(0) if event is None: + if (lastevent + pinginterval) <= time.time(): + ret, error = server.runCommand(["ping"]) + if error or not ret: + termfilter.clearFooter() + print("No reply after pinging server (%s, %s), exiting." % (str(error), str(ret))) + return_value = 3 + main.shutdown = 2 + lastevent = time.time() if main.shutdown > 1: break if not parseprogress: @@ -644,6 +653,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): event = eventHandler.waitEvent(0.25) if event is None: continue + lastevent = time.time() helper.eventHandler(event) if isinstance(event, bb.runqueue.runQueueExitWait): if not main.shutdown: From patchwork Thu Dec 29 17:07:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A30EC61DB3 for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by mx.groups.io with SMTP id smtpd.web10.208596.1672333654310683953 for ; Thu, 29 Dec 2022 09:07:34 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=MoxetGh0; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.48, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f48.google.com with SMTP id z16so1435470wrw.1 for ; Thu, 29 Dec 2022 09:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=mFin2loGDeMqi8JAlJmeDBXq8KlYP6njBHXSlSCNTrU=; b=MoxetGh0xUzCwvmELfHoE513Xm+gGiBssRc9MtRXFuIHkSpOUoqTwK5dKo1c/iXpsT xKMFiri+6z+EhJTZyXDZBOxVRK8kyNa8Vln3Ile57oFwOXgwGdA5uO4x0jcrmbsRJYE9 7MDyZ8Lc7FcANVd3CARZQNo80NHnPcvpzBeos= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mFin2loGDeMqi8JAlJmeDBXq8KlYP6njBHXSlSCNTrU=; b=LN/BpUyUqXhtNE1veSlDPy5yhwtGLKFk/mV2phRNUROi1qS4oshoDEp8KuZEfWcv6f MRppkRsLlQwfAcAW4Ruspp/PSr7Y+VFRkjeAPgKBNpoFvjoEws66E1ccbhEPp9eoa0k4 zmejlMyV/9IaFi9hryrZeylVW3weooN0V6LdYLmqwqmlg+tp4L51L9C8OPIONo49GbYl xSyRXoHZsKOrfCeEHUwLndjc1QLMG7jxwcCMJS6BOPuZe/cIEMkw7Q7rXqq+sKcE2t5U y4Psq/kaXhbcyEJ83eAslBIdqW4j7/BgXCvd9WfmU9WPHrejxyakN4Yuud6SqOHNmaE7 IIqA== X-Gm-Message-State: AFqh2kpP3uPgnkiSmI/PpuWpXbJXTNy4z+JM0IJ/NSqQqFO6ebZr6GJa Qkn1tnvUFSgfVkYNUHm4EXCKVlbK+qTr+1M4 X-Google-Smtp-Source: AMrXdXsWQNJdQNW1Tdz2AGCNe5NjX/wpyXV7xpfFG9VVdQSP7AuCIyaqAnhFPeykw39sBIttzAs0RQ== X-Received: by 2002:a05:6000:18a6:b0:280:4a9:c8dd with SMTP id b6-20020a05600018a600b0028004a9c8ddmr13868698wri.18.1672333652570; Thu, 29 Dec 2022 09:07:32 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:32 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 04/15] event: Add enable/disable heartbeat code Date: Thu, 29 Dec 2022 17:07:17 +0000 Message-Id: <20221229170728.880367-5-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14248 Currently heartbeat events are always generated by the server whilst it is active. Change this so they only appear when builds are running, which is when most code would expect to be executed. This removes a number of races around changes in the datastore which can happen outside of builds. Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 4 ++++ lib/bb/event.py | 9 +++++++++ lib/bb/server/process.py | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index df31a413d7..527be2f649 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1467,6 +1467,7 @@ class BBCooker: buildname = self.databuilder.mcdata[mc].getVar("BUILDNAME") if fireevents: bb.event.fire(bb.event.BuildStarted(buildname, [item]), self.databuilder.mcdata[mc]) + bb.event.enable_heartbeat() # Execute the runqueue runlist = [[mc, item, task, fn]] @@ -1500,6 +1501,7 @@ class BBCooker: if not retval: if fireevents: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, item, failures, interrupted), self.databuilder.mcdata[mc]) + bb.event.disable_heartbeat() self.command.finishAsyncCommand(msg) # We trashed self.recipecaches above self.parsecache_valid = False @@ -1545,6 +1547,7 @@ class BBCooker: for mc in self.multiconfigs: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, targets, failures, interrupted), self.databuilder.mcdata[mc]) finally: + bb.event.disable_heartbeat() self.command.finishAsyncCommand(msg) return False if retval is True: @@ -1578,6 +1581,7 @@ class BBCooker: for mc in self.multiconfigs: bb.event.fire(bb.event.BuildStarted(buildname, ntargets), self.databuilder.mcdata[mc]) + bb.event.enable_heartbeat() rq = bb.runqueue.RunQueue(self, self.data, self.recipecaches, taskdata, runlist) if 'universe' in targets: diff --git a/lib/bb/event.py b/lib/bb/event.py index 303b7a943f..db90724444 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -69,6 +69,7 @@ _eventfilter = None _uiready = False _thread_lock = threading.Lock() _thread_lock_enabled = False +_heartbeat_enabled = False def enable_threadlock(): global _thread_lock_enabled @@ -78,6 +79,14 @@ def disable_threadlock(): global _thread_lock_enabled _thread_lock_enabled = False +def enable_heartbeat(): + global _heartbeat_enabled + _heartbeat_enabled = True + +def disable_heartbeat(): + global _heartbeat_enabled + _heartbeat_enabled = False + def execute_handler(name, handler, event, d): event.data = d try: diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 586d46af88..91eb6e0ad9 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -382,7 +382,7 @@ class ProcessServer(): # Create new heartbeat event? now = time.time() - if now >= self.next_heartbeat: + if bb.event._heartbeat_enabled and now >= self.next_heartbeat: # We might have missed heartbeats. Just trigger once in # that case and continue after the usual delay. self.next_heartbeat += self.heartbeat_seconds @@ -396,7 +396,7 @@ class ProcessServer(): if not isinstance(exc, bb.BBHandledException): logger.exception('Running heartbeat function') self.quit = True - if nextsleep and now + nextsleep > self.next_heartbeat: + if nextsleep and bb.event._heartbeat_enabled and now + nextsleep > self.next_heartbeat: # Shorten timeout so that we we wake up in time for # the heartbeat. nextsleep = self.next_heartbeat - now From patchwork Thu Dec 29 17:07:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63B43C54EF0 for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.209994.1672333653060041532 for ; Thu, 29 Dec 2022 09:07:34 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=TieMcjAs; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.41, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f41.google.com with SMTP id j17so12539062wrr.7 for ; Thu, 29 Dec 2022 09:07:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=MCyqkGOQqLEt0IzOxLGA/sIHi6iZ3ZnElfqTvXE14hU=; b=TieMcjAsOCBBz3/Jb5cmfRDGR+cjV9fx9MlVB9F+vp112yh+Vn+FegFyIdZHWZTq1y c5jMQec7KHdWbT7UajERkoOKlxHO5AdDRQKLLkk49O1u7Z817XMBfkCjmoQPARgB+vAM vT5mnvCYeZNBCJokXuPBVjA3Q67j0nLdsjmeM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MCyqkGOQqLEt0IzOxLGA/sIHi6iZ3ZnElfqTvXE14hU=; b=OEmuthjPv5Pca9i8Og+QdtG2kY8XPSvII05zUiXs2UKcaukViVbofIF/yOIk5BoD3u PwE/UZA1SPdHFW7cmGNFHgs28+u2uCvMFzn9Uxz+eTxZeZ0pZk9muYC+b2IR8JLMkra/ i8OPbE9YU05OkMaSr8OSlE0OF+l43PZ+8hU8owWcuCVw+omNhhSQWy4DrJRZyXUYr8LR H1/3heQIF3IP6lZQoLqhqS2AcpqztOXF1MbRk3WhRRrzEd3Kw+4vAEKjAuEyUZPe1uwU Oco34hG3fj5u+Rxkx2yIarnBxPvnI4sqFMKrcO8KUU8O34Iy+/FUKUtvV3fLZTEtp9pN aOsA== X-Gm-Message-State: AFqh2kpU29hC+n5EBNByHWiWfrjk8dWGZ9h7TxBUXZypKn0nYE+Pf3hc JSEDV9ej+mrLdOpR2aR3vbRvlSU+nQQO5hW/ X-Google-Smtp-Source: AMrXdXuMMCWKwRAiusZJHfBEsUuB85gbprHpcvzllI0ObmtlUH5w4w143dCwSRTmxlXo1s0UxoS93A== X-Received: by 2002:a05:6000:1b08:b0:242:736f:d3c5 with SMTP id f8-20020a0560001b0800b00242736fd3c5mr17902958wrz.57.1672333653270; Thu, 29 Dec 2022 09:07:33 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:32 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 05/15] server/process: Run idle commands in a separate idle thread Date: Thu, 29 Dec 2022 17:07:18 +0000 Message-Id: <20221229170728.880367-6-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14247 When bitbake is off running heavier "idle" commands, it doesn't service it's command socket which means stopping/interrupting it is hard. It also means we can't "ping" from the UI to know if it is still alive. For those reasons, split idle command execution into it's own thread. The commands are generally already self containted so this is easier than expected. We do have to be careful to only handle inotify poll() from a single thread at a time. It also means we always have to use a thread lock when sending events since both the idle thread and the command thread may generate log messages (and hence events). The patch does depend on a couple of previous fixes to the builtins locking in event.py and the heartbeat enable/disable changes. Signed-off-by: Richard Purdie --- lib/bb/command.py | 4 +-- lib/bb/cooker.py | 19 +++++++++---- lib/bb/server/process.py | 60 +++++++++++++++++++++++++--------------- 3 files changed, 54 insertions(+), 29 deletions(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index cbac07f516..d2c01bf743 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -84,7 +84,7 @@ class Command: if not hasattr(command_method, 'readonly') or not getattr(command_method, 'readonly'): return None, "Not able to execute not readonly commands in readonly mode" try: - self.cooker.process_inotify_updates() + self.cooker.process_inotify_updates_apply() if getattr(command_method, 'needconfig', True): self.cooker.updateCacheSync() result = command_method(self, commandline) @@ -109,7 +109,7 @@ class Command: def runAsyncCommand(self): try: - self.cooker.process_inotify_updates() + self.cooker.process_inotify_updates_apply() if self.cooker.state in (bb.cooker.state.error, bb.cooker.state.shutdown, bb.cooker.state.forceshutdown): # updateCache will trigger a shutdown of the parser # and then raise BBHandledException triggering an exit diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 527be2f649..33e87981c5 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -220,6 +220,8 @@ class BBCooker: bb.debug(1, "BBCooker startup complete %s" % time.time()) sys.stdout.flush() + self.inotify_threadlock = threading.Lock() + def init_configdata(self): if not hasattr(self, "data"): self.initConfigurationData() @@ -248,11 +250,18 @@ class BBCooker: self.notifier = pyinotify.Notifier(self.watcher, self.notifications) def process_inotify_updates(self): - for n in [self.confignotifier, self.notifier]: - if n and n.check_events(timeout=0): - # read notified events and enqueue them - n.read_events() - n.process_events() + with self.inotify_threadlock: + for n in [self.confignotifier, self.notifier]: + if n and n.check_events(timeout=0): + # read notified events and enqueue them + n.read_events() + + def process_inotify_updates_apply(self): + with self.inotify_threadlock: + for n in [self.confignotifier, self.notifier]: + if n and n.check_events(timeout=0): + n.read_events() + n.process_events() def config_notifications(self, event): if event.maskname == "IN_Q_OVERFLOW": diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 91eb6e0ad9..6f43330fae 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -88,6 +88,7 @@ class ProcessServer(): self.maxuiwait = 30 self.xmlrpc = False + self.idle = None self._idlefuns = {} self.bitbake_lock = lock @@ -148,6 +149,7 @@ class ProcessServer(): self.cooker.pre_serve() bb.utils.set_process_name("Cooker") + bb.event.enable_threadlock() ready = [] newconnections = [] @@ -278,6 +280,9 @@ class ProcessServer(): ready = self.idle_commands(.1, fds) + if self.idle: + self.idle.join() + serverlog("Exiting (socket: %s)" % os.path.exists(self.sockname)) # Remove the socket file so we don't get any more connections to avoid races # The build directory could have been renamed so if the file isn't the one we created @@ -352,33 +357,44 @@ class ProcessServer(): msg.append(":\n%s" % procs) serverlog("".join(msg)) + def idle_thread(self): + while not self.quit: + nextsleep = 0.1 + fds = [] + for function, data in list(self._idlefuns.items()): + try: + retval = function(self, data, False) + if retval is False: + del self._idlefuns[function] + nextsleep = None + elif retval is True: + nextsleep = None + elif isinstance(retval, float) and nextsleep: + if (retval < nextsleep): + nextsleep = retval + elif nextsleep is None: + continue + else: + fds = fds + retval + except SystemExit: + raise + except Exception as exc: + if not isinstance(exc, bb.BBHandledException): + logger.exception('Running idle function') + del self._idlefuns[function] + self.quit = True + + if nextsleep is not None: + select.select(fds,[],[],nextsleep)[0] + def idle_commands(self, delay, fds=None): nextsleep = delay if not fds: fds = [] - for function, data in list(self._idlefuns.items()): - try: - retval = function(self, data, False) - if retval is False: - del self._idlefuns[function] - nextsleep = None - elif retval is True: - nextsleep = None - elif isinstance(retval, float) and nextsleep: - if (retval < nextsleep): - nextsleep = retval - elif nextsleep is None: - continue - else: - fds = fds + retval - except SystemExit: - raise - except Exception as exc: - if not isinstance(exc, bb.BBHandledException): - logger.exception('Running idle function') - del self._idlefuns[function] - self.quit = True + if not self.idle: + self.idle = threading.Thread(target=self.idle_thread) + self.idle.start() # Create new heartbeat event? now = time.time() From patchwork Thu Dec 29 17:07:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56EDEC54EBF for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by mx.groups.io with SMTP id smtpd.web10.208594.1672333652242653825 for ; Thu, 29 Dec 2022 09:07:35 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=Zv0c2p9Q; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.52, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f52.google.com with SMTP id bk16so4692588wrb.11 for ; Thu, 29 Dec 2022 09:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=3uqOpyvS9KkDXCZ1VdYD7kmnu7a46UEkOD9VjSvP24I=; b=Zv0c2p9QSSq0knSBHhSHYxWWxaYE4yLecy/KKFwbaLEb9y/nJCoYlkGceqeJtAc0EO FUgbTG/C9N6kDtYGtNW+D9jGol1e9z8OL6osxxzkLmlUBT7gLNE2WUuCm40cplGbpIUT 7T8vTw/QkD+xb7mO7Fua0Dm80iu2YyWFSDns0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3uqOpyvS9KkDXCZ1VdYD7kmnu7a46UEkOD9VjSvP24I=; b=J+iDkHJaDuGnU6iQjD6wH9G+e7ZQtKEg8mWV75W4zZ3b9Qu4K7nMO2nnYLk39cB6ff 2cmqur55PUZPFzoa/mo+oUu/iFyTXqSLX3VbWAeykTTyM7m3fGo0ChOv+18x3D2WG/MI 6F/CRs9AZOi8woOMfrpxb4GjL+mAoic24rbsxMM5ZpBTcCaZqbZ8kXI7x+la6KINLzH2 5EiMWnevba+gdaNBo0d9EavMyPeHs4yo0Ig5etdWLkFSacxpZ4tJP0u2ecYu9GUBWbvS ZiMZjhirBg7F1aGhE7yhTmmAmjYa5YqT/wqUmKSKtfMU4WftnMuNQTo3q2JMJyS/4L7v nSKg== X-Gm-Message-State: AFqh2kq9gOuL9REkYrTY+w6XHV1IrEp5JvzFgDEXktB7tlJzqjfNZe9g LFsLTNAwST1GZe1sbNquQN2s8Yk95gvI/Ukm X-Google-Smtp-Source: AMrXdXvwdRs+XXXNb1NDaoOdyrZAkn741VfAUv8i4ArqJeEUEark48RHJ8OLnq5r0EuBy25mQ0f/Eg== X-Received: by 2002:adf:fc85:0:b0:27f:effe:977 with SMTP id g5-20020adffc85000000b0027feffe0977mr7940441wrr.38.1672333653937; Thu, 29 Dec 2022 09:07:33 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:33 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 06/15] server/process: Improve idle loop exit code Date: Thu, 29 Dec 2022 17:07:19 +0000 Message-Id: <20221229170728.880367-7-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14249 When idle handlers want to exit, returning "False" isn't very clear and also causes challenges with the ordering of the removing the idle handler and marking that no async command is running. Use a specific class to signal the exit condition allowing clearer code and allowing the async command to be cleared after the handler has been removed, reducing any opportunity for races. Signed-off-by: Richard Purdie --- lib/bb/command.py | 13 +++++-------- lib/bb/cooker.py | 13 +++++-------- lib/bb/server/process.py | 8 ++++++++ 3 files changed, 18 insertions(+), 16 deletions(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index d2c01bf743..d03f35a896 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -128,22 +128,19 @@ class Command: else: return False except KeyboardInterrupt as exc: - self.finishAsyncCommand("Interrupted") - return False + return bb.server.process.idleFinish("Interrupted") except SystemExit as exc: arg = exc.args[0] if isinstance(arg, str): - self.finishAsyncCommand(arg) + return bb.server.process.idleFinish(arg) else: - self.finishAsyncCommand("Exited with %s" % arg) - return False + return bb.server.process.idleFinish("Exited with %s" % arg) except Exception as exc: import traceback if isinstance(exc, bb.BBHandledException): - self.finishAsyncCommand("") + return bb.server.process.idleFinish("") else: - self.finishAsyncCommand(traceback.format_exc()) - return False + return bb.server.process.idleFinish(traceback.format_exc()) def finishAsyncCommand(self, msg=None, code=None): if msg or msg == "": diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 33e87981c5..5e18c85bb8 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1502,23 +1502,21 @@ class BBCooker: failures += len(exc.args) retval = False except SystemExit as exc: - self.command.finishAsyncCommand(str(exc)) if quietlog: bb.runqueue.logger.setLevel(rqloglevel) - return False + return bb.server.process.idleFinish(str(exc)) if not retval: if fireevents: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, item, failures, interrupted), self.databuilder.mcdata[mc]) bb.event.disable_heartbeat() - self.command.finishAsyncCommand(msg) # We trashed self.recipecaches above self.parsecache_valid = False self.configuration.limited_deps = False bb.parse.siggen.reset(self.data) if quietlog: bb.runqueue.logger.setLevel(rqloglevel) - return False + return bb.server.process.idleFinish(msg) if retval is True: return True return retval @@ -1548,8 +1546,7 @@ class BBCooker: failures += len(exc.args) retval = False except SystemExit as exc: - self.command.finishAsyncCommand(str(exc)) - return False + return bb.server.process.idleFinish(str(exc)) if not retval: try: @@ -1557,8 +1554,8 @@ class BBCooker: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, targets, failures, interrupted), self.databuilder.mcdata[mc]) finally: bb.event.disable_heartbeat() - self.command.finishAsyncCommand(msg) - return False + return bb.server.process.idleFinish(msg) + if retval is True: return True return retval diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 6f43330fae..3c909942aa 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -71,6 +71,10 @@ def get_lockfile_process_msg(lockfile): return procs.decode("utf-8") return None +class idleFinish(): + def __init__(self, msg): + self.msg = msg + class ProcessServer(): profile_filename = "profile.log" profile_processed_filename = "profile.log.processed" @@ -364,6 +368,10 @@ class ProcessServer(): for function, data in list(self._idlefuns.items()): try: retval = function(self, data, False) + if isinstance(retval, idleFinish): + del self._idlefuns[function] + self.cooker.command.finishAsyncCommand(retval.msg) + nextsleep = None if retval is False: del self._idlefuns[function] nextsleep = None From patchwork Thu Dec 29 17:07:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56B8CC54EBD for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by mx.groups.io with SMTP id smtpd.web11.209995.1672333656344471503 for ; Thu, 29 Dec 2022 09:07:36 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=WNe1WN4d; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.52, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f52.google.com with SMTP id co23so17908703wrb.4 for ; Thu, 29 Dec 2022 09:07:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=qPLeies7zgP/LVrJy8QhnmJjtMZCF7Pb0/MpC5IRMok=; b=WNe1WN4drTnSjcBUIoiz+k7QER5mmpsWSX03IS6o7uTODppiwQu3XgAXHcZ3BPqYdD zblAijtxqOMHNiPRYxC2m0mi2+koHl7Pl+MU2EbQncx5Q3rqO3YoE0J6wbuEqeiRkf8j JCrhye34nzs3hylBI+PQaWMYxFYmTYpmcGmEw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qPLeies7zgP/LVrJy8QhnmJjtMZCF7Pb0/MpC5IRMok=; b=BTixzUQrxmTxEZVerjKoxdQp8zm5Ax0qRr2HLWYSbrLZmm3H4k6TlwkPPrlNkfRyR9 9cfWRaIFiGv6sQatDJgy6F+bm/uti7A8FIkI/8C7sqNdNHZ6Uuod29YyKf06HKwK6LVs Y1r5LfR80RnRU37ljJVh7EvLOSdCmJ3/MDyfiPmduDoKg5sTV2G1sKKiz115hZz86D0S R6bvX4WoU8yKBFBVE0A2+j7a+X3y3rsQZaChpnpfB0+puUrgIruJ+jHtnyUsWgw7ksbV L34PSpu52UNs6KF03hJbkCLesuJbM5iS6jcsEnpgkrsiDIWFkdgUDpPFexU29XQW/zwF qs/w== X-Gm-Message-State: AFqh2konwHzIYwXPt5gTqCi8swZBvAP1yDsSvcolGjJQ78o7R7c+nzwq 3zCHyH7PzVjRJH61VyEvA22ncqpbKNhqXQMH X-Google-Smtp-Source: AMrXdXvYDgCmLC0ZcxG4maPkIaRAb4tgz20rkwYqusc02vZPkH9eRuJxGqNmj+qDBac+ikT2VLFZfw== X-Received: by 2002:a05:6000:80b:b0:274:c846:4211 with SMTP id bt11-20020a056000080b00b00274c8464211mr19116093wrb.54.1672333654638; Thu, 29 Dec 2022 09:07:34 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:34 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 07/15] process: Improve client disconnect/idle sync Date: Thu, 29 Dec 2022 17:07:20 +0000 Message-Id: <20221229170728.880367-8-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14250 When clients are disconnecting, we need to ensure things are in sync with the idle thread. Use a threading event to show when things drop back to idle (there is only one always present idle handler for inotify). Later patches should be able to clean up the inotify handler to work differently and avoid hardcoded values. If we're not idle at client disconnect, shut down the server to solve the dilemma. Not ideal but shouldn't happen much in practise and would be restarted. Signed-off-by: Richard Purdie --- lib/bb/server/process.py | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 3c909942aa..503d5dd686 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -94,6 +94,7 @@ class ProcessServer(): self.idle = None self._idlefuns = {} + self.is_idle = threading.Event() self.bitbake_lock = lock self.bitbake_lock_name = lockname @@ -173,6 +174,13 @@ class ProcessServer(): self.controllersock.close() self.controllersock = False if self.haveui: + # Wait for the idle loop to have cleared (30s max) + self.is_idle.clear() + self.is_idle.wait(timeout=30) + if self.cooker.command.currentAsyncCommand is not None: + serverlog("Idle loop didn't finish queued commands after 30s, exiting.") + self.quit = True + fds.remove(self.command_channel) bb.event.unregister_UIHhandler(self.event_handle, True) self.command_channel_reply.writer.close() @@ -184,7 +192,7 @@ class ProcessServer(): self.cooker.clientComplete() self.haveui = False ready = select.select(fds,[],[],0)[0] - if newconnections: + if newconnections and not self.quit: serverlog("Starting new client") conn = newconnections.pop(-1) fds.append(conn) @@ -392,6 +400,10 @@ class ProcessServer(): del self._idlefuns[function] self.quit = True + # FIXME - the 1 is the inotify processing in cooker which always runs + if len(self._idlefuns) <= 1: + self.is_idle.set() + if nextsleep is not None: select.select(fds,[],[],nextsleep)[0] From patchwork Thu Dec 29 17:07:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A1FBC54EBC for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web11.209996.1672333657013717810 for ; Thu, 29 Dec 2022 09:07:37 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=PkMrTSwd; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.47, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f47.google.com with SMTP id y8so17878299wrl.13 for ; Thu, 29 Dec 2022 09:07:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=wnF4If3Dr/OaHy8ROycwg8tGZNXvY2VGiIzRtAMnOds=; b=PkMrTSwdhxs4dWL6aoRFkx9b7viRbPlujNuuATARaqRcKXlrVwhtsI2zEFWF3sijTE ZtEQ+H3xGGYbocxk+C4k94pQPSdZby5P5ARE09horQ+eAzOr46jFUQOJhBo5FZV24cjW 2Tgdqui5LgmuzifAAbMAC35l2BTw4eO4FGWhk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wnF4If3Dr/OaHy8ROycwg8tGZNXvY2VGiIzRtAMnOds=; b=44KKGDm4U6NJFUgdcuDL7dJ9AUHKlCU4s+ezAW9hE7WLM3u22cEdmxXncN1hmX1mUf +EHbCJkY5TdDepNQwpaCdyfn+rOse81fFVAMomDr7uM9HwfjsXqtDsiHxkzjlMUfk9RX 9LFhpuUVjjwfe6SfD/b1sCbYhn8Zaz9lo4cJnnTv9WogdF5S0x0pBVEbyQkz78D69BTi s95U0XwT9iIFEGAwlstmFXHtWKUgIJdOhBLi1YVgdtdDUtJqwzmaphni2Q44OphbNGRC J74HJC27IFz+Q/9Oja0o8gjwUnZfqM+eZ3SFK1QZiIhXwebDk3p4NUMxHg8jN/m+1o4F +IKQ== X-Gm-Message-State: AFqh2kqcCZ3DwV1M+cEIN16/X9qfINIQVUqXBVMi77t1NlGNMl8gbHFJ UTsfUTk7coJGiyLYhQeRGkwi1Nl752p3w2g2 X-Google-Smtp-Source: AMrXdXtxPCvdpR2+LbRPhuK29tqT3fD+Sk8DlBOJZLAuTyjDlMI/99ef0ByU/eBRvBEUed+Tcxxe3g== X-Received: by 2002:a5d:6808:0:b0:272:3a86:29c1 with SMTP id w8-20020a5d6808000000b002723a8629c1mr14818484wru.16.1672333655208; Thu, 29 Dec 2022 09:07:35 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:34 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 08/15] process: Improve async command execution with idle interaction Date: Thu, 29 Dec 2022 17:07:21 +0000 Message-Id: <20221229170728.880367-9-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14252 When running a new async command, the previous one might not have finished executing in the idle handler. Add some code to use the event logic to wait for the idle loop to finish for 30s before saying the server is busy. This avoids build failures when a UI quickly starts the next async command before the code has finished shutdown/cleanup from the last one. Signed-off-by: Richard Purdie --- lib/bb/command.py | 8 ++++++-- lib/bb/server/process.py | 2 +- lib/bb/server/xmlrpcserver.py | 2 +- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/bb/command.py b/lib/bb/command.py index d03f35a896..41903241ab 100644 --- a/lib/bb/command.py +++ b/lib/bb/command.py @@ -60,7 +60,7 @@ class Command: # FIXME Add lock for this self.currentAsyncCommand = None - def runCommand(self, commandline, ro_only = False): + def runCommand(self, commandline, process_server, ro_only = False): command = commandline.pop(0) # Ensure cooker is ready for commands @@ -100,7 +100,11 @@ class Command: else: return result, None if self.currentAsyncCommand is not None: - return None, "Busy (%s in progress)" % self.currentAsyncCommand[0] + # Wait for the idle loop to have cleared (30s max) + process_server.is_idle.clear() + process_server.is_idle.wait(timeout=30) + if self.currentAsyncCommand is not None: + return None, "Busy (%s in progress)" % self.currentAsyncCommand[0] if command not in CommandsAsync.__dict__: return None, "No such command" self.currentAsyncCommand = (command, commandline) diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 503d5dd686..fefc5a64fc 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -264,7 +264,7 @@ class ProcessServer(): continue try: serverlog("Running command %s" % command) - self.command_channel_reply.send(self.cooker.command.runCommand(command)) + self.command_channel_reply.send(self.cooker.command.runCommand(command, self)) serverlog("Command Completed (socket: %s)" % os.path.exists(self.sockname)) except Exception as e: stack = traceback.format_exc() diff --git a/lib/bb/server/xmlrpcserver.py b/lib/bb/server/xmlrpcserver.py index 01f55538ae..2e65dc34a9 100644 --- a/lib/bb/server/xmlrpcserver.py +++ b/lib/bb/server/xmlrpcserver.py @@ -118,7 +118,7 @@ class BitBakeXMLRPCServerCommands(): """ Run a cooker command on the server """ - return self.server.cooker.command.runCommand(command, self.server.readonly) + return self.server.cooker.command.runCommand(command, self.server, self.server.readonly) def getEventHandle(self): return self.event_handle From patchwork Thu Dec 29 17:07:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50526C54E76 for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.209994.1672333653060041532 for ; Thu, 29 Dec 2022 09:07:37 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=QejJvfrA; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.41, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f41.google.com with SMTP id j17so12539130wrr.7 for ; Thu, 29 Dec 2022 09:07:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Nvlhv/xI3RCqmomtv1v37EcS/G6BUEAe5DWKoRCliPo=; b=QejJvfrAUJ8Y+1Pmf9S8CSCVzkJstC2dJoM1sIeEgfLXlbWc8NxqZeR6uni9ggy39g xkOsyi3sOY1XzJYWsWAxWGGdYjTVt3O/HBPt0zuOwYxX70dgwKDUlqZ753Ruh/1sgXkg uXvop3twH/JUbhkUt5aAIFJIGtbdeH5WmQ9io= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nvlhv/xI3RCqmomtv1v37EcS/G6BUEAe5DWKoRCliPo=; b=B0svFS9z7OX22gLWqYj7XY6kdYXiHwss5GUKi0Ez8bkcf+7ARtf2uXztme0JRgEp9k DIUP9fvA8NWHotOtf7SUJioMhcpONcOB0HpYd6/6XsT2GV+Etm1EYyOAehRwk8jvWg2M zPJTaJ54YbXd5CtofWmqlwJNgoa+I+tGq5NH3XAH6AnF0r7shqPR98w0/3a6JF5EYno9 gtRarsC180edGx4fHbE/aLhrhofGFzfjc0zxUWNOPkUd1TU4gS98Ejvi/XPu96Ap8wzz iAhxoppn7QoHybs/JnjeMnyhQb27/vGjR6fEQg8+0Jd0/0YDvz3zV/KBWVj6jGt9EVQQ OFrw== X-Gm-Message-State: AFqh2kpkw3nERJOe2mVEkYJ1G7Lut7SSTWegqnB+UkUrHMwFDuMWvZCl NNNv9+rcwDel1frORBpRTEvundpm3jXK0itr X-Google-Smtp-Source: AMrXdXtu9PsNV/RIgjV8iAKqomDgLnNkJYXvFlTkeO1XS/UKC8SHVNBVhzW6zSyDTYsxrLgYwOhE/A== X-Received: by 2002:a5d:4351:0:b0:256:ff7d:2346 with SMTP id u17-20020a5d4351000000b00256ff7d2346mr16345300wrr.51.1672333655947; Thu, 29 Dec 2022 09:07:35 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:35 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 09/15] knotty: Avoid looping with tracebacks Date: Thu, 29 Dec 2022 17:07:22 +0000 Message-Id: <20221229170728.880367-10-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14251 If there are events queued and there is an exception in the main loop of the UI code, it will print tracebacks on the console indefinitely. Avoid that by improving the loop exit conditions. Signed-off-by: Richard Purdie --- lib/bb/ui/knotty.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py index ab1a367be0..c398f591e8 100644 --- a/lib/bb/ui/knotty.py +++ b/lib/bb/ui/knotty.py @@ -631,7 +631,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): termfilter = tf(main, helper, console_handlers, params.options.quiet) atexit.register(termfilter.finish) - while True: + while main.shutdown < 2: try: if (lastprint + printinterval) <= time.time(): termfilter.keepAlive(printinterval) @@ -646,8 +646,6 @@ def main(server, eventHandler, params, tf = TerminalFilter): return_value = 3 main.shutdown = 2 lastevent = time.time() - if main.shutdown > 1: - break if not parseprogress: termfilter.updateFooter() event = eventHandler.waitEvent(0.25) From patchwork Thu Dec 29 17:07:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41DE8C4708E for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mx.groups.io with SMTP id smtpd.web11.209997.1672333658478046324 for ; Thu, 29 Dec 2022 09:07:38 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=gsueRB3N; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.47, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f47.google.com with SMTP id ja17so13553477wmb.3 for ; Thu, 29 Dec 2022 09:07:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=BFeu+QStF9jw01bdqTGF/MIUtaK10LkrwHpk/9u9bTM=; b=gsueRB3NftMdxAMhbvdA7iOIb13KPFwUUF6HnoKKhXkEc8wxy3Klxi4SoRLLlgZPxP ETtH2O/pQBs2o0MEzQVIjcs+KFNvWu4zLml6QqF41sna3znZAQgs7JY5xl7nH0cqdxOm jUakwwLwCFz4oRoIAQ7sgPwy8TIzmWSoXs9E4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BFeu+QStF9jw01bdqTGF/MIUtaK10LkrwHpk/9u9bTM=; b=0eVDkdeAaXo56sWYwhmqNC1MG9jlaXfYuYBJHId+P5HrqzFvth6yXsjg4rjHdk88FA cWl5ha+/wcydTFco9FfJyRqG5EJvLAywWcklrs2YDDPmuCFZmwYoXFlXAV0FWxVot72I 5MDF3LGQPGt2KHCxOESfpZB1x6cNO6EFpcL2gfA7sWLj5xf2ER2yeeyyVAktjC6riBZS iarvboOCzoaRhQPCeCZKo1riIx0ciebjbgrA+jJd//D0d/Jf3ou64MlFxwWzTQwri8Tt 7aO8HI5Ag5WzGxHvqyVf6w3yb8ybI1FFiS3Vz/dhSp9IlomXLDNl2YFjBdNcWL2r6u9D kAMA== X-Gm-Message-State: AFqh2kqfsGKU/bmIgka4NwKwo1RbdOUraGf7K6afw3tPicFwOtm5Ivgi CWHekCbnNvQmIe/K/buIhGqeH3u4uO5vQdVi X-Google-Smtp-Source: AMrXdXsgpZn1Il/REMEoT5tKrLwvsv7MV8Wv+i7DfC6h+VJw3nQuztevVUHVxObV8J+0W9eXys9fog== X-Received: by 2002:a05:600c:8507:b0:3d9:6c7d:c9ee with SMTP id gw7-20020a05600c850700b003d96c7dc9eemr15741319wmb.25.1672333656569; Thu, 29 Dec 2022 09:07:36 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:36 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 10/15] event: Always use threadlock Date: Thu, 29 Dec 2022 17:07:23 +0000 Message-Id: <20221229170728.880367-11-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14254 With the move to a server idle thread, we always need threading. The existing accessor functions could end up turning this off! I was going to hold the lock whilst changing it, check if the value was already set, cache the result and also fix the event code to always release the lock with a try/finally. Instead, disable the existing functions and use a with: block to handle the lock, keeping things much simpler. Signed-off-by: Richard Purdie --- lib/bb/event.py | 73 +++++++++++++++++++--------------------- lib/bb/server/process.py | 1 - lib/bb/tests/event.py | 17 +--------- 3 files changed, 35 insertions(+), 56 deletions(-) diff --git a/lib/bb/event.py b/lib/bb/event.py index db90724444..7826541a64 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -68,16 +68,15 @@ _catchall_handlers = {} _eventfilter = None _uiready = False _thread_lock = threading.Lock() -_thread_lock_enabled = False _heartbeat_enabled = False def enable_threadlock(): - global _thread_lock_enabled - _thread_lock_enabled = True + # Always needed now + return def disable_threadlock(): - global _thread_lock_enabled - _thread_lock_enabled = False + # Always needed now + return def enable_heartbeat(): global _heartbeat_enabled @@ -179,36 +178,30 @@ def print_ui_queue(): def fire_ui_handlers(event, d): global _thread_lock - global _thread_lock_enabled if not _uiready: # No UI handlers registered yet, queue up the messages ui_queue.append(event) return - if _thread_lock_enabled: - _thread_lock.acquire() - - errors = [] - for h in _ui_handlers: - #print "Sending event %s" % event - try: - if not _ui_logfilters[h].filter(event): - continue - # We use pickle here since it better handles object instances - # which xmlrpc's marshaller does not. Events *must* be serializable - # by pickle. - if hasattr(_ui_handlers[h].event, "sendpickle"): - _ui_handlers[h].event.sendpickle((pickle.dumps(event))) - else: - _ui_handlers[h].event.send(event) - except: - errors.append(h) - for h in errors: - del _ui_handlers[h] - - if _thread_lock_enabled: - _thread_lock.release() + with _thread_lock: + errors = [] + for h in _ui_handlers: + #print "Sending event %s" % event + try: + if not _ui_logfilters[h].filter(event): + continue + # We use pickle here since it better handles object instances + # which xmlrpc's marshaller does not. Events *must* be serializable + # by pickle. + if hasattr(_ui_handlers[h].event, "sendpickle"): + _ui_handlers[h].event.sendpickle((pickle.dumps(event))) + else: + _ui_handlers[h].event.send(event) + except: + errors.append(h) + for h in errors: + del _ui_handlers[h] def fire(event, d): """Fire off an Event""" @@ -322,21 +315,23 @@ def set_eventfilter(func): _eventfilter = func def register_UIHhandler(handler, mainui=False): - bb.event._ui_handler_seq = bb.event._ui_handler_seq + 1 - _ui_handlers[_ui_handler_seq] = handler - level, debug_domains = bb.msg.constructLogOptions() - _ui_logfilters[_ui_handler_seq] = UIEventFilter(level, debug_domains) - if mainui: - global _uiready - _uiready = _ui_handler_seq - return _ui_handler_seq + with _thread_lock: + bb.event._ui_handler_seq = bb.event._ui_handler_seq + 1 + _ui_handlers[_ui_handler_seq] = handler + level, debug_domains = bb.msg.constructLogOptions() + _ui_logfilters[_ui_handler_seq] = UIEventFilter(level, debug_domains) + if mainui: + global _uiready + _uiready = _ui_handler_seq + return _ui_handler_seq def unregister_UIHhandler(handlerNum, mainui=False): if mainui: global _uiready _uiready = False - if handlerNum in _ui_handlers: - del _ui_handlers[handlerNum] + with _thread_lock: + if handlerNum in _ui_handlers: + del _ui_handlers[handlerNum] return def get_uihandler(): diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index fefc5a64fc..87b683ecf0 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -154,7 +154,6 @@ class ProcessServer(): self.cooker.pre_serve() bb.utils.set_process_name("Cooker") - bb.event.enable_threadlock() ready = [] newconnections = [] diff --git a/lib/bb/tests/event.py b/lib/bb/tests/event.py index 4de4cced5e..d959f2d95d 100644 --- a/lib/bb/tests/event.py +++ b/lib/bb/tests/event.py @@ -451,10 +451,9 @@ class EventHandlingTest(unittest.TestCase): and disable threadlocks tests """ bb.event.fire(bb.event.OperationStarted(), None) - def test_enable_threadlock(self): + def test_event_threadlock(self): """ Test enable_threadlock method """ self._set_threadlock_test_mockups() - bb.event.enable_threadlock() self._set_and_run_threadlock_test_workers() # Calls to UI handlers should be in order as all the registered # handlers for the event coming from the first worker should be @@ -462,20 +461,6 @@ class EventHandlingTest(unittest.TestCase): self.assertEqual(self._threadlock_test_calls, ["w1_ui1", "w1_ui2", "w2_ui1", "w2_ui2"]) - - def test_disable_threadlock(self): - """ Test disable_threadlock method """ - self._set_threadlock_test_mockups() - bb.event.disable_threadlock() - self._set_and_run_threadlock_test_workers() - # Calls to UI handlers should be intertwined together. Thanks to the - # delay in the registered handlers for the event coming from the first - # worker, the event coming from the second worker starts being - # processed before finishing handling the first worker event. - self.assertEqual(self._threadlock_test_calls, - ["w1_ui1", "w2_ui1", "w1_ui2", "w2_ui2"]) - - class EventClassesTest(unittest.TestCase): """ Event classes test class """ From patchwork Thu Dec 29 17:07:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48C36C5479D for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web11.209996.1672333657013717810 for ; Thu, 29 Dec 2022 09:07:38 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=J5iQriq3; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.47, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f47.google.com with SMTP id y8so17878357wrl.13 for ; Thu, 29 Dec 2022 09:07:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ovuyRg7h71SKWQRpouzEUKKFTjNE0toUW8yBGlQQYsQ=; b=J5iQriq313go9/IQnIADntH8/jly2ltcwnniNJN+GST4nr8rMGIgEFcHCu7lOnGomv Oq7ekQ6thnN4aISLy3FYayYWqPeczTiR7qgDSt9YNruS9Yx6jBBDhVpIJ+8yxs49iGb1 4M2MjxMRLShr28ddLq5TPo5bMMRO/AMjpyUXA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ovuyRg7h71SKWQRpouzEUKKFTjNE0toUW8yBGlQQYsQ=; b=4PX9l8qsVOsT1JVvPGb5anILBfVuKS/Zj9dGa9DB0ANLpz9heCsY+r9VmZRhjRd9L0 Qpjc+12r6b5LQ4XGqqJBIx57np2NT9YWPZTsqPvsmMnsrzvtyS48jliyrRPXANsN8XkH eEHltT9XYHs+sEn9auOl4Q9yoh+QCqLBC+alJ9P3vCsWXQ8zdoOXBlZ3fzxffKrttR4l VaOvvg6ar2PT1uk/37JRwT9TZ/VXPiar0SQ0MySHU8U8EOL+zdyFtOG1xKJ8sqgI7CWA n6ueifjq85QG+bFUriDYfh45GisGhDIXJW8pb+d85ydBHW88Mvf0JHjM8NxcRLXFxWAn i54w== X-Gm-Message-State: AFqh2kozgo+5qSqVCUfZVqzYEeshNw7nLth/e0CmSkAqrOgqK3nABrgX +iY27I6jTBCdal55NKDpdc+QXJOMPOQelRty X-Google-Smtp-Source: AMrXdXt+sHWwm/8XZ3zed3RpLWk8JZePlVWSIaUJPTgWuQOdnNJCV6lwf2svwzhJWk32GqjK1yLZuw== X-Received: by 2002:a05:6000:18d1:b0:242:1f46:fb8c with SMTP id w17-20020a05600018d100b002421f46fb8cmr17274050wrq.41.1672333657398; Thu, 29 Dec 2022 09:07:37 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:37 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 11/15] server/process: Improve exception logging Date: Thu, 29 Dec 2022 17:07:24 +0000 Message-Id: <20221229170728.880367-12-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14253 Currently if either idle functions loop suffers a traceback, it is silently dropped and there is no log message to say what happened. This change at least means the traceback is in the cooker log, making some debugging possible. Signed-off-by: Richard Purdie --- lib/bb/server/process.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 87b683ecf0..3038c63f55 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -397,6 +397,7 @@ class ProcessServer(): if not isinstance(exc, bb.BBHandledException): logger.exception('Running idle function') del self._idlefuns[function] + serverlog("Exception %s broke the idle_thread, exiting" % traceback.format_exc()) self.quit = True # FIXME - the 1 is the inotify processing in cooker which always runs @@ -430,6 +431,7 @@ class ProcessServer(): except Exception as exc: if not isinstance(exc, bb.BBHandledException): logger.exception('Running heartbeat function') + serverlog("Exception %s broke in idle_commands, exiting" % traceback.format_exc()) self.quit = True if nextsleep and bb.event._heartbeat_enabled and now + nextsleep > self.next_heartbeat: # Shorten timeout so that we we wake up in time for From patchwork Thu Dec 29 17:07:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D1EAC53210 for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mx.groups.io with SMTP id smtpd.web11.209998.1672333659761619065 for ; Thu, 29 Dec 2022 09:07:40 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=QFZY61DE; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.53, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f53.google.com with SMTP id z10so17882894wrh.10 for ; Thu, 29 Dec 2022 09:07:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=PHdDHXyxc/MBccgeHJ9yuRqXeSqLfGDw+8/TKtuL9wA=; b=QFZY61DEHZocP+xeXrX+EilSnTseg2vbfEMl25LyuFhTMXjZCBrQTxMhzJOWpGSFQb tB7MHPZSKltJGKcjMR6tjA7lJp6kv23eaSDxdPtVa+H4BQID8yQmUpB4Ide9o+0bjb6S zCzbta4Ej7HahuSB3NTi6xB8sLrv85YBKxRWI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PHdDHXyxc/MBccgeHJ9yuRqXeSqLfGDw+8/TKtuL9wA=; b=sWHpQWm+Q8IkUHfe/ByM5DbSBaXrW7LtcDBVZ920z3i+sx0tEThgjrBah+ce+nYBP8 cZSZprWlk7llCPLHKG6sqJeCH2ER8lXIsw4Nrb1aAQPtojAGqqzF3w0gxpnwqjwjVDGK qxtjwrS7xrNKh2XYO4PSfQ/+UVFjbNwMnZA9I19wmbIn1RoqvpUxTZVuzr+4nL21Xz6f 6QKlXreug+OOkKZh7qwyeYgytnhN8ZLIrcGzuhU7OCiItxdggNyZqJyYFCXOnk0hajRW Qml4ZMEKHgGfIEN3Ooq9hxUjp4nfaCZwU/szIaJ52nRukJ39NepNVN/aCbxRIzqFWiN8 /9ZQ== X-Gm-Message-State: AFqh2krrWEoU4v1rSRceYja8p9kIDTEiRBQnE2844GKNOBnpkvtivFC1 nzPGlc3M2KDugRdEm5625nbiYi+XGIEDofsv X-Google-Smtp-Source: AMrXdXvF0LgUNvdcrzmJJtgisLQdmtQGuST0WTWMR1gUVBm7cBJilAtGuUxQIUSOw3els1nQlijWiA== X-Received: by 2002:a5d:5002:0:b0:28c:d89c:ae48 with SMTP id e2-20020a5d5002000000b0028cd89cae48mr1564912wrt.41.1672333657961; Thu, 29 Dec 2022 09:07:37 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:37 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 12/15] cooker/cookerdata: Rework the way the datastores are reset Date: Thu, 29 Dec 2022 17:07:25 +0000 Message-Id: <20221229170728.880367-13-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14255 As far as I could tell, the current code could result in some strange situations where some data was set back to the original data store copy but the multiconfig data was not restored. There are also some changes made to the datastore which did not persist. The data store was also being reset at every client reset, which seems a little excessive if we can reset it to the original condition properly. Move the __depends -> __base_depends rename into databuilder along with the __bbclasstype change so these are saved in the original data. Tweak the databuilder code to be clearer and easier to follow on which copies are where, then save copies of all the mc datastores. Finally, drop the cache invalidation upon reset for the base config as we shouldn't need that now (oe-selftest -r tinfoil works with memory resident bitbake which was the original reproducer requiring that change). Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 9 +++------ lib/bb/cookerdata.py | 31 +++++++++++++++++++------------ 2 files changed, 22 insertions(+), 18 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 5e18c85bb8..ff7e48f6c0 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -409,9 +409,7 @@ class BBCooker: self.disableDataTracking() for mc in self.databuilder.mcdata.values(): - mc.renameVar("__depends", "__base_depends") self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher) - mc.setVar("__bbclasstype", "recipe") self.baseconfig_valid = True self.parsecache_valid = False @@ -445,10 +443,8 @@ class BBCooker: upstream=upstream, ) self.hashserv.serve_as_process() - self.data.setVar("BB_HASHSERVE", self.hashservaddr) - self.databuilder.origdata.setVar("BB_HASHSERVE", self.hashservaddr) - self.databuilder.data.setVar("BB_HASHSERVE", self.hashservaddr) for mc in self.databuilder.mcdata: + self.databuilder.mcorigdata[mc].setVar("BB_HASHSERVE", self.hashservaddr) self.databuilder.mcdata[mc].setVar("BB_HASHSERVE", self.hashservaddr) bb.parse.init_parser(self.data) @@ -1797,8 +1793,9 @@ class BBCooker: if hasattr(self, "data"): self.databuilder.reset() self.data = self.databuilder.data + # In theory tinfoil could have modified the base data before parsing, + # ideally need to track if anything did modify the datastore self.parsecache_valid = False - self.baseconfig_valid = False class CookerExit(bb.event.Event): diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py index 650ae05ec9..b4e0c4216b 100644 --- a/lib/bb/cookerdata.py +++ b/lib/bb/cookerdata.py @@ -263,6 +263,7 @@ class CookerDataBuilder(object): self.mcdata = {} def parseBaseConfiguration(self, worker=False): + mcdata = {} data_hash = hashlib.sha256() try: self.data = self.parseConfigurationFiles(self.prefiles, self.postfiles) @@ -288,18 +289,18 @@ class CookerDataBuilder(object): bb.parse.init_parser(self.data) data_hash.update(self.data.get_hash().encode('utf-8')) - self.mcdata[''] = self.data + mcdata[''] = self.data multiconfig = (self.data.getVar("BBMULTICONFIG") or "").split() for config in multiconfig: if config[0].isdigit(): bb.fatal("Multiconfig name '%s' is invalid as multiconfigs cannot start with a digit" % config) - mcdata = self.parseConfigurationFiles(self.prefiles, self.postfiles, config) - bb.event.fire(bb.event.ConfigParsed(), mcdata) - self.mcdata[config] = mcdata - data_hash.update(mcdata.get_hash().encode('utf-8')) + parsed_mcdata = self.parseConfigurationFiles(self.prefiles, self.postfiles, config) + bb.event.fire(bb.event.ConfigParsed(), parsed_mcdata) + mcdata[config] = parsed_mcdata + data_hash.update(parsed_mcdata.get_hash().encode('utf-8')) if multiconfig: - bb.event.fire(bb.event.MultiConfigParsed(self.mcdata), self.data) + bb.event.fire(bb.event.MultiConfigParsed(mcdata), self.data) self.data_hash = data_hash.hexdigest() except (SyntaxError, bb.BBHandledException): @@ -332,17 +333,23 @@ class CookerDataBuilder(object): if issues: raise bb.BBHandledException() + for mc in mcdata: + mcdata[mc].renameVar("__depends", "__base_depends") + mcdata[mc].setVar("__bbclasstype", "recipe") + # Create a copy so we can reset at a later date when UIs disconnect - self.origdata = self.data - self.data = bb.data.createCopy(self.origdata) - self.mcdata[''] = self.data + self.mcorigdata = mcdata + for mc in mcdata: + self.mcdata[mc] = bb.data.createCopy(mcdata[mc]) + self.data = self.mcdata[''] def reset(self): # We may not have run parseBaseConfiguration() yet - if not hasattr(self, 'origdata'): + if not hasattr(self, 'mcorigdata'): return - self.data = bb.data.createCopy(self.origdata) - self.mcdata[''] = self.data + for mc in self.mcorigdata: + self.mcdata[mc] = bb.data.createCopy(self.mcorigdata[mc]) + self.data = self.mcdata[''] def _findLayerConf(self, data): return findConfigFile("bblayers.conf", data) From patchwork Thu Dec 29 17:07:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F70FC4167B for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.web11.209999.1672333660484354404 for ; Thu, 29 Dec 2022 09:07:40 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=AdrdQNAB; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.45, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f45.google.com with SMTP id bx10so17932605wrb.0 for ; Thu, 29 Dec 2022 09:07:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=FotYd+yw6uokjiN37vk0vRJ4lgA31+9nocaqRYU1erg=; b=AdrdQNABYwOgHia5iXDdc7oZlx86ZO86s6m/JTriA9HyzcnanyjnFmltVw3gDaMtWa /DPwmu0z6wInGHb60QmJBAKT+fvIH0mssofaniozGJ84RfbcBie20EFDPXWxtUg3BjyO ZLkAKmgLRyebFLjtgkmKUhJQPf1IqO1DrxG+8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FotYd+yw6uokjiN37vk0vRJ4lgA31+9nocaqRYU1erg=; b=I21ylo98XXKjVFZO6ruI5+QjvYGbBSaxxunI1UCWB/2AJ//6ySkt9K9qVIvSj3SG5j fwq4RgqqCM4RlbPowUBo4YidQh3EtrdWAAcPXPawDCCR9PMFuBXuCKwL1URGKUIDeD2C JQ6jwYGTkcqLhcHPNijfiDf8CW/guD974hgkvrwvnAk6yLxnBfx5fiRMRCZ4EtxF5c81 lrDcO9CSOofEDPpxtwFjMyfY7xfYlLEy16bkTgzTdFRm9Tf0DbrPJVVXJvyGJBnmQBfS bBg9pvzwgPoBpnj91OBc2ZprKoDiJ5KcKpklY0myrqOP2wAwya+CSHelLpspNBWs+yj9 bsUw== X-Gm-Message-State: AFqh2krvp1ifoL1CZ3fenOH57xm7FKwpDbxQC1O8eBmjQ1NdzwjhhT6V le8uk9Rg5/3PMCiE9Q303vmdkl9yVX/aWKSD X-Google-Smtp-Source: AMrXdXua/Bnf3WW6oPWuTX89AJAV2UHd00wfVl0Z6nHicrH59ee7A6G8Yzs6TPcKaXyqVVIHfn0+nw== X-Received: by 2002:a5d:45c8:0:b0:242:4c28:c9a9 with SMTP id b8-20020a5d45c8000000b002424c28c9a9mr17724510wrs.46.1672333658755; Thu, 29 Dec 2022 09:07:38 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:38 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 13/15] cooker: Ensure we clean up active idle handlers Date: Thu, 29 Dec 2022 17:07:26 +0000 Message-Id: <20221229170728.880367-14-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14257 When the cooker shutdown method is called, ensure we clean up any active idle handlers before we exit as these could have trees of processes in them and these need to be notified and reaped. Ensure at actual shutdown when the idle loop has been terminated we don't trigger the idle loop call though as it will no longer be active at that point. Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 10 +++++++--- lib/bb/server/process.py | 12 ++++++++---- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index ff7e48f6c0..a5a635858c 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -149,7 +149,7 @@ class BBCooker: Manages one bitbake build run """ - def __init__(self, featureSet=None, idleCallBackRegister=None): + def __init__(self, featureSet=None, idleCallBackRegister=None, waitIdle=None): self.recipecaches = None self.eventlog = None self.skiplist = {} @@ -164,6 +164,7 @@ class BBCooker: self.configuration = bb.cookerdata.CookerConfiguration() self.idleCallBackRegister = idleCallBackRegister + self.waitIdle = waitIdle bb.debug(1, "BBCooker starting %s" % time.time()) sys.stdout.flush() @@ -1753,7 +1754,7 @@ class BBCooker: return def post_serve(self): - self.shutdown(force=True) + self.shutdown(force=True, idle=False) prserv.serv.auto_shutdown() if hasattr(bb.parse, "siggen"): bb.parse.siggen.exit() @@ -1763,12 +1764,15 @@ class BBCooker: if hasattr(self, "data"): bb.event.fire(CookerExit(), self.data) - def shutdown(self, force = False): + def shutdown(self, force=False, idle=True): if force: self.state = state.forceshutdown else: self.state = state.shutdown + if idle: + self.waitIdle(30) + if self.parser: self.parser.shutdown(clean=not force) self.parser.final_cleanup() diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 3038c63f55..365758f85a 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -150,6 +150,11 @@ class ProcessServer(): return ret + def wait_for_idle(self, timeout=30): + # Wait for the idle loop to have cleared (30s max) + self.is_idle.clear() + self.is_idle.wait(timeout=timeout) + def main(self): self.cooker.pre_serve() @@ -174,8 +179,7 @@ class ProcessServer(): self.controllersock = False if self.haveui: # Wait for the idle loop to have cleared (30s max) - self.is_idle.clear() - self.is_idle.wait(timeout=30) + self.wait_for_idle(30) if self.cooker.command.currentAsyncCommand is not None: serverlog("Idle loop didn't finish queued commands after 30s, exiting.") self.quit = True @@ -309,7 +313,7 @@ class ProcessServer(): self.sock.close() try: - self.cooker.shutdown(True) + self.cooker.shutdown(True, idle=False) self.cooker.notifier.stop() self.cooker.confignotifier.stop() except: @@ -607,7 +611,7 @@ def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpc writer = ConnectionWriter(readypipeinfd) try: featureset = [] - cooker = bb.cooker.BBCooker(featureset, server.register_idle_function) + cooker = bb.cooker.BBCooker(featureset, server.register_idle_function, server.wait_for_idle) cooker.configuration.profile = profile except bb.BBHandledException: return None From patchwork Thu Dec 29 17:07:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CA6CC46467 for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web11.209996.1672333657013717810 for ; Thu, 29 Dec 2022 09:07:40 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=NIb4BdP1; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.47, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f47.google.com with SMTP id y8so17878408wrl.13 for ; Thu, 29 Dec 2022 09:07:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=10952peMSbCjNKygbzkhlQaifYsAkrqPX0YF0GHq3fU=; b=NIb4BdP1BEw9cvUJUzKdF3Ej3AAcSgzPKjBnwnUbdBs79JAjaZSjHcxe4K31C7T9wO v4X5n6++q3eD3+z59rYBf6IlMavXNw//JeKsi5cLkCfn/lxyRq6T47JUtVdyQGYV619l TocedMMmbGUMcnXzzNPrlt20Ul3q0JY4GfinY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=10952peMSbCjNKygbzkhlQaifYsAkrqPX0YF0GHq3fU=; b=W43n/fQjvfcxjPK6eng4ODAJhDsOZnbOfiTfilx7poIaO7plee6ZdfKBNX3cBvb6b2 ZVP0mOo8xbAtIYNrK/IwHYILub6td1XB9F9jN+iykGfWJkdE/xM5OjXkYyn8QYyyMXjs 8mN/z4oYKnzsJmjnJtzNRgUFZFzlRlX9EqXCdTZT7Oqwuh6pll3/Hh80mdfxkZejzydD Aw6f3udH/ivka/ef2JGWNXcgI6EHNTYXVIOHJxgM1eu83rISj0zzNf8xK67je4WaMsvc W9x2NR8acdmoilNQIkqGVQVWld0J6ptIx2Gj8orlNgw7PI9A4y5ShPX1ADRGuaUn88s2 GO4A== X-Gm-Message-State: AFqh2kqLMHzkr93nfy8nZZZ/K9V1+YKAYPfZNpT2GxU8Lc0eHG1a/SBN 4JZFsup6fPxQ0grKUTRAKK3FIRx+p45OH3/B X-Google-Smtp-Source: AMrXdXt2v2h10Ozbz48m846jnJK7z8KUKg/i4M9DYTvR5Gs7qvMam/Uhdzk0bgtgV4PdRQlbt75wXQ== X-Received: by 2002:a05:6000:1109:b0:28d:b028:b16 with SMTP id z9-20020a056000110900b0028db0280b16mr946819wrw.32.1672333659457; Thu, 29 Dec 2022 09:07:39 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:39 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 14/15] cache: Drop reciever side counting for SiggenRecipeInfo Date: Thu, 29 Dec 2022 17:07:27 +0000 Message-Id: <20221229170728.880367-15-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14256 Joshua Watt pointed out maintaining the counting on both sides of the connection isn't needed. Remove the receiver side counting and simplify the code, also allowing errors if the counts do go out of sync. Signed-off-by: Richard Purdie --- lib/bb/cache.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/lib/bb/cache.py b/lib/bb/cache.py index 3fc097241a..ee924b2d2b 100644 --- a/lib/bb/cache.py +++ b/lib/bb/cache.py @@ -285,7 +285,6 @@ class SiggenRecipeInfo(RecipeInfoCommon): cls.save_map = {} cls.save_count = 1 cls.restore_map = {} - cls.restore_count = {} @classmethod def _save(cls, deps): @@ -294,11 +293,13 @@ class SiggenRecipeInfo(RecipeInfoCommon): return deps for dep in deps: fs = deps[dep] - if fs in cls.save_map: + if fs is None: + ret.append((dep, None, None)) + elif fs in cls.save_map: ret.append((dep, None, cls.save_map[fs])) else: cls.save_map[fs] = cls.save_count - ret.append((dep, fs, None)) + ret.append((dep, fs, cls.save_count)) cls.save_count = cls.save_count + 1 return ret @@ -309,18 +310,18 @@ class SiggenRecipeInfo(RecipeInfoCommon): return deps if pid not in cls.restore_map: cls.restore_map[pid] = {} - cls.restore_count[pid] = 1 map = cls.restore_map[pid] for dep, fs, mapnum in deps: - if mapnum: + if fs is None and mapnum is None: + ret[dep] = None + elif fs is None: ret[dep] = map[mapnum] else: try: fs = cls.store[fs] except KeyError: cls.store[fs] = fs - map[cls.restore_count[pid]] = fs - cls.restore_count[pid] = cls.restore_count[pid] + 1 + map[mapnum] = fs ret[dep] = fs return ret From patchwork Thu Dec 29 17:07:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30996C3DA79 for ; Thu, 29 Dec 2022 17:07:43 +0000 (UTC) Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by mx.groups.io with SMTP id smtpd.web10.208596.1672333654310683953 for ; Thu, 29 Dec 2022 09:07:41 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=MSBNR7NW; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.48, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f48.google.com with SMTP id z16so1435667wrw.1 for ; Thu, 29 Dec 2022 09:07:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=j3gxZxNly1tt9kHCQ0/8kkyqm5WtNyYvsg/tu1b/yo4=; b=MSBNR7NWTFhCriLTNxzLC5Z+4mnfNsBF9yBmqmIbg8tM8mJ/3Uhs1b7ix4aMadLSAX TM8EjI0Qh0mN3mnP71gTivRx1n6+Dt+c7HYPVVLeK7zlQ92xRpG8NxI9u21TLaOdxUCf eJjfwhoLVphJXzA8ANqlt0D9+vXAkaJ5Mjpzc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j3gxZxNly1tt9kHCQ0/8kkyqm5WtNyYvsg/tu1b/yo4=; b=8Jp6b7bpb8qn2bYore+Xgpk/ZaVCfkEGo3pgxjOavglQkQsEGbOGyWB+BA+9BowaYF JLC0JpoB8TY2VbsciELYUL3E7qlrQ3zz527eLBG64wTh3WC6z7nILTG4AHxJ/Qv7xb4X F8HK/5BkC/RHkZALJuOBuoWQfxLG/1geLsgs7LkpT/lNXizT62rWItYbFtnY7Kgz6Dlo 6WhA2Ep2WD67lxrrrlT9gxIohxOp4TLjUf6MpkMC9R3AZCDSpFRYgOX4DLIYhKQkpZC4 y4JDLgPJiTv8mcagUY89Hz+IjziIFz9AkPEwissaJWneT0y08+tysMDUaYfegO4nvd4v qxZA== X-Gm-Message-State: AFqh2kqHPPGSS5KIOOQ+/Q9JKRFN6nSytaQxOwMLYlJgIH/Goyxno9uG 8OskXJbJasccf7LQyUDTiaYjCymLbc3wEJEx X-Google-Smtp-Source: AMrXdXvVz3lYnGQY/EC0VOIUW9JBKfHvMbu2WTZS5IlPTg0KHq+x7Vw+n0CjeloBykGlQ/+AkiKRzQ== X-Received: by 2002:a05:6000:80b:b0:274:c846:4211 with SMTP id bt11-20020a056000080b00b00274c8464211mr19116322wrb.54.1672333660107; Thu, 29 Dec 2022 09:07:40 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:43af:59a1:5bcf:54fb]) by smtp.gmail.com with ESMTPSA id t12-20020a05600001cc00b0027b35baf811sm12308721wrx.57.2022.12.29.09.07.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Dec 2022 09:07:39 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 15/15] server/process: Add debug to show which handlers are active Date: Thu, 29 Dec 2022 17:07:28 +0000 Message-Id: <20221229170728.880367-16-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> References: <20221229170728.880367-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 29 Dec 2022 17:07:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14258 Add some logging to show when handlers are added/removed to allow a better idea of what the server code is doing from the server log file. Signed-off-by: Richard Purdie --- lib/bb/server/process.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 365758f85a..b69432ead2 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -112,6 +112,7 @@ class ProcessServer(): """Register a function to be called while the server is idle""" assert hasattr(function, '__call__') self._idlefuns[function] = data + serverlog("Registering idle function %s" % str(function)) def run(self): @@ -380,10 +381,12 @@ class ProcessServer(): try: retval = function(self, data, False) if isinstance(retval, idleFinish): + serverlog("Removing idle function %s at idleFinish" % str(function)) del self._idlefuns[function] self.cooker.command.finishAsyncCommand(retval.msg) nextsleep = None if retval is False: + serverlog("Removing idle function %s" % str(function)) del self._idlefuns[function] nextsleep = None elif retval is True: