From patchwork Thu Dec 22 23:47:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 17139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D4FCC4167B for ; Thu, 22 Dec 2022 23:47:34 +0000 (UTC) Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by mx.groups.io with SMTP id smtpd.web11.60899.1671752850177769269 for ; Thu, 22 Dec 2022 15:47:30 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=dnPp/g9W; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.52, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f52.google.com with SMTP id v124-20020a1cac82000000b003cf7a4ea2caso5044507wme.5 for ; Thu, 22 Dec 2022 15:47:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=pN4lANml8++f8XUrrE/hEDZrzvwj32RGT8aPCf00bUM=; b=dnPp/g9WU6G1nbqbofE9oFxKnPczONi8etRQcRgFazwaocA6qM/iyax5J5wdNOGFJ8 aBrnFZD3Y0BQRGAJlFy2Zhj9PRAwCjVfhq7hEp3g3Xp5+0YmY0BInSqhm+9OrYuMPOhd Q9n5LHOI5dEiA2xA06BsD1TAnIorV1nmMdas8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pN4lANml8++f8XUrrE/hEDZrzvwj32RGT8aPCf00bUM=; b=YYuu1xDsvjkxxZxoZZaWeLg8DC+TuU8Xlc8fLwkHmrE4gqZCd1EySDfU1fjZ1kaXen vvN7mo6nz3TKCI0OJdSmBBdhnQou4RqVgtBK7bub7JJ8ckBR+tLhrq7J4KOM61p+8pEt QyPGU+oXBIbNpr9YMO5Gu5Fs1hxcpryfHbf8WkLBcPV0xg58aypuuoMDQOY3Gf8Zm7uJ 4cca88J0J2fk3QqAQ61EQf9/c8aIX8S8LyLNdUZhlr+O+9xvPk+mH+9jleFx2PcwBi7b RvIUOu//T33oYw5egNEXy367AjIPUbs0IkWFfocrvgvLdWEDYZICSpG8ekDjsEGQFgyh fiZg== X-Gm-Message-State: AFqh2kqKsKJEv3Wlb3nuHBGjbmyy8tVQoagL1JY1tJgWfPhdf1KJj0IY CAi6cqefp2u7kzmwij2fOtdmQmlE5SiyKUAB X-Google-Smtp-Source: AMrXdXtMw1ht6NCY/ECvbHfCmoNZ64maBvNE6rdfJ1P96RjnGbH/Cb3+Sust5nVsb65DsnWgT4rQ0g== X-Received: by 2002:a05:600c:3596:b0:3cf:497c:c4f5 with SMTP id p22-20020a05600c359600b003cf497cc4f5mr5783882wmq.13.1671752848329; Thu, 22 Dec 2022 15:47:28 -0800 (PST) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:bc21:fec7:83cf:e0c3]) by smtp.gmail.com with ESMTPSA id bd25-20020a05600c1f1900b003cfd4cf0761sm7322189wmb.1.2022.12.22.15.47.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Dec 2022 15:47:28 -0800 (PST) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 02/10] event: Add enable/disable heartbeat code Date: Thu, 22 Dec 2022 23:47:18 +0000 Message-Id: <20221222234726.579702-2-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20221222234726.579702-1-richard.purdie@linuxfoundation.org> References: <20221222234726.579702-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 22 Dec 2022 23:47:34 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14230 Currently heartbeat events are always generated by the server whilst it is active. Change this so they only appear when builds are running, which is when most code would expect to be executed. This removes a number of races around changes in the datastore which can happen outside of builds. Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 4 ++++ lib/bb/event.py | 9 +++++++++ lib/bb/server/process.py | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 48c3002ce3..815610ff82 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1467,6 +1467,7 @@ class BBCooker: buildname = self.databuilder.mcdata[mc].getVar("BUILDNAME") if fireevents: bb.event.fire(bb.event.BuildStarted(buildname, [item]), self.databuilder.mcdata[mc]) + bb.event.enable_heartbeat() # Execute the runqueue runlist = [[mc, item, task, fn]] @@ -1500,6 +1501,7 @@ class BBCooker: if not retval: if fireevents: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, item, failures, interrupted), self.databuilder.mcdata[mc]) + bb.event.disable_heartbeat() self.command.finishAsyncCommand(msg) # We trashed self.recipecaches above self.parsecache_valid = False @@ -1545,6 +1547,7 @@ class BBCooker: for mc in self.multiconfigs: bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, targets, failures, interrupted), self.databuilder.mcdata[mc]) finally: + bb.event.disable_heartbeat() self.command.finishAsyncCommand(msg) return False if retval is True: @@ -1578,6 +1581,7 @@ class BBCooker: for mc in self.multiconfigs: bb.event.fire(bb.event.BuildStarted(buildname, ntargets), self.databuilder.mcdata[mc]) + bb.event.enable_heartbeat() rq = bb.runqueue.RunQueue(self, self.data, self.recipecaches, taskdata, runlist) if 'universe' in targets: diff --git a/lib/bb/event.py b/lib/bb/event.py index 97668601a1..21e9a1b025 100644 --- a/lib/bb/event.py +++ b/lib/bb/event.py @@ -69,6 +69,7 @@ _eventfilter = None _uiready = False _thread_lock = threading.Lock() _thread_lock_enabled = False +_heartbeat_enabled = False if hasattr(__builtins__, '__setitem__'): builtins = __builtins__ @@ -83,6 +84,14 @@ def disable_threadlock(): global _thread_lock_enabled _thread_lock_enabled = False +def enable_heartbeat(): + global _heartbeat_enabled + _heartbeat_enabled = True + +def disable_heartbeat(): + global _heartbeat_enabled + _heartbeat_enabled = False + def execute_handler(name, handler, event, d): event.data = d addedd = False diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 586d46af88..91eb6e0ad9 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -382,7 +382,7 @@ class ProcessServer(): # Create new heartbeat event? now = time.time() - if now >= self.next_heartbeat: + if bb.event._heartbeat_enabled and now >= self.next_heartbeat: # We might have missed heartbeats. Just trigger once in # that case and continue after the usual delay. self.next_heartbeat += self.heartbeat_seconds @@ -396,7 +396,7 @@ class ProcessServer(): if not isinstance(exc, bb.BBHandledException): logger.exception('Running heartbeat function') self.quit = True - if nextsleep and now + nextsleep > self.next_heartbeat: + if nextsleep and bb.event._heartbeat_enabled and now + nextsleep > self.next_heartbeat: # Shorten timeout so that we we wake up in time for # the heartbeat. nextsleep = self.next_heartbeat - now