From patchwork Tue Jun 4 14:28:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 44666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B641C25B78 for ; Tue, 4 Jun 2024 14:28:11 +0000 (UTC) Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by mx.groups.io with SMTP id smtpd.web10.23069.1717511286995462303 for ; Tue, 04 Jun 2024 07:28:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=DSiK28uY; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.42, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-4213b94b8b5so22175625e9.0 for ; Tue, 04 Jun 2024 07:28:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1717511285; x=1718116085; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=fuP5u9Q8awMP84BONylqqzQeYwLsCknEbIKHBhs8HTs=; b=DSiK28uYtzI/qI1mD9ddCTRPk8WfKeexajd5OIpgn4+tDHGHGhyZCM9ZjbRddCFFOJ 2j4qb+keY3CGVae8+HvBQJPVMyuuJOaVCWBwegkLf5T7lE1p1eexBUcuGv2Huv1r0bTt 1WatrT6zQvTUpVlXlkKX+2d26rxqKjIT/Q6oc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717511285; x=1718116085; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fuP5u9Q8awMP84BONylqqzQeYwLsCknEbIKHBhs8HTs=; b=u82Fve7iLe0JG1L47IgePJ+tNjq0WG9Tq96dwq26hwziOgziWYBwwURDzI//IyUXOB ldTbV7puBycl8cWOmJyMT0u4wMGpjHZmOdEIRi42mcgj+5i2WqJ5RXjCmDplQgtvs/uv fy1JRyVdtE4C8Tm7KOMmXVvZCmxQzKTQeRJfcOj4Y8CD2aTZ3J2x5hqXaruxA5Msg3nR dWKuDJdymlRrgsWP8QIuFadnffdsv46wq2dBdiWO0Hfou+In+pFQq4WZgscaGrF7ZRtn Etr5onfIsOXdd2CTiQ3a5piFE82NaMr2ZtzfODpaUOxXCGTbDxHwYzop1gElTOopf66I 2Qlw== X-Gm-Message-State: AOJu0Yw5BNgyHYiRzbHQMa0YUiP7qrnma9XZ0CUso9/G3VdtwlgGbYqS v922P5o0NGlNcu+rm3vJrCYF789HfzhC07DKjye6SInEcFejDJM46D8lspNH12eik9bNlsNyceA p X-Google-Smtp-Source: AGHT+IEY6oU4QBt7ZYiQAzUtF+v5+ttUQMomvXTZf8uTGuv4Ygksoc/+k3cSeDbSjwDEwqSjpfbhwQ== X-Received: by 2002:a05:600c:35c5:b0:41a:a4b1:c098 with SMTP id 5b1f17b1804b1-4212e076530mr108548275e9.19.1717511285071; Tue, 04 Jun 2024 07:28:05 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:9a5c:ae3c:1b90:53c1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42135c9dc18sm122043595e9.1.2024.06.04.07.28.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jun 2024 07:28:04 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 1/2] siggen: Drop copy_unihashes function Date: Tue, 4 Jun 2024 15:28:02 +0100 Message-Id: <20240604142803.3658144-1-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 04 Jun 2024 14:28:11 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16316 The code in OE-Core using this has been replaced with something more fit for purpose. Drop these function calls as they were never a great idea in the first place and cause IO slowdown for runqueue needing to sync the cache. Signed-off-by: Richard Purdie --- lib/bb/siggen.py | 6 ------ 1 file changed, 6 deletions(-) diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py index 92066da00c..89b70fb6a4 100644 --- a/lib/bb/siggen.py +++ b/lib/bb/siggen.py @@ -201,9 +201,6 @@ class SignatureGenerator(object): def save_unitaskhashes(self): return - def copy_unitaskhashes(self, targetdir): - return - def set_setscene_tasks(self, setscene_tasks): return @@ -418,9 +415,6 @@ class SignatureGeneratorBasic(SignatureGenerator): def save_unitaskhashes(self): self.unihash_cache.save(self.unitaskhashes) - def copy_unitaskhashes(self, targetdir): - self.unihash_cache.copyfile(targetdir) - def dump_sigtask(self, mcfn, task, stampbase, runtime): tid = mcfn + ":" + task mc = bb.runqueue.mc_from_tid(mcfn) From patchwork Tue Jun 4 14:28:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 44667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A00B4C27C54 for ; Tue, 4 Jun 2024 14:28:11 +0000 (UTC) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mx.groups.io with SMTP id smtpd.web11.22950.1717511287760044612 for ; Tue, 04 Jun 2024 07:28:08 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=cUUzaPWa; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.43, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-42155e7084cso133155e9.2 for ; Tue, 04 Jun 2024 07:28:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1717511286; x=1718116086; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=XLW355yoEJzF60+PYNejnUdrPxpupUaotjsBd9TeMkA=; b=cUUzaPWahzX4yD+oaS6iacn7HjSCwoslfrxGvrTn+ggxneOJ9sXA6cCGAbY/frPzFx M4Vo05DzYo/t5c3SrhKGspjoeBGWNgTtk0HpR5W3QSVeDjxPqX6G9QiLIWYF6hoZPLe8 HFuDKO7bhqXxe1g1JaqiBwhxOLBtRKUw98WLs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717511286; x=1718116086; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XLW355yoEJzF60+PYNejnUdrPxpupUaotjsBd9TeMkA=; b=cVW234OvsjEgW2RNPTg8CepAAaSA4BZJtxkZj4Fs5xxjKoeGEEgBgd/aU+oSU0G9jI PjlGHLCkdo21UwqnusaetbCjm56woVAB5gDb465osHBuaf35KvOvbgAUL6K4qWTbsai2 XGT6CnMYG7Wbyp/f1O0nLIku1g2uxpNbKUhg2qilJaWowvzOWhUvYCm7uRYxpa4Jid0z o8AIfdTm725VvztuosdcpVjqlH7lJL20e5EvmV4X9z4zJhZHiHR48ZO6LA/giFe20wco 4apSY+nJeyNPuRK+L1ekhUnULchCRoE3tHCReIxFjyOTd998t9EollJC+PRmJh+0YnMg goZQ== X-Gm-Message-State: AOJu0YyOf5hTPYtFvuu2sm29CccYePuset3sQRNiobRPS/QXFtHczsU6 ei8eaWfMwHMJJGMMnesznvKhiGdUZd2rhGTNxJBsihHV0XT2Sxqbo6jCoc8WWMlfeNZc9wkwglg H X-Google-Smtp-Source: AGHT+IGuF0+B+Si4dRqknU/ADLY7JqKqQ7ghqEjQqhseSjYFv3vDq63RtLMYCm3IJxzD29AwKnhPCA== X-Received: by 2002:a05:600c:4503:b0:421:48ad:60b3 with SMTP id 5b1f17b1804b1-42148ad6427mr26809065e9.34.1717511286074; Tue, 04 Jun 2024 07:28:06 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:9a5c:ae3c:1b90:53c1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42135c9dc18sm122043595e9.1.2024.06.04.07.28.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jun 2024 07:28:05 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 2/2] runqueue: Avoid save_unitaskhashes Date: Tue, 4 Jun 2024 15:28:03 +0100 Message-Id: <20240604142803.3658144-2-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240604142803.3658144-1-richard.purdie@linuxfoundation.org> References: <20240604142803.3658144-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 04 Jun 2024 14:28:11 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16317 The save comes with an IO overhead which can slow down the rehash loop in bitbake a lot. We only needed to do this when recipes were doing unihash cache copying. Now they aren't doing that, drop this IO pain point. Signed-off-by: Richard Purdie --- lib/bb/runqueue.py | 3 --- 1 file changed, 3 deletions(-) diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py index 93079a9776..3462ed4457 100644 --- a/lib/bb/runqueue.py +++ b/lib/bb/runqueue.py @@ -2550,9 +2550,6 @@ class RunQueueExecute: self.rqdata.runtaskentries[hashtid].unihash = unihash bb.parse.siggen.set_unihash(hashtid, unihash) toprocess.add(hashtid) - if torehash: - # Need to save after set_unihash above - bb.parse.siggen.save_unitaskhashes() # Work out all tasks which depend upon these total = set()