From patchwork Wed Sep 9 07:30:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 97654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AA72C79FB5 for ; Wed, 9 Sep 2026 07:31:14 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6472.1788939066171472512 for ; Wed, 09 Sep 2026 00:31:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=kiV1HxWO; spf=pass (domain: smile.fr, ip: 209.85.128.48, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-49d0b98d6d0so34874075e9.0 for ; Wed, 09 Sep 2026 00:31:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1788939064; x=1789543864; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=Z6Wl5+FBEJ14tfTnZD/ml3b9Yj2ybFHs1P+REAu7OYc=; b=kiV1HxWOuunVefM3LGd60SlgaujDeh5g0RI3HWkHRss9mVQZAcIPilz2kCmGXJB98W cOOhvuLtk8PLofsDll0DnlJIx0BOp4fWfts7jT4NP07gNbBco0cLgzcSD7EPBoeAPsq5 FbmKNm/npUz+WnLMGajqtI5QFlLDVi2b+kWG8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788939064; x=1789543864; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=Z6Wl5+FBEJ14tfTnZD/ml3b9Yj2ybFHs1P+REAu7OYc=; b=giDPeIsQm1Dt/3lFtWGpD9tJHSUgRaDp/9EnXyCprMxUUAmyqZqJhwHns19ktxnZPF uVhYtTRQ3UJrfEDAwmKSqNPYMTpgeDK5ao9tq6KxexUTO2iNrtdBEi69q+ySA7DX1Tb7 Zko7d9LFLpk7R+g9v3GKXkMGtGKzSod5jz6xi/6816EjJsECY3sjJ7ItegvMVw4TSdSM 4uKIYel0MSyGPOLa+qFn60esgcD9O0fILy4FQEwq2UjJAr2IjlL1pUKodlcFMxtQm9fy 00aN1XUST3Aax64tcnnGSEJQC9iDeIElQVluRBj9qhQOLXnpJhg+zeQ1UW2OiCP9WNtB 7DNA== X-Gm-Message-State: AFuF++lc4yathZN35Lo1WQEIylxquwV+0SnEr/l1OI0qjxcV71KMfHDW NgPeVAT+/XUr9Ucx8tGCM9ftlfZT+6IOPXfnXQNRjLTnPqarPCAdrDcbVF15+TYaAPVpYNPYMeW +l6Tpm6M= X-Gm-Gg: AYBFou34ctgOxxqlmVoPACifnz6JV9nRChfwOhcLr/raEHvRnDhNfDxGsEG86Ntjdgo mIeY83eAZJ4RsHbciHJGU3+Cl+6qZ7lnGuqNIOYnjBc13hSjKqOCZFCYlRETwbK5j/1/PGMmczY 7tqkFMNprcOrz8bl1rrkl7bAxIvgKfd4ibLUPVdkDoed8eevLVVY33i9jY0Dp88CpFUjVOzRO6H d43zzari6Pahx1wmUU/UOjArKiTuTCZJyNEPylrhifS7hhNC9Prsp9L0bQg1kcy/VHGWAIe0zVE GmFZ5FzE9tMIc6uj3CSsUW0O/Y7fFCy9zmkTzG38WXyA8iZ42wr9E22j3Qjcj9cyv/hHuKkDhaf pgtMsOBU0T5yoe6MaKo8oK6A9E7JXwSWtx6RxH/o8AYFL5qkZji79AVaqsNMq/IHsMCqanY9fdS IX9ELMIvh758/9lLpDo5Lx2HCSI1+Ue8m7XvKImu37y2XM4GhCcdzJ42fchzYP9k+cSh0DknmFQ dU+ThwfeeL05eyWDyTtKSwFXyGlbdzkXDkSj/DwW/2p9AaYRbyonMo5qAFv6L+HOnX3Z4aX9kQ= X-Received: by 2002:a05:600c:8b05:b0:49c:edd2:855 with SMTP id 5b1f17b1804b1-49cf823c411mr358311465e9.6.1788939064281; Wed, 09 Sep 2026 00:31:04 -0700 (PDT) Received: from FRSMI25-LASER.home (2a01cb001331aa00a2e4fb7b0d887544.ipv6.abo.wanadoo.fr. [2a01:cb00:1331:aa00:a2e4:fb7b:d88:7544]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cee5f912esm553491385e9.4.2026.09.09.00.31.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Sep 2026 00:31:03 -0700 (PDT) From: Yoann Congal To: bitbake-devel@lists.openembedded.org Subject: [bitbake][wrynose][2.18][PATCH 1/4] data: Return a list from exported_vars() Date: Wed, 9 Sep 2026 09:30:57 +0200 Message-ID: X-Mailer: git-send-email 2.47.3 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 09 Sep 2026 07:31:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/20173 From: Alejandro Mery exported_vars() was a generator, so each value expanded when the result was iterated rather than when it was called. bitbake-worker binds it before bb.utils.empty_environment() and iterates it afterwards, so any expansion deferred until iteration ran against the wiped environment rather than the one bitbake started with. The corruption is not uniform. The worker writes each variable into os.environ as it goes, so only the first value expands against a fully empty environment and the rest see it partially rebuilt, in exported_keys() order. Recipes whose exported variables expand a command during that loop hit this. A gitver-style PV such as "${@get_git_pv(d, ...)}" runs git while PATH is empty, so the git wrapper on PATH is bypassed and the real git runs directly. Under pseudo this fakes uid 0 against a repository owned by the real user, and git aborts with "detected dubious ownership", failing do_package intermittently (only on reparse, when the value is re-expanded rather than served from cache). Build and return the list in exported_vars() itself, so the expansion is complete before it returns and no caller has to know it was ever deferred. A value that fails to expand now warns at that point rather than during iteration, and a result nothing consumes is expanded anyway; neither is a change for the one caller, which consumes all of it. Update the comment at the worker's call site, which documented the generator that is now gone, and add tests for the expansion timing and for the warning that moved with it. Signed-off-by: Alejandro Mery Signed-off-by: Richard Purdie (cherry picked from commit c870f5bd96ad02efc02c58133c27bdae3a300e3e) Signed-off-by: Alejandro Mery Signed-off-by: Yoann Congal [YC: removed patch history from the commit message] --- bin/bitbake-worker | 6 ++++-- lib/bb/data.py | 10 +++++++++- lib/bb/tests/data.py | 28 ++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/bin/bitbake-worker b/bin/bitbake-worker index aa14ef191..ae21f277f 100755 --- a/bin/bitbake-worker +++ b/bin/bitbake-worker @@ -288,8 +288,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, extraconfigdata, runtask): else: logger.debug("Skipping disable network for %s since %s is not a local uid." % (taskname, uid)) - # exported_vars() returns a generator which *cannot* be passed to os.environ.update() - # successfully. We also need to unset anything from the environment which shouldn't be there + # exported_vars() expands each value, so it must be called before + # empty_environment() below: an expansion that shells out needs the + # environment bitbake was started with. We also need to unset + # anything from the environment which shouldn't be there exports = bb.data.exported_vars(the_data) bb.utils.empty_environment() diff --git a/lib/bb/data.py b/lib/bb/data.py index b12972c03..7c01e6bf8 100644 --- a/lib/bb/data.py +++ b/lib/bb/data.py @@ -193,6 +193,12 @@ def exported_keys(d): not bb.utils.to_boolean(d.getVarFlag(key, 'unexport'))) def exported_vars(d): + """Return the exported variables as a list of (key, value) pairs. + + Every value is expanded before returning, so a caller that changes the + environment afterwards still gets what the expansion saw. + """ + exported = [] k = list(exported_keys(d)) for key in k: try: @@ -202,7 +208,9 @@ def exported_vars(d): continue if value is not None: - yield key, str(value) + exported.append((key, str(value))) + + return exported def emit_func(func, o=sys.__stdout__, d = init()): """Emits all items in the data store in a format such that it can be sourced by a shell.""" diff --git a/lib/bb/tests/data.py b/lib/bb/tests/data.py index fd690a9e2..a83c71e2c 100644 --- a/lib/bb/tests/data.py +++ b/lib/bb/tests/data.py @@ -723,3 +723,31 @@ class EmitVar(unittest.TestCase): self.assertEqual(self.get_output(out), ['bad_chars="a\\"b \\', 'c\\`d \\', 'e\\$f"']) + +class ExportedVars(unittest.TestCase): + def test_expanded_before_returning(self): + # Called while the variable is set, read once it is gone: the value + # has to be the one from the call, not from the read. + d = bb.data.init() + d.setVar("TESTVAR", "${@os.environ.get('BB_TEST_EXPORT', 'gone')}") + d.setVarFlag("TESTVAR", "export", "1") + + with bb.utils.environment(BB_TEST_EXPORT="present"): + exported = bb.data.exported_vars(d) + + self.assertEqual(dict(exported), {"TESTVAR": "present"}) + self.assertIsInstance(exported, list) + + def test_unexpandable_value_warns_and_is_skipped(self): + # The warning belongs to the call, not to a later iteration. + d = bb.data.init() + d.setVar("TESTVAR", "value") + d.setVarFlag("TESTVAR", "export", "1") + d.setVar("TESTBROKEN", "${@int('not a number')}") + d.setVarFlag("TESTBROKEN", "export", "1") + + with LogRecord() as logs: + exported = bb.data.exported_vars(d) + + self.assertEqual(dict(exported), {"TESTVAR": "value"}) + self.assertTrue(logContains("Unable to export ${TESTBROKEN}", logs)) From patchwork Wed Sep 9 07:30:58 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 97657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E125C79FAA for ; Wed, 9 Sep 2026 07:31:14 +0000 (UTC) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6473.1788939066589844387 for ; Wed, 09 Sep 2026 00:31:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=DGpwz8K2; spf=pass (domain: smile.fr, ip: 209.85.128.47, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-49b965570d7so62283535e9.0 for ; Wed, 09 Sep 2026 00:31:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1788939065; x=1789543865; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=UHP5nm6A7C2oRk+g5YH6jBWyYD2p3QINf1KMLxW4Qbc=; b=DGpwz8K2IVGWlPufJ6vpOlPCH62oXRFHxkMn1Xk75ak0qMyhGETVHI6h7m8I8AYPaM kSe220UHkyGNdyrEeWa+kpBwZ2YhqKWODs5F7xBB5xj9qk1X0my9rBjGRJjS3Q/ym+qu 4D0SEI4r3Mp57MsgOiyh7+WleX/CbOcyejUDc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788939065; x=1789543865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=UHP5nm6A7C2oRk+g5YH6jBWyYD2p3QINf1KMLxW4Qbc=; b=c6QnUVHFYYr6pq2ny/aFh2wOld/zud+CVoqL33SJf83xNvj4yakCuj+Y7DZBb0XRc8 K5N6F35S056kuaaSXfB78iZ+5VFRLap4qHEr1f/5+9NLFU3UEUtctI5G2wYo0nN6yci/ 65n+T+DHzf0/dH2HqKmp8Ln4dUj9FEouQH4s23LDh8HIHH3dzJlOFcowRUM7WTSfG/M/ dW9ilR1qk5cjI/0OylktMcheU6Z+paEBae1Cx1PZuCd/Nx6gq9HyiZIw+e/7yhnL9dor YUctuPL5YxmfnSnIntaOny57Dr6yQCfz6xD9QQuw/qv1uJcSdhPIPvRVX2FGsmKz8yOf URgA== X-Gm-Message-State: AFuF++kamZlPaSszlOk6B2HOYy6jo4UrQiyGeW/qMhfdrmWjcLF0o1XF Pt7Geu+eGKkCFkgCd6/zXXAeWWLJuB0jVT4ki5uD5D/LPldJ+gna94S/a1JAX7pIbMYPJihjBlx X4EY9a44= X-Gm-Gg: AYBFou32h9Yh02E/7pT9+9+TEK4dtDvejao6iLZ3316uQLUpPUU57zzS9FC0b/v7k2s 2PJUyXggt7RwKvMwelEpUoWh7K1+2f5ejeUTv76EI1TvBI4CfI7yTyehgdhxBVuoX1/DybQq6UJ oLSsMpq7Jkc0TlGlQAF4Rf1yVQgykoVtcfqckfarjUv/BHkNtoKNv+40/06Ao6V8H3Tn/wR2KyV tcfo5HkvFwr05cwJCCJXRYUkSnw2344e79v0xV3Gv3qNoyuhQN7BeJZbqpuXWOPO7nv9PKM4TTV vavWP8fplLdrwH9HrVQ8Ozq7/tlhM2iOY4szEIXTgjsR1Np1SoRMbBH1VpOl0jXfiPk29F5z3wb lX49BruL7BmpVgrVRhCc3k0PRtCEvtgJksZeGeF9+BIuqBx8KVpiy5cxdumwVDTQwjRsnzPMhVg BdXRXWzlUnlAPbBjZEbWVQnIjASRGCGiQhKSwnWEs2HVghmy/s7jWcuZt8v5fX85EfIiRz8pIFF 1KyhejWR2Y3lr+Gcuzb5C5YbcHSnp1+634K/ofVDLLNVhDsSejq8QQsDyx1oVYhTrvc9AjOwBw= X-Received: by 2002:a05:600d:4447:20b0:49c:fc6e:8cbc with SMTP id 5b1f17b1804b1-49cfc6e8e40mr217289185e9.32.1788939064718; Wed, 09 Sep 2026 00:31:04 -0700 (PDT) Received: from FRSMI25-LASER.home (2a01cb001331aa00a2e4fb7b0d887544.ipv6.abo.wanadoo.fr. [2a01:cb00:1331:aa00:a2e4:fb7b:d88:7544]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cee5f912esm553491385e9.4.2026.09.09.00.31.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Sep 2026 00:31:04 -0700 (PDT) From: Yoann Congal To: bitbake-devel@lists.openembedded.org Subject: [bitbake][wrynose][2.18][PATCH 2/4] fetch/gitsm: Store the original url data to fix relative gitsm paths Date: Wed, 9 Sep 2026 09:30:58 +0200 Message-ID: X-Mailer: git-send-email 2.47.3 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 09 Sep 2026 07:31:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/20174 From: Richard Purdie If the gitsm:// url contains a git submodule with a relative "../" path and the fetcher is configured to use a premirror, it will fail as the host name encoded in the original url is lost and the correct paths can't be found. Save the original url data in the new data so that it can be consulted to fix this kind of edge case. The test case change that follows this provides a test of this functionality. Signed-off-by: Richard Purdie (cherry picked from commit 58872250f72f5760b4126552bdec5497f5a1a1f4) Signed-off-by: Yoann Congal --- lib/bb/fetch2/__init__.py | 1 + lib/bb/fetch2/gitsm.py | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index e0494b3cb..b9373d2a9 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1052,6 +1052,7 @@ def build_mirroruris(origud, mirrors, ld): newud = FetchData(newuri, ld) newud.ignore_checksums = True newud.setup_localpath(ld) + newud.origud = ud except bb.fetch2.BBFetchException as e: logger.debug("Mirror fetch failure for url %s (original url: %s)" % (newuri, origud.url)) logger.debug(str(e)) diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py index 5869e1b99..8b4ec6209 100644 --- a/lib/bb/fetch2/gitsm.py +++ b/lib/bb/fetch2/gitsm.py @@ -88,7 +88,10 @@ class GitSM(Git): # Convert relative to absolute uri based on parent uri if uris[m].startswith('..') or uris[m].startswith('./'): - newud = copy.copy(ud) + if hasattr(ud, "origud"): + newud = copy.copy(ud.origud) + else: + newud = copy.copy(ud) newud.path = os.path.normpath(os.path.join(newud.path, uris[m])) uris[m] = Git._get_repo_url(self, newud) From patchwork Wed Sep 9 07:30:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 97656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC082C79FB9 for ; Wed, 9 Sep 2026 07:31:14 +0000 (UTC) Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6474.1788939067145357977 for ; Wed, 09 Sep 2026 00:31:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=y+2oGKe2; spf=pass (domain: smile.fr, ip: 209.85.128.45, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-49a97714f5dso54042235e9.0 for ; Wed, 09 Sep 2026 00:31:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1788939065; x=1789543865; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=prHlDQSCWdtk7MIZW5fGZh31yv0LV3FTrB1H4yGpnV4=; b=y+2oGKe2CVR6AEaxn+9FcgdYtjpVQ9V2eFL8DvXSHRhH9aEfsDkuR3r7FQmipWkwsh DQGv3N/sV28Rv1dy9m2DQkLQ8yv506ppCqtr9WQY7Jk8dZi52xmWLj1+tv1Ds0qbK7AL kwi5x7iDLV/4pC2mIn0KSg5l2tA6WMo+1kGtw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788939065; x=1789543865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=prHlDQSCWdtk7MIZW5fGZh31yv0LV3FTrB1H4yGpnV4=; b=q0H2cEhHxbznQxEzwd8g2CgSUpka62a9mNlXgOT/Zn6vvObUpOKOXbkkWRk+5yfejT 3ZMJC/oqqECw2Q1lEvfMs5jKrr/itAcw29uX9ns2a3SvVsxXeR7O/KpJQ0iAv3QE4EBi bNTB6Fc3lWW13hKIx4QpkqRvNMZu3xg7rQcwuxWGTDL+3S/p/FHFpga+emyIt7OYZjMu BI3ENnQYhLG4hq8v592oN4dj2BevpG1QK5t7gmSsA+CqJ4VtzWxmYKCvNxwLGt0I0W75 v/1lmWCWHCAccvYqiVhGIxf8DVlFXfO/Rm9qgRzNC/EiUBxyasF6XQ9mA7pRdwyNc2YZ 5cCg== X-Gm-Message-State: AFuF++khyzU7XQqR0VPQBxI39tZ4/cXsCC3OJMZwZbBb0NvhDHFEjdQL Qe5y4Q62h5pqbDDsuDlqVIXM0abK9UZtWDxmC3qirV6mhEkmFre1b25zlzjF3DrtldBYv4dT9UN YbqUv+jI= X-Gm-Gg: AYBFou1Y9G3+3vP3nc2JOmEMiCrjk4xa71fh6/OMQjsmqk6DByWLkte9aqo0K2enSK0 /+F7CsVgW8Ur2XcDDTOrLfXRBF8sboH/jvm26VXgW19F4uWCOURyyMgXuPA/wCdT+Sj0IRMcL8K wVn1r9F+lBfhreSI6eE6nyAxhaf0o7iT4HBdyw6Mx0qqbPWz21HV+bkRPbBhupgU2q9oxZv7i8c d4Ev8g55SJzOACbmxkYVff7IuSBK5sikQt3f8EJXvriX99KuljCPWNo4sZ+DfF5lFRewQtjZ33K ID/8NGQGSIFQWn+rDXhDfl8dDnRe+sksI/ZfVyZUEPAu0EZs5y1kIRqnASYFIzsgWgF1TMi6KE8 QsR6jvcrV/FFoju+5fGBTHwic4Ig9Brp+sxWOgGJfr4CA8uRyAJ3NzV1mDphuGAFIxuT29mJ/dg ca0pqlHhMBOKJS0Ig6C7G99vs2lSKEPxkHrD9rmOEYGamUEts/r4RTmNy9iQQjJXri+RYm0G6dA qFyHGoyXECoyLZPTgBR8H5PU0OD/LtSccHRJXheKiS7DnbLuPIF5cA4G2dzRNjqVtK45J6GPQUh X-Received: by 2002:a05:600c:3f08:b0:49c:fa20:cc04 with SMTP id 5b1f17b1804b1-49cfa20cd10mr290700915e9.27.1788939065250; Wed, 09 Sep 2026 00:31:05 -0700 (PDT) Received: from FRSMI25-LASER.home (2a01cb001331aa00a2e4fb7b0d887544.ipv6.abo.wanadoo.fr. [2a01:cb00:1331:aa00:a2e4:fb7b:d88:7544]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cee5f912esm553491385e9.4.2026.09.09.00.31.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Sep 2026 00:31:04 -0700 (PDT) From: Yoann Congal To: bitbake-devel@lists.openembedded.org Subject: [bitbake][wrynose][2.18][PATCH 3/4] tests/fetch: Swap github grpc repo for our own small test repo Date: Wed, 9 Sep 2026 09:30:59 +0200 Message-ID: X-Mailer: git-send-email 2.47.3 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 09 Sep 2026 07:31:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/20175 From: Richard Purdie The grpc repo is 1GB in size and makes a poor choice of test repo. With github rate limiting, it is proving probematic. Swap if for our own gitsm test repo which is smaller but can serve the purposes of the test. Signed-off-by: Richard Purdie (cherry picked from commit a64b3157d74f883002318cc46c70bfa1c3609110) Signed-off-by: Yoann Congal --- lib/bb/tests/fetch.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index c071401e6..845741060 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -1199,7 +1199,7 @@ class FetcherNetworkTest(FetcherTest): def test_git_submodule_restricted_network_premirrors(self): # this test is to ensure that premirrors will be tried in restricted network # that is, BB_ALLOWED_NETWORKS does not contain the domain the url uses - url = "gitsm://github.com/grpc/grpc.git;protocol=https;name=grpc;branch=v1.60.x;rev=0ef13a7555dbaadd4633399242524129eef5e231" + url = "gitsm://git.yoctoproject.org/git-submodule-test;branch=master;rev=38e61644af90dccd73c03ed3acaed98c8dda9294;branch=master;protocol=https" # create a download directory to be used as premirror later tempdir = tempfile.mkdtemp(prefix="bitbake-fetch-") dl_premirror = os.path.join(tempdir, "download-premirror") From patchwork Wed Sep 9 07:31:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 97653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FC77C79F8C for ; Wed, 9 Sep 2026 07:31:14 +0000 (UTC) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6349.1788939067686061460 for ; Wed, 09 Sep 2026 00:31:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=Pm2VCigA; spf=pass (domain: smile.fr, ip: 209.85.128.53, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-49b8687630fso46284035e9.3 for ; Wed, 09 Sep 2026 00:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1788939066; x=1789543866; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=+enxi87nW/c8WyuswESt2WsoCe3FGwLPq75U9krjQ9Y=; b=Pm2VCigA/o+r95ApVE5oCdtNEAi3u2tsAuqzCfnfw0o5emuxpgpljnKIFkvzi/71FM QUGVGJ29M0HW7zRjZV54ho5a40Q75MwYP8ZpFn3cjbZWg5ahKZ9OcNZGunaH/LBuOeDR ezc8R8CyjEUuXpB9O3k1SAWAB9hP9kINsTLIc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788939066; x=1789543866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=+enxi87nW/c8WyuswESt2WsoCe3FGwLPq75U9krjQ9Y=; b=JDh/7DgK4FWRizIbfBmWzAfcZvpELZYxJqY0BdlyPiJIsQJdPKpHIxrpV7vHBiMdzv jyYoDz9CT9IdH/dHfQdjO8R6+5g/GguaeJx+ikGjw1u39nXvOTvtdsQnyw9OZWq65Qzy GnvLiltOJaUXY1D6jaQOexZ2vnK71X9JXTfx/yuY4EqT0QE/+dXOAcMNuUeIQArbT3+8 QMnBkri/SH0ljwn9u+0sdZ2OSRRBIJvJ1HYT+LfKFSIPMwSJ+T3NHgf1dgXjDpFfKRKj spuxBT3gKHxoUueJxwJYA5IepPgU/wX2UbU0LCA+hobyIKI8ss6MgUExfz+d0Beyvi9i faCg== X-Gm-Message-State: AFuF++mRVo9XmVyRhoWB9kC2W5iOyQaEghiLmYn9kM/X3Y+Lfx+G96yU +QHGs3/sowD6mdcvE3C+b5xpllY2PvVA/iNL5ozKvfKOl9iU/OwehTI4DJYUXktGuhUXxwrWm9b RrD1wWSU= X-Gm-Gg: AYBFou3AyGev+tTSdEQ0c2wxPQQvY4zjjM5M2YwkzDfrHrKfqPEUav/jxxC+kzX3skz 0V/unJIq4VWs7FJ2iIL35FynLaXu3Cn4mwva3P75C/IYNQ1s2mPayXANOTCwpbeijRG3NN74rNT +gbF3MFA5hCEHtvzVHug8XtGaKJoVvR9RQE5nMzmq7Hu7lbdFgIDrMs1KXJtcwighrc+wmuFeFz PTjoPNwYBGtalb4MSVqe/oIViGxXCgkjaKBhyj1/Y2DjwldhJYyuKjQJWF2UGptgaFYwi0Lprq/ Qqg7hAGMJ36kXMrhVTcFLtuY1CrNw9lJO50NKKu1eeJ3tdS3y1Ly0HhjFEczWO3quRCu2rZNEhk lrSSLkEr1KV9GvMj7GW+7xXaUbNjPkf8DL6D4FGuIfplAgrlC969+D5m2u1IboCBNVxTFDxhEio zZjEITgYlVsrCTOWa+0GquAtv5EpfgBsZ74n8tQEDWfHnhFqPWlHU3mv+SUypHW9Q/WH4NRpyFt HtERsZ3OdeAT8MrcMB1sqiXHtgfB7oJeD2JKu2eUCqchivrcXEJ0c1U2d4s5mLdwFo859X1+qw= X-Received: by 2002:a05:600c:6819:b0:49c:de80:b833 with SMTP id 5b1f17b1804b1-49cf81fb1acmr316990675e9.2.1788939065903; Wed, 09 Sep 2026 00:31:05 -0700 (PDT) Received: from FRSMI25-LASER.home (2a01cb001331aa00a2e4fb7b0d887544.ipv6.abo.wanadoo.fr. [2a01:cb00:1331:aa00:a2e4:fb7b:d88:7544]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cee5f912esm553491385e9.4.2026.09.09.00.31.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Sep 2026 00:31:05 -0700 (PDT) From: Yoann Congal To: bitbake-devel@lists.openembedded.org Subject: [bitbake][wrynose][2.18][PATCH 4/4] tests/fetch: Swap bitbake github mirror for YP one Date: Wed, 9 Sep 2026 09:31:00 +0200 Message-ID: X-Mailer: git-send-email 2.47.3 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 09 Sep 2026 07:31:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/20176 From: Richard Purdie We need a reliable mirror of the bitbake repo and github is no longer that. Swap to use the bitbake-contrib repo which carrys enough bitbake data to work for the tests we need. Signed-off-by: Richard Purdie (cherry picked from commit f311ff38b4bf79675f677f73f1d390723b1c4490) Signed-off-by: Yoann Congal --- lib/bb/tests/fetch.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 845741060..137505343 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -575,8 +575,8 @@ class GitDownloadDirectoryNamingTest(FetcherTest): super(GitDownloadDirectoryNamingTest, self).setUp() self.recipe_url = "git://git.openembedded.org/bitbake;branch=master;protocol=https" self.recipe_dir = "git.openembedded.org.bitbake" - self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https;branch=master" - self.mirror_dir = "github.com.openembedded.bitbake.git" + self.mirror_url = "git://git.openembedded.org/bitbake-contrib.git;protocol=https;branch=master" + self.mirror_dir = "git.openembedded.org.bitbake-contrib.git" self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40') @@ -623,8 +623,8 @@ class TarballNamingTest(FetcherTest): super(TarballNamingTest, self).setUp() self.recipe_url = "git://git.openembedded.org/bitbake;branch=master;protocol=https" self.recipe_tarball = "git2_git.openembedded.org.bitbake.tar.gz" - self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https;branch=master" - self.mirror_tarball = "git2_github.com.openembedded.bitbake.git.tar.gz" + self.mirror_url = "git://git.openembedded.org/bitbake-contrib.git;protocol=https;branch=master" + self.mirror_tarball = "git2_git.openembedded.org.bitbake-contrib.git.tar.gz" self.d.setVar('BB_GENERATE_MIRROR_TARBALLS', '1') self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40') @@ -657,8 +657,8 @@ class GitShallowTarballNamingTest(FetcherTest): super(GitShallowTarballNamingTest, self).setUp() self.recipe_url = "git://git.openembedded.org/bitbake;branch=master;protocol=https" self.recipe_tarball = "gitshallow_git.openembedded.org.bitbake_82ea737-1_master.tar.gz" - self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https;branch=master" - self.mirror_tarball = "gitshallow_github.com.openembedded.bitbake.git_82ea737-1_master.tar.gz" + self.mirror_url = "git://git.openembedded.org/bitbake-contrib.git;protocol=https;branch=master" + self.mirror_tarball = "gitshallow_git.openembedded.org.bitbake-contrib.git_82ea737-1_master.tar.gz" self.d.setVar('BB_GIT_SHALLOW', '1') self.d.setVar('BB_GENERATE_SHALLOW_TARBALLS', '1') @@ -2406,7 +2406,7 @@ class GitShallowTest(FetcherTest): @skipIfNoNetwork() def test_bitbake(self): - self.git('remote add --mirror=fetch origin https://github.com/openembedded/bitbake', cwd=self.srcdir) + self.git('remote add --mirror=fetch origin https://git.openembedded.org/bitbake-contrib', cwd=self.srcdir) self.git('config core.bare true', cwd=self.srcdir) self.git('fetch', cwd=self.srcdir)