From patchwork Tue Sep 2 06:55:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E717CA1005 for ; Tue, 2 Sep 2025 07:23:16 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.110]) by mx.groups.io with SMTP id smtpd.web10.71614.1756797794202570130 for ; Tue, 02 Sep 2025 00:23:14 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=MHghlYxC; spf=pass (domain: weidmueller.com, ip: 52.101.65.110, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oJjUGt6DCcMnqAcuYyc1lENrcLLw85EVf0dTgp9eEglK2hnstxeWydVPb7E5zgGGjTJyJu4lvTQ0s5elG7SiHeaFqAujCB3ni6tSnZszH24N+gp2SIyo3hxpcMxhnpdbisGHfibKFxjgrg+rOeVRYROlUdcDoHUpuayJ3yY1TQ1N3Z9W767PVFcrGxNCOGbu+RI+dOwXpCp89j2+wNroHqBfhhalqRtS31+Pozj7KAeduFAKDqrsFZjqU3DPLCS9KrD+frcLN/EEolSKqOskyxQsLbLRBTldnL2GSAUiVrAKnHmzQo4QfyVEu/A0qMxGSeO4sQxPWNLmNNRxnVQTag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bi9BGBSKl9qQyc3mrCBRxEu5LFf1qGOp6oGJWigkmug=; b=rDcGVlu6lOmrB5Hnvr8oOMmmpdoZn9ipdxxGq+U2GY8aF/ck0gkYWAvqWAboeYq6qCJo8hVr74r/DxxKAdlucj2eTCjBCGhQX88cp2K+Bb8qcjgtIp89Ab/+u+gQ6B3zL4P62n8yfdp6COduUes3FtaIas9hD/7DUPMo+ElUbkwlCg/AjfMwFEK/ZVBVtVDvwpqC0SrpSvLaOCAqamfYPKGG1bkl/Nh44O2L3dD7QedJTrBUoOx7IQGLej+RDBs+KaTsOnnJm3RrP0C26pgkBvIJK5mfgfzmi9bxvD+VkNw+oDVvhVkIhe44nyfMvP0eUzibPoTIP4mQ0vxboDz6Ng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bi9BGBSKl9qQyc3mrCBRxEu5LFf1qGOp6oGJWigkmug=; b=MHghlYxCovSn0uuW7TRwY5uhN+00WNIfi3P3y5QGhEWcu7dF5lJErT7t23TSw5B3ixeScJteHCVOt72lDY3yy/sejKBKviZBFXH5S1afnoqC8t/xHgyZiEuAhMXZ/Usqy3eipnCJF7jMqK1TGPTqFmijoeHZhj+ajXzQlgyH6LU+n8UAkLEP755CTaiMZkfKxxZKM8zHzjeYKNjuj+3e1BTjqw5xEtgwg6mkeTVmey1dGGV4tV4FnuI1/3tvz5uoMQg57SjaWljTpBSWHBBEXIPylO5ukt/Z2Y5zqILjzYXbz51OOJ6I9mrPpb64IXDSGIkOlxRx6b5/h3xRXwwnYg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) by GVXPR08MB7678.eurprd08.prod.outlook.com (2603:10a6:150:3e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9073.27; Tue, 2 Sep 2025 07:23:08 +0000 Received: from GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013]) by GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013%4]) with mapi id 15.20.9073.026; Tue, 2 Sep 2025 07:23:08 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [RFC PATCH 1/6] fetch2: rename u to url in Fetch class Date: Tue, 2 Sep 2025 08:55:02 +0200 Message-ID: <20250902065507.35737-2-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR5P281CA0012.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f2::9) To GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1PR08MB8426:EE_|GVXPR08MB7678:EE_ X-MS-Office365-Filtering-Correlation-Id: c4e8777e-180c-4a1b-379c-08dde9f19098 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: PqLB67jIMBHdyYuTC9eEcqy12J81O68MGxRrFqNUuoUxIhiUXbNw/sfubr//zaNW20r1bG8t/AqlhG+rPdHbTTqTMxhIwPBxDIdhVWPvMryZNFQFHma8GIVPwYWDv2sSmLmlfV8hzGG12b9gw8MGul8UsF9B3REMwWTrUoSs3l17qSlTyy8+/khH+uyvThq19vyE8xHILXLhS83rliTmIGFYHzAOEXgfjYdP3DC0aEEJ3IfuJ6uFNmomrY4mK5kQXlHAnQksH5xR613Sct2RNVHYuIgYfEOxlPAwaRW62l/+qgekFbNj3eo07L8VwgBwz5vUWx6wXYoBuvkGjkeVUCjEaomgvNznmRWmS+3BoyWPUZ3DETYvs4Tfgxmz4EF9DenJcNSavOgx1NkmdjU5bDOt9PDoRBTgETy3whJYNmOaXg7+ssGd6ca+03Qinyb3jR7tZU5rTRmhu7Da0WQ4lXYAlMGCSvw1mTfWosb24E/lxreOsnA/cVRFoAprEpl2tkWXYl0Xlkx5aXwU9p+sLCwEJs2SjguhCeji8YFQM2sNqSzuoKkkK3h7JV+XFjLUcgzYm73dXcUQlsxtqjq5TWRHKRjphEhe/JkIf7fwk6TcuqJ9wGCUmtQXtYPhySS87Bhsa/71FWhQEFwgWiRTZgd0k7ya+9UjeVkoeWeKzNwQKGOYqqGwL0gdi0+HkSAGf5bCmi4+fez2RMOFiZ41E1NCCdIcogMzApCjC7c91IJGbLL39QJXCW5+Xkv47jl09aUHYwyqnhSQ9jcpPFNgL1I6Fr2huJdeCnQt2Et4U+Dp1kEODtHDPT/luaptp2o57C9D8MyvaIMMFvKNWqWeNqc0C2HH7ghk/k3l/IIAO0azPy6js1ul2L+Z4wnKXXgk+2kWYXaUlsEch9KZl2IVTzizQrhQEw2JEmtBAGxK/gSve9Nw3B7D4wzX71vsAj1FK0Lu5RhY2yulS2JFYPGz2SPhUJ6TU/N6FCvIFPFWBpOxmfi4XEAmT7fYYHK8Mo7AVpGZ3jENu6YpsrhX/OHW2Zbj4+7BVqPixUQ1hfzJvmdBXmHN/vBQozpWo3trbVL5oLdUuWSVoGAkecHeHcJ8+ji/U8vbqbtJaWPBj5A7FNZ43ESJvwnmunEUicHX16GvKm4CUlv4mBdJFFGw/CAhfeeB4gVkPX/Mj9cdvS6gNy+qrx/dUWnfGJgCmzA691JquRULiUQrhqcJJMiTsMrbLPuH5UTOVsnTVlL96q8qrxH1cnKIVUJevF5IqBp+OL+eIqMw7bsWLmlz+UJyyFW0rWU4lk+/q0DumNjSmRZkUwQDJbeYrne3yFL4634s6Gz4IYEaMwwAnqES2brUqVf11NhR3/C8K58jViCXbLSbRxaeKKqhXRpUX0XHoQq2JBeY3McbP0wOMNUF5xUXdZtF7pUKPAzE+0N1aFvgNG9P5BTS9mnw2OZ5sQ4PWH02mIjFlGFz0S4LD5bJ86WmrUy1wA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR08MB8426.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Yfcos0+Ip41yW5uV0uykzI+MzZU/lz6t1pj7WzmW0JmYnxOMwn5w5Nl+s4ssPKZ25tuNNplu9sjxtjyx4QInc/pXR+FFq1UQyBR4jsZwcPOAtSOkgwVk5rThlpNPP7KS5VWrdyrVGruAieOuDEbDRUNcwvFopjHBLPk84b0eeizt2jcpQMNnw4fWDTijnlwFJWJxvZvAsBoYVXSd9EKzpIJcCaBwWS2BYtLNXFuRy8tdXBaWb648N2vDo+cqcwb2ZYMnvnJv/NxNYajttVDtOQ156DA5vuo6wr6pUbD2xnQCTyua7ZbalGGo+Fkny5XQ/OOzCS0CfHz9JsqnurG7Oe4c7JrZONPqsvNUF81LxN+HVt1S1BAQPxRID+C4CXukIlNNCKDAmDHtAJK58/GcYDkfa/TrIlJrtk0+9U/ddIIN5IwS4UF9mx80OOvFSYO2BV/LYauWZnYuCfUfl+KnsQ29jMkZo1Mt+3PN8oLU1Mt8l5d1mB/mEDCf/CX8YSlPOqDFrvhirYeO6YvN/vofa5PVo+qBsYJbLRM3vIysBCVMcol5MmwH/dk6ZUHDatPnVRY19TxpRHFKKsQ40THhw48fHCS+VJsAdnKE9DXhaiTxoiXovNo51KjUlEpemGs+TUS/DRxM8mIUtzkFoMwtj8/s/aqD0taz9SdclGHKMHNKIbVOmWCKLj69hTNFwDrl2SENMlv/m0dMQrAoZ7tzql1xSWxFTpl1uDQ16nQZMWkWtGHyIqv6dJdmQZFkHOF1RDdyRP667UPyBX0d6fr0Ag0F/H8ZEaBv93uerBfMzhIgH7mPtu1RfhjGrifoE9C/1wzzfIyr3HJyUlJLMHUxbVnC47P1A28lzaBC4ItXRugfPTUGCD0Kobwe5cb3EFGM5EOoU9icv9i6fsOEX/6bwkfhW1uTSI64kJu7AXEIgqhwkjlnLi6GdroC8cbavvJKCo1VCnnMszouui2jFepGlN+I0znOkJ/yivj5a1Wz/7W7EZ+tXAEcvZIraArvrHlBGXsCmCQPX2CJfxcZXbGGeLi2h5jce4LdL1CXHMmCeIJFppD5gDIxxBYO029edK2dcnjkVuixHTu7EDEG7kpGp38sKQ3jWRM9cRnrXjSqFm2Luefwf/3gwWy14noEai9lV7DFlAwGM5UBBMZlR2qk3ZlUxoFZ6JpDlyqvUs9V+B/Y5rs1sy3Dflbm4AiU7vdUoI+OKKVVQNCBbPi5tHG3m4a2LOCnsj1FxxoIt0BjVGrDXFxWQ504Nmzt9M2xnslfVfurOqvnX99iuNxsL9QHkPOi19HR4xvmdSDqm5ixZVTItJTEu7vLyA09CxUrFR/Yz7Iqcp11a1E9pACHl+dupIHgszVj8/4py9eU3u6aTFv4tVTjMmx+Bl0I9DlIBw7JEu/FgHqrzvmMoNRg9Dm7OJ0GMRvc9Ej5TZTmrxPuQ0zkwwvNBP1mflV8AHmIm34Y36RHNGQtmNa1zdO5MeOVdYdb/LHH3Vpbm8B9C86eVjz7871ffawJl9ersp2h9/YsDykSbvCDQnCNoEPflcD4ArGt50fNH9syOk4wwp5MwHjzoPnrRtY0+OYEbSWA61HCaUoet4FihRp5W5X8riMgKw== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: c4e8777e-180c-4a1b-379c-08dde9f19098 X-MS-Exchange-CrossTenant-AuthSource: GV1PR08MB8426.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2025 07:23:08.0425 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TzBXVCAN7AMO8AkL/EtroPNWDVW9+G/iPSgBOPO1jLJkHAFx2LbHaIvD3nO1oprjm66zPrU21gNAeaRD/jPhcQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7678 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 02 Sep 2025 07:23:16 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17935 From: Stefan Herbrechtsmeier Rename the varialbe u to url in the Fetch class to clarify its meaning. Avoid using short variable names over long ranges and simplify subsequent changes. Signed-off-by: Stefan Herbrechtsmeier --- lib/bb/fetch2/__init__.py | 42 +++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index de301543a..bc9808da1 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1841,8 +1841,8 @@ class Fetch(object): """ local = [] - for u in self.urls: - ud = self.ud[u] + for url in self.urls: + ud = self.ud[url] ud.setup_localpath(self.d) local.append(ud.localpath) @@ -1859,8 +1859,8 @@ class Fetch(object): premirroronly = bb.utils.to_boolean(self.d.getVar("BB_FETCH_PREMIRRORONLY")) checksum_missing_messages = [] - for u in urls: - ud = self.ud[u] + for url in urls: + ud = self.ud[url] ud.setup_localpath(self.d) m = ud.method done = False @@ -1882,7 +1882,7 @@ class Fetch(object): # contents mismatch the fetcher can still try upstream and mirrors m.update_donestamp(ud, self.d) except ChecksumError as e: - logger.warning("Checksum failure encountered with premirror download of %s - will attempt other sources." % u) + logger.warning("Checksum failure encountered with premirror download of %s - will attempt other sources." % url) logger.debug(str(e)) done = False @@ -1914,14 +1914,14 @@ class Fetch(object): except BBFetchException as e: if isinstance(e, ChecksumError): - logger.warning("Checksum failure encountered with download of %s - will attempt other sources if available" % u) + logger.warning("Checksum failure encountered with download of %s - will attempt other sources if available" % url) logger.debug(str(e)) if os.path.exists(ud.localpath): rename_bad_checksum(ud, e.checksum) elif isinstance(e, NoChecksumError): raise else: - logger.warning('Failed to fetch URL %s, attempting MIRRORS if available' % u) + logger.warning('Failed to fetch URL %s, attempting MIRRORS if available' % url) logger.debug(str(e)) firsterr = e # Remove any incomplete fetch @@ -1934,13 +1934,13 @@ class Fetch(object): if not done or not m.done(ud, d): if firsterr: logger.error(str(firsterr)) - raise FetchError("Unable to fetch URL from any source.", u) + raise FetchError("Unable to fetch URL from any source.", url) m.update_donestamp(ud, d) except IOError as e: if e.errno in [errno.ESTALE]: - logger.error("Stale Error Observed %s." % u) + logger.error("Stale Error Observed %s." % url) raise ChecksumError("Stale Error Detected") except BBFetchException as e: @@ -1949,7 +1949,7 @@ class Fetch(object): checksum_missing_messages.append(message) continue elif isinstance(e, ChecksumError): - logger.error("Checksum failure fetching %s" % u) + logger.error("Checksum failure fetching %s" % url) raise finally: @@ -1971,24 +1971,24 @@ class Fetch(object): if not urls: urls = self.urls - for u in urls: - ud = self.ud[u] + for url in urls: + ud = self.ud[url] ud.setup_localpath(self.d) m = ud.method - logger.debug("Testing URL %s", u) - # First try checking uri, u, from PREMIRRORS + logger.debug("Testing URL %s", url) + # First try checking uri, url, from PREMIRRORS mirrors = mirror_from_string(self.d.getVar('PREMIRRORS')) ret = m.try_mirrors(self, ud, self.d, mirrors, True) if not ret: - # Next try checking from the original uri, u + # Next try checking from the original uri, url ret = m.checkstatus(self, ud, self.d) if not ret: - # Finally, try checking uri, u, from MIRRORS + # Finally, try checking uri, url, from MIRRORS mirrors = mirror_from_string(self.d.getVar('MIRRORS')) ret = m.try_mirrors(self, ud, self.d, mirrors, True) if not ret: - raise FetchError("URL doesn't work", u) + raise FetchError("URL doesn't work", url) def unpack(self, root, urls=None): """ @@ -2000,16 +2000,16 @@ class Fetch(object): unpack_tracer.start(root, self.ud, self.d) - for u in urls: - ud = self.ud[u] + for url in urls: + ud = self.ud[url] ud.setup_localpath(self.d) if ud.lockfile: lf = bb.utils.lockfile(ud.lockfile) - unpack_tracer.start_url(u) + unpack_tracer.start_url(url) ud.method.unpack(ud, root, self.d) - unpack_tracer.finish_url(u) + unpack_tracer.finish_url(url) if ud.lockfile: bb.utils.unlockfile(lf) From patchwork Tue Sep 2 06:55:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77040CA100B for ; Tue, 2 Sep 2025 07:23:26 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.110]) by mx.groups.io with SMTP id smtpd.web10.71614.1756797794202570130 for ; Tue, 02 Sep 2025 00:23:19 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=VGE/UN4j; spf=pass (domain: weidmueller.com, ip: 52.101.65.110, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=zPYaOkww4E8H2gByh8XdFuBSNcAzXL/NOTGIF23MHW8jPGiugPfpGmuAk/5y1tTtOgNhWivR0B7RJY//0WS/+riCe4ete53Rsvmkd0+J+ohSRU3VAIlY5EY+Uv+85ubNRsZsYTpZ5jS0r+WY3Y9WLiQsKd0rUbmKmh8jJOODW/7RP8HqAoVfyxnE/vEhDMQhjIcaEHtgfO2kdYMm756JbvdJ/s+wmQhpmFZbFy2AghKIcmNkPgIPsAc9Reoef8Tu8g2/Ta/ELESY4xhBdDS2nznD02Ov10UOsEgMOeo8Z8kdcLiyQL5ppHguSPp4l9SjZLfGCWGBWU8vD4PAzqtfmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bNhJlNuntziFIuzpj/rkcazv3h8DaXRaQPq8UKwKwqQ=; b=JzYGPxxZEyLgdsQcskQ3lEXfkvq+WP4ctQS4UN+fDIiWlNNFHuZvl6s9fa1QYMry9x7rN8Q6YERwWax2VnDF40niUmPxhsXnyrF/BhlBZJDw7ELuQ3hN3p1wWHiYSbA1YWxhNVdv5scy2ou212+5Xc/IX3ByIeI0cIokZUkrZv2M4KUxdafBUihY8auktg7s7s7jXGkVFpXCq1BFnjUZKzWdwlPjoiOeNIbhltKIcfdf0D0Fs/wF4g6KQ+lttU3JNR/RiyK7eiNLjGD/QGLlf5hZSaDZCruxfYx1BY486Vb99R9Fo6RH++xkeaMUkvZ47dFVCc1ZqavvjLLC69JBQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bNhJlNuntziFIuzpj/rkcazv3h8DaXRaQPq8UKwKwqQ=; b=VGE/UN4jncTVohbke723hIGOwH6WFiF+qci5RPGpUYoc4LMa7tCBSuGi40c3bBTcHXAAA3bUeuFvvAMJ/JG/ZNcGE/3lfT+2PljxDED7nI0F9pOiuSqnKPVt4vVwwsmR1R5/xf+YGj/rjcMFwtd0IR3RLlhkFXYy2gF5n0GvJgmzKQHFTvsJ4/MnbhbupG/zYWPc3H6KlsrqSWh/5IQk7qNqMNoKLgl6vwkKOGktfVDD0EnpvJ40PnJP4o+GVWA/EUr7vDf+JbxB0cqkDJx2g0jozGbmlKI9pJ/35aT0VHTnBM11I7/tnUYs2sdDOcWfkTF+idYhYii5mbQXJLtWdg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) by GVXPR08MB7678.eurprd08.prod.outlook.com (2603:10a6:150:3e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9073.27; Tue, 2 Sep 2025 07:23:08 +0000 Received: from GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013]) by GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013%4]) with mapi id 15.20.9073.026; Tue, 2 Sep 2025 07:23:08 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [RFC PATCH 2/6] fetch2: call functions within loops of Fetch class Date: Tue, 2 Sep 2025 08:55:03 +0200 Message-ID: <20250902065507.35737-3-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR5P281CA0012.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f2::9) To GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1PR08MB8426:EE_|GVXPR08MB7678:EE_ X-MS-Office365-Filtering-Correlation-Id: aba03a68-1868-4fd6-3302-08dde9f190db X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: lJAhhqnccgjKRwiA/DqswXh8lAQvPfTtvTjnU+sIJsho7Yl+9LkhiwDbMBsCs2ZReSLO07muaZwzcA8SJBCykfyNOHq1F+neYQxks0W7/2pICqzQXhiwgq2OQmjxysQ6zGGeyNybSSUIhtjkZ2hDeZz2bFVEg+T4ru/0reNwPkKzIMu2YFwcsfSMaZT0OHqwtLQUhnpTlVDMcunlVXzV7JFqkMBo4zKeIaibnnSWDSEwPDKs6ggydOFbiUgVhCR3J+yzoXJfbgGo4TByY16Ol7c8oL3bn/QHRnsd21fdi9M+l1bS8Reile1EjE6oUa0upw12KLE9WGkJJmnXXiOEcoRoEgqkjKKRNbba+AOhF9BkEHMxWyb7z+3Z75M3nHhl3VZAWXI3DLKLNgsj8cWDUSIsH/DVdayAi3y2oxYTBHAEOj02CvnM1wV6MILlLCuFQihMwcL3UX4q/5CfBgJkJqS6yTe92gUNVG622W9vxg7Lij+SZsmDh+1Zg1GaZm65QN9uT6PuUapuUjSXNiJkGP9o8oo+h64Ci5eXb2ekCijuBsJLvxbMJWoFB+13i8qsy+qPIUtq2MguZHmhxi19T49GRojHURosv2Tak3YDIkxmZt7xO7ZrQDmYt7y7mBiojHGYYboQW8T3kuXO3Q0mt2pi4duCmf46wnugOA5aMorGIMQxyeWscRdvuGJhCu9cw68JTTQyPE/ZZ1TSYbZq61IWDnuseGVqEN1O/CFXBqjdD8us/Vw73SZONnrBa6WeclxDqJTEVVvQWQePbnfTdpEJBAKd1Cg+O4Cm4no2gSVWQtPPxJQ2JNAmPpNQgJzbv5VYB6KWl4f3kSv/bi682r/EuxDmNQDzAadwbvy2cCqUtlw3pzKvxUfz2HzutBZu49X4f0b3q1G+En54ZEkIP5ENXAPXjzQK8mIRDYs9hCD/5XaiNBCGUM8ZbPwtcYq7g52HJLmTeiGl4vfUfZlEB39wTJkti4Rk/SvJAi39dmtqvLR0rcYqwEHdwL+pgmvi7EYFp+fMomYXStfEEIWI90nvEwt3+853lbqpqc6oJjO5pNwpSSJ0VudlF4B7pGNv/V+riWAyEPjdxpkHirot7CPZWs2ml8XaaiNh5WCrSA/Bb7woMN0p0nFF9xf6+MybdwKB9EiKSiSttYrUaM3UKXKXMHe6IqIchmXJ42Ic/RZVn6dM5CgqQyOy5YuowL1Yb/78e5asfmOguK8fq2k4c5CEb4PEotHcEaceNgcUofRRnGA/6cLU5HGlb2N3onlCu3HN2zy5xWrH8NewlmDYfhCKbGgQcsi+1SV4deNro6oEmmydvIMpjwEG/41Jxk1DYzKtHUb1ul3Gh6L2zBKgBGO1uEptxwDtGSE8UYU8BEi51q99h0cPO1x6WJgIoZqH/rexqb3EixvLgj0xIZAG/jRXMxXkD93NX/+TFmzT2XHuUaR35fevgRkOGB1krrPL7pavNi+uFrtbr85IbIh/gA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR08MB8426.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: VbAqN2cNJLXhQehzE2A/v0yH9asllRg1AG7sfDhSFhMHCUl0qAOupc9psyebQyxRyFIemySQmsvKI2C1hfSj2FKcPl8n66n3w96fPz8jO2QP11GVexlnQaA32A5b2Nw1p0HY78JteGPtJvRtaXAJnpMKyz9+qpFYMVQbb64Vv1znrxl/aVUcN39+nX++AKtKX9aQmluP7i9Yr/49HoMnHHt31qK2UaM5Xaf4JUpGiv31YLGIQVwjIHgSnaq16Ie6Gt+RpmbqpPvzZReZHEN5UTzuqCIhhp3Drh4eIzY1Vi2dw2TcFuULBB1pqVBgaytgfgzUcOH3W0pq72/x7isQrO3QVnjqud6FQPPYo5WW/K1w6XJ7ZXN30xq770cJNu58EiW3JmfIKRQUhunAdkTpJd+L7E0XHbi5pdv1uIoRVJBHEp46f1oi2ggQhUzrdbM5oXZkbDpvZxmWu6csvvuTN9tk88CAlsx8P19rPutlbCiM33z+3QHNVF7+M2Tciog6rC1dLOrm6JIdap/IoK6o1JZj54aXakjmCDdht03jD9GtOUaI45ld5AMy2GRK2TGTWprS7oYsXettcTXmHgutrRKDoaf7NgenFH+DL4FaYmIU1afCJvWgjtKe8L2V2em6wQ4EYU73xbnE85jCKY2hjpM1uOzRxAKO9qsb1v3ltrpPjdm4cUb1sL2l3Pv6uinmuSwIf4xS5hO1Pd0HM+QrxbpsvaIVP77A5fhBCX3tJ32ZQxDf+vSLdq0Cfy66ARe3vBhFjdCTInwNDTWaBobtAH/ajqe5GqIk9W/bjh64Ge6nl2Qb/SLtvZOC6gUF5x9Pw5FzCuRurQdSaq2kgyulolc7zGTVP5s/EDtYTeYPV9Ulf/5slmLhGQpTCEsZOPGDOM1P7vQiEdwuoyjBcp6Py30NdYTriK+Ir0D1bUoJSMPcaYDoDWhT+MCO6cxGljGUU4pLCtaF+C/CqwpFuL1BnimMH+lpD8XTMBRE+lDVsoMTNCo3R0RtawMDLRaQLIPyQesRPUa7Z9pNCR716GzQO+EA3qwZbQye7Lut9A2CFOW47DGiFc1/ER6Nr+CuH+BqiLGpXN3d2Y556AaAAPCNS9Uu0q7cHqwKGDZi35pWkKpsYmPejvvZ3cNXWSzOJ54tsxLz+H4513GcbgLRts5XbJVQQuDO5omc8C27wS0u0LjjhzfzriLkcXMio95PO20YFPJfgkuThNZiNpXkPnS19fwhj+LgcuKv/qVrHlD7T9UorVPRr6lsVEWknX/eEeCvSyPVDiHmxosL03Cf2ZpE8ijAsbvuI7G/iiG8uT6QyDQxDod5OujVcuSrn0EGl1qnYDAYtxNzQnDS/m1SUNATr71EgQzCaJqyVflR5YRxTLeKsbaRKGqreRDblydf0+reo8K2n+brqei4/CYVoqyY0+n1x3mYaeqMKWyU+tCQTL00aLlonUx4wKCyEfvfUxy91dq1u2e0GgCrBVvXKzim4nhFZeGFzN3g354rkLYdXIbNPy2ojgkXYZt1NoW8Z3/Oo0A5hmgSQIMXstbSgG4GTDkooXLM18lACwlGcdHfwR7pAB23HfokP4SXeYQeCSVHttSDpzFV7VAkhaYBL6cvMw== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: aba03a68-1868-4fd6-3302-08dde9f190db X-MS-Exchange-CrossTenant-AuthSource: GV1PR08MB8426.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2025 07:23:08.5561 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VmXOo0Zoo2wczmpKe49tCAyUtCRF02NqMPH2M61diSjkgpn5XfoctTidycd20xg7KiGVMrsnD5afZqeBF1vFKQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7678 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 02 Sep 2025 07:23:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17939 From: Stefan Herbrechtsmeier Call functions within the for all URLs loops of the Fetch class to simplify subsequent changes. Signed-off-by: Stefan Herbrechtsmeier --- lib/bb/fetch2/__init__.py | 41 ++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index bc9808da1..80f1a0c08 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1841,11 +1841,14 @@ class Fetch(object): """ local = [] - for url in self.urls: + def localpath(url): ud = self.ud[url] ud.setup_localpath(self.d) local.append(ud.localpath) + for url in self.urls: + localpath(url) + return local def download(self, urls=None): @@ -1859,7 +1862,7 @@ class Fetch(object): premirroronly = bb.utils.to_boolean(self.d.getVar("BB_FETCH_PREMIRRORONLY")) checksum_missing_messages = [] - for url in urls: + def download(url): ud = self.ud[url] ud.setup_localpath(self.d) m = ud.method @@ -1947,14 +1950,18 @@ class Fetch(object): if isinstance(e, NoChecksumError): (message, _) = e.args checksum_missing_messages.append(message) - continue - elif isinstance(e, ChecksumError): - logger.error("Checksum failure fetching %s" % url) - raise + else: + if isinstance(e, ChecksumError): + logger.error("Checksum failure fetching %s" % url) + raise finally: if ud.lockfile: bb.utils.unlockfile(lf) + + for url in urls: + download(url) + if checksum_missing_messages: logger.error("Missing SRC_URI checksum, please add those to the recipe: \n%s", "\n".join(checksum_missing_messages)) raise BBFetchException("There was some missing checksums in the recipe") @@ -1971,7 +1978,7 @@ class Fetch(object): if not urls: urls = self.urls - for url in urls: + def checkstatus(url): ud = self.ud[url] ud.setup_localpath(self.d) m = ud.method @@ -1990,6 +1997,9 @@ class Fetch(object): if not ret: raise FetchError("URL doesn't work", url) + for url in urls: + checkstatus(url) + def unpack(self, root, urls=None): """ Unpack urls to root @@ -2000,7 +2010,7 @@ class Fetch(object): unpack_tracer.start(root, self.ud, self.d) - for url in urls: + def unpack(url): ud = self.ud[url] ud.setup_localpath(self.d) @@ -2014,6 +2024,9 @@ class Fetch(object): if ud.lockfile: bb.utils.unlockfile(lf) + for url in urls: + unpack(url) + unpack_tracer.complete() def clean(self, urls=None): @@ -2024,14 +2037,14 @@ class Fetch(object): if not urls: urls = self.urls - for url in urls: + def clean(url): if url not in self.ud: self.ud[url] = FetchData(url, self.d) ud = self.ud[url] ud.setup_localpath(self.d) if not ud.localfile and ud.localpath is None: - continue + return if ud.lockfile: lf = bb.utils.lockfile(ud.lockfile) @@ -2043,6 +2056,9 @@ class Fetch(object): if ud.lockfile: bb.utils.unlockfile(lf) + for url in urls: + clean(url) + def expanded_urldata(self, urls=None): """ Get an expanded list of FetchData objects covering both the given @@ -2054,11 +2070,14 @@ class Fetch(object): urls = self.urls urldata = [] - for url in urls: + def expand_urldata(url): ud = self.ud[url] urldata.append(ud) urldata += ud.method.implicit_urldata(ud, self.d) + for url in urls: + expand_urldata(url) + return urldata class FetchConnectionCache(object): From patchwork Tue Sep 2 06:55:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EE9CCA100E for ; Tue, 2 Sep 2025 07:23:26 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.110]) by mx.groups.io with SMTP id smtpd.web10.71614.1756797794202570130 for ; Tue, 02 Sep 2025 00:23:20 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=xKPFBJtG; spf=pass (domain: weidmueller.com, ip: 52.101.65.110, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Jap6TYkeh4vIuLMRrKx86pqYsoFMkxR11bbZvHvr1PoAAqcGlC6sgMVpvl4Oysq0XeiWbg4jzUfQgPx2K45EIsr7o8qvHL1Li3Dvl+VpmUw4ysz6uLA/sTSACMexomfZGirAH6/dZ8kEhZ0ZxPPKVVXtokensCzDNrrJmqUztwYlSeoj8F03zEg33mqe9CqIRBkrCo1wLWZZ1qStw2ji/eW30+BDiaINBRtW0JE7BgpJbi5FGM49nniwhbXGi2M5lyH+dPHZrzmv9bq1SynhOKLZQTuaBAfxsNX6LpR3AKrvH7cBSqbV+bdDSo/2ASwMLnJVGUcmDI0Qu3NFmELd6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kXc30iOzwHnw1svFcZ+NyNzL3YUsMSzvpacAe+E8W3E=; b=zRWbTOhBof0hRNhNqAJpJQ2uEWUdGrnhKA4awzXOksBi0OKBndOGga2VBSzBS65dJ9GZR/rsIai8OnVszR+dcocYAZYguJPv/bmD8aPgKygj0Aq07+Hw5HGhxBCQ9jP+73dJrwkRCsW+QJoW3ovgyxrv2StiUdAcBhrpY9yyCa4FVCSpt4fyAsRg/y438Wps2hYuF7JhAn/YpX1EWGiE8EP3XD2n6f+EnPsBZqIjPnYDjwdV361ApmZgoGiCG1zOYTTYl0YPDNwLp58mjslvR1A1H9zG/e4Juqc8QUX0Z8IKdQTe+BCxjxau9sJ0YeCdtvaZt5Rmxn1e9vFiRI+h3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kXc30iOzwHnw1svFcZ+NyNzL3YUsMSzvpacAe+E8W3E=; b=xKPFBJtGamr00Au8mhwTCkFYE7UMPsUKYSqwKvmpicLR5xiwAz0dh9mePoODHL+rxqwGgpGTcNa8GaNBjKgQ9TH+9j7enLAAgCb6TQFDmQPDlRM4jwhoUoYjQ9dXjB/zbDeZAXvhKzhtdSm/JRcoTWy78re8cET+2Uh8wEZzyCErUBFXuwvwzvn1wU4ohdcxlUWAT4GgvQfQaEW7s93Rud//3mHr1U1naI7mOpsFxtpyzLLZhKuxUTdhtwr+m7Bjel0rN1OWj2uTToABv9yNWKRgO/taZtrTqF3sxYZjlq4yFK8rWmJzXEia2pg70RSOwF++nYpt+L584FxKHb8nfw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) by GVXPR08MB7678.eurprd08.prod.outlook.com (2603:10a6:150:3e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9073.27; Tue, 2 Sep 2025 07:23:09 +0000 Received: from GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013]) by GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013%4]) with mapi id 15.20.9073.026; Tue, 2 Sep 2025 07:23:09 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [RFC PATCH 3/6] fetch2: add helper to get urldata in Fetch class Date: Tue, 2 Sep 2025 08:55:04 +0200 Message-ID: <20250902065507.35737-4-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR5P281CA0012.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f2::9) To GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1PR08MB8426:EE_|GVXPR08MB7678:EE_ X-MS-Office365-Filtering-Correlation-Id: 84e067bb-026c-4743-6319-08dde9f1912c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: m2lLxrxovwGv82/2O4mIpG8/F6yNOlcg6Vt148+5v5uEmbkMiFo26X+6Ldve97IsiGKuPW7lFXyXOH+J4fojEZ9yRsWx9rCCc6tSc5zEYeIo9fsXpNyeDmHdDl98G7iw/mEf2FNDEAaPiuUNISL2APd9fCyqySsTjqZMPZPldYa3gnMpstt4MBnwOn4RzYqzOhseIzFEWKIZy/5/G5aQyj4ReVCcH9sgOvnxTt0A0A3878M6u1Pxzp8E3xbq2UODB5az/MTMnUF4f9O4NJ8GoA/8q/uXpouI9gmyecQvRdE5UHEUXBbPOrVpV5OwsLxX5RqRVoxwYGFbVW9AS0naAUwEg4qA0hcyg9dobxk1sIsXRFlRMKVhsdDO6lznG4JIZnJSlgx2OQ0mLwpPZ6tpv8KwA7LiAaV23pqRnUItnWbJ91xKPDqaoqEW1VC8cNNQ3mKjhUSKqKtcbFsYzYGnzI4KSinyUooV6+ENsGFoDZOdOB/jUFDpwEMBFTclqpP3AYD0f8AH55jAnM8zkKjExUOieirmIypt+TuhWpZX+pPdxumK87s4MG16OGPWO03VgkhYsgYW/RYvZ0qVaLwQv2GIEHLc46JND5rSroFy1tOdPQ604gX3/mOMXTB605p5SCGWZEt0+9KSzd66jJZVgIBHlrGE0upTycwgzQDzwJreFPS2zIdysuA85BCqmcQCWMM8r7si+mbVtiojCVpBdvKqs+6ORNIlAz8ku7FVVhT4snsfJ8jddwTMFKFRUDjg2NJMFpN9gzNBMs9SuPy7tR5vU4t++xO6Q2Oqe95Y/nhtMoF7Eyw6Vqp2bV1zDEkowTbdSyju85vD5CbkeJr4mRDgF4eoJxC4Sa/aGm+JKUS3J60ZvGskBL096m/HgLf70lk81OtL1oeBFQpEXvt8UP4qvZmqjL5NhWhNK+SNy1qtF1o1lH7L/0Oxe8UJumH9unCQQWUCh1arktye2qjJrO53LovIa6y62IvveAWsusA7J4i+GuhYRay09OzeuevdvrrLw2wr+mlPEa2jFFtGuTVX+vFKEXT9YBV/hlvJH1eoDcht9in8acJczyISWE3EDB0hIfPy+r4x5frxLOP4wkfSCKRasR/ycYkeZjLzPCXOms9kxIJJvRaExCsWx2wX1NapLNDFWN2F2rRrnSx1WeZTdRxMapHnlsQcDLtVlLdEYYvPox+XRfgiw8G5v4GCzNTPqeH6FkBYtWJtmXKjk7a5TBK7bn7LVUwfwCif66Vt41jx02nNy3oZw4lrbd/SK5qzaxz4rtfxdH7lAD9Z+wto2TX6wZSyFzIHbY/SjA7DF+hd+G2ScJGeAwm2mucO0DoUYKUWOMnhuMHonDa+++kE5DfFSkcFG9V3nGw5/Lf+JLDlyaxUQikGw7sKQ+snr6eIEiYJVbtiY20RdF5OdX4EX/R5dizGZFpk96yQ4EfbSu11Dgh/VdKBFu06vNzBXx/emApOpgSJOHCjlwLPgA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR08MB8426.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gg9ahCYwyCoYnTE0xEUqgLnPjZxnHUIl7SDdeF6+/0FChcU6YaoFiQ3fufjhdjnFH9+emDVL7RJ0wOu0wYkP9vP9H9SSv0/nLZ6pmZe56F8KQdO+i8EaxJH7HxYep4gFipza09IJMDhlvTVr4PGpj06BJIk9fdqInK9dA5WS8X4542K4p6z5uO9Xj9pMdKH3P5ChR+sam1Jf1YPs7rrD4sfrCkOryT4y5fGTUsr8VOVf55D0HSX35HmfpPvTAjzjItOltiIRWVBPZ5DVM9m/tAYdr8/X7vipy6FSkZmTN3gI7W8XrHLlNp03QwnV2E4/JilnK09ImUjX41IbQqkZfkHd4Ji+Rk9Wtveu3SZCCXxHkuR0tGmzoiEebsYxZdpW64ahpX90WB7m/hjIXamxX+Ya0VvvWDmTzLTXXuk5h3L7H/bpZqevMS51cj0TAp17YaZLupsypwftbCm+mMMzn1uEl9aj660dL2P2zDG+urbqeUTpYGCN1qlx3tZQMRGy4Peqgo9/HhJclygRi6/WzH5QeiNfaT3zM8zF1N2bfLsyhojgVC6ChEO5Ep91my86rgJJBiXnG3ozpLzZEj5jepu5McLVsbLuxXaYHlSBe7faoGs+ip5dH9qJcY8FpBY+Ahrh6dmft+ducbsvOhaXUb/5w+Wqd4Gl8QLDwO+M0sOf0womyUdI6EWVzue2mB1xiPnG4hYivi4fEPnv2ON+l3F//Qzw3oqU3bnofUhFey7nD7gYXiG7SlWG9B165yrUDv3JCbg6txw9cSZM3ZA+B7XPp59v6JjU7RtiVNw5OKWL5z3NFuN1IfknbeoSh0ilzJXtC3iMFrvHLGxAOVA5hkxenemr64YxntHvgpp7XZjJ8O8go/I1DrkDVF5nzeMGnKd2I+56/4PKKY6o+3u96Y7QcIinEIQokOcxQ2rmbvTIPEb9DbM2ng9IuaUzoUcYAslA9qB/6PO8cGvn0CMsM1qzvlW9AmOxPRCVCbpoltm40wr5KZDT6Od3A9aG8Z04LOnzvEOwXlA2m4es/LJYHlEMZF9sD40ddy5sTIyT+G53YAbG+qi5Xp4qUvGqj/7cVkFDpcMNyBWDJYnTjkDsXrdCBg+gqxnkMW4M8G5r54vBXHhcMCeu+7UQHYxiaDRl94YWZw01Oh5UWDhoFyTg9ZCusvnS7KynqQK0CB+bHTQ5fScZzaK77vf/FG5unxFjOnSQfkXoxIAzd4FSu1DV4bTp/2rXxJH5iP6cfZ509UvU87hUdoG4/ULq2CiNXu7L5K+iES8nEo3ca/EQm0RXmgD00cZh/XvhbFUrWs4RV36eGf02AHgnWsVnnw2vwd1BVkjrEfYn/uq6hBdcQX/dOswG5KnBgg6w2lJUnbfpryyqb41YPp12vHdOl46EmF3eiff9QBhIEA0heS3BAOstg0rw3nE9Tqv5GfpKCOXkgpfXmF8z3VnXsd8d1H/+ZSVoSzp8pHaTkuiflWtupgn+ibDb7vzJ+Ygoat8HQktnV9EkD83oBFc1fgSpniC2oL2WRR5q6rA78l3nareqL1E8Qwci9DEptqhu9Ql6kGd4AqrvqdPjWfzxlN9UkuYtj5S6N2hgew2PjedJKUBthIxq0Q== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84e067bb-026c-4743-6319-08dde9f1912c X-MS-Exchange-CrossTenant-AuthSource: GV1PR08MB8426.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2025 07:23:09.0432 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tiKYgeKMw49dCzD2tlUCr4WyZROkMR0FZBvOOVb6sIG9N75W3bJ5AY+LPG1ywSe4er2QjgYYXwcJLy364us5gA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7678 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 02 Sep 2025 07:23:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17940 From: Stefan Herbrechtsmeier Add a private helper function to get the urldata object of a given URL. Create the urldate object if it doesn't exist inside the urldata object list of the fetch object. Signed-off-by: Stefan Herbrechtsmeier --- lib/bb/fetch2/__init__.py | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index 80f1a0c08..6da75008b 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1829,11 +1829,9 @@ class Fetch(object): urldata_cache[key] = self.ud def localpath(self, url): - if url not in self.urls: - self.ud[url] = FetchData(url, self.d) - - self.ud[url].setup_localpath(self.d) - return self.ud[url].localpath + ud = self._get_urldata(url) + ud.setup_localpath(self.d) + return ud.localpath def localpaths(self): """ @@ -1842,7 +1840,7 @@ class Fetch(object): local = [] def localpath(url): - ud = self.ud[url] + ud = self._get_urldata(url) ud.setup_localpath(self.d) local.append(ud.localpath) @@ -1863,7 +1861,7 @@ class Fetch(object): checksum_missing_messages = [] def download(url): - ud = self.ud[url] + ud = self._get_urldata(url) ud.setup_localpath(self.d) m = ud.method done = False @@ -1979,7 +1977,7 @@ class Fetch(object): urls = self.urls def checkstatus(url): - ud = self.ud[url] + ud = self._get_urldata(url) ud.setup_localpath(self.d) m = ud.method logger.debug("Testing URL %s", url) @@ -2011,7 +2009,7 @@ class Fetch(object): unpack_tracer.start(root, self.ud, self.d) def unpack(url): - ud = self.ud[url] + ud = self._get_urldata(url) ud.setup_localpath(self.d) if ud.lockfile: @@ -2038,9 +2036,7 @@ class Fetch(object): urls = self.urls def clean(url): - if url not in self.ud: - self.ud[url] = FetchData(url, self.d) - ud = self.ud[url] + ud = self._get_urldata(url) ud.setup_localpath(self.d) if not ud.localfile and ud.localpath is None: @@ -2071,7 +2067,7 @@ class Fetch(object): urldata = [] def expand_urldata(url): - ud = self.ud[url] + ud = self._get_urldata(url) urldata.append(ud) urldata += ud.method.implicit_urldata(ud, self.d) @@ -2080,6 +2076,12 @@ class Fetch(object): return urldata + def _get_urldata(self, url): + if url not in self.ud: + self.ud[url] = FetchData(url, self.d) + self.ud[url].unpack_tracer = unpack_tracer + return self.ud[url] + class FetchConnectionCache(object): """ A class which represents an container for socket connections. From patchwork Tue Sep 2 06:55:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FE1BCA100D for ; Tue, 2 Sep 2025 07:23:26 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.110]) by mx.groups.io with SMTP id smtpd.web10.71614.1756797794202570130 for ; Tue, 02 Sep 2025 00:23:19 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=3Q7YtIEc; spf=pass (domain: weidmueller.com, ip: 52.101.65.110, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZpSmyLS6OBlZtMiqrLicoXqxBuzGZRlTRxRk5YlSD1ckHodJe+1zSobvP64VqIAr44bkJykI8fIBVMqUN/K08kho7J5RqoDGzeBoob/qz/Zt6y0f+yA3B1eiu0vCulCDr0nxrNuIJEWYCmoDDqApCm6x9skWg9qVAzgv3o7DYRWq3LaGqnInMHW7jMVVUOjJd3oUyknvtZ5DaOzy7ffDqujEQhJbz2w8qeB8GByFX4+EqrDa6D13DSszvvBW4ZS5vlzAXaTET2Il7y5ZT+SGVCXZBT+AvilpIMRaynYlM1f78qCWuLZ8NcTCYyuTzLmK0IcR3bYpBm9CCeedAj3hiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8pyibmCDjVxcOUl8+AZATsxhEiXUybako2ZqMaqNKEU=; b=QzYqAP6BLxkFxMip7BJ8BC2Zo4kAAJ9io8MspD0DkzwwsT0oHUbcCwM8R8aaMJ/d+3ekpNXYuKpntMgW9ayLo8pDX4fnEZ2Ky7SQqdxxOMbEnk2HQyjlKyTi55+mWxjasErI0tZYeMEAEMdIARUtff77PuSmOFkxA0kaikPiKSdEMgxYRNQ4fw7gR+OA1DcWd1535O11vXTQNIv3avt/R9vLko7uO7fSpa+JuA63XRAWSW6Lz7pbJlVr4YB52VYW8gBFi/DK7osdz4fGmQXljSJAKZ4hFmWIJql+rThsYB/Yx3TLNo2PZU48Zlty8DwA5Ln+CzRx/wGk3wBc1BAzaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8pyibmCDjVxcOUl8+AZATsxhEiXUybako2ZqMaqNKEU=; b=3Q7YtIEcZ2lDuiQtI6XffDY1/Y8HvJaJgUBg9cVPI+oBWm76bP2pOH68Yoa6T6SqU+Io7qHE+pPDesWb0oPOoaRRzdR5mqh1tAN/LLUxIBZUcyMpVBu75dW05FDwVq+2ip8RRtCP/7VMZMuGV6KQDVz5nXKTLLCpM+7yDopfqgLA0pZtE3WMd8DBHbSfZuF5gELchGMtchFbOvd9Ycs4jU8PDsPtFaT7lMsjzCMn5skgVxwSIgT4C7ElKN6SLGIZ+3Q8L3m36OzATHhnOZdvBCzhUmi2ABQc2A6JXW7w11Fcf48wQL5B2+e/Q1Z2fLsfm8BMC7MlYw4fa4+bKpUxdA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) by GVXPR08MB7678.eurprd08.prod.outlook.com (2603:10a6:150:3e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9073.27; Tue, 2 Sep 2025 07:23:09 +0000 Received: from GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013]) by GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013%4]) with mapi id 15.20.9073.026; Tue, 2 Sep 2025 07:23:09 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [RFC PATCH 4/6] fetch2: add support for implicit urls Date: Tue, 2 Sep 2025 08:55:05 +0200 Message-ID: <20250902065507.35737-5-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR5P281CA0012.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f2::9) To GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1PR08MB8426:EE_|GVXPR08MB7678:EE_ X-MS-Office365-Filtering-Correlation-Id: f3add107-0abe-479a-4d49-08dde9f19176 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: 3Ks5U0/sVL6azFgxDeergpylC/Er96V3j7yKzvQPf8HxOrpGX3WxhzWIcNucautAQ6bvu6c2dtAp7QrG+aSi42c/YVmJ8UTGRfogp5HQ4hKptXlWJ5b+TJ9NAd2NLOLiQyVBDQUQGIAPAZhaZdZtKNyn3KZklpZ5SEwQC1TbavK94LJCzR+QyQA6FYIHE8oZeC8n/4ww1fjP+GA+iT3JLFmKePuklTYXmCNLFOGoo551AioJQe705hqFfENBmeck7jqpiaU781iWPnionG2B8QBnpMdJwEUJFuBBbldOnBCq450cb5YKlyuB3BapXiXudOxXquy2HVa7jOaw5nHdFp4bb62M8Q908NtmGNmefA0KqeraQ2x9xr+JVESbn4EwEtqAUEGtO3Z2cSLHV2jAo4bLnMZP6LkM/bKbRpUKUxG4DNwS8gAbl+iX/Z6NmvzEv4jIulNXbid7DQ5PeY3XCqEfjh8eHqCdJSgXMYis1HRiiT5EY/4lX1g9DHxGtbsDzllV5wnL+ID8MZdQR/VtPnm3R8mzJ0SWoSup9QZ/Pt8kAu0Ug/o7M1ObNlSoXqasDbh8dxKB7HN47wf63HjQAydSkJXYba9RaPXmV9pGYQCAG406xNmjS1wYFwnltWu+sS+ryFbjRHgwgH+RUGy0aHtA4gNlmHraRVJ9UH1yaxMe58UkQxmmQP9CEnHk3FVFmIXiZXR+n3TJ5b8NPc51KLYb5f3rTIBsCiyUlt5icSt3S+mENUXoEWmV0fnhIYReoLw17nHuPAz4Fw1JlwoSGmtZ4ughDG4dwhQjAOeEIVsJxt4m2Zx3n6GyTs3EF5b2alY/eO4Fio3uJrL9B7wuKgqByRFilpk2JgZCS2wqj43TDCZTEJiiApk1cXWZLaR1o1v2IGSrGvoblQL8F2VjRyp4kvb+OK9tLvczAG2swRpNaj04tNF/fujhJnHGwRiP7/pIwZTMA451/kMFhrGbXEKdA4e9XH+bWC9TJz9DO46PhVApEovfsUTMoqlGTyiXc8flfRh58YORwKAndj0OkzlQ9YqUnZlnz3pSNPd3QrXA0BmlrrfOLj9MSQZLPkDVmldozv3kv1MyiLVY4uD+7d1GPSFZrHuJ0cTuIg4qnMUYwWotUtBq5M5Ur0n8KkK82+wFa3PtpdYiTxT/J2PQevsHdMqxwjOpIA30lWnIQ/rEBgsBH+CY34vqlLTO8SdKbyjmRITDcCfJFXKsRryivkWxqShovNJzHVX8Yw2YQiMAwnAMSVcEK4owu5MNfZjc7mNnR14XzbkbHcPkmHHeoyiWqyaeHdtJBVLv4bT4B6lmDEle3Rt/Tc9ep0q4j7OsIKBCFrqTziQ0GxZ1NCrFMVvRqfaJRAhoHSF/iB73inzz4jNo+VpHd0qYltixjWWM5m6BDuNkFhDVzpB5TwLtqXzQfK9y/BtGbjsv3Rb2+sE8WRjN/9zDOv7t+0CxeNQu7mJxKEgufaFFksWFxpa+fA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR08MB8426.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LSW+Tbr5C5EI6eRWputAL9A+32/K0JqIfthKXvLXn2Hn/DkPLF57Qm9UZmK3tl04ZyLIs8klIXwOZGqkw2HYtUroKzqTzWgf28ts59HlryAd+Uv36zNbrWGfg+tLrwS7pkCN59xseKO5WAozNGJnXz4ht+XVE0J6jIwIeIT9bQs+RYg8WALUI2f1utOJo9JGRcnZSeEKkFCxvAVddNq7GSRfYuCmbuKF/SnElxpIynBJ6xWdwkwGFNBMn/0UBxkHETcSGZM/NbNScvM5g/0eDVN38eOd52hECI2DYBb5A5xzHTrNX0ye8qyT5noVjGftROjDMfCNLcWStgfmXApRV2PYKG6r/kx3FZL1YCSUnJ0wTWdGpSYMg0xFNl2AEKE4fD9tgpp/rXUGTdXewpJlwXHK+bZNsaM5MPPf7VwYWTLsBnU9lr0zQ1M64kMAUXqzoF9ulgfeE9+VZyeYGdsx9bkc0qEv16pWSOGfEhluDnmwbWr4XNemCYdwfD0VSQg44cSc8pYOrJqqW3VTSmKBi9x87/QacZT4TruOvg5Mln4y5xaZ4tzbCLPSIVdJCCmnUYWLJA4cE20OpRsVrYsfeyYfucrHOD8scVZib83tqsnxim/bfuNQb8dvwAv5oKyWc5Ata5dr5IYl4jU9o0q/bwsl+/6+x9sv+dagZAayxh3U1XnNUkjR9Z4lBHEpjShM95Kx7/qzq6jYIyZWTKzLFUA1G0KjDGdPJedZtAowRfOhy3Go1jcqwZZiaFx+GBdGUzHbkETlx16IfxzXxFG42/aURiEeji0fNujVZvFOtoLjYBHsETZWT+A4DlD2GL9PyCKkaaH80Jiy9ovlIrSdDLU/fPduM1ZK/iuiyYNr5zQQRft3PezWkj5jA5ZhQS4OqYBofsAkxE5JLSPne7dProHoR3oZ/xLY46yT+VHMB7OfsRTMBvE/Fgyo3aFkUAsC36AEMNNN3g8G9qUj6y/L3jpZ9LoLNQQeWy8kGyCEL3SG9KEeM7IJdhsOdwcgHqPz/eFZGTOemUMet1+Xux20rgtKG4pX+geEG7OmJbMZhImNaeZSzD6hB3L+FevgNq2Pq8pPO+wvlb82A951+TU4Pxmhu9Kbp0voHdHUgp+Pcdllxc3W+hjnVq7pAaE8seNI/rBAI9riTzKLSmXXK9+OJS4eKz01lYLdWJW/Ln5at2U4rOZ31yPi+ZU5/hNZeu9lTdAFQeil3LLCcyDsJNWpOtR3eWKNMVMOiizq+RpOQdtHvGhHxdpZRFyXAba6Ezx3dnDLy3QlJx4VqdJwqBMDNLYX0/L9X5nY9bcfe4BJp9KKenStOlttJ1bSvEfGMjxXVuB6Qwo4Jjpevb3RrP43FKJifCcv1WoKXDJ+Qiy9Qbz3HoYiOnwZBq5VkJk93csBnN0W/6gFK+jOYOYiGcyRVJTpMgne+wCbAqlc76G7pXZ+gsllyEBCY0uXr7Jo1YbEgX1iuQrhITFjBNR2x6WvhWC0cfjyTNbqxn0TxHKk7goRqtZSdu35ejR9UWRb1gMYEsBultQ7ixEwX1F7M1nhU5To1u1oRtLmH1p5baYWh1oSfAiIFcuG0QYc+K/Wqm8XMRqe138cr/xfS73U+RzZ/Q== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3add107-0abe-479a-4d49-08dde9f19176 X-MS-Exchange-CrossTenant-AuthSource: GV1PR08MB8426.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2025 07:23:09.4953 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: AZA3ctS55lOUiYCkmllqK3eDDsrXh2yxb0t+2ViYwXM9xdoFA1RpW8AZrqsIjAqqozBUdrwfjU5dzAF7JLFdOg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7678 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 02 Sep 2025 07:23:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17938 From: Stefan Herbrechtsmeier Add support for implicit URLs which are defined inside another source like a version control system (git submodule) or an embedded lock file (package-lock.json, cargo.lock or go.sum). The implicit URLs are extracted from the source and thereby only available after the download of the explicit source. The integration of implicit URLs beside explicit URLs simplifies the fetcher classes and avoid bugs because of iterations between the Fetch and FetchMethod classes. Signed-off-by: Stefan Herbrechtsmeier --- lib/bb/fetch2/__init__.py | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index 6da75008b..e7bbdac14 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1312,6 +1312,7 @@ class FetchData(object): if not self.pswd and "pswd" in self.parm: self.pswd = self.parm["pswd"] self.setup = False + self.implicit_urls = None def configure_checksum(checksum_id): checksum_plain_name = "%ssum" % checksum_id @@ -1409,6 +1410,11 @@ class FetchData(object): return d.getVar("SRCDATE") or d.getVar("DATE") + def setup_implicit_urls(self, d): + if self.implicit_urls is None: + self.setup_localpath(d) + self.implicit_urls = self.method.implicit_urls(self, d) + class FetchMethod(object): """Base class for 'fetch'ing data""" @@ -1727,6 +1733,9 @@ class FetchMethod(object): """ return [] + def implicit_urls(self, ud, d): + return [] + class DummyUnpackTracer(object): """ @@ -1843,6 +1852,9 @@ class Fetch(object): ud = self._get_urldata(url) ud.setup_localpath(self.d) local.append(ud.localpath) + ud.setup_implicit_urls(self.d) + for u in ud.implicit_urls: + localpath(u) for url in self.urls: localpath(url) @@ -1960,6 +1972,16 @@ class Fetch(object): for url in urls: download(url) + def download_implicit(url): + ud = self._get_urldata(url) + ud.setup_implicit_urls(self.d) + for u in ud.implicit_urls: + download(u) + download_implicit(u) + + for url in urls: + download_implicit(url) + if checksum_missing_messages: logger.error("Missing SRC_URI checksum, please add those to the recipe: \n%s", "\n".join(checksum_missing_messages)) raise BBFetchException("There was some missing checksums in the recipe") @@ -2022,6 +2044,12 @@ class Fetch(object): if ud.lockfile: bb.utils.unlockfile(lf) + ud.setup_implicit_urls(self.d) + for u in ud.implicit_urls: + unpack(u) + if hasattr(ud.method, "postunpack"): + ud.method.postunpack(ud, root, self.d) + for url in urls: unpack(url) @@ -2038,6 +2066,10 @@ class Fetch(object): def clean(url): ud = self._get_urldata(url) ud.setup_localpath(self.d) + if os.path.exists(ud.localpath): + ud.setup_implicit_urls(self.d) + for u in ud.implicit_urls: + clean(u) if not ud.localfile and ud.localpath is None: return @@ -2070,6 +2102,9 @@ class Fetch(object): ud = self._get_urldata(url) urldata.append(ud) urldata += ud.method.implicit_urldata(ud, self.d) + ud.setup_implicit_urls(self.d) + for u in ud.implicit_urls: + expand_urldata(u) for url in urls: expand_urldata(url) From patchwork Tue Sep 2 06:55:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AB0FCA1005 for ; Tue, 2 Sep 2025 07:23:26 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.110]) by mx.groups.io with SMTP id smtpd.web10.71614.1756797794202570130 for ; Tue, 02 Sep 2025 00:23:18 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=MpT4BtBD; spf=pass (domain: weidmueller.com, ip: 52.101.65.110, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vxF0MlmEox/37EogwkrgpjIEJV8IWQo7K0MWeeARd7hzWO6U2ciEmeW4Jxacd9zO0+EWoAAk3frI/R1arlG6/yTp2gK0C5zcc9TC2qRq1eliR5LVkkd6LuhtFuKzK6lH0nItsV/CfQcjzJZsSrjoY2smgHAjucUcJxo4z9H4W19L2lOaV+zwrFGbvMO0OQGxtajZzr0FWuuLlYINHJ0R66nXqem4+zHb/F8Py5F1lcWVWaIAp7DbqsCA5oXFlFqGOUD9iJnhFowWisX+2j+VGml4rLOILxBrtxH1OGEbCzCwEGXvIk7xmJEDZn4+qvNzpAKibGevPfby2grLzPahoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jP5p0G2Qi145I9pDmauGUYaFuRDckZbLSdt3To6PHz0=; b=ggEDK3BvWYLnjroNDTdnilLsc82zB0aEKBqtm9I7clc4kYGtOJWP3afYzyExrnijotQqytO6S+gy8AZuTUifdk+6Nkj0zVGd4AncFtwob+9bRZ+ZwqiRV8G/r6i4UfWIgddvg9UJcttae9dkSwYMb2nmvUQMbXQpBUjiYngtsPRgQ32APFVoy64jqNyNkhFs+nDN9TvPHyHagpl61/08uDMYpvha0jMtZQXOCBOfcuaS7SJN6dDuDY5FtJmRwIW47DpJrclxJgQkMt3zuvDYHAggIlrGHfpO+jW2MfvX4gAXIXBSLPl5J/qTbz3VIp8HRAdmINn32ZSsGh4kT2+hqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jP5p0G2Qi145I9pDmauGUYaFuRDckZbLSdt3To6PHz0=; b=MpT4BtBDS+x1ml8Lpp06JAEQW/kV8MDNZWXrUdDA6Ou3skVlIcFyVvrufQ0YYY4nNiQLi6Hgrt8cRTYHqKvxUHpNhELpKlEtI17MHSVP9u8qwBoXVgKWgcLpi1FyJg5DsbGrkk/xyGax3ftivjcGo7AQHxlAAa3gQI/OAB/n1XldH1YfH831Y/9G/TNa+jM1FOFrt3/OmOjvysQZLJX2Abw5mHru0Nm9zAbVOkgNH9qK/aI3vLnCmCoL5pT3Un+zB3OyQDNhBcen6OZumpwayCk6nGTwOkFgBm0I5wDzu4aWhPtX9Y6S1R1W8fzo1ImYpnWSFPnPSu9HkAfyZfnlVA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) by GVXPR08MB7678.eurprd08.prod.outlook.com (2603:10a6:150:3e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9073.27; Tue, 2 Sep 2025 07:23:10 +0000 Received: from GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013]) by GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013%4]) with mapi id 15.20.9073.026; Tue, 2 Sep 2025 07:23:10 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [RFC PATCH 5/6] fetch2: gitsm: use implicit urls feature Date: Tue, 2 Sep 2025 08:55:06 +0200 Message-ID: <20250902065507.35737-6-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR5P281CA0012.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f2::9) To GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1PR08MB8426:EE_|GVXPR08MB7678:EE_ X-MS-Office365-Filtering-Correlation-Id: f02677d8-cdfe-449e-05d7-08dde9f191b9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: Gh7AfGMYSFRsY34cETWZeCbsMZw8WwqQEcbFYH4TaAgAWmwhiugDilym57XlFIKoQjLhInKRcu4vTVci3KNl1eBvRzrHorcqtNjnsb9G675GfyxneYorgNfP55Ps4nNZY2WRVxwMFsEi+9dQoav1GVXNfvrQFbQM2QYBTy9vjLXUhcqGLFYsB2Ul2t6PikVVm7RRW51+9TEkRwLZQs/0LCOSRqIKTGQI684+VeXgE0YPzAxZSkSSaa0Uqf9OaFUxyxaShXMydAkQTt+UyzrOiX06hHv3l2oDQm056tL352+dbgR9m1UearZFgAdMyV4C+JQ3D/pVk6eqPWiEm2VDN30tHbx2G+2RVzpgq7KhB3JIGmBfIS95cfdnwij5slfzg8sUwLobqDTCCzCweE/bx2g5feGVMPzsPtEoKoiQaTmP6IiMww+zH8Bs4QGvrL5D4huBJAGh2x6/Dw4FiF/phNl449TnhJnN3riHPev5Z0ZJnVcnQ6aLrhNzb0bx52lSvqlflf+u6W2Bc6BAVchuGdlCNp40XxkxQ+fhWXmMqwkmc+pK8YLwwrUcpnx/PGIQoCD+S9SQu2V91h1Beg9sONqFQVjA3425s2t/WzCMF8GGCXZV3Gr3QayjYjR+cFMLjFMtBmq/MyedDP9ullXe7Rr+1XvILIY87wKlMCKvgQ69RSo5duCB/r0ytCck9BH92Xxk/CErPhuA+eqOAyWzNIHlN+Srcv/8Q+Q8pht0L2DRxpCDwGOvTp8lt5g9AVz/R8IsTectfx+nA55/R/Rpkhl4t6BU0GKAPXiOFUXdBEVG2GxJqRCicjT4HaiaF+5N8PnowI7Eovc7bi4E1erdlDtJ5uIz7LMqm+Am5hAJpRwPPJQo5fSDyEz+b2n2QzmbV3PGdi5MBmO5sbRtcX8x+ODEYuRQ17ebd5Xg3OoELVuBazmaS32sxz4WMCh7H9UOEo+ocinEzs6k9qYSPUcvNqfzwovNE7T34aLPs+ozO2Imhb8kDzBxByy1ELsx8+HxA204yQWhA4sdWQgoQ9IbfURRd2/u7cPLSreOgFEcz4oywU4/ka79L3MLdXso5iVvdhbsCzTuXa0cG9SsqiDUZJwCVyg5LQkiFpONgRKUs5PJ7J5tuJcdbcA35j5thsBB3qM8xsrj1vxghzypd9g4dUbNCVrnBENvQpCJkE5C56T2LHCQ3snekKaWkqiZPl/tpTYZ73qGorq3r7dvViHFgsxsL2absfuPdqF2UFb27aSD/it7vj0HXJz4uMBxLwee0ihG8eV8o25EKkpatXgZgSCj4PPkyaGzn2LSnGnBhg4LLw23YMj6LFULHFga1j/1ki5+WrPXr7cJt/A5zbPTP0lS4vM3QctIqXhqs9eNHYNRgoVrqt2qchzH8JsCLiEZ9e7VomNJ59+I+kyyBaqkAWmdXFSVT2Jyv2DHs3GuhBoNAI5SYfJ7DMDY7Mc0RHEDT550jDa4weh7MCmWyu8OVg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR08MB8426.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: sSGymrt5p1IvL4A99mrMPmMJzGyURQxwBPs5bjiWiLtxACHJUZbFck7NegTvKfHQ1VrlUXR0Y8il/X1fvGiISxQewPRuirkF7K9o/bru9AV/Jz+oOLFkRy85mYxt3qIv/fbN7ooUJrNArY3zLRh1OPa0u83WC2C69yV/DDyyx40qq0gwP8BdiEElmRQaCLHqrVSgyqjI/Fl8gQA1GOiU7YpGDkvN7hS5UfjANtPOSENosfll8xrhbJ+L9P1mOrLAIOyWvC3mqzlHxiCrAULy6AIh9yMx9a/7+aupHQMdQyB/lJy2XgDQYnaVvfEJH2L3RbFBcFc2VOsHMEW5ZZhJpTffOMwUe8DM1jbDvVTvn8/gnfmVDW8xdOT/kWprHdIxQf80+pKsA0eNvcBc9nxUpAOEqHnn+WOcTkfMvf3Ah0KuxdhRjAobb7tbzv9KiiAVjypItf+nm3Qm63xNj83NK129SrD8t7yNs+47yd21y4pUmo6ZmyoDlWDaD7J0evfPuFIcFCdS1mXYFJRi/km6r/IkqmvzH1URR9NNSQgM/FDOPAfOD7g3No9gGb9ChIe101y+4Y2QLd5aJIDhEM5P2Gk84bXUwhID4axLn3KI/bzPn3RgcRvAYryzDM0rKVV+IvNOv/z+M4SO3dFyLbGhj9K6jxkqhNSV+c52eo5aaH4eNGr/loRAF5m1R319DbvdCCX6GInvzCJsgw3AYaEMjDn4Xz+M1UpPLZCxVNNmiFjqvU+Ufsykxu8qZrV8qCMDS9QlFmydPbpjaxDwqNi5JtTM9l5qCgs2W2XWgXkAzWde4S9ktSokF2iXgAG6gZJ+IpV8xlXCm7ljCSq5Dm5BQEsGzwhSlYhJ6WqkKc2HOVqMsjbhvR5+g1SGA1Z1+irblXxmHc+OgpxKCaDuOnDU3MD+f9P1TEGq4Fkp/J71EyTVRlUR0Ojy4YdcLGtolT3sqrsGsBE6ubxtFMnaMzjaPemZ0htpbSupDpcCeoTBgjR4l5QRuf31b0vCdXwO6U4JiaTju9+50lkLNkX9fGlf2PblyXBKahFYmAPc9d9ccbOE6SoTwaDHRyoC3aK0nGODj/5E2KfMxMuflnfajQgAEbgtoyhWbU5UvZhlDdSjTd610YfpzVSB5E/E4Jq9YM8oYOCgDFo7OMhCG76QkA4v9Wi1g71Y5HyJUuHMv08NCkZTMLXkZ6egaAgJ+zCwiS2wN0Qcibt3SLvCNx+VfwhDfWW0AUBlTsaT2toJO3kxm3JUXiuhSteqO/4taUOI/Z8/qqMWxt/MbUte7wfG63Wy9Sa6qoHB46+Rm0+0aq+8FOMwwhHk4ncC844zZSsFypEB0oj0mPjchYzJ+B8UvWhzG7Q6EkO/ncoIufyV0qXuZAD4X1E83YByAmaXM45Z6U7O3EuTtfuc0EmGdn9AP7JoDswE7cn3SHTYnyAVz9m/OON7PJeanscJUrVMNwAvB4YA9bykegmsNYA+1Jt689FnhQrAlLpaX1OCpIlaQN1nVzf+EYyY2XADu4+js1bJDs94HjKLvW7keHzyclygedJ3SsJ+vpj5NW3Tkh6l8iGulYiMlDQSzL4YozMVXA4+6ZqjQtosl2n9J7XVG2vj+u4V0A== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: f02677d8-cdfe-449e-05d7-08dde9f191b9 X-MS-Exchange-CrossTenant-AuthSource: GV1PR08MB8426.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2025 07:23:09.9901 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sdbGCtQStQPCO1gedK7nOHWtvQOoVvSB0IZopSO9V+0x8LXaRYdcpPT2v7C+pg2oxgLbZWXaC9rDY034uGZ+yQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7678 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 02 Sep 2025 07:23:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17937 From: Stefan Herbrechtsmeier Return the submodule URLs via implicit_urls function and remove the manual handling of the submodule urls. Signed-off-by: Stefan Herbrechtsmeier --- lib/bb/fetch2/gitsm.py | 46 +++++------------------------------------- 1 file changed, 5 insertions(+), 41 deletions(-) diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py index 5ddc81f86..90fe92f74 100644 --- a/lib/bb/fetch2/gitsm.py +++ b/lib/bb/fetch2/gitsm.py @@ -24,7 +24,6 @@ import tempfile from bb.fetch2.git import Git from bb.fetch2 import runfetchcmd from bb.fetch2 import logger -from bb.fetch2 import Fetch class GitSM(Git): def supports(self, ud, d): @@ -141,9 +140,9 @@ class GitSM(Git): urls.append(url) - return Fetch(urls, d, cache=False) if urls else None + return urls - def call_process_submodules(self, ud, d): + def implicit_urls(self, ud, d): # If we're using a shallow mirror tarball it needs to be # unpacked temporarily so that we can examine the .gitmodules file # Unpack even when ud.clonedir is not available, @@ -157,27 +156,6 @@ class GitSM(Git): else: raise bb.fetch2.FetchError("Submodule source not available.") - def need_update(self, ud, d): - if Git.need_update(self, ud, d): - return True - - need_update = False - fetch = self.call_process_submodules(ud, d) - if fetch: - for url in fetch.urls: - urldata = fetch.ud[url] - if urldata.method.need_update(urldata, d): - need_update = True - - return need_update - - def download(self, ud, d): - Git.download(self, ud, d) - - fetch = self.call_process_submodules(ud, d) - if fetch: - fetch.download() - def unpack(self, ud, destdir, d): fulldestdir = self.destdir(ud, destdir, d) @@ -204,25 +182,11 @@ class GitSM(Git): logger.error("Unable to set git config core.bare to false for %s" % fulldestdir) raise - fetch = self.process_submodules(ud, fulldestdir, d) - if fetch: - fetch.unpack(destdir) - - if not ud.bareclone and fetch: + def postunpack(self, ud, destdir, d): + fulldestdir = self.destdir(ud, destdir, d) + if not ud.bareclone: cmdprefix = "" # Avoid LFS smudging (replacing the LFS pointers with the actual content) when LFS shouldn't be used but git-lfs is installed. if not self._need_lfs(ud): cmdprefix = "GIT_LFS_SKIP_SMUDGE=1 " runfetchcmd("%s%s submodule update --recursive --no-fetch" % (cmdprefix, ud.basecmd), d, quiet=True, workdir=fulldestdir) - def clean(self, ud, d): - fetch = self.call_process_submodules(ud, d) - if fetch: - fetch.clean() - Git.clean(self, ud, d) - - def implicit_urldata(self, ud, d): - fetch = self.call_process_submodules(ud, d) - if fetch: - return fetch.expanded_urldata() - - return [] From patchwork Tue Sep 2 06:55:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69699CA0FFE for ; Tue, 2 Sep 2025 07:23:26 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.110]) by mx.groups.io with SMTP id smtpd.web10.71614.1756797794202570130 for ; Tue, 02 Sep 2025 00:23:21 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=kBQbD21e; spf=pass (domain: weidmueller.com, ip: 52.101.65.110, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iAbqnBplvDCd83c0BPe9NuOSaZqyMT0SR7D0nyhb86R8mcIAYf7Z0ACJgmKasCVHbSp8+p410vOkIRZbbao7umT7pFBg3rcaqPU1+mmFKjQE9jjIwBbB3O4vA+8mdZ8bRANRDhrGVYH1iBLq+s7g8Q2oj7I2X+qbj4TLKjY8ep1uGPZRrJeipYT++eF8iFzJ6d/2M6RILxXDW/QVj4w+uq0gvMvBcnQFHhhgFK9bI17Jdf8Um3JqzNpkJGm6jI9oCVfsivkM9ILr6flns0hlrt0qsyAW61nxaAU1qb+2gUKEbaFlKpKHfz0TFbmFcenk83/ilx8pgHk+n4RghZc8GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RE/v3B0O/XkA4CJmoBvWFdObGQ+zP/Y5F5dbcQWlvgA=; b=Xqq6akZCkhTTGHnL9Lz31ArlWPuCreVIAVLSejub+Af/YpCY+D/ocjYtD2970IkwiA6LuoxD7A75uiYJqpXsqIVKMuyOurGtyTBoz4H3tFULFyHy+Hs/+USVflPRQSSdAZddLtBx2p+BfpAUVx9/VKdQL8b2RuvPX5k8jBVBBNXhEdyyDtzqpzRWAfNctB8DLIgJ5t+B6wAitWHmMxB8Yp8dGUjT5458N5McQhgbjn8TaPd028CRo7Fg5+Td982GPe2zKW04ynTvuGao/WTcBTUJ8zDWAEX+4iKpCiUK33YFsDGm1ZXnefoM7Z3+hHmLv+kmPv4aVvaQ3euSjnbNUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RE/v3B0O/XkA4CJmoBvWFdObGQ+zP/Y5F5dbcQWlvgA=; b=kBQbD21eiLl3loROzvdkIbTvAalbcdDU2uninmujYg4G3O5ZVh8zw0j2a6UQ3NbaBT6ZoF36HGvlekIWMKqJOgVOrCIBdwQz2eTDA/9cGTKhkaivp91/E9EAT09raDX9n+0awzzowItUVu3VX9KJxeqeHNLycOFdf4HTQlJLhS2WF5/AZksboUzYqiloEPM7NDrlto5hRi6fC6UyrXqfdaPaEazI0gbdJCwDIQ6zqFEpxD+YjAC41WbWK6VtlxUjOFYWOM7tYzwjYikQ2HnkCgTDhWZcJWNG79NRtRW56HrRNcmgZn/NXaZxWtRX8cay6+vvew3BL9iBwwe/7tQKUA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) by GVXPR08MB7678.eurprd08.prod.outlook.com (2603:10a6:150:3e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9073.27; Tue, 2 Sep 2025 07:23:10 +0000 Received: from GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013]) by GV1PR08MB8426.eurprd08.prod.outlook.com ([fe80::f9f5:b4bd:9e01:9013%4]) with mapi id 15.20.9073.026; Tue, 2 Sep 2025 07:23:10 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [RFC PATCH 6/6] tests: fetch: add test case for gitsm implicit local paths Date: Tue, 2 Sep 2025 08:55:07 +0200 Message-ID: <20250902065507.35737-7-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250902065507.35737-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR5P281CA0012.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f2::9) To GV1PR08MB8426.eurprd08.prod.outlook.com (2603:10a6:150:8a::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1PR08MB8426:EE_|GVXPR08MB7678:EE_ X-MS-Office365-Filtering-Correlation-Id: e4330e9d-5449-4060-f989-08dde9f19205 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: amCbgqsZnwocE08mOzTJ0iheOHKv4PgUkhIdUglFJ60IQI/UmKLk9IMAw+tP3BgmcFQGu13nWJyHU2ZiqSjivG5x6FDg60mlzQ4CBaa5c3RpMSH4ZJzJ48+nOH8w0UNUDP3tQ7Jaw83ZU/SYaZfWoM9icR9F7AWHGIA9bXVwMpdbgM8V0LCIE29zUvgbbgh4f/ubs2sX2WsQfJbKg0joM0zMD8gaAZD8faMCU1JSC5TdIPaEYTUvZmQqq6yaFRLKJlhGAB1I7fPMCJpGlseCkPZ89WwBX76Nh2FomCr3OQAK3+vmyUuLXnov8BUn9ew/6GK1qyL7AaEnu/cf9WDPQcU1s8WessGegvhZkzp0fhLwKxnCEDIvKW94dv4hQTaJ0Xx13JN4G20HTIveTi2XVGB4hF+6m3dEhRsTVjoX25j8EeHfHVBtJzujGInO8Vadu0I/jGGqsn07fYRjazQvUH1vvyHh+BTL7uIGH6JaU8isWfXAkvezv00sxNVWTAAug5rhMknXtVnxBo6M6ZiZEH02gMEGRZRTmtuSXtcTXVJfHxLbUWQxdzIPF/hjiPsMksQ9MdP33hOch73r5XLBWbj2vHZJRkWeCIN9/asdurpAn2rviR5sm0bhxXvpa3FC67hKlewunMxR2vJaz6DZgW9XKGzDRz4vfIU0/X8dTBuUXpgAgZCxZ+gDwqLBtOhL0xgm4KueYNMqbRHjzm8o+TpXqb9iSICGO2ZjoCyIG/80vG6YmhnpHsisjhCnz4KVxsfBxdgk6b3NvAtwTOJ3BwwItQ+X4QXvg1a3uykZN2RnadydkaNSKLt7KZBRYK3NE4uAaqr3ClZW/vQA2Euw8Ng5gMUD0ix7/O5f/FJvovRY6NiGJHdWzjiuaDH/wYV2rpdwhYA5OI+utPYxzhcwW60GKwev/IlCLwIlPJmAO8Uv2WHtlrBrMa8zJy7OQNXKoZdaEpX6Yl0mODi50pWJj7K6R9FFaRmTL/AeTLt+QdaZDGeEt3lv7KAN2HOHHHrOsqxn7NjddZTtm/X7MgOMDu1/PU/tDeIRJJUn4qw8litp6ApSkpuQIFWPSk4LjxcHFlsHt3a3z7augG3S063bEkG8l6QOVHLLDV2vLHN08DqU6tg5/5qEpM6PSPwCz/olf4av6UPcYl2X5g1Fjd4mzQnhlKUojGbslB8AqSjg57BsStUscO9AVJUj0iru0evSYEEDWrFc6HNnsuaDEAni0J0g/BEYUU0COkW0Nt16D6/NdiqkV5DKocAn5gDd2YUunRz/c6etMk3FY0FqzSo4Tdj8O4gVLAoH3zDTuJnHyOLbi1Gg0xAzYhCrdk+/2HVPbG+oHjjlATTWIYkXyLJ4r/vQrOWGCSsLHuartxWDas41a4Ai9GXsrnE8QAHcu0OcDCU/TKxRrs74YHH8dmMYOK3Fdy1Mv0y6J7xONFmZrM8UiHaQGPIkb1cI3f/g/pH1aqmqA1P3m74ErTn5loAKZw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:GV1PR08MB8426.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hjoK2zfnalKGsndFsJrGdf2nfeh8ysDJcerrdWQPqk9glkrVdS9CHcSKwXq8U3R2awU4foroGy+duXjOSG7+eqSwMduqwaFC8lkqBV2yXLyuAAQssmIramg3sPh5+emKLgLqOA446RS5yb67uMTUr1PLF/vjabU56hEoVSOUjjCq5Mh+rpADCWUY9Q7X1G1VswzMXuzmNEnKJy3xnqjV0Kaphn9wnpbT3TZkkFDw6P6JhmMtcda7/FJFPnOXgjogodDVzCTLBcwBXeooWVozctK67Tn/ZQbUgfTvuz8OrNiPxqGR0CodwY6sb8g5RXaDz6nszFcq2fNb2It1q0EXg/XbiZWNroHGdlJ5oa/HUVfP4oYUZSu5nW/OEfzFz0DVbCUT9LsmN8r+kdmpBF6Bs1rgywAMiwTk9EvTAtsYCeJhioBlf8m9MRrky77tl44t1DnjPMS7gCmenyPUqu7i6kCP91kAVN10T/mTMsgLtbxGwTWUpxhusdfWNX4kdsHvXMwUReVSlU0ZR8rD6r/XEtiA+wQBAKey4AyeAY/tKoZIbMesr0uEfkD2aED3qwiURO2vd9IP+OtVyZcNbduN4Rgl1xEWAXvfjywIdSKlcHxMOKnZHeyl2nyo6L+zdk0t8ufsHJWqiMUd4r8/MHoeunajjC19wac75/Zt9j4Yn82qQ0Xc0KJd7cfGg4U5bnY0cSkeTa+xUU1Ij44cSW1K4gcWqilLYJr1RzpOlcMyrbC81Jk+NUESQi8kDkpBYWLko6XFf8qEdflvBVJz7E7eUH64ws3ug7SWOQ/67DebnfhhDkAxR6hNQKYb6DfgZe7FkpvA39nMvWRRIDU3i9QuMM1jpZqaRfLVzH1krPTBUqdsnAfP7R+dAbrcPMwmSKjblWaRCUEskTT2RZNLSeRnscYa4V4xXn1BADmwX8wtMXirRzKCx5PmMZG+B2Iol6hk+n24j8ff60A5ZpCICM5fH2CFaxKytZ0d9cABr2i0+98F0sA0FntLf/JZD4D/bSBn5MXnht1zm0qe64m6u+o/aj9GMZyfsq95ycrjtzLoTVYQ4YcP1IFhUhlNHhAzwJb3Ouzbr0tp2WcYno9UMeTsnJJF2uc6XQVhKb3vqBosDL9HHiXaHX4jSwrf2egvCIMgnfborVPfb8W1mVOioN4tGAvNtOo3EDqrhGD1pInna2vSSUgNwxzSszHt+K0j51f0/78GwNTiKCF9M6+dHwe9bPNaqFTGICKBHfoyyX2CZF+x7f7O2F0bJd6GO8fKlq8re9LC3Jrpr5XY78jluzvKD5CxYS1OXtrOR797ZduXmUj9tjKooDk3+D96Te41pwtAEL7x0Klaf+SlJelYPuZ7ZBP5y8uOjOX1Vjvi55b6vt6gocb9kaX8P4nxQoB55aRGXUacN+/VV/4JWdoqijNRlNwe+Xtof0ocmkVDH7svOLtkYxJ3mpjaqEH23J1CcsH+cuw8hyVs+c/5j8Xxi5joNCVvj5C1tYBAYY73KiK/yKkqJTjSuQeaU4NmXnpE1L0jqXgB54/sHR8kH9sJz7i66pJZo8hKQaKag8mStaAs2TWivmBEk5ec4NufMRiTvHBcHFaVkpdV8dISgVqpo+bS8Q== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: e4330e9d-5449-4060-f989-08dde9f19205 X-MS-Exchange-CrossTenant-AuthSource: GV1PR08MB8426.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2025 07:23:10.5365 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2Kg7o6lAqzd0OfEEPAUeSvnCqJDwtgUmF3vJRNxqNL/2wz1B34JoNVZiyvePE6fdlNiG4B888SDKpHXSNfcW5g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7678 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 02 Sep 2025 07:23:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17941 From: Stefan Herbrechtsmeier Signed-off-by: Stefan Herbrechtsmeier --- lib/bb/tests/fetch.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 6067b02d2..e181a9a2a 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -3818,3 +3818,15 @@ class LocalPathsTest(FetcherTest): } fetcher = bb.fetch2.Fetch(localpaths.keys(), self.d) self.assertEqual(fetcher.localpaths(), list(localpaths.values())) + + def test_git_submodule_CLI11(self): + url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=bd4dc911847d0cde7a6b41dfa626a85aab213baf;branch=main" + localpaths = [ + f"{self.dldir}/git2/github.com.CLIUtils.CLI11", + f"{self.dldir}/git2/github.com.google.googletest.git", + f"{self.dldir}/git2/github.com.arsenm.sanitizers-cmake", + f"{self.dldir}/git2/github.com.nlohmann.json.git" + ] + fetcher = bb.fetch.Fetch([url], self.d) + fetcher.download() + self.assertEqual(fetcher.localpaths(), localpaths)