From patchwork Thu Aug 27 14:29:55 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "chris.laplante@agilent.com" X-Patchwork-Id: 96571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71287C61DB9 for ; Thu, 27 Aug 2026 14:31:16 +0000 (UTC) Received: from MW6PR02CU001.outbound.protection.outlook.com (MW6PR02CU001.outbound.protection.outlook.com [52.101.48.41]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.37737.1787841070472930182 for ; Thu, 27 Aug 2026 07:31:10 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@agilent.com header.s=selector1 header.b=Ebf84X3E; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: agilent.com, ip: 52.101.48.41, mailfrom: chris.laplante@agilent.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YCiNLfMplF1kHZogOZ8ZLCEj2XJa0uxP9qFb/VgaOrcMoA11Qyup9aMFTRhCFlRmBzEOA2PENykuKiat4Okbn0elHzO04+p/rTpMi2mUOszEDQpbpCVIUmodA17IryJrgZh7EPckQBzmkK94adevzh8sICcxqyuNNwXoKgv9SJthY67p9kl34/XmerpR9efqVGXw6cNAVVgzHl0+ObwlXyiFnmF4S3WEfXIFMncy0T/TJDbDBSOMpzKSr7bhWA+OBEVTrob5ZetAdfTnVBPIX4IU9VTqSmh23SRgH3j+VeJqzyAPP4ZPJl6Gs7Fpz/QSPsHVKuuLYshJh4K3ZKxz0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f0Ea0qrmXRvhCcsfM/aZMwhxyof7zYAetbN8i+jfE1w=; b=HTGzsrOMYQHcC/npUhOkaGlTuxtwVQ6Cjwito/r/qDhuluDdoQi9eYQN7WBw0CCVqh7cgAE9S6U9mQ7isuqTjZib219xQyLxv/5upldxeu78ysLb0ha1Znu0h1v9X7Ymft04eKcTw4XYbrHc66NB7i1MwEffOpQ8St43715RxnYP/1Oq9UGGrTgokUahE1HBu8XJadK1BE5lF2K57EPEQobjoATRmSbB7mn5CuIRiMYcz/1yQdjud9K4QTC7PUkOQekHzKQTlV7rMbii+f7FIHEdp+7e4ogvNYZKZVaxy9zJxMI87eehL+mCoq3QU/6Xu3PwqbKvNJQEaCP7s0vlZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.25.218.36) smtp.rcpttodomain=lists.openembedded.org smtp.mailfrom=agilent.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=agilent.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=agilent.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f0Ea0qrmXRvhCcsfM/aZMwhxyof7zYAetbN8i+jfE1w=; b=Ebf84X3EpHk9BTqKUPk9E6rC72e+UZmHziezP90C0SlAt+Guow2uASLN1piK10h0xYaxRh1c7z7GQ0CrJOVgihFugx24JmKLjPjCteVt44H1qETEk6UaJcId939tgJQ5X1KVVwJ5G9PxI1+H7yBEBUb9a9CzkKQPda1f75BUo6kymcUR6rG7sBRkjQGfIIHEtsqroniFrtw5l+sEU2T71fyxjFG4VBIumBhCII8pln6YW7/qTAWL8NARXMD5Gt52LCSzjBE0pifUankolgrRdW8AVQh42uo6CrvrqGv/0o8UcYokPzcYkDXVOhZdkslFoDpKE8FqCjqDSgKa1PUdAA== Received: from BLAPR03CA0158.namprd03.prod.outlook.com (2603:10b6:208:32f::24) by DM4PR12MB5914.namprd12.prod.outlook.com (2603:10b6:8:67::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.10; Thu, 27 Aug 2026 14:30:50 +0000 Received: from BN3PEPF0000B373.namprd21.prod.outlook.com (2603:10b6:208:32f:cafe::f) by BLAPR03CA0158.outlook.office365.com (2603:10b6:208:32f::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.360.10 via Frontend Transport; Thu, 27 Aug 2026 14:30:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.25.218.36) smtp.mailfrom=agilent.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=agilent.com; Received-SPF: Pass (protection.outlook.com: domain of agilent.com designates 192.25.218.36 as permitted sender) receiver=protection.outlook.com; client-ip=192.25.218.36; helo=wpcosapexchedg5.windmz.agilent.com; pr=C Received: from wpcosapexchedg5.windmz.agilent.com (192.25.218.36) by BN3PEPF0000B373.mail.protection.outlook.com (10.167.243.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.0 via Frontend Transport; Thu, 27 Aug 2026 14:30:48 +0000 Received: from laplante-B650.lfs.agilent.com (192.25.126.4) by edgeappmail.agilent.com (192.25.218.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Thu, 27 Aug 2026 08:30:13 -0600 From: To: CC: , Chris Laplante Subject: [PATCH 1/4] lib: rename fetch2 as fetch; invert compat shim to make bb.fetch2 = bb.fetch Date: Thu, 27 Aug 2026 10:29:55 -0400 Message-ID: <20260827142958.547182-1-chris.laplante@agilent.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B373:EE_|DM4PR12MB5914:EE_ X-MS-Office365-Filtering-Correlation-Id: 85af5391-92bd-43dd-71e0-08df0447ca2f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|23010399003|82310400026|376014|36860700016|13003099007|6133799003|10067099003|11063799006|56012099006|18002099003|3023799007; X-Microsoft-Antispam-Message-Info: bWT8J+z7iXQ910+3f+eM6EliaVL7iWjygFHBpnP/55ed+he82w4akCyD7oCeldlzL3H3UF+OlnZlk6KXnS25B0AJN+UNeycyGdmiCQ3NYdjxE19KD6itPfyhiEs9acxfTuIOlJyG4GhGl2izCDkkZO1aDCNc8UjdhwHwGsjlKHfiG3bX91Vm9qXYlbT2/5AHCkR4A30bFI8DM07j1OJYQBN9D3DcaWbTXsFAeTROUjXwtVD1bWLbzJpi4PI0idU8UnHCY/4YW5fS5yxKaC7Bv4jam8CojgHvn/cqfcILLkX/+OZOuaCv/Alsf4H7Onc6VejIY4U4W26mHyxZElPuSyz0TECwjpSDVOfWunEiyxDyf/R6f7KuzYAwrSulC3vhb0TCVgMS80jK4rymoRNz7mN8hnpx0CXfvBvJFHhujGy+6AlDZrEwE6D7lUBsfwnde3XcDYPT5ovNXXSrkJz5vd9XAdqhi6boyrn3NBJI0FWiwVpnY35cAWo03h08azoP/030lQYb1mYSWcCYGNZLtIuO2ln0Y/+zsOmLQtSIPba+P/UgfSuLMFuIwdyvNdY+WpkZkU9FrExm8cat1Vftk11Zi0JOwfRJ/V321DsCdrXRNfPMj8v2KIWD3pPH1bVi3Neec4D+CE3jByTU5oo3Cw== X-Forefront-Antispam-Report: CIP:192.25.218.36;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:wpcosapexchedg5.windmz.agilent.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(23010399003)(82310400026)(376014)(36860700016)(13003099007)(6133799003)(10067099003)(11063799006)(56012099006)(18002099003)(3023799007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kgNW+2DNuErC2HyTcFwCiLIMjcCsrGxp037B/0fuD3J+AkeaMztQyRHw7eEKorzVU1oE3TvvAyJRBxmld0YQKk3T6LyPljaiI4TdcMaTEQmcDYrVFrdLURC/XfiLsU3DG4VaY7fuQRkRx21MwHkodke8pFZPCTXlXbwNKTRdGdKMOPhvG0zvWw0e30V6oGqh03sojXcDeR8W9qS4Aajs+VdtDJAvB1sSFG46yAlNwZBQuRJMxVjLlR1cmwSWtVp7LyfybdB7rhZThJ1bXzPFq7rHrwWImKfPPHuIYbflxeupix39N9DBVywc9Qo9hnx95Jtl+muOHRG8SiRaCZEdtc1TgGY3BXHZN3CqmdEGhqoOkKKO6Q4lmSRaj8SF7/IzcZZp/vpsJZ8Y++bSRwiRgJ+f16TfNKJ8Y9LOdJBByJi+0sUG/CgDQD3taOgZQ7Ei X-OriginatorOrg: agilent.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2026 14:30:48.9813 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 85af5391-92bd-43dd-71e0-08df0447ca2f X-MS-Exchange-CrossTenant-Id: a9c0bc09-8b46-4206-9351-2ba12fb4a5c0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a9c0bc09-8b46-4206-9351-2ba12fb4a5c0;Ip=[192.25.218.36];Helo=[wpcosapexchedg5.windmz.agilent.com] X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TreatMessagesAsInternal-BN3PEPF0000B373.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5914 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 27 Aug 2026 14:31:16 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/10387 From: Chris Laplante bb.fetch2 replaced bb.fetch back in 2012, and the compatibility shim of bb.fetch = bb.fetch2 has been in place ever since. There's no longer a good reason to keep the '2' suffix, so lets drop it. We also invert the bb.fetch = bb.fetch2 compatibility shim to now make bb.fetch2 = bb.fetch. We do this by creating a new lib/bb/fetch2.py, which lets us get rid of the ugly-ish shim in lib/bb/__init__.py. Finally, we add a deprecation warning for good measure. Signed-off-by: Chris Laplante --- lib/bb/__init__.py | 3 - lib/bb/{fetch2 => fetch}/README | 0 lib/bb/{fetch2 => fetch}/__init__.py | 20 +++---- lib/bb/{fetch2 => fetch}/az.py | 8 +-- lib/bb/{fetch2 => fetch}/clearcase.py | 18 +++--- lib/bb/{fetch2 => fetch}/crate.py | 6 +- lib/bb/{fetch2 => fetch}/gcp.py | 12 ++-- lib/bb/{fetch2 => fetch}/git.py | 80 +++++++++++++-------------- lib/bb/{fetch2 => fetch}/gitannex.py | 4 +- lib/bb/{fetch2 => fetch}/gitsm.py | 10 ++-- lib/bb/{fetch2 => fetch}/gomod.py | 14 ++--- lib/bb/{fetch2 => fetch}/hg.py | 18 +++--- lib/bb/{fetch2 => fetch}/local.py | 6 +- lib/bb/{fetch2 => fetch}/npm.py | 18 +++--- lib/bb/{fetch2 => fetch}/npmsw.py | 18 +++--- lib/bb/{fetch2 => fetch}/perforce.py | 16 +++--- lib/bb/{fetch2 => fetch}/repo.py | 10 ++-- lib/bb/{fetch2 => fetch}/s3.py | 12 ++-- lib/bb/{fetch2 => fetch}/sftp.py | 12 ++-- lib/bb/{fetch2 => fetch}/ssh.py | 2 +- lib/bb/{fetch2 => fetch}/svn.py | 18 +++--- lib/bb/{fetch2 => fetch}/wget.py | 18 +++--- lib/bb/fetch2.py | 19 +++++++ 23 files changed, 179 insertions(+), 163 deletions(-) rename lib/bb/{fetch2 => fetch}/README (100%) rename lib/bb/{fetch2 => fetch}/__init__.py (99%) rename lib/bb/{fetch2 => fetch}/az.py (96%) rename lib/bb/{fetch2 => fetch}/clearcase.py (95%) rename lib/bb/{fetch2 => fetch}/crate.py (97%) rename lib/bb/{fetch2 => fetch}/gcp.py (92%) rename lib/bb/{fetch2 => fetch}/git.py (92%) rename lib/bb/{fetch2 => fetch}/gitannex.py (97%) rename lib/bb/{fetch2 => fetch}/gitsm.py (97%) rename lib/bb/{fetch2 => fetch}/gomod.py (97%) rename lib/bb/{fetch2 => fetch}/hg.py (95%) rename lib/bb/{fetch2 => fetch}/local.py (91%) rename lib/bb/{fetch2 => fetch}/npm.py (97%) rename lib/bb/{fetch2 => fetch}/npmsw.py (96%) rename lib/bb/{fetch2 => fetch}/perforce.py (96%) rename lib/bb/{fetch2 => fetch}/repo.py (89%) rename lib/bb/{fetch2 => fetch}/s3.py (93%) rename lib/bb/{fetch2 => fetch}/sftp.py (93%) rename lib/bb/{fetch2 => fetch}/ssh.py (98%) rename lib/bb/{fetch2 => fetch}/svn.py (94%) rename lib/bb/{fetch2 => fetch}/wget.py (98%) create mode 100644 lib/bb/fetch2.py diff --git a/lib/bb/__init__.py b/lib/bb/__init__.py index a9c31815a..049c87ef0 100644 --- a/lib/bb/__init__.py +++ b/lib/bb/__init__.py @@ -153,9 +153,6 @@ class PrefixLoggerAdapter(logging.LoggerAdapter): # can result in construction of the various loggers. import bb.msg -from bb import fetch2 as fetch -sys.modules['bb.fetch'] = sys.modules['bb.fetch2'] - # Messaging convenience functions def plain(*args): """ diff --git a/lib/bb/fetch2/README b/lib/bb/fetch/README similarity index 100% rename from lib/bb/fetch2/README rename to lib/bb/fetch/README diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch/__init__.py similarity index 99% rename from lib/bb/fetch2/__init__.py rename to lib/bb/fetch/__init__.py index 9cb268be5..153700dfb 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch/__init__.py @@ -543,8 +543,8 @@ def fetcher_init(d, servercontext=True): try: # fetcher_init is called multiple times, so make sure we only save the # revs the first time it is called. - if not bb.fetch2.saved_headrevs: - bb.fetch2.saved_headrevs = _revisions_cache.get_revs() + if not bb.fetch.saved_headrevs: + bb.fetch.saved_headrevs = _revisions_cache.get_revs() except: pass @@ -578,7 +578,7 @@ def fetcher_compare_revisions(d): """ headrevs = _revisions_cache.get_revs() - return headrevs != bb.fetch2.saved_headrevs + return headrevs != bb.fetch.saved_headrevs def mirror_from_string(data): mirrors = (data or "").replace('\\n',' ').split() @@ -885,7 +885,7 @@ def get_srcrev(d, method_name='sortable_revision'): return pkgv def localpath(url, d): - fetcher = bb.fetch2.Fetch([url], d) + fetcher = bb.fetch.Fetch([url], d) return fetcher.localpath(url) # Need to export PATH as binary could be in metadata paths @@ -926,7 +926,7 @@ FETCH_EXPORT_VARS = ['HOME', 'PATH', def get_fetcher_environment(d): newenv = {} origenv = d.getVar("BB_ORIGENV") - for name in bb.fetch2.FETCH_EXPORT_VARS: + for name in bb.fetch.FETCH_EXPORT_VARS: value = d.getVar(name) if not value and origenv: value = origenv.getVar(name) @@ -1059,7 +1059,7 @@ def build_mirroruris(origud, mirrors, ld): newud.setup_localpath(ld) if hasattr(ud, 'unpack_tracer'): newud.unpack_tracer = ud.unpack_tracer - except bb.fetch2.BBFetchException as e: + except bb.fetch.BBFetchException as e: logger.debug("Mirror fetch failure for url %s (original url: %s)" % (newuri, origud.url)) logger.debug(str(e)) try: @@ -1154,10 +1154,10 @@ def try_mirror_url(fetch, origud, ud, ld, check = False): update_stamp(origud, ld) return ud.localpath - except bb.fetch2.NetworkAccess: + except bb.fetch.NetworkAccess: raise - except bb.fetch2.BBFetchException as e: + except bb.fetch.BBFetchException as e: if isinstance(e, ChecksumError): logger.warning("Mirror checksum failure for url %s (original url: %s)\nCleaning and trying again." % (ud.url, origud.url)) logger.warning(str(e)) @@ -1814,7 +1814,7 @@ class DummyUnpackTracer(object): class Fetch(object): def __init__(self, urls, d, cache = True, localonly = False, connection_cache = None): if localonly and cache: - raise Exception("bb.fetch2.Fetch.__init__: cannot set cache and localonly at same time") + raise Exception("bb.fetch.Fetch.__init__: cannot set cache and localonly at same time") if not urls: urls = d.getVar("SRC_URI").split() @@ -1934,7 +1934,7 @@ class Fetch(object): # fetcher still have chance to fetch from mirror m.update_donestamp(ud, d) - except bb.fetch2.NetworkAccess: + except bb.fetch.NetworkAccess: raise except BBFetchException as e: diff --git a/lib/bb/fetch2/az.py b/lib/bb/fetch/az.py similarity index 96% rename from lib/bb/fetch2/az.py rename to lib/bb/fetch/az.py index 81fd14468..76c411adf 100644 --- a/lib/bb/fetch2/az.py +++ b/lib/bb/fetch/az.py @@ -5,7 +5,7 @@ BitBake 'Fetch' Azure Storage implementation # Copyright (C) 2021 Alejandro Hernandez Samaniego # -# Based on bb.fetch2.wget: +# Based on bb.fetch.wget: # Copyright (C) 2003, 2004 Chris Larson # # SPDX-License-Identifier: GPL-2.0-only @@ -15,9 +15,9 @@ BitBake 'Fetch' Azure Storage implementation import shlex import os import bb -from bb.fetch2 import FetchError -from bb.fetch2 import logger -from bb.fetch2.wget import Wget +from bb.fetch import FetchError +from bb.fetch import logger +from bb.fetch.wget import Wget class Az(Wget): diff --git a/lib/bb/fetch2/clearcase.py b/lib/bb/fetch/clearcase.py similarity index 95% rename from lib/bb/fetch2/clearcase.py rename to lib/bb/fetch/clearcase.py index cb3f8b650..54f347952 100644 --- a/lib/bb/fetch2/clearcase.py +++ b/lib/bb/fetch/clearcase.py @@ -52,12 +52,12 @@ import os import shlex import shutil import bb -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import MissingParameterError -from bb.fetch2 import ParameterError -from bb.fetch2 import runfetchcmd -from bb.fetch2 import logger +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import MissingParameterError +from bb.fetch import ParameterError +from bb.fetch import runfetchcmd +from bb.fetch import logger class ClearCase(FetchMethod): """Class to fetch urls via 'clearcase'""" @@ -189,7 +189,7 @@ class ClearCase(FetchMethod): if os.path.exists(ud.viewdir): cmd = self._build_ccase_command(ud, 'rmview'); logger.info("cleaning up [VOB=%s label=%s view=%s]", ud.vob, ud.label, ud.viewname) - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) output = runfetchcmd(cmd, d, workdir=ud.ccasedir) logger.info("rmview output: %s", output) @@ -215,7 +215,7 @@ class ClearCase(FetchMethod): self._write_configspec(ud, d) cmd = self._build_ccase_command(ud, 'mkview') logger.info("creating view [VOB=%s label=%s view=%s]", ud.vob, ud.label, ud.viewname) - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) try: runfetchcmd(cmd, d) except FetchError as e: @@ -227,7 +227,7 @@ class ClearCase(FetchMethod): # Set configspec: Setting the configspec effectively fetches the files as defined in the configspec cmd = self._build_ccase_command(ud, 'setcs'); logger.info("fetching data [VOB=%s label=%s view=%s]", ud.vob, ud.label, ud.viewname) - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) output = runfetchcmd(cmd, d, workdir=ud.viewdir) logger.info("%s", output) diff --git a/lib/bb/fetch2/crate.py b/lib/bb/fetch/crate.py similarity index 97% rename from lib/bb/fetch2/crate.py rename to lib/bb/fetch/crate.py index 830612c98..16ea02d8a 100644 --- a/lib/bb/fetch2/crate.py +++ b/lib/bb/fetch/crate.py @@ -26,8 +26,8 @@ import subprocess import re from functools import cmp_to_key import bb -from bb.fetch2 import logger, subprocess_setup, UnpackError, runfetchcmd -from bb.fetch2.wget import Wget +from bb.fetch import logger, subprocess_setup, UnpackError, runfetchcmd +from bb.fetch.wget import Wget class Crate(Wget): @@ -76,7 +76,7 @@ class Crate(Wget): # break the URL apart by / parts = ud.url.split('/') if len(parts) < 5: - raise bb.fetch2.ParameterError("Invalid URL: Must be crate://HOST/NAME/VERSION", ud.url) + raise bb.fetch.ParameterError("Invalid URL: Must be crate://HOST/NAME/VERSION", ud.url) # version is expected to be the last token # but ignore possible url parameters which will be used diff --git a/lib/bb/fetch2/gcp.py b/lib/bb/fetch/gcp.py similarity index 92% rename from lib/bb/fetch2/gcp.py rename to lib/bb/fetch/gcp.py index 462b8e97a..dd85b9fd2 100644 --- a/lib/bb/fetch2/gcp.py +++ b/lib/bb/fetch/gcp.py @@ -10,7 +10,7 @@ Additionally, gsutil must also be installed. # Copyright (C) 2023, Snap Inc. # -# Based in part on bb.fetch2.s3: +# Based in part on bb.fetch.s3: # Copyright (C) 2017 Andre McCurdy # # SPDX-License-Identifier: GPL-2.0-only @@ -20,9 +20,9 @@ Additionally, gsutil must also be installed. import os import bb import urllib.parse, urllib.error -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import logger +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import logger class GCP(FetchMethod): """ @@ -62,7 +62,7 @@ class GCP(FetchMethod): if self.gcp_client is None: self.get_gcp_client() - bb.fetch2.check_network_access(d, "blob.download_to_filename", f"gs://{ud.host}{ud.path}") + bb.fetch.check_network_access(d, "blob.download_to_filename", f"gs://{ud.host}{ud.path}") # Path sometimes has leading slash, so strip it path = ud.path.lstrip("/") @@ -101,7 +101,7 @@ class GCP(FetchMethod): if self.gcp_client is None: self.get_gcp_client() - bb.fetch2.check_network_access(d, "gcp_client.bucket(ud.host).blob(path).exists()", f"gs://{ud.host}{ud.path}") + bb.fetch.check_network_access(d, "gcp_client.bucket(ud.host).blob(path).exists()", f"gs://{ud.host}{ud.path}") # Path sometimes has leading slash, so strip it path = ud.path.lstrip("/") diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch/git.py similarity index 92% rename from lib/bb/fetch2/git.py rename to lib/bb/fetch/git.py index 0bb2ea39a..30618f073 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch/git.py @@ -76,10 +76,10 @@ import urllib import bb import bb.progress from contextlib import contextmanager -from bb.fetch2 import FetchMethod -from bb.fetch2 import runfetchcmd -from bb.fetch2 import logger -from bb.fetch2 import trusted_network +from bb.fetch import FetchMethod +from bb.fetch import runfetchcmd +from bb.fetch import logger +from bb.fetch import trusted_network git_hash_re = re.compile(r'^[0-9a-f]{40,64}$') @@ -162,7 +162,7 @@ class Git(FetchMethod): bb.warn("URL: %s uses git protocol which is no longer supported by github. Please change to ;protocol=https in the url." % ud.url) if not ud.proto in ('git', 'file', 'ssh', 'http', 'https', 'rsync'): - raise bb.fetch2.ParameterError(f"Invalid protocol type: '{ud.proto}'", ud.url) + raise bb.fetch.ParameterError(f"Invalid protocol type: '{ud.proto}'", ud.url) ud.nocheckout = ud.parm.get("nocheckout","0") == "1" @@ -174,7 +174,7 @@ class Git(FetchMethod): ud.usehead = ud.parm.get("usehead","0") == "1" if ud.usehead: if ud.proto != "file": - raise bb.fetch2.ParameterError("The usehead option is only for use with local ('protocol=file') git repositories", ud.url) + raise bb.fetch.ParameterError("The usehead option is only for use with local ('protocol=file') git repositories", ud.url) ud.nobranch = 1 # bareclone implies nocheckout @@ -185,7 +185,7 @@ class Git(FetchMethod): ud.unresolvedrev = "" ud.branch = ud.parm.get("branch", "") if not ud.branch and not ud.nobranch: - raise bb.fetch2.ParameterError("The url does not set any branch parameter or set nobranch=1.", ud.url) + raise bb.fetch.ParameterError("The url does not set any branch parameter or set nobranch=1.", ud.url) ud.noshared = d.getVar("BB_GIT_NOSHARED") == "1" @@ -206,10 +206,10 @@ class Git(FetchMethod): try: depth_default = int(depth_default or 0) except ValueError: - raise bb.fetch2.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH: %s" % depth_default) + raise bb.fetch.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH: %s" % depth_default) else: if depth_default < 0: - raise bb.fetch2.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH: %s" % depth_default) + raise bb.fetch.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH: %s" % depth_default) else: depth_default = 1 ud.shallow_depths = collections.defaultdict(lambda: depth_default) @@ -224,10 +224,10 @@ class Git(FetchMethod): try: shallow_depth = int(shallow_depth or 0) except ValueError: - raise bb.fetch2.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH_%s: %s" % (ud.name, shallow_depth)) + raise bb.fetch.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH_%s: %s" % (ud.name, shallow_depth)) else: if shallow_depth < 0: - raise bb.fetch2.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH_%s: %s" % (ud.name, shallow_depth)) + raise bb.fetch.FetchError("Invalid depth for BB_GIT_SHALLOW_DEPTH_%s: %s" % (ud.name, shallow_depth)) ud.shallow_depths[ud.name] = shallow_depth revs = d.getVar("BB_GIT_SHALLOW_REVS_%s" % ud.name) @@ -343,7 +343,7 @@ class Git(FetchMethod): for rev in ud.shallow_revs: try: runfetchcmd(ud.basecmd + ['rev-parse', '-q', '--verify', rev], d, quiet=True, workdir=ud.clonedir) - except bb.fetch2.FetchError: + except bb.fetch.FetchError: return rev return None @@ -415,7 +415,7 @@ class Git(FetchMethod): if not bb.utils.path_is_descendant(toplevel, ud.clonedir): logger.warning("Top level directory '%s' is not a descendant of '%s'. Re-cloning", toplevel, ud.clonedir) needs_clone = True - except bb.fetch2.FetchError as e: + except bb.fetch.FetchError as e: logger.warning("Unable to get top level for %s (not a git directory?): %s", ud.clonedir, e) needs_clone = True except FileNotFoundError as e: @@ -438,7 +438,7 @@ class Git(FetchMethod): repourl = repourl_path clone_cmd = ud.basecmd + ['clone', '--bare', '--mirror', repourl, ud.clonedir, '--progress'] if ud.proto.lower() != 'file': - bb.fetch2.check_network_access(d, clone_cmd, ud.url) + bb.fetch.check_network_access(d, clone_cmd, ud.url) progresshandler = GitProgressHandler(d) # Try creating a fast initial shallow clone @@ -473,7 +473,7 @@ class Git(FetchMethod): else: fetch_cmd = ud.basecmd + ['fetch', '-f', '--progress', repourl, 'refs/heads/*:refs/heads/*', 'refs/tags/*:refs/tags/*'] if ud.proto.lower() != 'file': - bb.fetch2.check_network_access(d, fetch_cmd, ud.url) + bb.fetch.check_network_access(d, fetch_cmd, ud.url) progresshandler = GitProgressHandler(d) runfetchcmd(fetch_cmd, d, log=progresshandler, workdir=ud.clonedir, extraenv={'LANG':'C'}) runfetchcmd(ud.basecmd + ['repack','-adk'], d, workdir=ud.clonedir) @@ -486,12 +486,12 @@ class Git(FetchMethod): raise if not self._contains_ref(ud, d, ud.name, ud.clonedir): - raise bb.fetch2.FetchError("Unable to find revision %s in branch %s even from upstream" % (ud.revision, ud.branch)) + raise bb.fetch.FetchError("Unable to find revision %s in branch %s even from upstream" % (ud.revision, ud.branch)) if ud.shallow and ud.write_shallow_tarballs: missing_rev = self.clonedir_need_shallow_revs(ud, d) if missing_rev: - raise bb.fetch2.FetchError("Unable to find revision %s even from upstream" % missing_rev) + raise bb.fetch.FetchError("Unable to find revision %s even from upstream" % missing_rev) if self.lfs_need_update(ud, d): self.lfs_fetch(ud, d, ud.clonedir, ud.revision) @@ -595,7 +595,7 @@ class Git(FetchMethod): # The --depth and --shallow-exclude can't be used together if depth and shallow_exclude: - raise bb.fetch2.FetchError("BB_GIT_SHALLOW_REVS is set, but BB_GIT_SHALLOW_DEPTH is not 0.") + raise bb.fetch.FetchError("BB_GIT_SHALLOW_REVS is set, but BB_GIT_SHALLOW_DEPTH is not 0.") # For nobranch, we need a ref, otherwise the commits will be # removed, and for non-nobranch, we truncate the branch to our @@ -673,7 +673,7 @@ class Git(FetchMethod): # If 'subdir' param exists, create a dir and use it as destination for unpack cmd if os.path.isabs(subdir): if not os.path.realpath(subdir).startswith(os.path.realpath(destdir)): - raise bb.fetch2.UnpackError("subdir argument isn't a subdirectory of unpack root %s" % destdir, ud.url) + raise bb.fetch.UnpackError("subdir argument isn't a subdirectory of unpack root %s" % destdir, ud.url) destdir = subdir else: destdir = os.path.join(destdir, subdir) @@ -721,35 +721,35 @@ class Git(FetchMethod): source_error.append("shallow clone not enabled") if not source_found: - raise bb.fetch2.UnpackError("No up to date source found: " + "; ".join(source_error), ud.url) + raise bb.fetch.UnpackError("No up to date source found: " + "; ".join(source_error), ud.url) if update_mode: if ud.shallow: - raise bb.fetch2.UnpackError("Can't update shallow clones checkouts without network access, not supported.", ud.url) + raise bb.fetch.UnpackError("Can't update shallow clones checkouts without network access, not supported.", ud.url) output = runfetchcmd(ud.basecmd + ['status', '--untracked-files=no', '--porcelain'], d, workdir=destdir, extraenv=extraenv) if output: - raise bb.fetch2.LocalModificationsError(destdir, ud.url, output) + raise bb.fetch.LocalModificationsError(destdir, ud.url, output) # Set up remote for the download location if it doesn't exist try: runfetchcmd(ud.basecmd + ['remote', 'get-url', 'dldir'], d, workdir=destdir) - except bb.fetch2.FetchError: + except bb.fetch.FetchError: if ud.clonedir: runfetchcmd(ud.basecmd + ['remote', 'add', 'dldir', 'file://' + ud.clonedir], d, workdir=destdir) try: runfetchcmd(ud.basecmd + ['fetch', 'dldir'], d, workdir=destdir, extraenv=extraenv) - except bb.fetch2.FetchError as e: - raise bb.fetch2.UnpackError("Failed to fetch from dldir remote: %s" % str(e), ud.url) + except bb.fetch.FetchError as e: + raise bb.fetch.UnpackError("Failed to fetch from dldir remote: %s" % str(e), ud.url) try: runfetchcmd(ud.basecmd + ['rebase', '--no-autosquash', '--no-autostash', ud.revision], d, workdir=destdir, extraenv=extraenv) - except bb.fetch2.FetchError as e: + except bb.fetch.FetchError as e: # If rebase failed, abort it try: runfetchcmd(ud.basecmd + ['rebase', '--abort'], d, workdir=destdir) except Exception: pass - raise bb.fetch2.RebaseError(destdir, ud.url, str(e)) + raise bb.fetch.RebaseError(destdir, ud.url, str(e)) return True # If there is a tag parameter in the url and we also have a fixed srcrev, check the tag @@ -763,14 +763,14 @@ class Git(FetchMethod): output2 = runfetchcmd(ud.basecmd + ['rev-list', '-n', '1', ud.revision], d, workdir=destdir) output2 = output2.strip() if output != output2: - raise bb.fetch2.FetchError("The revision the git tag '%s' resolved to didn't match the SRCREV in use (%s vs %s)" % (ud.parm['tag'], output, ud.revision), ud.url) + raise bb.fetch.FetchError("The revision the git tag '%s' resolved to didn't match the SRCREV in use (%s vs %s)" % (ud.parm['tag'], output, ud.revision), ud.url) repourl = self._get_repo_url(ud) runfetchcmd(ud.basecmd + ['remote', 'set-url', 'origin', repourl], d, workdir=destdir) if ud.clonedir: try: runfetchcmd(ud.basecmd + ['remote', 'get-url', 'dldir'], d, workdir=destdir) - except bb.fetch2.FetchError: + except bb.fetch.FetchError: runfetchcmd(ud.basecmd + ['remote', 'add', 'dldir', "file://" + ud.clonedir], d, workdir=destdir) if self._contains_lfs(ud, d, destdir): @@ -830,7 +830,7 @@ class Git(FetchMethod): cmd = ud.basecmd + ['branch', '--contains', git_ref_name, '--list', ud.branch] try: output = runfetchcmd(cmd, d, workdir=wd) - except (bb.fetch2.FetchError): + except (bb.fetch.FetchError): return False return len(output.splitlines()) > 0 @@ -876,7 +876,7 @@ class Git(FetchMethod): output = runfetchcmd(cmd, d, quiet=True, workdir=wd) if len(output.splitlines()) > 0: return True - except (bb.fetch2.FetchError,ValueError): + except (bb.fetch.FetchError,ValueError): pass return False @@ -885,7 +885,7 @@ class Git(FetchMethod): Ensures that git-lfs is available, raising a FetchError if it isn't. """ if shutil.which("git-lfs", path=d.getVar('PATH')) is None: - raise bb.fetch2.FetchError( + raise bb.fetch.FetchError( "Repository %s has LFS content, install git-lfs on host to download (or set lfs=0 " "to ignore it)" % self._get_repo_url(ud)) @@ -930,10 +930,10 @@ class Git(FetchMethod): if search: cmd.append(search) if ud.proto.lower() != 'file': - bb.fetch2.check_network_access(d, cmd, repourl) + bb.fetch.check_network_access(d, cmd, repourl) output = runfetchcmd(cmd, d, True) if not output: - raise bb.fetch2.FetchError("The command %s gave empty output unexpectedly" % cmd, ud.url) + raise bb.fetch.FetchError("The command %s gave empty output unexpectedly" % cmd, ud.url) finally: d.delVar('_BB_GIT_IN_LSREMOTE') return output @@ -943,10 +943,10 @@ class Git(FetchMethod): Compute the HEAD revision for the url """ if not d.getVar("__BBSRCREV_SEEN"): - raise bb.fetch2.FetchError("Recipe uses a floating tag/branch '%s' for repo '%s' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE)." % (ud.unresolvedrev, ud.host+ud.path)) + raise bb.fetch.FetchError("Recipe uses a floating tag/branch '%s' for repo '%s' without a fixed SRCREV yet doesn't call bb.fetch.get_srcrev() (use SRCPV in PV for OE)." % (ud.unresolvedrev, ud.host+ud.path)) # Ensure we mark as not cached - bb.fetch2.mark_recipe_nocache(d) + bb.fetch.mark_recipe_nocache(d) output = self._lsremote(ud, d, "") # Tags of the form ^{} may not work, need to fallback to other form @@ -961,7 +961,7 @@ class Git(FetchMethod): sha1, ref = l.split() if s == ref: return sha1 - raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git repository in git ls-remote output for %s" % \ + raise bb.fetch.FetchError("Unable to resolve '%s' in upstream git repository in git ls-remote output for %s" % \ (ud.unresolvedrev, ud.host+ud.path)) def latest_versionstring(self, ud, d, filter_regex=None): @@ -974,7 +974,7 @@ class Git(FetchMethod): try: output = self._lsremote(ud, d, "refs/tags/*") - except (bb.fetch2.FetchError, bb.fetch2.NetworkAccess) as e: + except (bb.fetch.FetchError, bb.fetch.NetworkAccess) as e: bb.note("Could not list remote: %s" % str(e)) return pupver @@ -1031,7 +1031,7 @@ class Git(FetchMethod): commits = None else: if not os.path.exists(rev_file) or not os.path.getsize(rev_file): - commits = len(bb.fetch2.runfetchcmd(['git', 'rev-list', rev, '--'], d).splitlines()) + commits = len(bb.fetch.runfetchcmd(['git', 'rev-list', rev, '--'], d).splitlines()) if commits: open(rev_file, "w").write("%d\n" % commits) else: @@ -1045,5 +1045,5 @@ class Git(FetchMethod): try: self._lsremote(ud, d, "") return True - except bb.fetch2.FetchError: + except bb.fetch.FetchError: return False diff --git a/lib/bb/fetch2/gitannex.py b/lib/bb/fetch/gitannex.py similarity index 97% rename from lib/bb/fetch2/gitannex.py rename to lib/bb/fetch/gitannex.py index c378cc36d..d5917e54f 100644 --- a/lib/bb/fetch2/gitannex.py +++ b/lib/bb/fetch/gitannex.py @@ -9,8 +9,8 @@ BitBake 'Fetch' git annex implementation # import bb -from bb.fetch2.git import Git -from bb.fetch2 import runfetchcmd +from bb.fetch.git import Git +from bb.fetch import runfetchcmd class GitANNEX(Git): def supports(self, ud, d): diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch/gitsm.py similarity index 97% rename from lib/bb/fetch2/gitsm.py rename to lib/bb/fetch/gitsm.py index d4340e116..734b35364 100644 --- a/lib/bb/fetch2/gitsm.py +++ b/lib/bb/fetch/gitsm.py @@ -22,10 +22,10 @@ import bb import copy import shutil import tempfile -from bb.fetch2.git import Git -from bb.fetch2 import runfetchcmd -from bb.fetch2 import logger -from bb.fetch2 import Fetch +from bb.fetch.git import Git +from bb.fetch import runfetchcmd +from bb.fetch import logger +from bb.fetch import Fetch class GitSM(Git): def supports(self, ud, d): @@ -115,7 +115,7 @@ class GitSM(Git): proto = "file" url = "gitsm://" + uris[module] if url.endswith("{}{}".format(ud.host, ud.path)): - raise bb.fetch2.FetchError("Submodule refers to the parent repository. This will cause deadlock situation in current version of Bitbake." \ + raise bb.fetch.FetchError("Submodule refers to the parent repository. This will cause deadlock situation in current version of Bitbake." \ "Consider using git fetcher instead.") url += ';protocol=%s' % proto diff --git a/lib/bb/fetch2/gomod.py b/lib/bb/fetch/gomod.py similarity index 97% rename from lib/bb/fetch2/gomod.py rename to lib/bb/fetch/gomod.py index 5cdf8f998..a62d643fb 100644 --- a/lib/bb/fetch2/gomod.py +++ b/lib/bb/fetch/gomod.py @@ -67,12 +67,12 @@ import subprocess import zipfile import bb -from bb.fetch2 import FetchError -from bb.fetch2 import MissingParameterError -from bb.fetch2 import runfetchcmd -from bb.fetch2 import subprocess_setup -from bb.fetch2.git import Git -from bb.fetch2.wget import Wget +from bb.fetch import FetchError +from bb.fetch import MissingParameterError +from bb.fetch import runfetchcmd +from bb.fetch import subprocess_setup +from bb.fetch.git import Git +from bb.fetch.wget import Wget def escape(path): @@ -115,7 +115,7 @@ class GoMod(Wget): else: ext = '.zip' path = escape(f"{module}/@v/{version}{ext}") - ud.url = bb.fetch2.encodeurl( + ud.url = bb.fetch.encodeurl( ('https', proxy, '/' + path, None, None, None)) ud.parm['downloadfilename'] = f"{module.replace('/', '.')}@{version}{ext}" diff --git a/lib/bb/fetch2/hg.py b/lib/bb/fetch/hg.py similarity index 95% rename from lib/bb/fetch2/hg.py rename to lib/bb/fetch/hg.py index 5c65a1985..9813b0edd 100644 --- a/lib/bb/fetch2/hg.py +++ b/lib/bb/fetch/hg.py @@ -16,11 +16,11 @@ import os import bb import errno import shlex -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import MissingParameterError -from bb.fetch2 import runfetchcmd -from bb.fetch2 import logger +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import MissingParameterError +from bb.fetch import runfetchcmd +from bb.fetch import logger class Hg(FetchMethod): """Class to fetch from mercurial repositories""" @@ -159,13 +159,13 @@ class Hg(FetchMethod): logger.debug("Running %s", updatecmd) try: runfetchcmd(updatecmd, d, workdir=ud.moddir) - except bb.fetch2.FetchError: + except bb.fetch.FetchError: # Runnning pull in the repo pullcmd = self._buildhgcommand(ud, d, "pull") logger.info("Pulling " + ud.url) # update sources there logger.debug("Running %s", pullcmd) - bb.fetch2.check_network_access(d, pullcmd, ud.url) + bb.fetch.check_network_access(d, pullcmd, ud.url) runfetchcmd(pullcmd, d, workdir=ud.moddir) try: os.unlink(ud.fullmirror) @@ -180,7 +180,7 @@ class Hg(FetchMethod): # check out sources there bb.utils.mkdirhier(ud.pkgdir) logger.debug("Running %s", fetchcmd) - bb.fetch2.check_network_access(d, fetchcmd, ud.url) + bb.fetch.check_network_access(d, fetchcmd, ud.url) runfetchcmd(fetchcmd, d, workdir=ud.pkgdir) # Even when we clone (fetch), we still need to update as hg's clone @@ -203,7 +203,7 @@ class Hg(FetchMethod): """ Compute tip revision for the url """ - bb.fetch2.check_network_access(d, self._buildhgcommand(ud, d, "info"), ud.url) + bb.fetch.check_network_access(d, self._buildhgcommand(ud, d, "info"), ud.url) output = runfetchcmd(self._buildhgcommand(ud, d, "info"), d) return output.strip() diff --git a/lib/bb/fetch2/local.py b/lib/bb/fetch/local.py similarity index 91% rename from lib/bb/fetch2/local.py rename to lib/bb/fetch/local.py index fda56a564..dfc5b564c 100644 --- a/lib/bb/fetch2/local.py +++ b/lib/bb/fetch/local.py @@ -17,8 +17,8 @@ import os import urllib.request, urllib.parse, urllib.error import bb import bb.utils -from bb.fetch2 import FetchMethod, FetchError, ParameterError -from bb.fetch2 import logger +from bb.fetch import FetchMethod, FetchError, ParameterError +from bb.fetch import logger class Local(FetchMethod): def supports(self, urldata, d): @@ -33,7 +33,7 @@ class Local(FetchMethod): ud.basepath = ud.path ud.needdonestamp = False if "*" in ud.path: - raise bb.fetch2.ParameterError("file:// urls using globbing are no longer supported. Please place the files in a directory and reference that instead.", ud.url) + raise bb.fetch.ParameterError("file:// urls using globbing are no longer supported. Please place the files in a directory and reference that instead.", ud.url) return def localpath(self, urldata, d): diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch/npm.py similarity index 97% rename from lib/bb/fetch2/npm.py rename to lib/bb/fetch/npm.py index a429a8292..c8995ba6b 100644 --- a/lib/bb/fetch2/npm.py +++ b/lib/bb/fetch/npm.py @@ -35,15 +35,15 @@ import re import shlex import tempfile import bb -from bb.fetch2 import Fetch -from bb.fetch2 import FetchError -from bb.fetch2 import FetchMethod -from bb.fetch2 import MalformedUrl -from bb.fetch2 import MissingParameterError -from bb.fetch2 import ParameterError -from bb.fetch2 import URI -from bb.fetch2 import check_network_access -from bb.fetch2 import runfetchcmd +from bb.fetch import Fetch +from bb.fetch import FetchError +from bb.fetch import FetchMethod +from bb.fetch import MalformedUrl +from bb.fetch import MissingParameterError +from bb.fetch import ParameterError +from bb.fetch import URI +from bb.fetch import check_network_access +from bb.fetch import runfetchcmd from bb.utils import is_semver # Preference order matches strength; the first one present in SRC_URI wins. diff --git a/lib/bb/fetch2/npmsw.py b/lib/bb/fetch/npmsw.py similarity index 96% rename from lib/bb/fetch2/npmsw.py rename to lib/bb/fetch/npmsw.py index f09ea5794..fde4fe65b 100644 --- a/lib/bb/fetch2/npmsw.py +++ b/lib/bb/fetch/npmsw.py @@ -21,15 +21,15 @@ import json import os import re import bb -from bb.fetch2 import FetchError -from bb.fetch2 import Fetch -from bb.fetch2 import FetchMethod -from bb.fetch2 import ParameterError -from bb.fetch2 import runfetchcmd -from bb.fetch2 import URI -from bb.fetch2.npm import npm_integrity -from bb.fetch2.npm import npm_localfile -from bb.fetch2.npm import npm_unpack +from bb.fetch import FetchError +from bb.fetch import Fetch +from bb.fetch import FetchMethod +from bb.fetch import ParameterError +from bb.fetch import runfetchcmd +from bb.fetch import URI +from bb.fetch.npm import npm_integrity +from bb.fetch.npm import npm_localfile +from bb.fetch.npm import npm_unpack from bb.utils import is_semver from bb.utils import lockfile from bb.utils import unlockfile diff --git a/lib/bb/fetch2/perforce.py b/lib/bb/fetch/perforce.py similarity index 96% rename from lib/bb/fetch2/perforce.py rename to lib/bb/fetch/perforce.py index 52ea061c0..aa46e0ebc 100644 --- a/lib/bb/fetch2/perforce.py +++ b/lib/bb/fetch/perforce.py @@ -27,10 +27,10 @@ Supported SRC_URI options are: import os import bb import shlex -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import logger -from bb.fetch2 import runfetchcmd +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import logger +from bb.fetch import runfetchcmd class PerforceProgressHandler (bb.progress.BasicProgressHandler): """ @@ -97,7 +97,7 @@ class Perforce(FetchMethod): logger.debug('Trying to use P4CONFIG to automatically set P4PORT...') ud.usingp4config = True p4cmd = ud.basecmd + ['info'] - bb.fetch2.check_network_access(d, p4cmd, ud.url) + bb.fetch.check_network_access(d, p4cmd, ud.url) output = runfetchcmd(p4cmd, d, True) ud.host = None for line in output.splitlines(): @@ -201,7 +201,7 @@ class Perforce(FetchMethod): 'p4 files' command, including trailing '#rev' file revision indicator """ p4cmd = self._buildp4command(ud, d, 'files') - bb.fetch2.check_network_access(d, p4cmd, ud.url) + bb.fetch.check_network_access(d, p4cmd, ud.url) p4fileslist = runfetchcmd(p4cmd, d, True) p4fileslist = [f.rstrip() for f in p4fileslist.splitlines()] @@ -234,7 +234,7 @@ class Perforce(FetchMethod): for afile in filelist: p4fetchcmd = self._buildp4command(ud, d, 'print', afile) - bb.fetch2.check_network_access(d, p4fetchcmd, ud.url) + bb.fetch.check_network_access(d, p4fetchcmd, ud.url) runfetchcmd(p4fetchcmd, d, workdir=ud.pkgdir, log=progresshandler) runfetchcmd(['tar', '-czf', ud.localpath, 'p4'], d, cleanup=[ud.localpath], workdir=ud.pkgdir) @@ -254,7 +254,7 @@ class Perforce(FetchMethod): def _latest_revision(self, ud, d, name): """ Return the latest upstream scm revision number """ p4cmd = self._buildp4command(ud, d, "changes") - bb.fetch2.check_network_access(d, p4cmd, ud.url) + bb.fetch.check_network_access(d, p4cmd, ud.url) tip = runfetchcmd(p4cmd, d, True) if not tip: diff --git a/lib/bb/fetch2/repo.py b/lib/bb/fetch/repo.py similarity index 89% rename from lib/bb/fetch2/repo.py rename to lib/bb/fetch/repo.py index 9b96cd824..c7eaf8b3c 100644 --- a/lib/bb/fetch2/repo.py +++ b/lib/bb/fetch/repo.py @@ -14,9 +14,9 @@ BitBake "Fetch" repo (git) implementation import os import bb import shlex -from bb.fetch2 import FetchMethod -from bb.fetch2 import runfetchcmd -from bb.fetch2 import logger +from bb.fetch import FetchMethod +from bb.fetch import runfetchcmd +from bb.fetch import logger class Repo(FetchMethod): """Class to fetch a module or modules from repo (git) repositories""" @@ -63,10 +63,10 @@ class Repo(FetchMethod): repodir = os.path.join(codir, "repo") bb.utils.mkdirhier(repodir) if not os.path.exists(os.path.join(repodir, ".repo")): - bb.fetch2.check_network_access(d, "%s init -m %s -b %s -u %s://%s%s%s" % (ud.basecmd, ud.manifest, ud.branch, ud.proto, username, ud.host, ud.path), ud.url) + bb.fetch.check_network_access(d, "%s init -m %s -b %s -u %s://%s%s%s" % (ud.basecmd, ud.manifest, ud.branch, ud.proto, username, ud.host, ud.path), ud.url) runfetchcmd(ud.basecmd + ['init', '-m', ud.manifest, '-b', ud.branch, '-u', '%s://%s%s%s' % (ud.proto, username, ud.host, ud.path)], d, workdir=repodir) - bb.fetch2.check_network_access(d, ud.basecmd + ['sync'], ud.url) + bb.fetch.check_network_access(d, ud.basecmd + ['sync'], ud.url) runfetchcmd(ud.basecmd + ['sync'], d, workdir=repodir) scmdata = ud.parm.get("scmdata", "") diff --git a/lib/bb/fetch2/s3.py b/lib/bb/fetch/s3.py similarity index 93% rename from lib/bb/fetch2/s3.py rename to lib/bb/fetch/s3.py index fdd450429..ea3dc28fc 100644 --- a/lib/bb/fetch2/s3.py +++ b/lib/bb/fetch/s3.py @@ -8,7 +8,7 @@ The aws tool must be correctly installed and configured prior to use. # Copyright (C) 2017, Andre McCurdy # -# Based in part on bb.fetch2.wget: +# Based in part on bb.fetch.wget: # Copyright (C) 2003, 2004 Chris Larson # # SPDX-License-Identifier: GPL-2.0-only @@ -20,9 +20,9 @@ import bb import urllib.request, urllib.parse, urllib.error import re import shlex -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import runfetchcmd +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import runfetchcmd def convertToBytes(value, unit): value = float(value) @@ -89,7 +89,7 @@ class S3(FetchMethod): """ cmd = ud.basecmd + ['cp', 's3://%s%s' % (ud.host, ud.path), ud.localpath] - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) progresshandler = S3ProgressHandler(d) runfetchcmd(cmd, d, False, log=progresshandler) @@ -113,7 +113,7 @@ class S3(FetchMethod): """ cmd = ud.basecmd + ['ls', 's3://%s%s' % (ud.host, ud.path)] - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) output = runfetchcmd(cmd, d) # "aws s3 ls s3://mybucket/foo" will exit with success even if the file diff --git a/lib/bb/fetch2/sftp.py b/lib/bb/fetch/sftp.py similarity index 93% rename from lib/bb/fetch2/sftp.py rename to lib/bb/fetch/sftp.py index 82c76bb66..4054382d7 100644 --- a/lib/bb/fetch2/sftp.py +++ b/lib/bb/fetch/sftp.py @@ -39,7 +39,7 @@ SRC_URI = "sftp://user@host.example.com/dir/path.file.txt" # Copyright (C) 2013, Olof Johansson # -# Based in part on bb.fetch2.wget: +# Based in part on bb.fetch.wget: # Copyright (C) 2003, 2004 Chris Larson # # SPDX-License-Identifier: GPL-2.0-only @@ -49,9 +49,9 @@ SRC_URI = "sftp://user@host.example.com/dir/path.file.txt" import os import bb import urllib.request, urllib.parse, urllib.error -from bb.fetch2 import URI -from bb.fetch2 import FetchMethod -from bb.fetch2 import runfetchcmd +from bb.fetch import URI +from bb.fetch import FetchMethod +from bb.fetch import runfetchcmd class SFTP(FetchMethod): """Class to fetch urls via 'sftp'""" @@ -67,7 +67,7 @@ class SFTP(FetchMethod): def urldata_init(self, ud, d): if 'protocol' in ud.parm and ud.parm['protocol'] == 'git': - raise bb.fetch2.ParameterError( + raise bb.fetch.ParameterError( "Invalid protocol - if you wish to fetch from a " + "git repository using ssh, you need to use the " + "git:// prefix with protocol=ssh", ud.url) @@ -106,6 +106,6 @@ class SFTP(FetchMethod): cmd = basecmd + [remote, lpath] - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) runfetchcmd(cmd, d) return True diff --git a/lib/bb/fetch2/ssh.py b/lib/bb/fetch/ssh.py similarity index 98% rename from lib/bb/fetch2/ssh.py rename to lib/bb/fetch/ssh.py index 56e455fb4..304872ad0 100644 --- a/lib/bb/fetch2/ssh.py +++ b/lib/bb/fetch/ssh.py @@ -31,7 +31,7 @@ IETF secsh internet draft: # import re, os -from bb.fetch2 import check_network_access, FetchMethod, ParameterError, runfetchcmd +from bb.fetch import check_network_access, FetchMethod, ParameterError, runfetchcmd import urllib diff --git a/lib/bb/fetch2/svn.py b/lib/bb/fetch/svn.py similarity index 94% rename from lib/bb/fetch2/svn.py rename to lib/bb/fetch/svn.py index 973f76f00..73ce83f4e 100644 --- a/lib/bb/fetch2/svn.py +++ b/lib/bb/fetch/svn.py @@ -14,11 +14,11 @@ import os import bb import re import shlex -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import MissingParameterError -from bb.fetch2 import runfetchcmd -from bb.fetch2 import logger +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import MissingParameterError +from bb.fetch import runfetchcmd +from bb.fetch import logger class Svn(FetchMethod): """Class to fetch a module or modules from svn repositories""" @@ -141,7 +141,7 @@ class Svn(FetchMethod): except FetchError: pass logger.debug("Running %s", svncmd) - bb.fetch2.check_network_access(d, svncmd, ud.url) + bb.fetch.check_network_access(d, svncmd, ud.url) runfetchcmd(svncmd, d, workdir=ud.moddir, extraenv=extraenv) else: svncmd, extraenv = self._buildsvncommand(ud, d, "fetch") @@ -149,7 +149,7 @@ class Svn(FetchMethod): # check out sources there bb.utils.mkdirhier(ud.pkgdir) logger.debug("Running %s", svncmd) - bb.fetch2.check_network_access(d, svncmd, ud.url) + bb.fetch.check_network_access(d, svncmd, ud.url) runfetchcmd(svncmd, d, workdir=ud.pkgdir, extraenv=extraenv) if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"): @@ -164,7 +164,7 @@ class Svn(FetchMethod): bb.warn("To disable this warning add ';externals=nowarn' to the url.") else: bb.debug(1, "svn repository has externals:\n%s" % output) - except bb.fetch2.FetchError: + except bb.fetch.FetchError: passs scmdata = ud.parm.get("scmdata", "") if scmdata == "keep": @@ -199,7 +199,7 @@ class Svn(FetchMethod): Return the latest upstream revision number """ cmd, extraenv = self._buildsvncommand(ud, d, "log1") - bb.fetch2.check_network_access(d, cmd, ud.url) + bb.fetch.check_network_access(d, cmd, ud.url) extraenv['LANG'] = 'C' extraenv['LC_ALL'] = 'C' output = runfetchcmd(cmd, d, True, extraenv=extraenv) diff --git a/lib/bb/fetch2/wget.py b/lib/bb/fetch/wget.py similarity index 98% rename from lib/bb/fetch2/wget.py rename to lib/bb/fetch/wget.py index 0d2b7f5c8..7b05a9ef2 100644 --- a/lib/bb/fetch2/wget.py +++ b/lib/bb/fetch/wget.py @@ -23,10 +23,10 @@ import socket import http.client import urllib.request, urllib.parse, urllib.error import subprocess -from bb.fetch2 import FetchMethod -from bb.fetch2 import FetchError -from bb.fetch2 import logger -from bb.fetch2 import runfetchcmd +from bb.fetch import FetchMethod +from bb.fetch import FetchError +from bb.fetch import logger +from bb.fetch import runfetchcmd from bb._vendor.bs4 import BeautifulSoup from bb._vendor.bs4 import SoupStrainer @@ -94,7 +94,7 @@ class Wget(FetchMethod): def urldata_init(self, ud, d): if 'protocol' in ud.parm: if ud.parm['protocol'] == 'git': - raise bb.fetch2.ParameterError("Invalid protocol - if you wish to fetch from a git repository using http, you need to instead use the git:// prefix with protocol=http", ud.url) + raise bb.fetch.ParameterError("Invalid protocol - if you wish to fetch from a git repository using http, you need to instead use the git:// prefix with protocol=http", ud.url) if 'downloadfilename' in ud.parm: ud.basename = ud.parm['downloadfilename'] @@ -118,7 +118,7 @@ class Wget(FetchMethod): progresshandler = WgetProgressHandler(d) logger.debug2("Fetching %s using command '%s'" % (ud.url, command)) - bb.fetch2.check_network_access(d, command, ud.url) + bb.fetch.check_network_access(d, command, ud.url) runfetchcmd(command + ['--progress=dot', '--verbose'], d, quiet, log=progresshandler, workdir=workdir) @@ -167,7 +167,7 @@ class Wget(FetchMethod): # Try and verify any checksum now, meaning if it isn't correct, we don't remove the # original file, which might be a race (imagine two recipes referencing the same # source, one with an incorrect checksum) - bb.fetch2.verify_checksum(ud, d, localpath=localpath, fatal_nochecksum=False) + bb.fetch.verify_checksum(ud, d, localpath=localpath, fatal_nochecksum=False) # Remove the ".tmp" and move the file into position atomically # Our lock prevents multiple writers but mirroring code may grab incomplete files @@ -177,7 +177,7 @@ class Wget(FetchMethod): def checkstatus(self, fetch, ud, d, try_again=True): check_certs = self.check_certs(d) - newenv = bb.fetch2.get_fetcher_environment(d) + newenv = bb.fetch.get_fetcher_environment(d) class HTTPConnectionCache(http.client.HTTPConnection): def cache_id(self): @@ -568,7 +568,7 @@ class Wget(FetchMethod): try: self._runwget(ud, d, fetchcmd, True, workdir=workdir) fetchresult = f.read() - except bb.fetch2.BBFetchException: + except bb.fetch.BBFetchException: fetchresult = "" return fetchresult diff --git a/lib/bb/fetch2.py b/lib/bb/fetch2.py new file mode 100644 index 000000000..e64017016 --- /dev/null +++ b/lib/bb/fetch2.py @@ -0,0 +1,19 @@ +# +# Copyright (C) 2026 Agilent Technologies, Inc. +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import sys +import warnings + +import bb.fetch + +warnings.warn( + "bb.fetch2 is deprecated; use bb.fetch instead", + DeprecationWarning, + stacklevel=2, +) + +# Compatibility shim for code using bb.fetch2 +sys.modules[__name__] = bb.fetch