From patchwork Fri Feb 28 22:30:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 58108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAA55C282C5 for ; Fri, 28 Feb 2025 22:30:56 +0000 (UTC) Received: from EUR02-DB5-obe.outbound.protection.outlook.com (EUR02-DB5-obe.outbound.protection.outlook.com [40.107.249.84]) by mx.groups.io with SMTP id smtpd.web10.1447.1740781852735994171 for ; Fri, 28 Feb 2025 14:30:53 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@axis.com header.s=selector1 header.b=Q4Di01R7; spf=pass (domain: axis.com, ip: 40.107.249.84, mailfrom: peter.kjellerstedt@axis.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=c0vRBcdmExsk6s1C2/QEPbmFtEHljKGWVPyGzT3oT8tc7VD100Eklf5DcoNsgzyuVY0FA+yJOQ40c01U6lqOzYwkdIRVDgJHuYVMCarCRFyx88vRegiGNLxN8wIEPTl6+vp/K/Rhlu80p77KVjuvREc/of2gUrUiHgOPSyHCBL0t6IM6Ubvv+eY7IWHe+QER4Z/cezezzsXQw/+0GCl3ivHm/uPLRD8l9ugbRB1r2GC3gEXKfV/fXk1wEZ4WmrqK1WIr339yFpYmyfU3XvJIUTcd0J2pZ0WzTQQhKIOId8IW3v7ZtwkbDjF98M4pK1J2pieM5Dk9AdcKRikasxtDjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SoUB7DZWMjt0D5CR6k12keV3fayAF65J8wIjWlikRaQ=; b=wmJAdn32++zkTOGL29FiKOG5ZlMmZML8zs2HcVBVjdXpnIJ8eWIwva2xl0TJoEQcHF+FnJSZSdvPrRfUIWtt3zc4KLFdWgt4e5+IfhClfUjkXX0v/+POwIrmA5jbUgQ1lx7I3Scm95yhr+JMiGZJJZE0CC1iEeBO/LQMZCYnLeM3sNmcepClwpIqSsXgFsAkq8c89BN79PU7/eK/+majXQhRFIN89MZnhEnSQJMqDNIv+fpDyPmRrb/EICi86bfyGZ0gbzxRCfbmYtffoft++vVScQwTgZFHFKSnhsY9pUWK7F91VgBz1czonhM7CB0oXMbuodRoO5g4/3Y3QZOfgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 195.60.68.100) smtp.rcpttodomain=lists.openembedded.org smtp.mailfrom=axis.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=axis.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SoUB7DZWMjt0D5CR6k12keV3fayAF65J8wIjWlikRaQ=; b=Q4Di01R7pCDZQ9oiQkb9kJ03OC50PX+vqvnkEhX1hF696FJKLhbmcx8HipBq7n+JKEvvc2Ed5iG8W9NMN75hTaynlCx66QifbPaJLMFrWHwOzBmT/AIQP+LYG+0xsfop1ESusb7EsWqc+CjmiwAKJAGHI4YQlybvqriNdIC+D60= Received: from CT2P275CA0082.ZAFP275.PROD.OUTLOOK.COM (2603:1086:100:27::8) by AS8PR02MB10318.eurprd02.prod.outlook.com (2603:10a6:20b:63c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 22:30:47 +0000 Received: from DU6PEPF0000A7E3.eurprd02.prod.outlook.com (2603:1086:100:27:cafe::60) by CT2P275CA0082.outlook.office365.com (2603:1086:100:27::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8489.23 via Frontend Transport; Fri, 28 Feb 2025 22:30:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 195.60.68.100) smtp.mailfrom=axis.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=axis.com; Received-SPF: Pass (protection.outlook.com: domain of axis.com designates 195.60.68.100 as permitted sender) receiver=protection.outlook.com; client-ip=195.60.68.100; helo=mail.axis.com; pr=C Received: from mail.axis.com (195.60.68.100) by DU6PEPF0000A7E3.mail.protection.outlook.com (10.167.8.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8489.16 via Frontend Transport; Fri, 28 Feb 2025 22:30:43 +0000 Received: from se-mail02w.axis.com (10.20.40.8) by se-mail02w.axis.com (10.20.40.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.44; Fri, 28 Feb 2025 23:30:43 +0100 Received: from se-intmail02x.se.axis.com (10.4.0.28) by se-mail02w.axis.com (10.20.40.8) with Microsoft SMTP Server id 15.1.2507.44 via Frontend Transport; Fri, 28 Feb 2025 23:30:43 +0100 Received: from saur (saur.se.axis.com [10.92.3.10]) by se-intmail02x.se.axis.com (Postfix) with ESMTP id 20F9763 for ; Fri, 28 Feb 2025 23:30:43 +0100 (CET) Received: from saur.se.axis.com (localhost [127.0.0.1]) by saur (8.17.1/8.15.2) with ESMTPS id 51SMUhQ63902440 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Fri, 28 Feb 2025 23:30:43 +0100 Received: (from pkj@localhost) by saur.se.axis.com (8.17.1/8.17.1/Submit) id 51SMUgjw3902439 for openembedded-core@lists.openembedded.org; Fri, 28 Feb 2025 23:30:42 +0100 From: Peter Kjellerstedt To: Subject: [PATCH 1/2] fetch2: Revert decodeurl() to not use the URI class Date: Fri, 28 Feb 2025 23:30:38 +0100 Message-ID: <20250228223039.3902407-1-pkj@axis.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU6PEPF0000A7E3:EE_|AS8PR02MB10318:EE_ X-MS-Office365-Filtering-Correlation-Id: 790741b5-b6ee-4d92-f26c-08dd584789ed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: wFE4PklCz2oFb14baaip+DorCLDvPmvQY5BoFgVd55YbvtQpZHHVAa5Nxp/eEf+mbrC9wx6UoY1664tO1X/KgLrybpSNx6AONXfX9vpjFfUpyGVfOSAco072ittAa9uSJT+dWQ/N2R9iKkb0DWxJh4atbSrWfpSJ+5Sk2dD5Cw4WQTtpoMQTtzReHHnFReLemUjcqunv1mJ8WEOtSltwDUGtfDoBXVIxFLVhUpuw1VPoYj5+m910sqWXHuU9x0/2s/l3spt5hkFpWf0FS00oe356ijjG5Q4an2B+txwti6YG9M3vCgkLToSkPqbz9fI0dYhYh65/y3ndYRdlcw14k3neOomOI5Fx2aCc4TiHptcthEpt6LHc6Os0KicQbMKHIcQFRffHxrvaH/U+jcQz/NmqyO4mDP0HAjiOkW72rWH/r6XKvxuA0f8U7HJtKs35tlCZeHdLkdIFIIccQ7unlDd5gFuJEs0aRoTkxSs+GqRQ5TAp2oQFn0GtaUyBCWvUl0fXWJUM47ubO31En+K6bcDWfEx8aFIq7eJo5THHArLQW6wGr0KHKt1Lex7Pq12b2JCreKAy2Vn4wK+jhX9urq7w63rPawuErakPIVKg7wFe2gXbg6qBM/z4wrLGAlsfFRoBzy1zowAZzMNJS/yDn9XqaORyjqoiC5R3QEcoOWXgRpF0LK720BCbyMOSb3vK+dfK7/8e3vPkRMNDXQo72/991joDQRsV59Vw0SlB4Y0rpy+alVGYz8T5ZoaDkgY6/WvSxZiJcahuyqcD1rGIJ1PrCAHW3hwpwGyaVajVTWefRKXEBUqc8caP0h6jk0phhAM6MTOB9cQVyzxaIggjPHFwCVoKMVAqcmihYMdAwLQFDed9bG1HaGRkSj7174+HFI12uT6Plo/Msslme6sKABI02prDCEJBOhr/eW7CP5Z6pT0zHeyDMoSAYc6GzG0Q1aIsG9PmQLbbnnCmdXz44rNiVOSOmg/xJ0qYkVEyRHZCmV1Md/zSAJzha1pG4hZ/hgYMHOaHYkPMcxNoeDixBqIGbZOsiLy5mf8b7xueq+v8xOyDibTZ9WoE257J7RqYTbfVZikU+ximf0AXWoMl9jUhSMbClMvr795NxXdm1wVT517ofDH1S42BHwcmufEyebu6L3rjd7s3QxutUmftOXFz7E7vN5GIELW4+ALjjCm1jcFY1K3aSVIkcGxuPL8M9TfEP2a7Tl1/5j1kOSEotp058EjYqx+NILiXHOsryXT/DOyxnMeK2uP0tG0zLxehBl8/Vj6+4TI0Jpez7fK/A0WM3Nx+OKYgPiNv3MpTknNX53wftHzCrJ7eGoGF8+Ye3/vMD+GmDaOCkRxILCbpytLbNkARgSJbAvAfbXdH7aJdpEGN24UW/jQ/sJkn4FJIG7M7SWISEXSDpLlyOWrZ6gik3IYsL2IWP3V0ACQsBzj0VYCwxHvY+APBvqiD7oIXCJZ4epswPLyLHYkmFaAGbiEH7itFVHsqVsi2GpTYa7M= X-Forefront-Antispam-Report: CIP:195.60.68.100;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.axis.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: axis.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 22:30:43.5848 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 790741b5-b6ee-4d92-f26c-08dd584789ed X-MS-Exchange-CrossTenant-Id: 78703d3c-b907-432f-b066-88f7af9ca3af X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=78703d3c-b907-432f-b066-88f7af9ca3af;Ip=[195.60.68.100];Helo=[mail.axis.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000A7E3.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR02MB10318 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 28 Feb 2025 22:30:56 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/212081 This reverts the recent change of decodeurl() to use the URI class to parse the URL instead of doing it itself. While reusing code is generally a good idea, using urllib.parse.urlparse() (which the URI class does) to parse the regular expression "URLs" that are used in PREMIRRORS and MIRRORS does not work. A regular expression URL containing https?://... would be silently ignored, while a URL using a negative lookahead such as git://(?!internal\.git\.server).*/.* would result in a cryptic error: Exception: re.error: missing ), unterminated subpattern at position 0 The problem is that urllib.parse.urlparse() treats the ? as the start of URL parameters and thus stops parsing whatever part of the URL it was parsing. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/fetch2/__init__.py | 46 +++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py index 93fe012ec3..464b9ee4c3 100644 --- a/bitbake/lib/bb/fetch2/__init__.py +++ b/bitbake/lib/bb/fetch2/__init__.py @@ -353,9 +353,49 @@ def decodeurl(url): user, password, parameters). """ - uri = URI(url) - path = uri.path if uri.path else "/" - return uri.scheme, uri.hostport, path, uri.username, uri.password, uri.params + m = re.compile('(?P[^:]*)://((?P[^/;]+)@)?(?P[^;]+)(;(?P.*))?').match(url) + if not m: + raise MalformedUrl(url) + + type = m.group('type') + location = m.group('location') + if not location: + raise MalformedUrl(url) + user = m.group('user') + parm = m.group('parm') + + locidx = location.find('/') + if locidx != -1 and type.lower() != 'file': + host = location[:locidx] + path = location[locidx:] + elif type.lower() == 'file': + host = "" + path = location + if user: + path = user + '@' + path + user = "" + else: + host = location + path = "/" + if user: + m = re.compile('(?P[^:]+)(:?(?P.*))').match(user) + if m: + user = m.group('user') + pswd = m.group('pswd') + else: + user = '' + pswd = '' + + p = collections.OrderedDict() + if parm: + for s in parm.split(';'): + if s: + if not '=' in s: + raise MalformedUrl(url, "The URL: '%s' is invalid: parameter %s does not specify a value (missing '=')" % (url, s)) + s1, s2 = s.split('=', 1) + p[s1] = s2 + + return type, host, urllib.parse.unquote(path), user, pswd, p def encodeurl(decoded): """Encodes a URL from tokens (scheme, network location, path, From patchwork Fri Feb 28 22:30:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 58107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB9BCC282C6 for ; Fri, 28 Feb 2025 22:30:56 +0000 (UTC) Received: from EUR03-AM7-obe.outbound.protection.outlook.com (EUR03-AM7-obe.outbound.protection.outlook.com [40.107.105.68]) by mx.groups.io with SMTP id smtpd.web11.1399.1740781849425442209 for ; Fri, 28 Feb 2025 14:30:49 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@axis.com header.s=selector1 header.b=K7wXNdLx; spf=pass (domain: axis.com, ip: 40.107.105.68, mailfrom: peter.kjellerstedt@axis.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=weGUvPRkWX3eCGq+Pk/qiRaUGY5aKUmzt9NKSnn87nYub/+ETHWHMiF1dR9/tTT+olGXU6HBd0actOCtY2gjSkth22RA+nwocBoQI/UuXLWq5J0K6OHT8B6HB7nz4szj030XfiKg1nz3G5UI5n5slrMt4Bto+fvLS20n8c8re2NQrqd12U8V02afseC7dep6tvvjkSqNA4HtFuXDSQ0p3pyn1vBs3gGAzR4O4K0dPvDN2dJh0UtTNPKtXijVLNIuxZefExOtjoAEg2ApASPBE5aDJNCluon6Jc8rS/SbiJ9si31llT2vIaKXfD/VLpu7yK5ozTZqYIw3cIMg/GHGzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eFMAosinZVmTiaQjTOqo3pQ8qUAKnmlLYxjTdFm/nFA=; b=bq7wRjT+wSMzXxdtfNRe/6hfiKM11Oj0+rcTWnYNIGkTNd92xg/BOFQD2w2Mv33SFl7R6SACNvJiWei0qW7aHfZpnHUHeeJrICM8DiVT7R3jiECOd9omcTK7N8Pbf4OfBXrIm9KoisXfqlHa+MswkYJ7yYcP1rYmm5D0LaXgjbXaF9V00bHuMrwLg5ZNPkE/kBxUmBh6WXdHYTa/Gi61AU4RUl/QxAQsQauWSr2HDnuPjHBX/vjzMlKWRPIhLUN7gI7FERRVnZXIcWgpMgZj3VOfquKvEVWx5iJjzNXx7HLOlsx1JWfshFE5SWODB3PMeWOO3e4n1d6zg1kOz42ojQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 195.60.68.100) smtp.rcpttodomain=lists.openembedded.org smtp.mailfrom=axis.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=axis.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eFMAosinZVmTiaQjTOqo3pQ8qUAKnmlLYxjTdFm/nFA=; b=K7wXNdLxAToVKiqjlPQ7DEetJw6wdq+67OBvZLPcpYwrtelSjWKpu2U9tp+5dw12OvsiuCv3lccXbkCqU1tLR8XA4wXjGYlSVu9/Gui5ZMYFrlzm+IpLgsKJ36MMLgEAKapdpqQnix8qg5ZDoiaNoQGiDWR+0d0OKSu57bdKNSI= Received: from AS4P189CA0029.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5db::20) by PAWPR02MB9102.eurprd02.prod.outlook.com (2603:10a6:102:33a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 22:30:44 +0000 Received: from AM3PEPF00009B9C.eurprd04.prod.outlook.com (2603:10a6:20b:5db:cafe::b2) by AS4P189CA0029.outlook.office365.com (2603:10a6:20b:5db::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8489.22 via Frontend Transport; Fri, 28 Feb 2025 22:30:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 195.60.68.100) smtp.mailfrom=axis.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=axis.com; Received-SPF: Pass (protection.outlook.com: domain of axis.com designates 195.60.68.100 as permitted sender) receiver=protection.outlook.com; client-ip=195.60.68.100; helo=mail.axis.com; pr=C Received: from mail.axis.com (195.60.68.100) by AM3PEPF00009B9C.mail.protection.outlook.com (10.167.16.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8489.16 via Frontend Transport; Fri, 28 Feb 2025 22:30:43 +0000 Received: from se-mail01w.axis.com (10.20.40.7) by se-mail01w.axis.com (10.20.40.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.44; Fri, 28 Feb 2025 23:30:43 +0100 Received: from se-intmail01x.se.axis.com (10.4.0.28) by se-mail01w.axis.com (10.20.40.7) with Microsoft SMTP Server id 15.1.2507.44 via Frontend Transport; Fri, 28 Feb 2025 23:30:43 +0100 Received: from saur (saur.se.axis.com [10.92.3.10]) by se-intmail01x.se.axis.com (Postfix) with ESMTP id 423A9256 for ; Fri, 28 Feb 2025 23:30:43 +0100 (CET) Received: from saur.se.axis.com (localhost [127.0.0.1]) by saur (8.17.1/8.15.2) with ESMTPS id 51SMUhdf3902444 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Fri, 28 Feb 2025 23:30:43 +0100 Received: (from pkj@localhost) by saur.se.axis.com (8.17.1/8.17.1/Submit) id 51SMUhiU3902443 for openembedded-core@lists.openembedded.org; Fri, 28 Feb 2025 23:30:43 +0100 From: Peter Kjellerstedt To: Subject: [PATCH 2/2] tests/fetch: Verify regular expression "URLs" that contain a ? Date: Fri, 28 Feb 2025 23:30:39 +0100 Message-ID: <20250228223039.3902407-2-pkj@axis.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20250228223039.3902407-1-pkj@axis.com> References: <20250228223039.3902407-1-pkj@axis.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF00009B9C:EE_|PAWPR02MB9102:EE_ X-MS-Office365-Filtering-Correlation-Id: 0781ac77-9956-4184-2b40-08dd584789ed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700013|1800799024|13003099007; X-Microsoft-Antispam-Message-Info: mT25yr+lbzqtd38HUVP9FLyfWewZeTX69vqPOdaFpbsV9LM0RK00B5urP8gDrEvpHHjxLQcaY3aYMqXXqQB+tPNKOi+EEUq8TTkQpMcOveEBk15V39hFKrH4YwvOkD011/rlw6YtXoyC5Xy5x6q6dLcLVB9W0DrJblyNlzqbW/w5h0OSWDISbVQI3ID6dd2KXM7MmLfk1QGYgvW690/vZK0iktrOS3BPYaQ/KeFkLZgYXGeRKYYrDUNGUNTomPPLtAzoPlOTO0qL5mRtmz3P04yYkQWGLPFy8Lj7RxzlGUle6GnXhliNz0Lo4HOk5R6///QZ/lZfnHNT+XQ6L93rnq122o+Ft45RSpoyz0LLSU3ASunzSj3HmNH0zgzJqG4jlucdZsneHomFpQtRvb4bZ7NpUnPQDddBFKJNYYQDZyqWabsMe+Y1upFGw20A95hWKSwFEbM+A3xYqajDTI5W9ycy6SNeMXXV1lfwmlHk5vW+15q/K39CMx6hqcYLtGWFIeLOERmjvojDm53OZMvFqmC62hQmYwZBu9BWvxXCG1I0NmSB3SIkgjz6IysiIaIYobE6PMcEoF+eHsRDEvyu6lmrnHXX1COsiT7qDTNB8rdNv5T8bkw3L9DVPMWIGF3wiM9zWb9t1xT8jDVY6meum64jM2Y/GgQ+hHW+oq5cGOvPTxu3ptgthkK1mc43rE1V7pC60lj7EQ9/0h/ir0XZf8YBP2cJJvju13XgJcw1HkyElaeS1cWbc3QJlP7UQQzsZGTW3wF9oqwTxqjxwko5Fjua49mDUaCzbrxkSQ0vBvLGTtoRJK4EbKI3bYNItYCmJwF7BXm9BBhyVvk/TF9mHBAP6XQ5ThNWuJ8lD+1qyASV9a3dgpga+qgEOiSDiBhBGNz3iRljbOx+bgRaG2vuYL4RSwNKpY/3V2pXCPZD9eIRFNhhMRoTMKHRSysFANwWIN7V+3eoWwXS6Gwk3w8QAdMt6GxHFslxJfDnkgD98z9ORA7w1batk22kSpNQxfFc3ackRbJseNwEleBlr9vQw/LrnJBoqs5ZJshbQn72AVEU4dwgNmswo0/lxSO9QKxVBK/1QVdSlQ06jSJxoVaJDaj0E8iRhHvdWnKyKBbSrJkSWQ3WJ/JBrEzZVYeNnTnaEwmKOhOGEwUHjN1sHNSnBejiT/SKSJvmx9jb9pt3NX4mSpIlbIuVvl0doi4G4g/+NoF961WU7S87+jYR7XJYML+rYaQQzoP0tcA34dptaOU4ZLhLI7OTg13xeSN8UyzsX4NdzuTSbW5EKnjqLCoiYGiGQ/tdyyB/T1vs8IoBKao3u4qSsHkyD+eZwUyNguJxmxqbAqoZI9YAqXStmxysBwH5ot5S7Lq9Dv+xF1CRyclVlL5y8fftvr4qGiUhRZhCFWuHE8BhnS6fqDBAsVbp82kgFP4xs3NCYBInBDSNVNo= X-Forefront-Antispam-Report: CIP:195.60.68.100;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.axis.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: axis.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 22:30:43.5953 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0781ac77-9956-4184-2b40-08dd584789ed X-MS-Exchange-CrossTenant-Id: 78703d3c-b907-432f-b066-88f7af9ca3af X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=78703d3c-b907-432f-b066-88f7af9ca3af;Ip=[195.60.68.100];Helo=[mail.axis.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF00009B9C.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR02MB9102 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 28 Feb 2025 22:30:56 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/212080 A regular expression "URL" as used in PREMIRRORS and MIRRORS may contain a ? as part of the regular expression. Make sure this does not cause problems when parsed by the fetcher. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 48b9e4af16..be09a2964c 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -525,8 +525,7 @@ class MirrorUriTest(FetcherTest): mirrorvar = "http://.*/.* file:///somepath/downloads/ " \ "git://someserver.org/bitbake git://git.openembedded.org/bitbake " \ - "https://.*/.* file:///someotherpath/downloads/ " \ - "http://.*/.* file:///someotherpath/downloads/ " \ + "https?://.*/.* file:///someotherpath/downloads/ " \ "svn://svn.server1.com/ svn://svn.server2.com/" def test_urireplace(self): @@ -1415,12 +1414,17 @@ class URLHandle(unittest.TestCase): "https://somesite.com/somerepo.git;user=anyUser:idtoken=1234" : ('https', 'somesite.com', '/somerepo.git', '', '', {'user': 'anyUser:idtoken=1234'}), 'git://s.o-me_ONE:%s@git.openembedded.org/bitbake;branch=main;protocol=https' % password: ('git', 'git.openembedded.org', '/bitbake', 's.o-me_ONE', password, {'branch': 'main', 'protocol' : 'https'}), } - # we require a pathname to encodeurl but users can still pass such urls to - # decodeurl and we need to handle them + # We require a pathname to encodeurl but users can still pass URLs without + # a path to decodeurl and we need to handle them. decodeurl is also used to + # split the regular expression "URLs" that are used in PREMIRRORS and + # MIRRORS. decodedata = datatable.copy() decodedata.update({ "http://somesite.net;someparam=1": ('http', 'somesite.net', '/', '', '', {'someparam': '1'}), "npmsw://some.registry.url;package=@pkg;version=latest": ('npmsw', 'some.registry.url', '/', '', '', {'package': '@pkg', 'version': 'latest'}), + r"git://.*/.*": ("git", ".*", "/.*", "", "", {}), + r"https?://.*/.*": ("https?", ".*", "/.*", "", "", {}), + r"git://(?!internal\.git\.server).*/.*": ("git", r"(?!internal\.git\.server).*", "/.*", "", "", {}), }) def test_decodeurl(self):