From patchwork Tue May 27 09:59:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 63695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B8BDC54ED1 for ; Tue, 27 May 2025 09:59:18 +0000 (UTC) Received: from AS8PR03CU001.outbound.protection.outlook.com (AS8PR03CU001.outbound.protection.outlook.com [52.101.71.31]) by mx.groups.io with SMTP id smtpd.web11.48346.1748339954448454056 for ; Tue, 27 May 2025 02:59:14 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=XEQDwRe5; spf=pass (domain: siemens.com, ip: 52.101.71.31, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YNVN98joj80LCDj1f0xFdJpExwYKksZ5FNZ9XRbuvqyCRYvO/vfzXKIH8eb5yKylXzuHzm5vUIvpWUSRicuZVEMczY9j2kVCBHFI+gw0ZpZlN6D8Gi9oorypjfeMKAO5C7L00POUvbbRkAj3Bz+xRfve9R+vTwoRbDXtxxQLJbfE8By/8eMITVUllbB3zfYdkfCJDT4DjltyEqf0ESD0fuut9MNa+VLtNQBoLS6FUIsi8F3EbTwYY73SdMjaQLfzA1pvOWYqLzOAjLYAVdrYQh2kb0IGcsdZuVLf57XnaTciNHo/2mJWDevtTAtuTKT/gR0xRaqvNZrjUhq5pi5bMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MSp96LgtxxVk8ZjHjXWIFcqAEsZWhrsZ3Ozav0Nkdug=; b=VGAA68ltaekS/fxSj0jXq4CYL0ZZ73iqQyN37L72S5pMo3SE3rWKxImDQmLX+KKkNZ37Bq5g8egFsHZANcGmYg4DPhiuSEpGvsQhucNq8iExce5StA9qWNxenbPgma0po8V5XK4ycLBkwVafSxRbs57tQdjKB1SSBXK7LaHN459v4zIap3dVseTXHYE7OM6n7vA83exjNtasPHivOR6+n0VKmHcBZLiHyabdgqT7yqhqylfh1UTiu8wNQRevkYYC3RHXD48nmsd9WJumGCoqK7DQYKlod6ZoA0nTy9ZKNDOdlLSG7ly+Gs2F7zgJQswOgH+k0ZopKH1rhqYYhm3k/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MSp96LgtxxVk8ZjHjXWIFcqAEsZWhrsZ3Ozav0Nkdug=; b=XEQDwRe55hdQFQoa0Sq0TCszWxivkTiUqq2hYHExi+fR1fIpppT2Kzq0EPNRa2MepbHAt1ACif+nC44Vk0tWeiRtKUtix01gFszmunGvmB7nTK7Hk3YQyLaj7yKmeSBVa/oFulv1taTDmcy3T+j4JDfnIhMVXpfLpD6x0LeBtMWxTObjKmpNyBjJH6zBvpRjsSgWEH3USluHMaecPoebmkHWzh0z6q1y8cerVhXuoh6rD/xeETrikVeDUrAUqYXpjHOGaXVhEpj3G/9SuMZ3/EPnPjO6GfrrgyUM82VvauA9piuNzI6fqAUpKo7HH4Jo+cgSHrfkxENcaCdChegtqQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by AS4PR10MB5442.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:4bd::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.27; Tue, 27 May 2025 09:59:10 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%4]) with mapi id 15.20.8769.022; Tue, 27 May 2025 09:59:10 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: docs@lists.yoctoproject.org, stefan-koch@siemens.com, richard.purdie@linuxfoundation.org, liezhi.yang@windriver.com, niqingliang2003@gmail.com, alex.kanavin@gmail.com, quentin.schulz@cherry.de, jan.kiszka@siemens.com Subject: [PATCH v3 2/3] fetch2: Ensure a valid symlink in `PREMIRRORS` case when using shallow cloning Date: Tue, 27 May 2025 11:59:02 +0200 Message-ID: <20250527095903.2300556-2-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250527095903.2300556-1-stefan-koch@siemens.com> References: <20250527095903.2300556-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR4P281CA0412.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:d0::20) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|AS4PR10MB5442:EE_ X-MS-Office365-Filtering-Correlation-Id: 279a2508-5b0d-433f-eccb-08dd9d052086 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: /XHXBwWPfVBZKIvM3KJA6XozlHf9+aQPEJMooK5zfnhk3yO6l9p6r+YCHZzxLB/PnmfH5U9Rll1rIM4oOegZi9OvlwytzwoGz8MbpKEibTyDaxrx4PQoqs8dAvdc/1t1GnvOzUQjwJPm7nqgNn69slVCY+srvQ6lSYuHhdDucDj2LjpPIXQWzxAL1fg6YD+OPnZvlo5eQRQhBXcY2t6y3fX9jDVAsGnvwf/jtLXVwnPMVr7tiHejr30eqq+NAqj12m9MLknZpobUfGvW4kmF605QAH0RXGH86i/+OXXy67KoHfxSRcqfXQ3vuC5GIu3xXXPJwUy5X8kLtKxkcjBruFJhDxzn+mZpunctc8RPYhpAdDOkIdoGj9jJ7j41HmQwnhn5ZjOTxFhZ6qwCwg3M3JaewRhkfRhjvpJDWKauOevlHlX3bPcC3tfeHJKbvMzPN8CaAPYITbMaOQ4s71Dj1z9ppdNir9pRT9XqoCiVP1aSx3iAnB/bP//SbqBMxzHVyJg/PlDnZQDrl6E9dYgq5y3wCxqv/2PAMgry17gZOEFXtexPVs4O5D8vUkINxnD9k8dvhd0JdiJMu6SFC8AWfB5wcYgFNPXqjxzl/LwlE6t+A3X1wj+w9PNWhh5WedJWBgLUJ1++MOT5wnshqdJbRd6OQYVwptwpE+eFPi2l/oR/+kofuusOQ+O5IWvtBMOBpvsldDboGsephBltWpOnA7sEqrcqNKCK9p01maGhDKMM3h9yZog19X1Vfn0OXX250KeAh2YfyxHE/AsLxeJKSB5hOQXbnQQTyhvb9pa9hzf08TY6azoOjAq+HsVz7H+b5EdK5mXjQKk6W0YttJUxdsX+mlobRNJHkk0BkOFLXwjs1YVK/sZH10/RessWALkxbUZkrxZkRi496Z8bNsWWZ1gJU01ZrazL0HHPzGaSut9jwNOx841FFCjojzxu7HxkT7BCo5YaJcrZltUge7psfSyKQ/qlGENc/ZMHPO5XBXR87F7Goq3Ie4Ln+Mc91kINjx/Q28gtOrVqx9X5b41XSqwLEQm6S/cj/MNMXyv4qRUb1mUdbLY+E1XM7Sws/H9tgjETBdY10hMiYKh9F5YuTi5LeWEhFip++4mhhn3xQhFFnkscBsKyh/0kuSFMFPWJgAd+Z1UeudSl02MVFyBT9geDrHCNLWzWnVIoeGMOiZwS8sXRWVqWaC/AITmk78rOOqOEchBinLoq+z6gtq5AkvcLfY39SqzYEJzSutg/6geqUpJMrJZwg/IMPKoZR7wphl7HUqO6r07M63/HGxFuj1RYg1eu6Ht6RwFKWZDIbCfC10cozEX2WM/G6priuqEwTMkvgnjQ2EXG7AxJ8ougy0C7hD0ufGCr6LtjNke/ERuTLtdHu46ogHe0EZcnhGc7cJ8L851P7nxRcara6h3TRUi+IrqwwKZnN6oK6EBEkaY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: N/6vGQbXtmS1duPAlEVMdN+DEhm2oIdTDmm97diFpTVIZGtsBuSI2KKaBZ4FNMD0BmEY8VjDAdt9PgRdWMqC1VkCuwEB2xbn8KB5gczlZTobYg39Xo9xuHikvcU+VPpxvDZfp2Hx6RYFHM78EFvOfeyIH54rg1Xun+op4OyaahPUH3Se3fsodjLvtPLZI3iT0gKM6XlITOkg13rFtg/Sb8tAMZiLEOMHJMNRnCfJA1RCTGXp3wtGbg+E9yPpTLrXGvymCBtkL8p/PIjKiKs/8SB/9tsxR+rY/dVhQJ11uknFETikBeta03QcwaMIfs0n8voXFDkKcWOscMP4kezNE/S5Fv8iV9YJ9nGcS2QSxTX4Lgsaxq+YzCTQu56t/6jyDX3UVjKiC+UFt7HdmhObRzRHT634o65UjO4QQq/SHbc34Imz1ukk8Mdb6OoaIq6HJ/0aPo0m1HcEOR28NkThN7SwuY4en3jPPhCOt+b+mn5i5kjDxJ1SS2je9SRPx+VIyA3WKrCVl6u2NQp6M4IZyXYk/vJpPamtL+tIYAeoRMj9R9VA3WWon8WQA0SCdPmYolVtmPbsZ36GKwU5xxMeOc+hFAWiqEOGibnAbCLBet9/Ln9boYi+cECcLw1Fu2eMBznuXOsHZZzECZkdXVNedKpKncWevd8ujL8nmbOGP77uvoW50r6LGWZqARfDBjfJNfTLQy/zrGLBFSKDOmtdt6YUvdNz4ZLnD7VV3lVxPfE28eq7OKA0S/PIz8O6gTp4qmi8c7ExNH/cf5xG9VEflSgONERwy//Mc3NK2oAy3U9iDA/SzZyiJ0Ig+0FGI6jiUSlOJ9ZJ6QJzCp3RmUoBNY6d1IbHdPdbNwZS16y6bPR7sAPIHZqxmFKG6timQSea/LTg3MGjMwngwzHG95bF2o/qzsnzUD64PTh1OToAQlOsDdsk39Q1lat966DoKLSONlsd1M+B/Kxhb9hKHXrXJAw8l5BUzHeenSzvZmpmly49sGvDktHZX4d7tAVKqyNUydF+DeV52oLRKI++lRSJGeAGVO2jfdbh4QTljLt2cPBFLSMW0rBdH2EPCIYPJq7DWhXP1+SeamEQB5x/M0unIG6Q766crDnNRLqtYffS6wAGiIEzIdRQJzFHhc4MUNCShMiTYGnMycqy3cATfkmgGjhNaE5sQZjyaeNVpbiO7KWwtyUzyAC+LY4DUz9R3dSOoR2DUx2snwq3sODNEiLQ2xA7ldVRA0nJ2zrXvg/wGrkFFangQqXpQUXf/mu78bzkaO+KrWlO/dWb71hnkrvzbo801GEC6wQMbRLtzZbVwwHyUx++phpHCE/o0zjCL9hc6JdfrhTczpGjvcNRnf5dO/mw8nBwxKgVsYtAipfbMZAL1Tf3zV9yt/yGsZqy78w7pZeNrZcMM8lqzt7ox0r8bYtMhI873jMHdkaa2MZ7Gjt26XBtB+rxORBSurTWwB94YkdIIKV2zE3GqT3zwBAM7vrxiyXqlmiGnzeU8nOr0tmMUX/CifwBPuv/6DxjKoxHDk32mJGj8qZfVYmRkMNOYB5b5R4JUUnuD1tRVpGRxF2j/NbEK6wZwf2vxveFpR54mCPuZYo2TAX/+KhDBMg3Sg== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 279a2508-5b0d-433f-eccb-08dd9d052086 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2025 09:59:10.4250 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aBHJXZxzIyg+sDz6rmqXtbBkMNFRZDxiREB4JhQ909dYIBIrOEXsSykmIiLqEOEu5W0+/DeX1dV+zqcwe9fk6w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR10MB5442 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 27 May 2025 09:59:18 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/6939 - Since `ud.path` contains in that case the `PREMIRRORS` prefix path, this change ensures that a correct symlink is set up. Signed-off-by: Stefan Koch --- lib/bb/fetch2/__init__.py | 10 +++++++++- lib/bb/fetch2/git.py | 7 +++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index fe6c1cee2..dd2dbaabd 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1117,7 +1117,10 @@ def try_mirror_url(fetch, origud, ud, ld, check = False): origud.method.build_mirror_data(origud, ld) return origud.localpath # Otherwise the result is a local file:// and we symlink to it - ensure_symlink(ud.localpath, origud.localpath) + # This may also be a link to a shallow archive + # When using shallow mode, add a symlink to the original fullshallow + # path to ensure a valid symlink even in the `PREMIRRORS` case + origud.method.update_mirror_links(ud, origud) update_stamp(origud, ld) return ud.localpath @@ -1653,6 +1656,11 @@ class FetchMethod(object): except FileExistsError: pass + def update_mirror_links(self, ud, origud): + # For local file:// results, create a symlink to them + # This may also be a link to a shallow archive + self.ensure_symlink(ud.localpath, origud.localpath) + def try_premirror(self, urldata, d): """ Should premirrors be used? diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 784a45bda..55dd084ab 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -348,6 +348,13 @@ class Git(FetchMethod): def tarball_need_update(self, ud): return ud.write_tarballs and not os.path.exists(ud.fullmirror) + def update_mirror_links(self, ud, origud): + super().update_mirror_links(ud, origud) + # When using shallow mode, add a symlink to the original fullshallow + # path to ensure a valid symlink even in the `PREMIRRORS` case + if ud.shallow and not os.path.exists(origud.fullshallow): + self.ensure_symlink(ud.localpath, origud.fullshallow) + def try_premirror(self, ud, d): # If we don't do this, updating an existing checkout with only premirrors # is not possible