From patchwork Thu Feb 20 17:27:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A80B0C3DA4A for ; Thu, 20 Feb 2025 17:27:27 +0000 (UTC) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (EUR03-DBA-obe.outbound.protection.outlook.com [40.107.104.56]) by mx.groups.io with SMTP id smtpd.web11.2757.1740072443352419987 for ; Thu, 20 Feb 2025 09:27:23 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=Dn/Yyqe6; spf=pass (domain: siemens.com, ip: 40.107.104.56, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dvhgGPWNaCsQ/Bp7JKcFe/rtNyK8xUoQ5UblUEffAMv+Gu+XHFgngXMW4+BRsqcknS9XmoJPQLHsUfHerDBk2PbT4YwKfN/LGideHWwmEclQ2UQR1/A4EilrBYJCy+g2V8tW702wRh8qw6XOJvBaAQqIWt7A56bl48GVKDFw+tyVUrnEhkYKof0MxvgABd6pvULIEwMH/oR+n7aRiutB4VBbu1JJh2OewKvCxnmiFZqE5QU9Zz1UgrTwkUU2Yv4meW6Dh/oHNt3HwoTETbO1I3WFgnKOWY8BA1/B7uc4OtQzCUIteB0M++dXyDkovl2jQGLeBSRatndndGsswDgY6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fm0ObBeGDZ85BodGI91rkrc31PpouF69sKNu8AHmCrw=; b=jlL/Mpv14ZB/dR9VO8g6XShiXNTdeC1WgkTxRvHfEuAvNmox0wJ8Pa2xfr0xMpAT2w0E6oujdZgJG6OKbOCGztSZeRsUtrWqtyASpNTpQn71xOoXm5v3KnuHYdCco9pfsSn5dRcgWc+ScIO9Z5FLZ8GCohV2aEMfsZTetmK0ahSoLhRSIIA16v+OCW8Q1lQVLes+oYs2XtlrUNIC1EJWpomimPmaIVERDF6jYPbT/P1T+4+6QQSCyC9gzEJKcPHtvicnVgoU7Fs3CYMNob7mC9x9BDwS/ZgdO/GLZMecYFKzOvyq0P7yR4lWou2OYZ9WniZhGuzU8WPWqYIWSS8Jew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fm0ObBeGDZ85BodGI91rkrc31PpouF69sKNu8AHmCrw=; b=Dn/Yyqe6761uTu0noqMQw+UEb/x1YAtPclLOwM8osCcp58eFvrAdtEAkL8TxIjG8F+PncCmCYKsMmd5ygJQ3xJgiggVCH6Q7sf+ZRj4QriQ9rbuJtMCCx6yqWumy7p3KRfVBW+Gn8pHXjyOc464i5gMyD00ylaGbqfFCPUDV+M+r0qTc7J7VO0QuLHMZsElaSRkTmHbzFZCW665vNmpLuHZq7Cehp2M6Gzz+H0zrzi8mHkehl56OAE6dvwus4lVoFlEe8t+fA8f8+sRtEyNX1KFpA5u77J8eLo/aw7vUZAPG2IqgA8kYJOEySqQM5ZHHJg4GPZvRT8BNpUOf2Za+hg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DB9PR10MB5551.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:30a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.16; Thu, 20 Feb 2025 17:27:19 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Thu, 20 Feb 2025 17:27:17 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org Subject: [PATCH v3 1/4] fetch2/git: Add support for fast initial shallow fetch Date: Thu, 20 Feb 2025 18:27:03 +0100 Message-ID: <20250220172706.3850722-1-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 X-ClientProxiedBy: FR3P281CA0196.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a5::14) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DB9PR10MB5551:EE_ X-MS-Office365-Filtering-Correlation-Id: 7f9f878d-f46a-4770-0872-08dd51d3d281 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: R+R0OE1fUMtoulDJYqScvK+a4yp5KqU+nwDyvm3emGF651jvyh/fRoy+VYWrkbsY5+qPq5uK9VqW7NdErzHaV00C4iqnbMlBrtPEztcYCDsHyOnO3vGiTZSEyQkbnmkv0KlozmfiqYrKbjExduuD+x1hkPCBOdwqTiCvs92U6R6fxpYnwKwzPrkY2XJxJ5sV6YAIY5/B7i8iN4vb6/Erfj8wxIkuN8JqzhV3elU1+P6XRd/4RPFG4yaeFuibRyEUaHRgnjRafGw2dQwR6Q/CMt+cZ3WmoJqCL47t6xlqnV1J8c6qhDZvwxPixY9tWjTg5NkycG3snTThJCAQbL5nebMQZL6NlELG1YSs1qPyEUfC6glUvC8EnfJNAyD8nS/3pCm7+skl+W3BvcR89t9+GLRt/u/64jb3dAYDp4xQx0wWveZVUg2PRuvQTGs6ThkBOiChOV/y/Oj24L2WFx+PQ8dU2IOW9qDxzE5g5qIYYzCzSFZIhMnw/HTs4+XMY6SKMoZuZo1o78IcNAv6N88Ft9hfvMHCho/7FoxdWbAMqxD3pB82yzIuk0bV3xqE0jsqCj7bqmHMn+uaffmwHH/PN0BUSK/SXfQfI20RWY5ByBSAdhQsK4+4tqTYQMYwCIxsEFWcO0asQGFDevz82pAc3JNk8wH1vRV7av6JHNmPwXZvmN9iJY4faoNaceaRz3NxtxDb5IP31tcV3zOkB/YvInwqHp5c473Wbg+qAQncoO56tVPh1nyKZ6VetPd3W6gcx4EEq014PKzuhAJM6EVBYrhjiyP3Dpy3NdudJ43P73tT5Sg1wow0QLbRMrZxVj01LXHOAVIj3rHULt6jWAvzaakQbkNoAtaMXUUCPDHCXBTGrXvhRLbZ0iibQMF0f9F5z1LYkPLIj55A7P9e30ri5FvFO+GjzZpdnYfYcFe1HZBhJN/WiGg2cJR7qTE+Fp6tihAzQ/OSif66etF7zcPZT/cPHeZPsr0m2Yn+/QbrSAcup7jdGBxhjTQcsnKYhaJyxe2zlkmsUjz3XRMppKdXP7bJgjOks797Pn1Ca4YoWdWgeEE4eZPzTo9M/BalqXGUc2JCIhBdYF4zwMbrNFXXLZRVY3zu12ARCvU10wR2FW8DxnxtTgwT7/yGgrSDVHNbzJMswHoKWBtjOYLOWGHPcQb2RyIeDjxESUto2dxFKLQpWmGmXbBx1Jn+ZBkRnF7FUUk7RJqX4cMYv813punHK+9xAdbKHik4wJKMCYMW93ZP/T+cnYNIxBERhlPR+hxOQEOKhWuRs0xXnK/+mwq6rzTeDbSZxIBmnokGqMmn9jhmF2BzfqlEoymrn1McIPzZ0u6ruY2m+nOSWHe9gU7V/WtQPQJTN3XozTLbgjfqGV3I/rVMV4hLac7E4aMurUC/ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qGazxaACjwww1KhJsid4LQyyB+L+/koKnvxj32H7QCZ4Cdk0c0DFQ3H6Sv4Ti6/Br7wvci+wSJxtTfCE9GfW6fTlOmDVf83P/KY0vuCpECrBSTNVc3pIROWiZ0ZG+F/FsAvU5fkDRTkg0S68HxuxZOpXkFeQvBHRzZgKASM6AsZeSbNFcPFsOhY3VKsc9Nuf8v1ouWYDJRrLfAWLGO3wofbvotUAj0VNUlckUxpKABfN/U+eRgIsNNA4Xf8Ijs5IfTVJWzomU5NyrQuxn8u6ZB5aUXkFc/TV1RC3zvPp2sY65Ud9+WZUfxQNONGSGEpqUPhz02NV8q3EIyGbrYbHMviHXCgAIl0lC1catkpOs4yT2X7QbSHZAR83vN5va41g1eLCcoJOqN5i+eGRVEo8GvPPjtwzjTeHzBS3j/+3xEy0dEjQnjm6w3/UV7C6KUGeJJfBz2iu1pB9bXqb0wGAw9mV1gvissHoss/OvFIhvQS+EXlCn5B0+HmpuWx6TXgMWI1Zk4n93zEvGssZD5khKByH9GHWjngLePNffyt9Z/rnJoXo3zl+XkwIlCgJnFG1FqTDskVP4Qfxg0NncQRgP/anyMiYoeuuv+sOMPXyKznp6Bv13MXSq0K7pCVB7oCa00WvFm77PwFRnA0zPYtRfaa0SQ9AgVW7iN/WeE/wuuTBrv2bL9j2GK7cfA15RymmNDoTlSiS1wfriiPj32CPC3J72u1xCQQAFxKxtlIecRbVqfqM4NBY5sdLmO6wEcJHrDpWaSlfb03pWgedoFsgqlbmArzM0CsxcSLZH0Llu3KpGcPDfjt/GjhAMCeSBAI1ak1FAvxdVoaJlLOM640DSQgce1XxSyfBoLXpKjPsUgiL1goMni+AANh4PykjNu/eLYDngbvivkCYdFDlpP8C2+V95ebILYSMWJ7SH2/KCPKsIm65dBIWfNohTEOI3TQM7ioNCyi0H8SZbBsV2OBFKkhvmJDLMmfBKFLjku9s/Yx01nuUJjHMADBgLSkuYTM1KdtP2hnHF2uMh7a9zLiQZfISaiDr/ECMpEAM9E+wx44FruAONN16y84l5fCKZXDubdzlrXNtmxYtm3eDXQodWZVjfMXuu6Bc2jWZXspijt8oCxcOEivz5+3Rkddn1UHa851iZ+dO0ClEddFgVoO5NDlC+6H0s34Fth6m0Sfr1mxhTB3/2EIdwJFdtJjsvUJMQiX/YI/LDmFxlruWJQrlqbeb2qA96B9332NTKPOFSMeyZSXpcQbpWVeT/GCW2xZqG/ySMoTl1EaRGbtzfCWImPocQP8qL73CSEL2XY3+SznLMAG6fylAHIP4HEvhi1+IfR737i0uNeT/RHtF6+wBF4214eVT7Ke86zTdgLG7HVcQ9lavY/jSniWyiPtSDflvnDGbg6zPe7PjB9VW4ul1RqxBF3HflFXkBFBbbyIt3hZWeiN6ZdxQV5tAFCCQN6m2Wm46vH0ZZFbFnWUe8RDGlOI8XZN+aPbe9FJf835awEmfJg0XO4l3ePuGIFwsat13CXCJRAIGkTfrt25DzZ7fQaLG+ABC9pmpEO5tJRo+6/H29q9r6bsEvyi/wjVJ5nOvhl5RR42Nx9To1zsapXD0ng== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7f9f878d-f46a-4770-0872-08dd51d3d281 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2025 17:27:17.0916 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 59Ke4WBSiZVF6QPa79eW6xfYN7XM5H94WwIAemySMqSIYCkPFGZ+fZc2iUZ/YPDggfjASBsEWKhuZ/S6NvwtTQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB5551 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 20 Feb 2025 17:27:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17262 When `ud.shallow == 1`: - Prefer an initial shallow clone over an initial full bare clone, while still utilizing any already existing full bare clones. This improves: - Resolve timeout issues during initial clones on slow internet connections by reducing the amount of data transferred. - Eliminate the need to use an HTTPS tarball `SRC_URI` to reduce data transfer. - Allow SSH-based authentication (e.g. cert and agent-based) when using non-public repos, so additional HTTPS tokens may not be required. Signed-off-by: Stefan Koch --- lib/bb/fetch2/git.py | 100 ++++++++++++++++++++++++++++++++----------- 1 file changed, 74 insertions(+), 26 deletions(-) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 6badda597..d5c7f4332 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -366,6 +366,33 @@ class Git(FetchMethod): def tarball_need_update(self, ud): return ud.write_tarballs and not os.path.exists(ud.fullmirror) + def lfs_fetch(self, ud, d, clonedir, revision, fetchall=False, progresshandler=None): + """Helper method for fetching Git LFS data""" + try: + if self._need_lfs(ud) and self._contains_lfs(ud, d, clonedir) and self._find_git_lfs(d) and len(revision): + # Using worktree with the revision because .lfsconfig may exists + worktree_add_cmd = "%s worktree add wt %s" % (ud.basecmd, revision) + runfetchcmd(worktree_add_cmd, d, log=progresshandler, workdir=clonedir) + lfs_fetch_cmd = "%s lfs fetch %s" % (ud.basecmd, "--all" if fetchall else "") + runfetchcmd(lfs_fetch_cmd, d, log=progresshandler, workdir=(clonedir + "/wt")) + worktree_rem_cmd = "%s worktree remove -f wt" % ud.basecmd + runfetchcmd(worktree_rem_cmd, d, log=progresshandler, workdir=clonedir) + except: + logger.warning("Fetching LFS did not succeed.") + + @contextmanager + def create_atomic(self, filename): + """Create as a temp file and move atomically into position to avoid races""" + fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) + try: + yield tfile + umask = os.umask(0o666) + os.umask(umask) + os.chmod(tfile, (0o666 & ~umask)) + os.rename(tfile, filename) + finally: + os.close(fd) + def try_premirror(self, ud, d): # If we don't do this, updating an existing checkout with only premirrors # is not possible @@ -446,7 +473,40 @@ class Git(FetchMethod): if ud.proto.lower() != 'file': bb.fetch2.check_network_access(d, clone_cmd, ud.url) progresshandler = GitProgressHandler(d) - runfetchcmd(clone_cmd, d, log=progresshandler) + + # When ud.shallow is enabled: + # Try creating an initial shallow clone + shallow_fast_state = False + if ud.shallow: + tempdir = tempfile.mkdtemp(dir=d.getVar('DL_DIR')) + shallowclone = os.path.join(tempdir, 'git') + try: + self.clone_shallow_local(ud, shallowclone, d) + shallow_fast_state = True + except: + logger.warning("Creating initial shallow clone failed, try regular clone now.") + + # When the shallow clone has succeeded: + # Create shallow tarball + if shallow_fast_state: + logger.info("Creating tarball of git repository") + with self.create_atomic(ud.fullshallow) as tfile: + runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) + runfetchcmd("touch %s.done" % ud.fullshallow, d) + + # Always cleanup tempdir + bb.utils.remove(tempdir, recurse=True) + + # When the shallow clone has succeeded: + # Use shallow tarball + if shallow_fast_state: + ud.localpath = ud.fullshallow + return + + # When ud.shallow is disabled or the shallow clone failed: + # Create an initial regular clone + if not shallow_fast_state: + runfetchcmd(clone_cmd, d, log=progresshandler) # Update the checkout if needed if self.clonedir_need_update(ud, d): @@ -509,20 +569,6 @@ class Git(FetchMethod): runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.clonedir, d, workdir="%s/.git" % ud.destdir) def build_mirror_data(self, ud, d): - - # Create as a temp file and move atomically into position to avoid races - @contextmanager - def create_atomic(filename): - fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) - try: - yield tfile - umask = os.umask(0o666) - os.umask(umask) - os.chmod(tfile, (0o666 & ~umask)) - os.rename(tfile, filename) - finally: - os.close(fd) - if ud.shallow and ud.write_shallow_tarballs: if not os.path.exists(ud.fullshallow): if os.path.islink(ud.fullshallow): @@ -533,7 +579,7 @@ class Git(FetchMethod): self.clone_shallow_local(ud, shallowclone, d) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullshallow) as tfile: + with self.create_atomic(ud.fullshallow) as tfile: runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) runfetchcmd("touch %s.done" % ud.fullshallow, d) finally: @@ -543,7 +589,7 @@ class Git(FetchMethod): os.unlink(ud.fullmirror) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullmirror) as tfile: + with self.create_atomic(ud.fullmirror) as tfile: mtime = runfetchcmd("{} log --all -1 --format=%cD".format(ud.basecmd), d, quiet=True, workdir=ud.clonedir) runfetchcmd("tar -czf %s --owner oe:0 --group oe:0 --mtime \"%s\" ." @@ -557,12 +603,20 @@ class Git(FetchMethod): - For BB_GIT_SHALLOW_REVS: git fetch --shallow-exclude= rev """ + progresshandler = GitProgressHandler(d) + repourl = self._get_repo_url(ud) bb.utils.mkdirhier(dest) init_cmd = "%s init -q" % ud.basecmd if ud.bareclone: init_cmd += " --bare" runfetchcmd(init_cmd, d, workdir=dest) - runfetchcmd("%s remote add origin %s" % (ud.basecmd, ud.clonedir), d, workdir=dest) + # Use repourl when creating a fast initial shallow clone + # Prefer already existing full bare clones if available + if ud.shallow and not os.path.exists(ud.clonedir): + remote = shlex.quote(repourl) + else: + remote = ud.clonedir + runfetchcmd("%s remote add origin %s" % (ud.basecmd, remote), d, workdir=dest) # Check the histories which should be excluded shallow_exclude = '' @@ -595,15 +649,10 @@ class Git(FetchMethod): if shallow_exclude: fetch_cmd += shallow_exclude - # Advertise the revision for lower version git such as 2.25.1: - # error: Server does not allow request for unadvertised object. - # The ud.clonedir is a local temporary dir, will be removed when - # fetch is done, so we can do anything on it. - adv_cmd = 'git branch -f advertise-%s %s' % (revision, revision) - runfetchcmd(adv_cmd, d, workdir=ud.clonedir) - runfetchcmd(fetch_cmd, d, workdir=dest) runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) + # Fetch Git LFS data + self.lfs_fetch(ud, d, dest, ud.revisions[ud.names[0]]) # Apply extra ref wildcards all_refs_remote = runfetchcmd("%s ls-remote origin 'refs/*'" % ud.basecmd, \ @@ -629,7 +678,6 @@ class Git(FetchMethod): runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) # The url is local ud.clonedir, set it to upstream one - repourl = self._get_repo_url(ud) runfetchcmd("%s remote set-url origin %s" % (ud.basecmd, shlex.quote(repourl)), d, workdir=dest) def unpack(self, ud, destdir, d): From patchwork Thu Feb 20 17:27:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6B1BC021B2 for ; Thu, 20 Feb 2025 17:27:27 +0000 (UTC) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (EUR03-DBA-obe.outbound.protection.outlook.com [40.107.104.56]) by mx.groups.io with SMTP id smtpd.web11.2757.1740072443352419987 for ; Thu, 20 Feb 2025 09:27:24 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=XA6w0pVI; spf=pass (domain: siemens.com, ip: 40.107.104.56, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=m4V1w7nXZn3ivrktoowhwis+Hhpg2E0DHa7xB3krfzVj2Ey1WPxDjQlMJsAWx1y0o/+9awwXwfHATMMbposZQ8rSkA5aJMQB5wg+oe0OWGeQtlDeFwwdI9iuQNgzVDoKGwNC7LCtRQhVUdHUqPfdkBoDg5/Gyik9diSHzoS9S1FEYlUaWFWrfZQ+ZO0YD3xq1+ZEVVD+Mp04zkkn9eSUf21nVVwP7M0KUSFtYl0nQ4i3Se1WQDowroOf/6YuIwNeaRP3qDH1kDIRNbU12Qos8ltTWszmEcU/yUwhlJTbBfdNq49RKTKiUSMJLmlLSCz4PxL2VYvLwZfEqVN0D5yw4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DHmX0A1A2JmdJp+WPKA5qQK1V39QZvrldLsfRc7xdBM=; b=A+EE2VN6pICeW9Semk79RW4perhgCukVLBSeIynrsp6TxAP0zm2YnvyAXW2JI3Pu+5/UgAJTJWx1HLFytF02ixhCd0MAMYu4I5fiGD6TtDMfdY9pQPaXRdklCE5GbBt0Pfki7bvTHkD5ZLtRhphVVygmnklr7KPtdWHtU8E42eGeWoa+92Os+T2XDimNWQZn1aSxajw4fl+zYfC56mfqS6ZBbXmvERqSJDGCfmSb2FXcYQu5p6agXZnHFU7iVFuH2Bws6c4IQUT2H+r9qfJ2XBD+ZD/2rl8Obc0c4KBFcWGdhMmUrP0Dl+HvaP4bU344/fp2IguvubrItDrPw6d42w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DHmX0A1A2JmdJp+WPKA5qQK1V39QZvrldLsfRc7xdBM=; b=XA6w0pVI+56dBgcOpaFemXrXs4PpF7xZBmKmZ2ippXdv3/OqFzk+xd8zCU03KRH/F4HLBIUzIfAL32azrv0stlHgr8JIxRNjBW8aEZqfGbSHApsKG6052ln2Y8lrNhhWgsvtxn6u7qoihHxxzZNrQ2sIUSM+nP8OOivWo34xvz7b2+pYWNFqXeiF7CvjUIgai2Q9q/AxWojEWx0LBBX5dgG76ACmpg0NzsLMSV53DROwd4XjBpDCLO1dHrnzUeeKjvDsmkP1ftPDzxqFlVvidVyiyzy4ZJwHjHgnpaBSmt5ZYNS3MLaR8hECevskBRFNs6F1tUoWfbgvoVm/Gw7H4w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DB9PR10MB5551.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:30a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.16; Thu, 20 Feb 2025 17:27:20 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Thu, 20 Feb 2025 17:27:19 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org Subject: [PATCH v3 2/4] bitbake-user-manual: Update documentation for fast `BB_GIT_SHALLOW` Date: Thu, 20 Feb 2025 18:27:04 +0100 Message-ID: <20250220172706.3850722-2-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250220172706.3850722-1-stefan-koch@siemens.com> References: <20250220172706.3850722-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR3P281CA0117.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a3::8) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DB9PR10MB5551:EE_ X-MS-Office365-Filtering-Correlation-Id: b2eea9a2-299e-4f0c-8463-08dd51d3d32f X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: Sb2etOhjOPvoZJWIIiHf13sOcP9N8riAxO6orAs6JtJso5qXAXET5avCynHVVOzzRBJKZGghZNK7gNqT1RsMg0cxjYMpFiPbhYb3ywbuOPFSBZFHUSH3uR+Sc+tqTrYU1WJpwnymxDINpc3GCBYo6XLq4C7sfKuvu6h4RRcEAb7ikSENWKy0FPnygPUE+9tIabDjFlYqgCi4BuhiDFm3dsaEuyzLZZlHQ6Qf3WQXfBH85drlr9uv83xK3Ju/Hv493KmpbkX9/Kx/+Uil9rjEXcjsTZb9zLR7qOuH7KUM4KMcAujYy159YI6oPsepv+zuBhBLiO5w7kIV7CuEDm2lBLup6nHlvHVIfI9hW/XZw5BnOFkQ1D3u+S77pMZlT/ItU3uJFgo4CmflAwFaGzqEiD0dqr8o6M+esGPaIjbxSL6nYxcYT4dAf2Li6x/55+U4EWMAir/dalqKMf+q0XHodhY+57qvMM0TiG4TF2GE1JNDkcQPojrhI4GSKOUzYXw9mktjRz690qtqS7Ffo/Yd3+MTJrzFPYXodH7ltDV3feFaktGmBBnb3+wsLl+FDEW+0Dk1EGWlsvs7vBwBi+xvQzibu0m5C0tCMHx6QgCBvkBkeirmb6L9dF4hKhShfpSWZQuLuB2Aeh3WMUOlFvBWHXoBsyb3eBXayavmavL3i7bn7c9p/P263vAMaFowR9N2hG/NopQANqn30N5ZS1gj27YbovEhCCdEH11ARVqQqyiR95St6MFDdB3vssk4bTbXBihrxQEKdRhAMfSzFeTS+xhC2OHzB3KsSNrMyspTxuE9HyNkH2s1f1RrSPrzUQ44n4VmOQQMoVzogT3SxYMBiwl2iC/8VMQ8EJHeTHzWeF7C8sLoEco15sa++UWB44gu8EGJh+exOQeI+YJzbPLZu63r23Jutk4qIoFyVSwX0/90KNZi9Wrs6fDC7pVeEiWTMh5OiFEg0/O9eNzseNBmAKNMCFdF37giSsL9ZUGpgKn1nvK60bzMdQgC2dIJxxcbEhT052V82u2qRTmUvx/KlZuzRCrqiDyDXnZhT51Zz7GU6mdl8JTVdVMiprUdM5yduW4NTeCs5bKVRtMIzslrYCsIN7RUJuNIcZS5Vrs0nEEtioiZ0Dqp8c2n3ViEN3cLWsaB6Lf6vyNUkO2Jm4nJLRaRTJxGuN9JzdI62/EzUqylCX93UZFvlMX4CyTLk8clSAkXhCUEoDdMLt1dVA84rSvDrap/76AGBQcm26q9f7ezGpbIKY1Iuwc6on8+FlEVnLHvjs0O+O6jlTSkVbhJzS1FVoXyIH5jGKsj35vZu/3fNr3aEO3jrvNzGvtyCmOaBhrI+HHHaqpEiREUR8jGPnDEVXvnJXVMtesZH+1WwQbp4zrLAREj8/h9ds8mZlbs X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HB6Ro6WihXiQQau2e9FgS4+Is5CGxbolzg2UKObL0hiot9+qH/C3JK/+lZZV4E8P/WPQQgHU/EgDOpqpYkWNLe959B25SrszXfeGpd7m0CDosdxi0aleMHr62dNSiXbXhdtwDB6GpVLuL8eMIcpidtmPg5ZVMMpM6X/nEWQR0uLv2XPs4hG9Zsh0UJJy6eDO8SOShyOtndqBh6AOqgkzwGFGs3/P3HN7/sp8M9guprJwT7XyYdhNJt/Q4Kh37eONA9c3D0+XAXzTRS/MwtU2Eovci+sEFcjydC/HkIZokb5YkKki5BZ9vi99AJKUWc9LnY/XNg5e4xTWXnoSvKUKDgTnyO2veRzqR598bD1Aws1oij590FH0jUGJFA82iAUAbsEAkGVyuLqK74pcB3QuIi3GhCUAqcPwoDRoyFMtWKqKdk3tQt5VPf33M9mhQY7M98KUzKJ1wwTmp5v+wHKQvlp+7NJZWlNiaimbB1nDOCK1HT2cVDz+DUylgMf6U2bsFH63Ffn42Z2ulK7rlZIEVynophxzq35jZFabMO4xjb1GBUoMCgu/2e0LvAafLFYFUq/xRHry6ekdppOnDQ2jJ1CupFXgnHo1GCSgk6Dw6wFZ5GWILJmOhM9AmmSBnWs8yUdheEcdxTdt1Q8vycvhV5QI1DSnYlZ8jIajx3upjnmcls7QJms5/WFkh0+TIn8+wK/203C/UHMHw1LLMvnnHGt/09o+c99edW69/UOzyQA2n2mKPV2lz0F61PU34hR54TqXP2sIKZw9FJYis7XQMSjxkPCSSMr6DZHlnTZwgwEtHHa8ei4qNVIEMX4YA43U6poqzCUIjjqRZEP14ikcjr39plrTNp1gmr1pGf9KDL3uhL6pWFkLmYjrnpLrNHSxgO3IdmU61/2PgSMH/km6hej3dUZ83mjN37EwBChWIXXwRWEZxDnU4wCs8dRjQ2B49wtFSeihjsl0U8C6ZMZYki21apJ/Ky3BrTkLJekYFMGIDnNfisQc4HeTcNRzlWqkCpFRaOB7aaAYjPa/RTAPS9Bn9UP83RT+hc+0c7LKZNzu/cfu0pjXjicOJtEUeRUc/3Owvl7NycAS1rIfQpGKTGf9QgxnqX6hPDl0RGgkaJDX+nDrHeZ3/mp9tZ4WlSfQaggHZeA/0acBAzRkhq33RTdxfByFqmt2jBLJo8oSfwSmftVPiqad0l6qMeOUAB+20WYBlrvTxVXCYmv1+abj8BQ3X7NWPUv4dc0loDfIZUOFTEEWLWpgmmQuY4SJ/WPLoQUHpmGu6ltqgJz2rIslM9yWxbOyNczfkW5uLaXdbwYlxZ3TpcH2uj9cm7SrMsAqOIW7vO7Zmln0rKJ3r+cxaCMziV4nP07yFy50E5f3IStS00UFkklYXdZXR1MLQ5c4fv6i+hsVkYtJK4i5KptdE6mgzCENmsRhk8e3awn/zBl/btEUqwSGT1+CwdlHbVWo14eGTD7L0bdVEREzC1OdgyePJBX5IKaQAYjGrLr3xOssoGTkOstPYTgskIBd9O1MD5H7GkAnF+O+caWqkeLAlsEKIEdS1X9L2+mPH9qODUJBMD+CW2E3YFwar/hA/w/3ccQ0/9XZDLxn+ih+SMTvVA== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: b2eea9a2-299e-4f0c-8463-08dd51d3d32f X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2025 17:27:18.2271 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: azCGDitC3j4je55Z3akKrxuKtWUxANTDy0oV/3TlwDGgAyBOx/ttVhqDRGxn0ytIdM9p+mvUMoaWjaym2GpWcw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB5551 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 20 Feb 2025 17:27:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17263 Signed-off-by: Stefan Koch --- .../bitbake-user-manual-ref-variables.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst index ad219b531..f781c004e 100644 --- a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +++ b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst @@ -315,11 +315,17 @@ overview of their function and contents. mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. - When a mirror tarball is not available, a full git clone will be performed - regardless of whether this variable is set or not. Support for shallow - clones is not currently implemented as git does not directly support - shallow cloning a particular git commit hash (it only supports cloning - from a tag or branch reference). + This setting causes an initial shallow clone instead of an initial full bare clone. + The amount of data transferred during the initial clone will be significantly reduced. + + For updates, when keeping the cache within the download directory, + the data transfer may be significantly higher because entirely new shallow clones are required. + Over time, numerous shallow clones may cumulatively + transfer the same amount of data as an initial full bare clone. + This is especially the case with very large repositories. + + Existing initial full bare clones, created without this setting, + will still be utilized. See also :term:`BB_GIT_SHALLOW_DEPTH` and :term:`BB_GENERATE_SHALLOW_TARBALLS`. From patchwork Thu Feb 20 17:27:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4152C021B1 for ; Thu, 20 Feb 2025 17:27:27 +0000 (UTC) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (EUR03-DBA-obe.outbound.protection.outlook.com [40.107.104.56]) by mx.groups.io with SMTP id smtpd.web11.2757.1740072443352419987 for ; Thu, 20 Feb 2025 09:27:25 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=eHON2IHM; spf=pass (domain: siemens.com, ip: 40.107.104.56, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f/fXeOv4hXb3GjCkFLOmU6QRZZrUWWdpfukQV5MK1KW1ry7Gi5gDQ5f6dB+LYgIUfKWsiBT9ArzClxPz6ObpKnjX5GsE/zZb0eH32XSmXVpR1PuG6DO4Tgrz7DlKwpmo1B6ga33fvk9W9UCVXWhBvFSZ4FP6BqbN7ln5hcS/1JWe3agL6XOTzXQG4x71NbJYY0LbV3RI9sEPuE6V284XAmXi3COCeqhBDVMbPe5390rbzV3HJROJ0FHYJcaWxcGwP8W1+0Kh+PXell4Twvk13Hug0KamH6SiZydp/GExPZOswvJH9mtuR09WptCa5hVpKBnbwCStFkaKWnfFtPrB7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UsxrczEO4YFetLHXnXNy5nZF3OU81aT1xLFybME5Ldo=; b=Bet/XgzUp4UyA+KisszW+9DeeJBh9e60BSB7aAxGrXOXMLUy6jYWRiWTKKDleIB4W47rvRowMM6oidsIycmHCcUUh9UOpl5HfSIJgcX2fKLGhNlO/SXkWxE71utziWZA0Ul09kPxV9xi05REQWbSkSl1JAOqN97zhfHfCVT7d7li04hg4U9YR1Vtr1vgexJWhmSWM11Rn5XXJBsNJLrxwPejWa6pICfl8zBJhjEjZ01Xf3aES3FFAaGkExGTP1aOKfBR0+1ue8UmHO+pG0X5/4b2pW7jTmcuTiXNFVCKAk02T+MFBPHJu0K1UuJgiH+309LNmBa2JiPwFfaBsOJ45g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UsxrczEO4YFetLHXnXNy5nZF3OU81aT1xLFybME5Ldo=; b=eHON2IHM1XZNzuvtV8KQku5+QEX6Pz4jMCB2pcuE9BDCjhV/XmSrJGQMH9MQW8r9P3fEkO+hP1sWJpx56UPZncHpDwnnxlB6XWhxlYvRw7yrhatrxqbIinvR+lW/kIKN06BL8ABOcA77l0YKYABjKcmTb/w+yBRJw4v2Pg2QSLZMTbK1ZvVmz3L2VJSOo6K+Ja5w/Jv0OyBhUnCCra7Ztjfi0x7gRVzRLmmBOmcXOLW18SPztWqEf6aE7FVvwKzFq0+rkGR6QBWA6eoFKtkgbbnj8AsqM3ulAcV4HGkSjsUu/LEEkp05ipAw4IO8UAjD8g8cHDpGB5F7L8p4lfqHIA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DB9PR10MB5551.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:30a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.16; Thu, 20 Feb 2025 17:27:20 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Thu, 20 Feb 2025 17:27:20 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org Subject: [PATCH v3 3/4] tests/fetch: Adapt test cases for fast shallow fetches Date: Thu, 20 Feb 2025 18:27:05 +0100 Message-ID: <20250220172706.3850722-3-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250220172706.3850722-1-stefan-koch@siemens.com> References: <20250220172706.3850722-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR0P281CA0165.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b3::13) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DB9PR10MB5551:EE_ X-MS-Office365-Filtering-Correlation-Id: 0df2251f-a017-466d-91a7-08dd51d3d430 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: P68INBlnrXj5kZnNzEJaLrD7hiONf+kAJym/FedJpahQl2A33irtuH2YRX9l+zXLZgdEJNSMqYogCj4zgMVcFGV3hfK6PO/kE7HwoGYtN91uoIZ7Zssyvhp+YB1Hcv1QVzhy6WXe+DSXrY8swzjdD3wL77mf4JMm3FpQxzG6qNmSc8Uu7zp8Q/fi0TJzHTAaO4ylNaD5jEqVwddyuyOOIjaVG/SqJ98GvE6smRaUJi8lDq6er2IrOvLxsiPe8yNNv8YhhRibCphYzujxkmyJHAxORfzKkVFOtzsSQL1kMQkIOKXgPkt+XIggwZwsQusC3kvzeUrHG56FoHDxCX0iAZt1MXPUCpkfbQg5RDjw4dV61yhLzbB0kbDAneC5XzB5/H7bTkrGfis07FT1Lj3w+iGtf/2jaT+NMz8xKRNLrFStKAxG7hz+S0C22mROcaPGjTpr50TUg18G5NxAVjkv5HyseNqYgBKLEIs2yixJ/03z1gOPefcnmrmORTPcVXLKpDwLClVSf5d7dGPHlElv4OMA/ghmdR6YSbfxAlf9nw8hN2sX3WugvP6K0xb/iUNdXCj8ltGnqrrUq4wrRMppXqcOdccEUHlQKcOE19ajslKUZgfXqoj4O6hJEflTczlri1zsCBsiTX22Ggd2fDAEh/u8DYDugWRY2B+wi8s9pqozCwe7PEGlD4If7/GLsP2psO3GAhPbPRQ6UapionlWO3jkgVz61U8SqDUgHg0tV1kCjiPCx/XDZpLxlS6dJhqbuys6JN08MIpHt6YColPbceCwXmUIJ9ohiPTBFhPSopOLZsDbpAeyWtX7bLqyuBWM+lBZk4EnAL95NcAYjDM10KdbHcCAnzQFbzp+/WFEoTbtOh8/RLGgDDoTImAcJIS+iERGkZN37hPrNBRLOECsqHJvgs0qYnIE+0fs59Z3f7L/LSHssAyZ5je5ggdBYhHYjDVGg1m7tEIctpukkKlFWktsSNPRHZbK3ix9UMqYuRzxtzjdZWdwljS05hBq+DxMPkMaGt4KDbtmoJADjfMYM9XqZpitj3tb//MrcvUnGdaKtRfc5hizB4F6KztLEanwfjHDh5sGYihW/J7cEiQTTmkVaaZD7I3141UI2vlm6VAzRcZD054fy2wE26PJ21H02j9bdxQzmRzT3PWtBK+zanoqnWofYmBvRphtJbKTZqHxGcBepylbXI6ccYkvqFGzAybxhL1BdU6ifMjPj6yTN+eprV4EgFJrKbf7JOT3C55pLXMvCcvqtC6N0zTyAOZkh2pJkpEe7rHvpXbrmbj1J40G5MvPIZwEuSgSoE17mK86TX/JZ0//TY0+CMryQahU5OetAcIDgODNTHrOdQH+StI0qSxIvt0TTF6GeAD/agfjXJ3FwdTdQfKxyAjETPdR X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oQq6s5ozXYf7MROHnOAkPzyYJ/emxkzkIuF/gLVtdBsZr+9TYvreVhOkyn6oP+UiESVeC99AZBLUvcFzZv4ALk4yd02xA1mw9lzlG73TWqlNtVQwOM33pcs+E1+hrvL+rm6e23ITukwib+CcDpgQVfBrTXtpe97yJD9CmoPhyVa8OyjvvuKFsAw+eDyzaQxEYvTVStOiLstSUI0WF3HMv79T6z2rvB9+8PmtM+s5p1ciCDBt3ZejI1qRCz6yn/6yw325UUWOTD2sIp81NzbI1EsFyQj2NposLbhj1smG7e1uA32DGawhhoIew3vmOhffsd3nWcWyWbpj0u1WA36/mU43UI8lp+qLwtCIB6r3uxHZQFThbihfJFRVV038BGLleQ3d+7yqOW55BtB0+4Seb+iPiQPriv6eZ9rADzvNVWPE621nxXN5IlNduEtR6uEr59dREzFq3GzwkRegRH4vuLhdtI+V4ZfryCJ5m7xJ80IlzTqqAzw5jq02UtyGaF46NBen0uzJNvBLshs9Gv6ennl7wdg9r+8jC0Biu88jUrIb0daFLkfeawW/dmQK7O1+pDthjAICWLjbqxrtBgCUQAe7Q/EcxgPds3cpm6kse1i1x/LdXpgozglTkjXM2Wphn6lL2rVombXvQEUQwl1f0tg9DsO1OmQf+KNJ1x1pcZsZYwQ3LjCua03GX3+qIADGUwYg4o3Qi2UJIjVLoRSNjgoDnKcCx5AKNIHT0amnhxDW0yPuNorG5W6jq2CBYe3oT9r6RBS/CRL+gkZ3yDxYoxuq1d2wIdeBnCiPXUVSqXHN9vPtaJX5xEf3mlY3/fCQfxPReA4dr0cnRmtz+vi1/FIcw2kw0STJ11cTGUhP7LZYk0w3fPJcyM3QtzDNAjRHEHyt1AkIf1cqz8OKmJpENoyc08lbhCV0bdCNvmRg/ilnGXxedxZiqNQbSb3PatwQ5mKbt+Emuq9icmC+mL5Kx95OuF3/h/Cj/xhrDT0ZrVosuwWOH5DXKADAkN4CJJx3Zp1i4eyzzMUM/OY73Jx1OOj1TtFUt2p5S6j1NEUFZFdpR7fOkS6r89Tx19aaGsfFNmZO67gGYLOksl/OeRv0h/aRo+/HvQmcWLk2DuUCvvOQAnIniUkPKWA1zj63NLoIZeSBAL9Ff7xTv6uQbMSm29rfIayG+lYNdBLK3bD+bHxBqaZV7leyAk9x6+lEyc72Zp+xpSUx1lfX9Jjpg8Ii5NFppR0dHMCLBlQbRLKEL775dFWgogYjsCoZnJ3h/i+hM2KZn6bP9uLXbj2I/DZuHpMekel5G5Bpu0NCif4sEsfp7kDbx+DWJYGVIpbFwzBu/gxCpBlNO2ks45Ee+ziR+yDgcFlL2zOFSoPdSYvDZMbrDhBbiS6jASye/Bc2KnrwoNCgKjUkCjYAFCpyZBoy3aOclLBl1OmDj/CtJwr2NFcSo51gZWUiOU+xYxcNdVztMhCI5xemDIn5x+T6BiaRttF+gxUmpjPi0T3NNhs081OmYQAbyIqbDm1J7tRVxZo6vidWNHamOOBlUpf9UgIXAYx0/fPJ4YebQKvQUnYl0FNE9TdhB4aTKP21J6wXwd4TBJUf51KG1261sMeE5SyToA== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0df2251f-a017-466d-91a7-08dd51d3d430 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2025 17:27:19.9392 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +jRvY70RiLUJXyGltDvJQratsTHyoYi3ozYzS0t/RIAH+nWKSQboiPI0rGhz0PzgyYwShFQ86Kq6x6C2PI8XBw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB5551 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 20 Feb 2025 17:27:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17264 - Address the absence of an initial full bare clone - Utilize the initial shallow clone - Modify existing test cases for this behavior - Remove incompatible test cases Signed-off-by: Stefan Koch --- lib/bb/tests/fetch.py | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 48b9e4af1..c0b53da31 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -1814,7 +1814,6 @@ class GitShallowTest(FetcherTest): def fetch_shallow(self, uri=None, disabled=False, keepclone=False): """Fetch a uri, generating a shallow tarball, then unpack using it""" fetcher, ud = self.fetch_and_unpack(uri) - assert os.path.exists(ud.clonedir), 'Git clone in DLDIR (%s) does not exist for uri %s' % (ud.clonedir, uri) # Confirm that the unpacked repo is unshallow if not disabled: @@ -1822,9 +1821,6 @@ class GitShallowTest(FetcherTest): # fetch and unpack, from the shallow tarball bb.utils.remove(self.gitdir, recurse=True) - bb.process.run('chmod u+w -R "%s"' % ud.clonedir) - bb.utils.remove(ud.clonedir, recurse=True) - bb.utils.remove(ud.clonedir.replace('gitsource', 'gitsubmodule'), recurse=True) # confirm that the unpacked repo is used when no git clone or git # mirror tarball is available @@ -1907,7 +1903,12 @@ class GitShallowTest(FetcherTest): self.add_empty_file('c') self.assertRevCount(3, cwd=self.srcdir) + # Clone without tarball + self.d.setVar('BB_GIT_SHALLOW', '0') + fetcher, ud = self.fetch() + # Clone and generate mirror tarball + self.d.setVar('BB_GIT_SHALLOW', '1') fetcher, ud = self.fetch() # Ensure we have a current mirror tarball, but an out of date clone @@ -1919,6 +1920,7 @@ class GitShallowTest(FetcherTest): fetcher, ud = self.fetch() fetcher.unpack(self.d.getVar('WORKDIR')) self.assertRevCount(1) + assert os.path.exists(os.path.join(self.d.getVar('WORKDIR'), 'git', 'c')) def test_shallow_single_branch_no_merge(self): self.add_empty_file('a') @@ -2116,11 +2118,12 @@ class GitShallowTest(FetcherTest): self.add_empty_file('b') # Fetch once to generate the shallow tarball + self.d.setVar('BB_GIT_SHALLOW', '0') fetcher, ud = self.fetch() - assert os.path.exists(os.path.join(self.dldir, ud.mirrortarballs[0])) # Fetch and unpack with both the clonedir and shallow tarball available bb.utils.remove(self.gitdir, recurse=True) + self.d.setVar('BB_GIT_SHALLOW', '1') fetcher, ud = self.fetch_and_unpack() # The unpacked tree should *not* be shallow @@ -2295,19 +2298,7 @@ class GitShallowTest(FetcherTest): self.assertIn("No up to date source found", context.exception.msg) self.assertIn("clone directory not available or not up to date", context.exception.msg) - @skipIfNoNetwork() - def test_that_unpack_does_work_when_using_git_shallow_tarball_but_tarball_is_not_available(self): - self.d.setVar('SRCREV', 'e5939ff608b95cdd4d0ab0e1935781ab9a276ac0') - self.d.setVar('BB_GIT_SHALLOW', '1') - self.d.setVar('BB_GENERATE_SHALLOW_TARBALLS', '1') - fetcher = bb.fetch.Fetch(["git://git.yoctoproject.org/fstests;branch=master;protocol=https"], self.d) - fetcher.download() - - bb.utils.remove(self.dldir + "/*.tar.gz") - fetcher.unpack(self.unpackdir) - dir = os.listdir(self.unpackdir + "/git/") - self.assertIn("fstests.doap", dir) class GitLfsTest(FetcherTest): def skipIfNoGitLFS(): From patchwork Thu Feb 20 17:27:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82E4DC021B1 for ; Thu, 20 Feb 2025 17:27:37 +0000 (UTC) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (EUR03-DBA-obe.outbound.protection.outlook.com [40.107.104.56]) by mx.groups.io with SMTP id smtpd.web11.2757.1740072443352419987 for ; Thu, 20 Feb 2025 09:27:27 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=gbvPM8V3; spf=pass (domain: siemens.com, ip: 40.107.104.56, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=g2pas2Bq8SoRIKcrH+dh+le18aPbohz/MFeKnz+rl0LjtNUL1U6VbAHfoWQBvju84TP7rtLV47ftJFVZvPGFarI1o3JU9t9G8KWxyRKCPxnSzHsvDRI4Hl7kjWJvIbI+jVMPhSistyZ3ZtCeDAUj1oVcwmX3u0tetMvF+Yy5L4l0t5LxhBxAkjr7Y0wHJN57tcuKamClLhP6sxflP0z6Nx7eqCdQ6QtyY6btyc5bcGL37fqT0iOTkksSuJ0Q2kew3Es2oQ+wYlr3pOPlzMSHN+Do0bJny1rCZQzPy0daBniM/tZhLkrFZmRkEhUZ7jnnxihJM4c5etvpg+QL+lwlTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0qQjgfJtFeDpQs1UDd3grJQeTE1d0BRDq6YHo/nEurw=; b=DU0Z/2X1lwogESFFWSWu6MCkudZOv38F2bAvx5e51GpqC7KKG5kfV3IfE1OqYzSrxhFap36NzDcFfDPf5hdJpk/tC6wmc++nHIyVyaF91vk+4QNoslmHBuE4GxfPWvW6GSZLwjhZjX5U9ukMMvSQ/Haloiammnm5f+BuiN2Uqiqh//DqAkN4tUJAGNCn55wYMroFEeDnuNHepS5qM0rv5YvIMoS081EfYWjKKpiTIa8+Vlym/CgQXIs3gJhraugvoZbFUGK971nUZQtrWQZ6/WvEJg+j5xSr634ib88tF+cVWHA9RnJ+X1+O3uMoyns7G5PsCDPdYdgjyMxVlZsK6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0qQjgfJtFeDpQs1UDd3grJQeTE1d0BRDq6YHo/nEurw=; b=gbvPM8V3aGA8mvx4LtKacQGxQwFRnEkncF5kWnLef41MUCnDkA9fJigqeI4hca2AP3kdQ1U1GUbek2FUTtMEeJezdzImnyQL0xgXzyqqtXndVg8u3GKbEBKO46yXV3wt7EUnbhd51mCPNkv24TCt+GJ1E+O+5VWBY/U4cUlkMk1FgzXF4bmPLXBhtXyaAcaD1Bycx3ViESzqpcqy/vzoTLAM9Cg+tLbseCPMJTtgclJ5FUXRnqt8GWGCWX/UvyTQUCM0UgwlOVbBkrL75EW2Vc6R+gG6OIsrJOKf356y+CRZWRM1CfNzq6JGmVrpZWPzRKS6U2+XZXlnzt0mAHXy4g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DB9PR10MB5551.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:30a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.16; Thu, 20 Feb 2025 17:27:21 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Thu, 20 Feb 2025 17:27:21 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org Subject: [PATCH v3 4/4] tests/fetch: Add an additional test case to check whether the fast fetch is shallow Date: Thu, 20 Feb 2025 18:27:06 +0100 Message-ID: <20250220172706.3850722-4-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250220172706.3850722-1-stefan-koch@siemens.com> References: <20250220172706.3850722-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR0P281CA0160.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b3::14) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DB9PR10MB5551:EE_ X-MS-Office365-Filtering-Correlation-Id: 2541aa35-517d-4da3-c553-08dd51d3d4eb X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: GGgqb2aO1elr84aGs9Pff0dsQapj/6I6TUgJdTA54sXVZYN1s2stpcjgX4F7JEJ312BEBbKU0mpAjhrTV0ubmUDOOKhbPB25XDQ7LGfshQ/4BEWL3ZEBEBC/3LannaejBZ2HcELs5leGlj4CmZlQz/9FNGRana+HUU3Se1XV82j/wyhtoO33qPDYsr2cqOd1QGkJFKq9hg2oSg1VW7ywLaOvUYl+lB70hfuP792GVmY44Matw5VBVolxtFBL+syxjaZBRFjGdm+SVJsZIJ+dPUd9vz43cuKPbLnXLpYdesL4mJWSbOy+HjmBKjrjlrkmNYpxcIsCuOXHRaUOvPH91WyfHmffWN/35YJyeqBKPvF9/0hG1vu/ix2b0Mitper6S5JxaFv80UVlwqulF5Fzbx4lQYN1DohFIOWkM/9GfaWOrfdWp9UipneqNligcANc77z69Gr/R+XXJa3Y+hlw4QHIfEd9wDUYuHtWJjc9BQhFWXxRU0Ow9Ts0z0ehK1tiSG4OTDa6KVZB969ubjXx39559K8Rg+nYB5jN3fWAaw9AtEYsPkWET2klFq8axpCYeM6t/RmwnwEJ/sXCRsVbkg775AshYFFo7i6t2c786Jf5j/kgUtShzDCZ4PAVik6wSeMm7rBg+2mUahouQRk+WF7UR1nNI7DgOV1gYLYHKVOTV8Vs21xjVskOhKIKkSlMrtsFK3vhiatRq3lPeysBTAebQu15ZWchgXhhYWa3pNDtRAOC+aAy80ltxC//0Gm6fBbnRpZ21vFCiwVNl+oi8FtobJ4JEfeqq85YOBqFQnq0fDwulPAcB9NQoTtnfxif09smo/t79eLUm6hbk8e2gl+550ccdzQaH0LKFQfcmc1gNWfCz8T6W1+z5DyO1O5fJrwxbgstfBe/gy0loWpBfKf1uxJNmVQTQqi/tVwEksne5JgzFnbv/oKk2KBPgthmF+sIud9Td4YnZ3tRM72D0llXu2/KJbKOT9zu3UoosHOuNy8MNs8nLmlA0oOI1VWA3iPlULltLF+Uym9t3Yoax9yPpgyNC7LGie7et5nq0sX6/XmaG+1NKxRynmQEY1Viaa2xY+dEdPh+nCIiJdozZyZZAfsvY0gsq3KJvwh8d2m/FBKogkhVmXXWXL8Rtknt7caiyKYva8vs7lk6qjg1bmpNhOaavE0c0CQTCS36upNczbVorqp/HnOfvepiT3osSzjpi+k6QDrYX2r04OXgiYx7PK3Z4o8KL95W8trBaCQoM7eSpCdj76OEy/vNecSo6oqKQ4dxWgt8Ku4T38oF+LWosCRQiH1x4cTufqRK9R6DfDqJpc3GFwfND7K/SCXoMdj46E1X/CIlNFpoAeGmRUTmvpQApnV/hbaI3mh18oFG6a8K60MxkAV6tsZrdOnN X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: sUimxy7e5X6qAem1/vBhxbmJlHaJM0vKWj8mEsYmucC++06seaC1ftx6vLGOh5z53837KU2zAHIKcdmpxVoTcrukr7nBJ8SneVUranS4bspW45AcUIP1mxOCDEDqS9vsduYdT4IHDKcu+Av0HCo2DtIKQuL73l6URJVz/ng4txZLYnEe0J0ZBex159+hZm/YfpKn++3851ewM6l3svlPw77u4ER2WqUNCp6OqOKd9EjksuXhcAkvP4n5qijSD4coKso6f2SEZQs2MpiHWUk9TzmKyxbHG7CKb3IHyg2YVHztk/8vT/oBPPSNQiyDjVVR8Ujt6CCQDXlPGQ8Q4QDDAmnNgbnqizRxvhlKTqYi2/tyW0ag6Mfd0aw7e7VLBQeQxjERXhOv8gTJ4e3wdWYJKDbM918p8irKcYBMNFLpfFgc3xogzWiDFgYqbUUlMMLIuq2TopMZC3zT9LIyH3frcv9Jpnw6E6ysXmIUWkkaRY3MtAr6wTRgSOIfihmAaMdyFQYR17iAXa+U60TZpIoBUasn4uBlv56wp2sxkDgyfiMXqt5ocvB9J4zpm4wMVYkgLFTFNu2h4fTjR7iQcoklTfFzGg7dwhdP9njstpRWYuczRBbklavrlATJRxdEwoQc8c+beHFbz94G+hZx/cvjRhvhdHhccQLUDIwQp1t0Fs1FpPercvuEJt0/ZCk7Ma8JEEPYIxqGrmB2fEhwwPYuyu+8LK4swOURV6uFrn9hc4KVtofdgKFKwwqewcs+tJafUWKGU3ntxxtIgZfsYCemJNyVyeE47XyOV7hIwtWMr+oTThC2RR33q7kPfzLwbAQtUqh+FY4TNamZaOj2aCDfwRfRr7xnxEn/OX1Xxr6cht2WjVNaBnRuWniyh0x4zvwG4cdk/f8/y7HtHT14H7I2b2dM80YVpAkUm22lQPG0CkChYafZNRro3SfNdhwakIDEEOfspvP2OJLNNYkeue2lbyPowvIf4wv2EmUx8C7e+Iq9EEYYRBxZZ7tf9odvpazT61lOK8AQSMlxnDEyuGDmVtEcQ8WwGFGkntr3qFzrbOJ4XryHtoLMa6geUj7UW9S/QsKsRlpahbvTPABzhKxG7oTzzNGWKMdU/RfhjFSIayWGQ7XwiBIhVATI8qfdZlALmp5JuTffLovPBpDOVtQ1o84K9SfmHTK7nvxznaKQwaP7JvQCTTBfF3YVAC4+R/afdzFscfXA9jyIX9dZ6+vEYo5kYVyM5YthIu9Yxfm0HGsgiboV5lUIu4M3xoDIk1xrQwsLoHAd8DRR6RAmaGBN36hXIGdEvRd6Jxal+I4rLPXZqv8/rJU/kO9B8miqZerk2T+tlCbLCklzQNrUnk3k9td2K3nysbdOJCqGZbQD2+KCk9gaYgrySmDdJn+6LVs4Ke9NG43/7t58niJimyrfh6B0p63k0wH7mrlOceGuPUI1kgJOuZN7ng6ptt9S1B8tEvWXddtjIkEct+ng9U8miefJnMwuvcuOK2TATbr5PGHaQaaRK0ytmgm3415EZgLaebUe3aKwXe5kyk8jumJNryqJSS9iRYFZOJbWSZ2HJitY2lnr67ttuP5XqQKs04wYPIR75A9J9ISUGJM7Xqxggg== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2541aa35-517d-4da3-c553-08dd51d3d4eb X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2025 17:27:21.0216 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bSonqOb4EAz4zujXkOWz8aUYnUozdq6oqK+P6+TrxMYOPU3UJ0AY8hPndl8FB7SI1yL5+wMILWdSixz8VNYg4g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB5551 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 20 Feb 2025 17:27:37 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17265 Signed-off-by: Stefan Koch --- lib/bb/tests/fetch.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index c0b53da31..27924881d 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -2298,7 +2298,17 @@ class GitShallowTest(FetcherTest): self.assertIn("No up to date source found", context.exception.msg) self.assertIn("clone directory not available or not up to date", context.exception.msg) + def test_shallow_check_is_shallow(self): + self.add_empty_file('a') + self.add_empty_file('b') + + # Fetch and unpack without the clonedir and *only* shallow tarball available + bb.utils.remove(self.gitdir, recurse=True) + fetcher, ud = self.fetch_and_unpack() + # The unpacked tree *should* be shallow + self.assertRevCount(1) + assert os.path.exists(os.path.join(self.gitdir, '.git', 'shallow')) class GitLfsTest(FetcherTest): def skipIfNoGitLFS():