From patchwork Fri Feb 21 17:25:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEB11C021B7 for ; Fri, 21 Feb 2025 17:26:15 +0000 (UTC) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (EUR05-DB8-obe.outbound.protection.outlook.com [40.107.20.78]) by mx.groups.io with SMTP id smtpd.web11.29032.1740158767905191544 for ; Fri, 21 Feb 2025 09:26:08 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=btTNGA0w; spf=pass (domain: siemens.com, ip: 40.107.20.78, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HhUmOHOSIvMoUhjaGmmTKHSZzAIbR40+OQCDpoyhwiB3aTSq1bR3Q+9X1DdPcKwFiuRjNm3RNSO90sa5rbMixskD4djSmwMBN9Z0Jh7gUhBQk4OgHGxXddnktds8RFoZtafDf8tFA6/APjiHD6NERn2vD0lBQlVlj5rCevboqsWuMw+co5k816qIMeRnUAo4ig+9842nXXvKNT9sjksTZKlur0jmlD/DiqPN1DF8fPNhQavl/Do8YYK6RXoHZvIw0ZI5o3KVsSsLctRlHPRRyydTxTkwPpByPeA3X1DjV4ga9/9CiXllJiXp8+ajpqFOg/5jaFM/8ikWW0ud0XkySw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WoYpRWWxQtycvWwvMlOYLdgUTe46p0PunvlHEl+rttE=; b=rAijn5se+ttksDm11vLLEXWM6nyn7CXHL0fE50Yw9dIhgD8Jz2Z2uklsLuV+qOvLWXdQB5m259JohwL1uMfXtnjnm+6MQkrlxIe8nS74j/4DviGVv9XRq4sHf61Gdjo7QIshVvdk3iR22VyarZwaber9Tj/kEW725LpbYEOoib/LmTz4HSBkLRo7b7L+c18ArYs85EzucF81+hA4HwgrdMHaxzWMuOjVZZctn0IYRzVnpT1lUvcxniallHRWd4nAQh1i9JIj8RhpE5cedDq+CwOanRtVAjymIn9vv0xKjJjOItzojxhJlXvqsuVcRXtaYUuG13tH+xDw3cfyDHEcQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WoYpRWWxQtycvWwvMlOYLdgUTe46p0PunvlHEl+rttE=; b=btTNGA0w+vj4iW3niCTkkynE2cA+rfz1DOSVJxyi9qYU1B3LD2rYczD5byQa1FPNMRa6TLDy/KPoB6/lHolk/hygMO4CTMmPkIJ4+itLGMLvmQnE0sCXNitaPFYeoBaRUJW5/nwnfIyKuWdANKafafHA6toC+QtHDCaLPzljG1bXE4PBEKGbxen/71DxAj8GDwMw84NhHTInPomfWg3Avt3wByeHHX+NYipBRTHxw5skNE7NKHKnC1wTbCpuOEcGHHh29o/kZCkKTlgABGro9iI6K8CtnC6e99cnJrFWlzX99BxvDbAjeEWYWw4xG0qiTWZvb/HP8TeuksPbEFl/+w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DU4PR10MB8514.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:562::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.14; Fri, 21 Feb 2025 17:26:05 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Fri, 21 Feb 2025 17:26:05 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: docs@lists.yoctoproject.org, stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org, quentin.schulz@cherry.de Subject: [PATCH v4 1/4] fetch2/git: Add support for fast initial shallow fetch Date: Fri, 21 Feb 2025 18:25:59 +0100 Message-ID: <20250221172602.527924-1-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 X-ClientProxiedBy: FR4P281CA0306.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:f6::17) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DU4PR10MB8514:EE_ X-MS-Office365-Filtering-Correlation-Id: 82a6032d-626d-46ea-025c-08dd529cd202 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: FHSu6As1mqR7EpzknTtjm7ntGOimDXmGgtK9+IgsLm2APfttUwDwBbxVg+SD6a+8EEyOQHvaLdJQJS26HOAR8HR1Ut3ZwjF7KHTFLi6lDPPg2u621zz86HYkHfdCLAz42xFHKsfi6zmFBGWA3dCuX0ViEitEfvMDf6sshBkRiU33aQ458JMZjoGhWPH/PtXkF7jYCAO5X39POUyiRlXh3jbBUlDCAxR/15w5VyauQGd8HbJLkQLSBOPCaTFl2prQYCuMbEDRRTYngeeFe4leo/mhqqH0oHQGO7H6geT/0jeCNzUkS17JyvYrtXoVYOcRSykuZnY02XftTIT7piWrv6FJeZrQNt8ZAKQx1cP9pVSwXOCCWYEQ2U5GgzwgsniZv6uALzUkrfE7ob2NSsMn6yjtucgxndCI2Hwytz/g/XPY2kXFdaw7OhOqoN8Q5m1dWWA1QH8ReXvAp+qm5NBU3dcL/6O3fSIvNBlFFInBezDwa7ZYBd9qyoDDSEpLuzm22Jit13mtHkPdmo0sTvFSy7p7u9Jnw8ZUuxr1t93Ci3eiFoaAU7vnv6hX7mR/aW0RqwBK1cT+VC3AoLMxYKGuvXbnTr8tASeevleLimsn+pNSsjJVdwdcNtdYfVOr/Vwy/65vUEFjuq9mjCQDLGJAKVAFPXumYEK5LSv4g51Ixj5CVH7M/DQQAFdBK8R4oB/gPDlHHILgnCOh00ofs5N8LqEaqYoQ7ppLebjGXUzypJNTJi00m+V8rze67eBvFEFYMbtbRbJ64dtbnafN+Ll+P66XBFwGDH6EZ+RUliAVvqMpg8OIHUokjEdrpQEDVxBs34e1Olbt1dt9tc5/1WXajCZbmG60Tf/0u96eonjlYsxOSgbCB8STCIfDlIuJHg7x0Z5ZHosdHFPsq/Mb8PVZ/5KSxl+EAX8D2s5dTGVxEp/r6yvvBgxUW/QydQL50O08l8qA338r/Zi587ucvLK9aZNRctESB+/myDkmJj7515U8xjV+nNR6WvKf/MshqbkBBsHTazgFecJvWS7ljiYGa37ITPNYAaOeGkTu7Q2xtG9PCTMIQ95mPVSYWc3oknFfMmqJvrUP+TyTRExN3TY0HRf0d/Cf9WUvXAMQb2fgNrz/HXh1yEi/HjHjup6zZTFuEkteFlc7Y31S/baFD99gx86DtFfGoGa05GgAfzsCzIl7UCV+BlDUMK7mjdyJ4nSaPPFT5pjb7BBqT8Yh6HDUzQH2a02/KLN3J7IV1Pu9aTne/+DzVPcztaLUJ3NP630jGEI79uwjDXe0kbUMImFMFU+EOhPKQ4lfq8gz8SBpFIXfAgAFt6TShlRD8mXy1+ye0CUPzef3VzGanQswwnxQ27xyB4OYuRpDnWaVm4A4oxdxhrFbe2yKT7XaypbmWEGJ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2SG2kq8iQZP6lEetRmO1ePjarC5juu0IqWXJNy1yXZ8eOYQq9YpldX2Zg2cmBrIr6k1sEOWIudPiGzMzjczzituBKWoMP6C7/p64DEQo8ap1o0nyMfIJwHslXMJumCBK9zcBtMgwzIVHeooFITNh2VbZdCQVR9SsSpT++GHr//qKS3XDzpOQLQhtC89Isb4ErflPMkDGvY/3pqZ4giO01AZNdn6p7PXrHyuG9eO2ZnpNGfeL/m5/mbRp20PUG5x2eSpPiqAvcdehO4+F4F/Sw57EwDuTGpkE9V430PbQzmjH2P+aETtJZZ58SxMWgiSACnDIZa5JbAdmlW/tLvzXg3i5WQGJQtnErxSJFr9gjBBTE8HGqBG9FfBlaOWr5h4bVpodLZKSXRLVDECfRXWChKqUybqA81X7CWfKqQ8Tild33Me73iTDDdKn57ynZbFDcID7uFgl+YZnm4yiCzr4Y8twaM3HX6AFjJAR/Z04UgJUUY6Wj2kQYLjzRa29mi+N0PJYUj3xSY3rp6yMs6iuYC2RCsNZjxSMom3jz0od1IpSczaACcexERyUulxt8aezY8x1mxWcMJo9eXd/k2VBDKUAxNktOgoH469XKEhXnvDvZ7w7cWu6P5KjvOztB9W9O4T1ahvN0YZkQTw+qWqbTvNvwi0/F1bPD0MgLqWRVia9GZhpRLRNNPx60eg8epYvWTV6uUp7CXnUJ0885obwJ+V+qNEwl75OQiG+1iwjr0p0NjGZqNO3aSXZCHF+VVqZrQqN1rvFp8EBgZIM1KMUU0YrLysiFe4SOrAYc8cziirP6GHCUaTHkBlo1J+9ZvC3aoDUcEW1YjFNJiPRUi6TlfqIC6GU/DT45NJIoxRHl6GBbQASY0ZLiEiajr+sEY7K+05JHBevp9m+0FU5NhSRFjYxqx3CMFgb1QZYl63JJ4ddRGsCooUysNnRZpjhUG4Qa6GeF+XlM4XJSILwc7ylwpeiGjyKaMRpl2yIu8fJEH70gahi7X5jqbKb64RN+7OrXvWDbBJbVVddEDcsR0ZCoznaC4fG++75EEtPP+MsuBa3OzvunHfhP0uBpR+JKfr6DYlyrzaw8h1oPruzEe7aDrlsowaTW5qEK7ugVPzgbbJrtT+WFmvR2R2zF4YOYSkRdzkPpDsZoBl6Enhc9g79r6nj9F4ziXOTxNwebK4fmxSRoE5silrneuDHTmE/TK24re/UoFjU06sx5/LwYRF+wClivXr0XwG+dQx7LXHR1iRfG3kgO0UMSVFMGoJCaWKaNMID+a1GjCJD+4WfNFPRDwWCiqTUfwBidrJX32F111VHBBTDA/uHxeg104hkSSIlD75k21dMoU5DCvySsjDvRjqFl3Wlne+8n3G3pB0RFn4HK54zZeBG1xnvOtnmIXDNIEmDSANMSpT2qYldK/DEMeyIwSm7/epmduqjNUFa6UH5MAsXdNQwyIkPRsvhihEeN5/oQtaQh3ZrTM80JHyV4qIzJwgrTYlEyzmtnup6hS/CpmIQSPvIcbhKe65+wxvHvQ6c24gKoIvrtXRdRE1mFyI5cJYwm40E2Wj6YSMAt2hvzZ4VkzD4sm/zSxzAsbVjR89HhXUTVXGqLxXCmbsDHw== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 82a6032d-626d-46ea-025c-08dd529cd202 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2025 17:26:04.9587 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ACHIi4MhSWqHA2WsRKwB7H96FlT86siEZYQbLfQc0bPbj2iFTvGCRDuwnx9rOZatDReMtdDOUSPSwYRrURxEtA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU4PR10MB8514 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 21 Feb 2025 17:26:15 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17286 When `ud.shallow == 1`: - Prefer an initial shallow clone over an initial full bare clone, while still utilizing any already existing full bare clones. - This will be skipped when `ud.shallow_skip_fast == 1` This improves: - Resolve timeout issues during initial clones on slow internet connections by reducing the amount of data transferred. - Eliminate the need to use an HTTPS tarball `SRC_URI` to reduce data transfer. - Allow SSH-based authentication (e.g. cert and agent-based) when using non-public repos, so additional HTTPS tokens may not be required. Signed-off-by: Stefan Koch --- lib/bb/fetch2/git.py | 100 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 80 insertions(+), 20 deletions(-) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 6badda597..5ef2c2147 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -207,6 +207,7 @@ class Git(FetchMethod): if ud.bareclone: ud.cloneflags += " --mirror" + ud.shallow_skip_fast = d.getVar("BB_GIT_SHALLOW_SKIP_FAST") == "1" ud.shallow = d.getVar("BB_GIT_SHALLOW") == "1" ud.shallow_extra_refs = (d.getVar("BB_GIT_SHALLOW_EXTRA_REFS") or "").split() @@ -253,6 +254,7 @@ class Git(FetchMethod): all(ud.shallow_depths[n] == 0 for n in ud.names)): # Shallow disabled for this URL ud.shallow = False + ud.shallow_skip_fast = False if ud.usehead: # When usehead is set let's associate 'HEAD' with the unresolved @@ -366,6 +368,33 @@ class Git(FetchMethod): def tarball_need_update(self, ud): return ud.write_tarballs and not os.path.exists(ud.fullmirror) + def lfs_fetch(self, ud, d, clonedir, revision, fetchall=False, progresshandler=None): + """Helper method for fetching Git LFS data""" + try: + if self._need_lfs(ud) and self._contains_lfs(ud, d, clonedir) and self._find_git_lfs(d) and len(revision): + # Using worktree with the revision because .lfsconfig may exists + worktree_add_cmd = "%s worktree add wt %s" % (ud.basecmd, revision) + runfetchcmd(worktree_add_cmd, d, log=progresshandler, workdir=clonedir) + lfs_fetch_cmd = "%s lfs fetch %s" % (ud.basecmd, "--all" if fetchall else "") + runfetchcmd(lfs_fetch_cmd, d, log=progresshandler, workdir=(clonedir + "/wt")) + worktree_rem_cmd = "%s worktree remove -f wt" % ud.basecmd + runfetchcmd(worktree_rem_cmd, d, log=progresshandler, workdir=clonedir) + except: + logger.warning("Fetching LFS did not succeed.") + + @contextmanager + def create_atomic(self, filename): + """Create as a temp file and move atomically into position to avoid races""" + fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) + try: + yield tfile + umask = os.umask(0o666) + os.umask(umask) + os.chmod(tfile, (0o666 & ~umask)) + os.rename(tfile, filename) + finally: + os.close(fd) + def try_premirror(self, ud, d): # If we don't do this, updating an existing checkout with only premirrors # is not possible @@ -446,7 +475,41 @@ class Git(FetchMethod): if ud.proto.lower() != 'file': bb.fetch2.check_network_access(d, clone_cmd, ud.url) progresshandler = GitProgressHandler(d) - runfetchcmd(clone_cmd, d, log=progresshandler) + + # Try creating a fast initial shallow clone + # Enabling ud.shallow_skip_fast will skip this + shallow_fast_state = False + if ud.shallow and not ud.shallow_skip_fast: + tempdir = tempfile.mkdtemp(dir=d.getVar('DL_DIR')) + shallowclone = os.path.join(tempdir, 'git') + try: + self.clone_shallow_local(ud, shallowclone, d) + shallow_fast_state = True + except: + logger.warning("Creating fast initial shallow clone failed, try initial regular clone now.") + + # When the shallow clone has succeeded: + # Create shallow tarball + if shallow_fast_state: + logger.info("Creating tarball of git repository") + with self.create_atomic(ud.fullshallow) as tfile: + runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) + runfetchcmd("touch %s.done" % ud.fullshallow, d) + + # Always cleanup tempdir + bb.utils.remove(tempdir, recurse=True) + + # When the shallow clone has succeeded: + # Use shallow tarball + if shallow_fast_state: + ud.localpath = ud.fullshallow + return + + # When skipping fast initial shallow or the fast inital shallow clone failed: + # Try again with an initial regular clone + if not shallow_fast_state: + ud.shallow_skip_fast = True + runfetchcmd(clone_cmd, d, log=progresshandler) # Update the checkout if needed if self.clonedir_need_update(ud, d): @@ -509,20 +572,6 @@ class Git(FetchMethod): runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.clonedir, d, workdir="%s/.git" % ud.destdir) def build_mirror_data(self, ud, d): - - # Create as a temp file and move atomically into position to avoid races - @contextmanager - def create_atomic(filename): - fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) - try: - yield tfile - umask = os.umask(0o666) - os.umask(umask) - os.chmod(tfile, (0o666 & ~umask)) - os.rename(tfile, filename) - finally: - os.close(fd) - if ud.shallow and ud.write_shallow_tarballs: if not os.path.exists(ud.fullshallow): if os.path.islink(ud.fullshallow): @@ -533,7 +582,7 @@ class Git(FetchMethod): self.clone_shallow_local(ud, shallowclone, d) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullshallow) as tfile: + with self.create_atomic(ud.fullshallow) as tfile: runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) runfetchcmd("touch %s.done" % ud.fullshallow, d) finally: @@ -543,7 +592,7 @@ class Git(FetchMethod): os.unlink(ud.fullmirror) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullmirror) as tfile: + with self.create_atomic(ud.fullmirror) as tfile: mtime = runfetchcmd("{} log --all -1 --format=%cD".format(ud.basecmd), d, quiet=True, workdir=ud.clonedir) runfetchcmd("tar -czf %s --owner oe:0 --group oe:0 --mtime \"%s\" ." @@ -557,12 +606,20 @@ class Git(FetchMethod): - For BB_GIT_SHALLOW_REVS: git fetch --shallow-exclude= rev """ + progresshandler = GitProgressHandler(d) + repourl = self._get_repo_url(ud) bb.utils.mkdirhier(dest) init_cmd = "%s init -q" % ud.basecmd if ud.bareclone: init_cmd += " --bare" runfetchcmd(init_cmd, d, workdir=dest) - runfetchcmd("%s remote add origin %s" % (ud.basecmd, ud.clonedir), d, workdir=dest) + # Use repourl when creating a fast initial shallow clone + # Prefer already existing full bare clones if available + if not ud.shallow_skip_fast and not os.path.exists(ud.clonedir): + remote = shlex.quote(repourl) + else: + remote = ud.clonedir + runfetchcmd("%s remote add origin %s" % (ud.basecmd, remote), d, workdir=dest) # Check the histories which should be excluded shallow_exclude = '' @@ -600,10 +657,14 @@ class Git(FetchMethod): # The ud.clonedir is a local temporary dir, will be removed when # fetch is done, so we can do anything on it. adv_cmd = 'git branch -f advertise-%s %s' % (revision, revision) - runfetchcmd(adv_cmd, d, workdir=ud.clonedir) + if ud.shallow_skip_fast: + runfetchcmd(adv_cmd, d, workdir=ud.clonedir) runfetchcmd(fetch_cmd, d, workdir=dest) runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) + # Fetch Git LFS data for fast shallow clones + if not ud.shallow_skip_fast: + self.lfs_fetch(ud, d, dest, ud.revisions[ud.names[0]]) # Apply extra ref wildcards all_refs_remote = runfetchcmd("%s ls-remote origin 'refs/*'" % ud.basecmd, \ @@ -629,7 +690,6 @@ class Git(FetchMethod): runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) # The url is local ud.clonedir, set it to upstream one - repourl = self._get_repo_url(ud) runfetchcmd("%s remote set-url origin %s" % (ud.basecmd, shlex.quote(repourl)), d, workdir=dest) def unpack(self, ud, destdir, d): From patchwork Fri Feb 21 17:26:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0DF8C021B3 for ; Fri, 21 Feb 2025 17:26:15 +0000 (UTC) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (EUR05-DB8-obe.outbound.protection.outlook.com [40.107.20.78]) by mx.groups.io with SMTP id smtpd.web11.29032.1740158767905191544 for ; Fri, 21 Feb 2025 09:26:09 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=PLnt7U72; spf=pass (domain: siemens.com, ip: 40.107.20.78, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Y+Tx4pBw4n8AiLefkjEl0jqv0FMgwksVb4yFmqIn2GpumhJQNoiJldawm5z2DLN248SYRlnmS3aYmsi3Qf0PhiD8VwtB1BeAbsI9spUI/iCAJpndebAsXw1O7k3WOq/UaVN/zfVHQpv1BdVw/q5KZux7+iumWFNUQ43rJQ0abG5KiyaejDiByraaFBMDtUPVMFNXu+f7Znbz5HfqAcCbmFDLyY9aCngibe1dPeZPeWuDuzLYikfFNuuKn3KZbz3fw81P3kwWybcc5rH+NaEzD3a7ebNOV4Icl3BzRYW4BjOHWrtlS7fFepqteXhGoBo0s36oI+HZBF2wbffdtAiVdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ep9gVdmoez24eqEvUdrVa5lIjDVPIIxFdg0UsYElOJ4=; b=awlCGPSnoCzf506LK1UTsrIsVQZzBu8hXktVRMMALnFgjq/bY7XLYtEjhG63KZDIl2SLUFOH24mkc2tn/icsyIeCmtFZ3mYyu+gZ6y48ioW/KckKG3mU0Cx3YlQsDCfTn05hluoVna1OUfux3GYL/ZoD1q35QGv89Y+SuwMZ+IN5T8EskEt+vwWpTI4ucD2y9wUwYkE6lI5LPG5yeXCziWb0RccGTMUDrxiMr2r44NRCX80aI70S9Tn041ahrJ+s4vvTXIEipY8VFkN8ys5KfiSg3O0UFa1uP3l/y+Ya9A6pxjmuVMXFzqUMJpx/bqLGRcbmp45fYf01Q7zUlNEklQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ep9gVdmoez24eqEvUdrVa5lIjDVPIIxFdg0UsYElOJ4=; b=PLnt7U72HXyy/uLNMEhNZKtFiXuFyUKSGTFniJ2Uv1MGjA7ISEB3dcbxb2ylJTrtOgMHivW7EPu/8wqxs6hb/MXqxkpsljpbWbeCUNaBWbY6Enjy68ftVsWaxsVQIgRWijuXEDPyFbCQogBz/iF1z3qFIEPXmzgzTobt0M9hboFfyY2gnBOJdRVopZoe49JLuUR8zfJCIc6W3cyIaQKAUoPa7nEA9kUHM9t/3kMenJ87KH6ypS2biIFtC9l+V9dBbzQJj7NAxSLnOgtuUDHjC/s4skwVCMDTbR0zQfucLI2CG8EUboLVRq8Rhh2ounqrrlH7aewBPK6rSOhfWNUL3w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DU4PR10MB8514.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:562::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.14; Fri, 21 Feb 2025 17:26:05 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Fri, 21 Feb 2025 17:26:05 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: docs@lists.yoctoproject.org, stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org, quentin.schulz@cherry.de Subject: [PATCH v4 2/4] bitbake-user-manual: Update documentation for fast `BB_GIT_SHALLOW` and add documentation for `BB_GIT_SHALLOW_SKIP_FAST` Date: Fri, 21 Feb 2025 18:26:00 +0100 Message-ID: <20250221172602.527924-2-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250221172602.527924-1-stefan-koch@siemens.com> References: <20250221172602.527924-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR2P281CA0094.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:9b::20) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DU4PR10MB8514:EE_ X-MS-Office365-Filtering-Correlation-Id: dd1bec3f-cb5d-4224-2a99-08dd529cd290 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: X6PgI6/T08cL2hVGOiLsOxaH9BWCVvcIg41UZYZyrlYwUW4mjXUmXIocV4c9eN6ViENtKOK2sY4m43dIiWbbyJlGSjxS6iS2wRlLSkDQcyR8hdmZ/D4/HIEyJRhych6LOGbXTl/yP9LsZoBrwat4jVrk72rlNDgi/Iinjk3FSgU0C0+ti2bSySOAAsjtwKXls9KklK7mVCwQireGe0DMTrlvc2Xw0WUpIM4U3yF97L6AxygjfUeWVVme0SK2SQbcS05fz3z/GlA3PUkWK7YNsaRjj22VjZRxy/7q9niuvEQrSfrA4F7zcJeYwBvJkwsPLGxFWfGnMj+G674MSZjzSefe9ZwgqD/1UXL16NCruQZ6DH8ECMT6G9r4pcD9mPsAvosQZojEBNqCLZLAcKzX7E0SznPfRJzsG2pp+8SY5ucvtzviqJFYmiKMKdKCGRcC5vDRTfpoRL46FyzBmMwahrG2OgjiOMfwLgUquCA6sLtou3uvZeuElrAL5aDHlh+nVAnokR9ehnaqB8L8in691Bnh5y29e9h3dFlkuL6l19zk/7c00VE2gFVxGCvTNKBsbughw8YuddTMGQN2XI5EgBAjX5Hb+KUmKjUsv0mhtiH2ggLexCasXl0P8VzC3N0pVHBshc7kazxI2skDMZ3Di2ZTONbuq3UpLtBGK3gjRtDCEeAZmkWcfa/J1WjisSA3TAQoNZ97aCrtfxRcMnbvVhJD3RuchrpjCJqYsRrlRHpMPHo3mb6rc0WRuUyxXwaekJwhmZ8tHe1XR26LCLqY9gYlYrxACYwCcHcl7RoTbbAuEyncEgPjzITH98Db0EqnwlBzF21z4xO4cJ2E4oDey1d0XCbAQlG0P0cOrNTDCuVVVscibiSWD46FuS85cobGi72+pW9Nv3bxCGJlKxKrbNS/3OeRe4kvfwPbM/OzByiRLzJxDBeK+izDBhAmpohM0+0SbhGqO7MdWAhmd9QUnT+REpty3Sb5Z2xYh8RJdbCBuMDgcHlPT22lOUgKbO11XOv4Q/pOkswXDRX2Led1wZUeoUCQxoNF3+NzXRYut4vVNJpZyvRo3OPGeOozAluY+2jbzgMxuHrvap0vY5oLllzJy83MZUURbTG1cOin+RQBHbl2N/1/T5o2/khRcOyVhNJrdd/ttZFT+pW9cvyz8hhnaB8doLH1nVZZ9dhni7f/RTn5Vjq/eeEjgWnj981bOM8KgyWG98MDgmcCvcq2MiKZZjD6alyvtiad2yPSQp5AfPVpPP0cfhQXdoVZQMNsFIxKinAME/wosoEEo1z6hfaCsDkCaSTgPLEQ8wCT7jPvZ+cugbOY67xpRM1+qz20O14pOdz7PhoN/0UKqFFa+Z7Zr1ntFpddRzl/z0yGjgtuK0clKok36jcDAPWl9uyq X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JWPiwMDYUGbp3maPIG9IME1U14235oiK9OcTX2zmQaFcI3dPM+jOQYNrc2GV80zYWHZFoBLrdYzhpMvXcl/Ynk+G7MDsu7M9k27nZwZmQnLKWhwW11gNzOYGGbiur9s4t0rdZXq40SpzMc7XnCrWwp3IBFbfP+ej9iGRSODgaSF1keEsik3Y1K8zkSfauTvPg3AylqZIR1KYe8olUkNCLG0BhLpEHx8OtFVGtIpVnScLzXvY2nZMmNcF5Rn8Al27wcyU6xBVVJJ/c93BY7NCSpQhx2atiEzDiR0NXsbO0DqkSmwzQvPRqbttBDu+11wp23oJvO1HyIUvVDSCGO4T791F9d5LX77WC0FuOw//CeJy+8lboS8URAKwbJdyHd2RGAaDQEsQyZVgKehN0BboX66s/9TvM2JjsEBkcRUsKZgrFGZkSC+L31NiEPpid/vaK2pPKfTuWCsp2bqXG0K8W/qKWVQbPowe8V/A8lbsuvqpTNFkTy/Ml4iSPZq5BZ2n8qgisN5+o7FQ7AaOkfp+a+QX+jf44PcwqmgMKxXiQZJoA2Z/RKt6wav18uTLFpkQmGdmyYMeX8CSTSzD06U4ta0Xpt7YHY062b/rI+kFaN4VOyB8HZGES5qLNDW4rzuPhzN3Hn40T+wHm/KSxonxwxQzsu7LpmBXM9Ph8jjdTqd1KtQOUt2MgrVzuVNeiAFVcx+LaTv5oPCWxWxNeZO5qqWf/ZMrmf1CIzwgdfWEWgBjNW3XIRgONtPelwR9arYW2ChvFxR8yULLZ0xY0uwsXfQdgkWyhoF+Up0M+UE4a8YrApXvqXTNF6kfoYFvWDQPVodXXH7VUVKEk/gobf6J5zFqVOB9oO67rIHATRA1KlR+XKd0VdmVGPQhzhcHE2nYnUbJHfuFWI60gDMY/8A1qExBJHLB+BTqg5fYCY1+1xPDLypIADs4n9yzLxt/NsNJMxyMgeAnWODTvAy6ZQwUCVYWbn7ytGs6FnNQItnENS5tYKgm0LWXW5gJ6E31j/X1Nsm7eSGDFGWqLMEpt17B3acwd1rnqIWUZvCz0J0kH5m3lMNwLqtwAtyATbKPl1167B/DHdAAlrXAvb4whcJFt9IIMr+kufDy9M3dpWZ1E8rahQrKreT5J9ulVRkmzBlc+2Zb488K3BgHDm0UWJP/a+/9pSMU/hNenldFDs9c2AVB/XW/lt85nKEkqxo+A85NyYvI8xn48m0CbK34dh1Xy7Z63P+RaSgRpaOOCGkI6NFsPwz68A3CNuiXqsiqyZkwpyrrhDqsb5l7UdlJo6jmg8YELXXhpkT18OZLl2K9d/wTVOOJ6Jon2JU0JRwyn/uXltNDFvvkZPaabM8EHROw6WEUM6wsyUlqcElBtTN+HXV7JOCd09kEsXECHARHbea6xOwsbH1/JGElF76njrLKIvon2z2W9Qgv14471Fl1tODM52iQghV3xyLTpWN6Lp2TwNcsseTJxWpU1/LTjugy+0bwI897HWvg4LpAMVcTS+lPLimOaONbdvAdR5w64Rb4e+5t2koPtldq2pJZFOfeUO6DfJPFjT3ymWo+BdXJrwhFiQAX+1WdLWw1LS9huqT3h9IyLsugUK18Tc/wEvJ3ew== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd1bec3f-cb5d-4224-2a99-08dd529cd290 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2025 17:26:05.8643 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0ksa/0wtM5XJGw5D9KiMWNmFwF4V/KuOPvCbnqs1cZ2sb2FwhxeS0wFqJ7J8rjVzfiDjP9t30ogs4ONDM0t1Zg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU4PR10MB8514 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 21 Feb 2025 17:26:15 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17287 Signed-off-by: Stefan Koch --- .../bitbake-user-manual-ref-variables.rst | 27 +++++++++++++++---- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst index ad219b531..47e327277 100644 --- a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +++ b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst @@ -315,11 +315,21 @@ overview of their function and contents. mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. - When a mirror tarball is not available, a full git clone will be performed - regardless of whether this variable is set or not. Support for shallow - clones is not currently implemented as git does not directly support - shallow cloning a particular git commit hash (it only supports cloning - from a tag or branch reference). + This setting causes an initial shallow clone instead of an initial full bare clone. + The amount of data transferred during the initial clone will be significantly reduced. + + However, every time the source revision (referenced in :term:`SRCREV`) + changes, regardless of whether the cache within the download directory + (defined by :term:`DL_DIR`) has been cleaned up or not, + the data transfer may be significantly higher because entirely + new shallow clones are required for each source revision change. + + Over time, numerous shallow clones may cumulatively transfer + the same amount of data as an initial full bare clone. + This is especially the case with very large repositories. + + Existing initial full bare clones, created without this setting, + will still be utilized. See also :term:`BB_GIT_SHALLOW_DEPTH` and :term:`BB_GENERATE_SHALLOW_TARBALLS`. @@ -347,6 +357,13 @@ overview of their function and contents. For example usage, see :term:`BB_GIT_SHALLOW`. + :term:`BB_GIT_SHALLOW_SKIP_FAST` + Set to "1" to skip the fast initial shallow fetch. + This setting forces an initial full bare clone + instead of the default initial shallow clone. + + For further details, see :term:`BB_GIT_SHALLOW`. + :term:`BB_GLOBAL_PYMODULES` Specifies the list of Python modules to place in the global namespace. It is intended that only the core layer should set this and it is meant From patchwork Fri Feb 21 17:26:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1DADC021B6 for ; Fri, 21 Feb 2025 17:26:15 +0000 (UTC) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (EUR05-AM6-obe.outbound.protection.outlook.com [40.107.22.71]) by mx.groups.io with SMTP id smtpd.web10.29015.1740158771028340956 for ; Fri, 21 Feb 2025 09:26:12 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=EAAvSKHz; spf=pass (domain: siemens.com, ip: 40.107.22.71, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DYzberVLvnyWe81aCQ4K3mJYcWCctip7imnQyOaFvJdbtySnNOI2Pa1hSYTB8EOZEgC7LM2Vwpu2dZCyzKrM4n1CDKUfUWabSuoxHdgOlEPiiVM8dsxV/P3s62vKw849fi6dSUGTiCbcoBFAF3mj6pZ4A7/qchIfyAXeOfUPGwTCpyiJBQo9ODSoPTK7/4GjgA/3dKQPLCn7EN90QDGznly6/2N6tfLKVMHEoloPnOtWkaSjCOItmrJVaUqoMlzXpy3Lii1MB8HZCJ26Kw+GvumKpaRafmxylrIJhXmOKIcia0WQisaVEy4hDyMFTOT9MrJT2MF3YynGfMi6uSlxvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DQ1xQNI7+4lQGUeKabrkZnRlyiuO6amkDVfvyn0PtSs=; b=JtQXE+o+B/teuED8W7GrXmlKuTR/pGI1NO+FvAlxc0p3m2ARL0XpUImcypXUi+8p8vg0nxwlxI5EH9dPYe291ogDjuEnCjYqw/6gS5p+jNriPQnR5CQmyah2OjJ/zYOhS2btFkaB6QgAk8DHqu7auU7YNrWMlGXLXRaXtOG2FKnq5hgt1BBTLpUyVeD0r7vxFHTxphFHAleta2jtf3dKWc1f2MRrOoYhnNmttx38PSAp+H5gd0fgGh7aPw9T+1SuHmnFXvp/smuhxVaAEI7oSJwH7UHZ9StgDOLUB/xQ9bwd1HRAJsO1I9e8uZpO0s2jB/5alzk2FNL9y/C5J/6E6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DQ1xQNI7+4lQGUeKabrkZnRlyiuO6amkDVfvyn0PtSs=; b=EAAvSKHz11veaXvNFbU72i/vVAvnwCMYmCVYRLlL3dxdNZX871JITxJ8YCjiZktWaCAunuQe0D7bZllByL9BmAgHw9nMTensQBJQaf3YJF3gTN3QVifm+34GJaHf+RUQkNvNnOAOqrg48JyCHMVTv2hyHTRtwYJD0oLDoFqHXcyJy/QocMIMV707GPSIen8FwYmv1Hqm/dHSLpB7A56ckBRv5F3rCTcbHxdtbB1lKZtsbRgKwLJjUF9tSg0VDJ9Jj/FsPZmaiNQqD28xhPCJ8YFBK+zXX0jS4HVfkyqa62NpmXrGqI2LQfD/zDDO0PYOralJQUPgEs73kZQBzh5ZQQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DU4PR10MB8514.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:562::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.14; Fri, 21 Feb 2025 17:26:06 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Fri, 21 Feb 2025 17:26:06 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: docs@lists.yoctoproject.org, stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org, quentin.schulz@cherry.de Subject: [PATCH v4 3/4] tests/fetch: Separate test classes for default fast shallow and skipped fast shallow modes Date: Fri, 21 Feb 2025 18:26:01 +0100 Message-ID: <20250221172602.527924-3-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250221172602.527924-1-stefan-koch@siemens.com> References: <20250221172602.527924-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR4P281CA0402.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:cf::14) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DU4PR10MB8514:EE_ X-MS-Office365-Filtering-Correlation-Id: 6b07f482-74d2-4016-ccab-08dd529cd30b X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: 1LeKjg0VhlF0ECOOJ9yUle3O8qMZ1Tp8N44KaKoIVtilo7PPfqEMAeD2jf+AJqcknw7aRg1UELLqhKX7nMVaddRpb/X3vOGn8+FHNecDv/y0UvHZd7fw8VpHGCg3HyEpvyMXdA8wnF1MBViD/zqrKQ0jhGRuJGoV5gCJrtUHwYylLGPvvuhfFogElRVBuzGEnRdLJzlZ+coWrsD89KFBuUwfw5Mk8GYXzv7ehBXZ5ML2IrO5IETRj0hQSlOqf+fmXZtTMqJ2sAtYWyph+CkK3sJj60Gm1BKGJ97soS6hQLEQoX/Y3e4QN+DS84C5cCDzXrtqYrv/VUPkXNKoohTh6J0OOEsW4iDcJk8NCL4rQ/85YIlNi5s5G5d7cmNZE5iSTvsApflQTnU4rdqIJBUCOjB5p7HQIYZrH+lFPn3mpVEQLBUsxf6WyMG13c+5jcf84U28Ezm6g0oofIU78PQF+p/XIlpp61lDUJBTznb7kiueb2YjB/V3QPaW+Z4epL6hZQWp2bOmZqnInxYv3YQnZnRNTjnzvozukdT+VpPsnBpkAeWQHjnAuHfYit/5j4gXYtmQ/uyh9FsMMPYs6e+iImxwPJXPQt+YZ4zhyjmhhAvDkVtcaD3xIQBjl6qfDJgXy229sfXDOYyOf5KSQD7XjX0cWMEMelfxua+jDweS9qrRvqqOy6D9Qzk/C6vsZyU38gP9Hn+O/CRVGGMVhvPPhxCk5GZf3WFG3Oi6k8MCOC0cx3SEGj7g0NfLkITyfeZ2EeS1JAxHXXUesrEffxHasnyDVFhro/4uCr/TIhajQmHFXg0JzEDs0rvCOsWklPbkNEYgXY1G3uBGnXp7zLVkyO6nmcSEUpoh3qVq2z4gIGrT98jWTPgIyKv7hjRXf/kzIeo7nCzJcWJ2acV9uuj4WiDi30oKEBN2ZsHNVaghZ9bfXqyQ0qqzh6Hl6czoBv0dHCNEWqfU2rk4zBYZDvC8rA9w0Qx5aIiiVm/oA6U2e7YH7vXCrqGXyDEPPgVJxxTpkEl93iaNJDxc4ep/i6jukezWD6nXXuIzJtNcc5DyubDmWr6MotbKGEMw2c9HOtClR8JBsZQZY/p0GaiinQf7vHS8E4lfL1wSRJqHBHXWUSDtXcDljSSDI7C7Ch+Y4f1FlRIlBHmPRQn6FkoykY+ZXzBMPdY8YOi0elBz0Sl2KQAGTFv5RuWm6wUN0+ih2yiae+5tExMGqNmYG2qDH8PJAYCOkCwoRnPCzjX3bIuNX5LoOX5a9E8cNmlpdLVzW0E/eW+TFAuDEAoteL+ZqYxcaCBHuQPl3hqUdGMQjxrUy+uK2XdeymR5HlMOPkcuGlTUBWFm/ge2TKTazCrv8rDWJ0+LCqM+TaDcXgLtB3woqIfcLuJ2DWpgOJn85zk5j6lV X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /N2C+yNmSSl7gi4O+J2C1jRH/a3VwI4R3smNVqluUdkGTAbeRjDrLpKuNESc3z2UhpBYk9VpNn+vQzq0TgYFCWdsb8KuXV8O3Ox5lsptIWtbtFRUES1SwBFMqnePv1W8MAyh9UiW7vVeU+jB89fNflvMNB+s3B5p3doiKe9IxzOF+Zt1ano4v4RW5sg9EMyaH91hJ9iPRPwl/sUpOTrOja90AOKmiD0uzTLjVNsXxuPAdXpolvClsBjjTBU+znJKFbnbw7faLUz92W7yM2NLT+u3fXSi7wlXGiZVf7O7sAA3NgNKai4gYTML0L7LPa2gSrrDsWa+eoJUNLbc+YwRnuaafnXg8dhrGus77K4y2eJ35auVp5SIMKnJZfSBM14S2TvDjHonos/sIdCH4+NaGgIghDn97cEzBwB9SU0iALJQ6yO4Y3X0FLJ1he9IB40P/yFN5Dn6O+qRteJrT3/zTNCj8CjIeAFhBjPDqxKqlnvQTPUn4Ib6JRjpOyLHueSH+tsTE5nZQHEOQz0vzGoXoMM1sokbUWpdyqJswXtU5Xv4OavWHg4jcrbEoee9O1QS63zKjh17kohxvNWSoBiT3ycaKGjKrNrPKXIyPzOq6idSme+TtCvsAX6NDMaV6yJnZyoYLhN8mDvyb1VP1kD/PJutcvbr2cx5hfj64shgw3QaPDee3bduCVDFctPB7ioV7B4gb8kL9wwiqKEFewPtkk0Iyq8IU0Ix65vYb2hj49Zaw255eDR9TeeWSwui03L6eM8NBKkNLaeuH7WoBzWYEuOzZLwP+uuQVmrwG4PQwE9++OmVkMQ9Fj/fFWpB5F2Is+jJx8MQCuZUFxBlJGEk1mArcaxUULJvoi1puFXmO533TYYG0J2hjCdGJc2HzB3yZhjurbjTqAhO1ouDEOf09VyjGMGvv94ECOJDpzRlzwn8vdYp1hGe6Jmr4x7WH6Bmz4xnZJr9+wJgrjXyRM3/HTHhV1GRg3pO2GQZtS69VYzug2A2vTFJGvzmuf+a4u0zAlh59u+mwkI3h2ckdAv8TulFxJPWhGZbEMmThbmB0H4FZCribSb5qbbDziEZ7s5HPsh2YvTvlo+uP/O02ZYX4iGv/dA+35TnymGou5eaxFT4L8CcUUOLjFurSg3R3hhvj30/qGrB6pAgGJwaWKef/4BczRIJgOHXf5TZPuhHZpknRhHah7jNUMZu7wKIhqef7ytygYY2jQd1ejCF9u1h6z+BkSKpey27Ns9w/3+u7JIUAJyfUrzKBtrFjDfGU7svNP78NEXItVppIqAWDh+LPuCchHGL3L9GJ2BWf5h5UQrGH/PUYucNL7+vpebl0EQp/QHUn9cXyYIeigjO9PTsbfZX5zh5pD/llIxze87c+WxFlgIwLpoKpPxmPgT4ZcvetJpIIi0QYYy0SSQWhu/GCaAYTh2q3DfLmBjZmqNZMbJ6gWIsHahH7jT4bkffBRISb88QmbNnXM1qoOUkLh9OlUpgNRL2nRuH8i6dfN8JHkSVrkdz+tU9njQzFNycKNJPBkZJX5DlpLiPsuBVGmrQQzKX6uSeqTpRJlCD+qZStCFlpMBLUOhTA2fK0GGUCIU+nun7FSEkVTuzCSClrBKfEw== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b07f482-74d2-4016-ccab-08dd529cd30b X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2025 17:26:06.6696 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sxB8O2RZP2VRZI75gKq3evCKAU8c+2eDdCTuvTnrzMF6B/Y0LYqofp03LUi5ApM7k0absGpgFrmvAIMy7m9BsQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU4PR10MB8514 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 21 Feb 2025 17:26:15 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17289 - The base class GitShallowBaseTest contains all test cases for the default fast shallow mode and the skipped fast shallow mode - The class GitShallowFastTest contains all test cases used with `ud.shallow_skip_fast == 0` - The class GitShallowSkipFastTest contains all test cases used with `ud.shallow_skip_fast == 1` Adapt test cases for fast shallow fetches: - Address the absence of an initial full bare clone - Utilize the initial shallow clone - Modify existing test cases for this behavior Signed-off-by: Stefan Koch --- lib/bb/tests/fetch.py | 49 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 43 insertions(+), 6 deletions(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 48b9e4af1..ba08cee1f 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -1739,7 +1739,14 @@ class GitMakeShallowTest(FetcherTest): self.make_shallow(['refs/tags/1.10.0']) self.assertRevCount(orig_revs - 1746, ['--all']) -class GitShallowTest(FetcherTest): +class GitShallowBaseTest(): + """ + Test cases for the default fast shallow mode and the skipped fast shallow mode. + Test cases must succeed with `ud.shallow_skip_fast` set to either `0` or `1`. + The class GitShallowTest contains all test cases used with `ud.shallow_skip_fast == 0`. + The class GitShallowSkipFastTest contains all test cases used with `ud.shallow_skip_fast == 1`. + """ + def setUp(self): FetcherTest.setUp(self) self.gitdir = os.path.join(self.tempdir, 'git') @@ -1814,7 +1821,8 @@ class GitShallowTest(FetcherTest): def fetch_shallow(self, uri=None, disabled=False, keepclone=False): """Fetch a uri, generating a shallow tarball, then unpack using it""" fetcher, ud = self.fetch_and_unpack(uri) - assert os.path.exists(ud.clonedir), 'Git clone in DLDIR (%s) does not exist for uri %s' % (ud.clonedir, uri) + if ud.shallow_skip_fast: + assert os.path.exists(ud.clonedir), 'Git clone in DLDIR (%s) does not exist for uri %s' % (ud.clonedir, uri) # Confirm that the unpacked repo is unshallow if not disabled: @@ -1822,9 +1830,10 @@ class GitShallowTest(FetcherTest): # fetch and unpack, from the shallow tarball bb.utils.remove(self.gitdir, recurse=True) - bb.process.run('chmod u+w -R "%s"' % ud.clonedir) - bb.utils.remove(ud.clonedir, recurse=True) - bb.utils.remove(ud.clonedir.replace('gitsource', 'gitsubmodule'), recurse=True) + if ud.shallow_skip_fast: + bb.process.run('chmod u+w -R "%s"' % ud.clonedir) + bb.utils.remove(ud.clonedir, recurse=True) + bb.utils.remove(ud.clonedir.replace('gitsource', 'gitsubmodule'), recurse=True) # confirm that the unpacked repo is used when no git clone or git # mirror tarball is available @@ -1907,7 +1916,12 @@ class GitShallowTest(FetcherTest): self.add_empty_file('c') self.assertRevCount(3, cwd=self.srcdir) + # Clone without tarball + self.d.setVar('BB_GIT_SHALLOW', '0') + fetcher, ud = self.fetch() + # Clone and generate mirror tarball + self.d.setVar('BB_GIT_SHALLOW', '1') fetcher, ud = self.fetch() # Ensure we have a current mirror tarball, but an out of date clone @@ -1919,6 +1933,7 @@ class GitShallowTest(FetcherTest): fetcher, ud = self.fetch() fetcher.unpack(self.d.getVar('WORKDIR')) self.assertRevCount(1) + assert os.path.exists(os.path.join(self.d.getVar('WORKDIR'), 'git', 'c')) def test_shallow_single_branch_no_merge(self): self.add_empty_file('a') @@ -2116,11 +2131,12 @@ class GitShallowTest(FetcherTest): self.add_empty_file('b') # Fetch once to generate the shallow tarball + self.d.setVar('BB_GIT_SHALLOW', '0') fetcher, ud = self.fetch() - assert os.path.exists(os.path.join(self.dldir, ud.mirrortarballs[0])) # Fetch and unpack with both the clonedir and shallow tarball available bb.utils.remove(self.gitdir, recurse=True) + self.d.setVar('BB_GIT_SHALLOW', '1') fetcher, ud = self.fetch_and_unpack() # The unpacked tree should *not* be shallow @@ -2295,6 +2311,27 @@ class GitShallowTest(FetcherTest): self.assertIn("No up to date source found", context.exception.msg) self.assertIn("clone directory not available or not up to date", context.exception.msg) +class GitShallowTest(GitShallowBaseTest, FetcherTest): + """ + Test cases for use with default fast shallow mode. + The class contains all test cases used with `ud.shallow_skip_fast == 1` only. + """ + + def setUp(self): + FetcherTest.setUp(self) + GitShallowBaseTest.setUp(self) + +class GitShallowSkipFastTest(GitShallowBaseTest, FetcherTest): + """ + Test cases for use when skipping fast shallow mode. + The class contains all test cases used with `ud.shallow_skip_fast == 0` only. + """ + + def setUp(self): + FetcherTest.setUp(self) + GitShallowBaseTest.setUp(self) + self.d.setVar('BB_GIT_SHALLOW_SKIP_FAST', '1') + @skipIfNoNetwork() def test_that_unpack_does_work_when_using_git_shallow_tarball_but_tarball_is_not_available(self): self.d.setVar('SRCREV', 'e5939ff608b95cdd4d0ab0e1935781ab9a276ac0') From patchwork Fri Feb 21 17:26:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3ADBC3DA4A for ; Fri, 21 Feb 2025 17:26:15 +0000 (UTC) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (EUR05-AM6-obe.outbound.protection.outlook.com [40.107.22.71]) by mx.groups.io with SMTP id smtpd.web10.29015.1740158771028340956 for ; Fri, 21 Feb 2025 09:26:11 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=Y/Gs6To2; spf=pass (domain: siemens.com, ip: 40.107.22.71, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XAVUMelK5OWnXMr8nM5CSojA4VqHQ7Ru+oKm64GSp7vLOflEdebaIsgcFHthZee1034WmaMMjVYHn2Lnuw0JzqWc9HSVmWBaYNcoA2/QUU1aIMwwmz+h2bKHdR60XxlYpT1ikCPdpnSSMLUmYpypWRKAvzuQP1bYEWsA35QM6jifyUMxnw6+9A+OEJY46WikbMccjYZBKTuWCO886i48vykQ+eowq3sJonJbYpic2LRYTrH5Hd7pKuNI3ZuxUL8yhDF4PpMOPjh1J4/dnQOWXOX/7dfA9FYGykLqDScSAp8bLYdlLvdRZVYUWkMzDmki/E8NtEN9IaH2W3VPuyqo/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Grmg1xFFB8GESgdgnHvP0EwaggoI+O4wp2OGbohbnkQ=; b=rwzH50AU10nDRG6qoB9O91b15m9EC4nveRvSsPqNLO8NUj/0gjuHhrEa5Gd30i45Yq50Hi20mkZNlkYrLIjnyRwAnpU0gorVf01NnK+vEPZWEQylMkJlwJXK8cFr5PH7qGoWFZn+J+Hw0AxsgnkAVf0gHCe9NFiEnBuEol/tEGtpI+aSfBiBjaxcSx9PA2+ZhYxDJ0U94KYdCConfEV5IwDPUvhsyhNeWy/trtYzS+UY0xhJscXGDfmkxr9ap2KEuhhRY1qEkCnY3HDawwmAr83aurSO+D6c/d1BlbzjYa+IR3O5AZB4ITcG/qg795ZTIdoXhU4mlc7Yrm8ZwMC/fw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Grmg1xFFB8GESgdgnHvP0EwaggoI+O4wp2OGbohbnkQ=; b=Y/Gs6To2oh+iP0eO0rX8YtWJyhnNCs2+L6QQ76xrV8MYSuWdphpwdFHOhP2GenSjw2L3C933Aj6Y3Tzsq3heOIfiuPSkc4y5pCMIQPC8+UW1SA4CWRHt6ytO9NCjwE9ZbeDlLma7UvmCfsODfAEgwpmnYBkpR00BMEmACH1MAgS1dUsJRu5SZbh3MKv+d7WjZxqSQXD4TYI790wvshWuWGcwpjLUldrxY1UAlxlY3DS6+oV5tdM0MK7r2VDwNeySWuZ60lRuxUCHhz2f4anQcsdByGIsAtaTBcHdxt6nQDH26g09xV4FtUVlcdmdKh2C9mkwUdRdOR1VK6AtPTNAMw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DU4PR10MB8514.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:562::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.14; Fri, 21 Feb 2025 17:26:07 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.015; Fri, 21 Feb 2025 17:26:07 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: docs@lists.yoctoproject.org, stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com, richard.purdie@linuxfoundation.org, quentin.schulz@cherry.de Subject: [PATCH v4 4/4] tests/fetch: Add an additional test case to check whether the fast fetch is shallow Date: Fri, 21 Feb 2025 18:26:02 +0100 Message-ID: <20250221172602.527924-4-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250221172602.527924-1-stefan-koch@siemens.com> References: <20250221172602.527924-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR4P281CA0044.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:c7::8) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DU4PR10MB8514:EE_ X-MS-Office365-Filtering-Correlation-Id: f5a1718a-e883-43bc-362c-08dd529cd392 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: 4mG14D9enCncCuNW4UD8lvlWtoEONfFhDBd4yr14OqvBP9vL8nVcHC90FjHrRTZ8ylJgDm0vS9wVci8K5+FV/d1lFLqWQpcMCU4w3iyvsfmn7g8YwWlwwKTCb+Xpa5lL02+JYWpiBTe4vKHts0akwiAHVTDad3GgPLXwY+yPY0+/77LH8dEFxXabhGUY/VbE4NtVh0wNeLUjAF+Z77iRa+OBUfCVUYpOo19A7+fLQVWmeRKnmsfRoJKHadOqy3GFHaXu54DzcWYhLZvedzlJVIpBUuRhRRrYnBXXHcBSWnMVbv9pFTum3s84cOkL8XPEP7PK/8VXfc1n/Pp5YAklUTbezcsmhQxIj0gzr6PYWJZV9uvZxa0RWDzP79/Rb9r/pRCFf236ZmyWLUeeH0V+/DBHGhTx0GnnZx0BdtNcigOc9CswIx7HG+fxrLH41xz1x3n2OvPTpYwDvHRc9lDOnfnuFQ105Pp8cjV69W2i59joatTAwZatWzNqDK3qRBaD+Oud9MiAWLjbnGQBbnzsKrRfdvE5sNR+p3aFTURgJ4LoPbMcxSAq4Pno0B9hZ3KqHdTOzPFn6SAmahORGAw07h28UL/Jp4PA7NgmjPBpYiMkYu7hanvx9EZaBga9QzrBoy5WDdZb79B+7NlHUWAUdpO4oDT+OYglc41b7jey8bKtzpSpTnVbuaZkxMus2rHqiJ0IcYZzdccSZo17VITuTWYDviMbLZOMi5ykRttm/6qWkqVT8MBOqILwO50Cajrphx3JMro7xdbRhPCRrhDzRuP0pQNyh8hNAierGGCijE5UlZZzntdszZAI0oIETJAjXNcuLtcR4rCKx/4HtSRQQWIyYCr8Bel9i6LUyS7gzEB7qdq+C/BHnmtYRdZzMq8ET1x3QK0EGMjGHB+NTzjCYZUxVTmsFKZ8IL7p0+7Ij84bPMw3C35sVXSVYzHsgFH3st+vtezFS0GdR3jgiasGY/HobduutbMB5da1aEdjUf8U/RKUUxqFtNUPpkRJ/vbj5IMuC0CbsqmeIoOPsVPiSwTd4zyhnd65T/8bnRa+adGGQefXSrUTHAgJp64gOtigUCZ+2Ahg7xDl331damTxeoSF8cDIV1oMJH/2oMUfs5oqSYBeLd1URqvPMdI2bL56a0aHConReCOL7swKa+FxZT3i+rYyaI4kudjEGaPUAQVaE8w+ycxxLdmcdvNaB1hkRijKFuq5SBOiBHUUISJevbYF6cVns2ZJgEynolDdFgzRRW5h6wnOhp520S4+nhA57zntSU1+KGAdmOkmn7gFdqePGZVdsGSXQ3ht3iHMiPqFbfc/Jkw0z6NBPvkVTIM1+bs1UB7mD7bM482LkjTTjxE/v5gfJ6tC4MoxF1aAj5q62LUSxARFCCdLnXunxzAC X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JJ40xijMd7rJi3HKjFoC5BcyTseeLu9pslqyhJOzb0IzU16mkvB2Ge/lKixFN9SgWAP5d4uNm/1MECSPv/X+9JPvaeEoM3GfrgJyxLaiNl/b1lnQ93fjfT7SraZpwm3HYvJl4d4CtpfL4HEiCeOhb95TNzyjUXOOFEy/V8bYaSuvoB0fXnbqS9VQDyxgjSWdhUuidrKCHjofAK1LAxHsEOZ+gP35IP5aop9GsFPSnFWw5gebuxn/PAqMcoXGzw83tpZFBHA6nQ1SG9+Tx2ai2Vgl/0bg6/DdeOhPRxNx0V74d3JA6+aj7xw5pjpZWy0JBbUnpGyPc1Zg/jiiFE8cdHvFhg1IWiS4FmBkuzOD3dWNtcJb8S+f7wkjEPbO+oNnQ/qF89mme/Gh1DZw3Q8X/r8l24xIGWHj9X99vL+Nv7S0gNKE52PrVoi5JpWpNJLB+s7qIIb5BkmmrGssHAeonHqAtDd4zdZihzmPAK4x1c9m9jr+HLhXdyUjlc5RdXfRTFpoQbPnqyFCC++1tvKasdpgQv2RC14D64wf7/+VFzGz73Pd50ygp5D2uUm6ZN1Sh39c2xR9QZxtNsHN6RRsKRHZfxEWxQTvalNxGtQTNKXbRajuIui40aEhdZDaHw5W8U/rNyn7K+n6h07IGlCsmzneIT5D9EA2SSyVVtfEesAtQgQ45ZMzepf3PHvV/hUAcBmQuJ5T9g7ozVCYoGZtfZZdpJiKkxx7dpmir+l6TcFdP6i7UiJZ1+y7gVNNwECpq3uB/UxJRz2gNcvFSBVJzaKYwy0ggUlRfP9kE3VGcLBnaGrH9Nti35pl0O/EczMdimD1WDkZ/FuZQSUVCs8U1YQ36XeDBMaXt3UOignjOjpOC5Yc3apsN8PVBDuyelKviLFPIGGW1u+UJvN3RExBRs+UMo/9l4v2PxFXmv5y7ULBoly5f+ZE8b7eKYlatqCqDK0qyRJdhl+cfhPuRy01BPwJDtECVwN1qDZE/y0U9WYdv7NVo5O2iN19c2IVlbgAruGuLCO1GZlQDsowRPY4UvEZsXwdCIdURP8JZoVUqQjFpFTjYmRCkCupx1uV9iqXG4+fnIQUnivlFYMfig/OGRlFBviOHbFoBSOMgSa01geJgAaMe2Zfll4+15Pf+hlkWwO+rU5YvvXOEDt0KgtVgDT9C3b5BmRMCevsJk8dKZseDr3mjorp+vgVzakhhKzwYzOqrTs3pqLBEX2+1FEgRVTC9EfpQKDKcV2P30HdDZC+8+YIYhfJo/Iokhd/4EemiDvuRaDPMGNIEuvglDcOnKYw+3Bnan+esN1gK+9y04X5qVAgLOwskFiNLBdaBeki6x5it4lXio3JJOJ2pV8gzkDpiviQYyNhfT4kVAROEtbCPXhlrvqgkOEcozbTRE6m2XU9ghNelZ7bgAKfmQLxmHX7yrP5TMLuW1A6oSAJ1RqrVeniwApRjSNcqWwFgxQlBm8zgeLEUUnctJ5HiQb5O7EvQiqAmnFilwFLk2r7K5I75BufOaIV4sMDhg7NkzWvj/V3F3/3cjAQ4GZsGf+eicX0GldwHXHVhqvBvGkrxaSQaPOAH/8cN7aqXG546zUzFj55GWks6tag8AxSNMhZUg== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: f5a1718a-e883-43bc-362c-08dd529cd392 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2025 17:26:07.5452 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F27abWkD0E5D+YTndnkQnHsCHONJJX/53hJeE6/oWAeOyln5CPnSvCFIAa6y1rzChdrwQOQlCqAIWnbtUru74A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU4PR10MB8514 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 21 Feb 2025 17:26:15 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17288 Signed-off-by: Stefan Koch --- lib/bb/tests/fetch.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index ba08cee1f..91d1c3c7d 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -2321,6 +2321,18 @@ class GitShallowTest(GitShallowBaseTest, FetcherTest): FetcherTest.setUp(self) GitShallowBaseTest.setUp(self) + def test_shallow_check_is_shallow(self): + self.add_empty_file('a') + self.add_empty_file('b') + + # Fetch and unpack without the clonedir and *only* shallow tarball available + bb.utils.remove(self.gitdir, recurse=True) + fetcher, ud = self.fetch_and_unpack() + + # The unpacked tree *should* be shallow + self.assertRevCount(1) + assert os.path.exists(os.path.join(self.gitdir, '.git', 'shallow')) + class GitShallowSkipFastTest(GitShallowBaseTest, FetcherTest): """ Test cases for use when skipping fast shallow mode.