From patchwork Fri Feb 14 15:50:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57336 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 234D7C021A6 for ; Fri, 14 Feb 2025 15:51:10 +0000 (UTC) Received: from AS8PR04CU009.outbound.protection.outlook.com (AS8PR04CU009.outbound.protection.outlook.com [52.101.70.50]) by mx.groups.io with SMTP id smtpd.web10.24130.1739548265353865883 for ; Fri, 14 Feb 2025 07:51:05 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=VwcECcvV; spf=pass (domain: siemens.com, ip: 52.101.70.50, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BZDcJu4XrkZJkuFiJzv7o2hvnfY6Amqm7l2DrO4UjWp1s1maoP+V1hgSbAr17CjKnIPwyO66VNJpPJJ4rerM5qvkB2YXFdlzojvFze3VsYij1E3RG5QX9S/ycc0X0+ySZQvZHdaVvgrkDdDmRWxw/moEBseNBmqgXfRB0IbvUUllfW2Aapn17S4Mitu26maxcTaXX38/i74l4sNz72ky5m8YljdoWLnEBlmWf3zikmnebwipREx8dZsiww1HSHB+RM76WdPM2Pz/lgDjOTaJADZTZ6Si0pU2thbmoBha1uULJbFs+H8+g/q2p64MHmWtGCJAJGD2NGiwaAv7KSdHSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pPPuFC3C4li9vlkLtECDcFuz5JBQzy/XjbHqPv+rJOc=; b=NhEY0pUePWhimdulnnMLLABcSqyrsImpP8a/V1kTiDpvHylw2DvENfwkTEfzPdy5EPL1rCCCDupxPc/UFPSfUuLMmOdMab4e4FdscvTsp1++J75KrXCj2SC+DhJTIloYrsoJ/hyZFNNwiuSNOgi0WwPvS8I3HVsmTgGufrkAAELtyHZBqp714SMEEQFWz+g7bXnPzZFqmVboah6Le9j7gbkIFeyGA3tlmNC80dnZcknnKhdddZiKP7hMx4icIPq1FIzslQKn6DC6gskbQApNTPAcyv/TXBMTE2/0kaRYP0rumi65Ntw8xZkCXvhvSg6D4Izab06VElI2WOEW4NRGwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pPPuFC3C4li9vlkLtECDcFuz5JBQzy/XjbHqPv+rJOc=; b=VwcECcvVN+SMX3YBOrKX9D8KixTixWI9VF9s/1iqgr4LzRXwKaFApOLuqpGxpDelaSa3N1E1e0PjtB/4fvD7i/UvQatwFeW8Bd1vtw/NFdtYqu59vobklDMLYz36hCrzAFaLtjj71K+1QixX16JpMGsAXOg9+bjLBkTZxuxOlL2+cf3qauJdrDI4mdMHVDtgNQQ1OuUvid2LnIEFi3/K4EfzLGCVm5LOSIr/L3kAEQuf0dFs6NbdDPKiRMadlVZHwouKxDxtbXCIOymwVWkQOy0DWoUlYIivLgmeah/hYZZbVo5XsghAR3SBSY2dTE2aPwizysGjzl7r70q8g9KBfA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by AM8PR10MB4081.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:1ca::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.5; Fri, 14 Feb 2025 15:51:01 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.009; Fri, 14 Feb 2025 15:51:01 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com Subject: [PATCH v2 1/4] fetch2/git: Add support for fast initial shallow fetch Date: Fri, 14 Feb 2025 16:50:54 +0100 Message-ID: <20250214155057.1748020-1-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 X-ClientProxiedBy: FR4P281CA0174.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b7::20) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|AM8PR10MB4081:EE_ X-MS-Office365-Filtering-Correlation-Id: 845f7e4b-86a3-456a-8bbd-08dd4d0f6160 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: 06eiNIFeYmgoXN/eh/FezMoW0wFjUTOMgAkBwC4cmtA46MqZqJFnYokPFpzmOaboM2PrCvZu8HlWdyMrMXfh6YlFX38BLMHFnFYs4gc+VRJOSHJX//AEJEsMPWklTSRfwbwegohKVX5OvoeXKiwWYJOoWidxT+6AXs7rfdBkNSRaEp3o1jVdD7B1lkQyVYQstRU3L58FTUCOtqzYi5iFviVCX4Q76nf7CRwW8pbXsm5oaUyF0kzdKjuJ2+yxI09/e0Pf/qVFXkq3SjtgwxQFfgP6zeRGWJg12ZGCuvSQJSnPDEnVfEm+QbD3KXW4C5zVc63LVc8H9NYQ4YcPFhKTuMb/WgwNz3fMPBLHG9l2eR4Z9+lA4dGUdzBmxnmsL6YCIFm5sE4WIP/D+Z09Qbn7/YXjKqhFxXF8T6NfvjCQtgEsyTqysw/w96eXXMWOB9Vlhj2ZUJL2SGqTTdpSlXeVu1yzOcmW2ZH7Lb60ipvlvska8QyoPCEXQHQn2pPcvqM/hWF28rBwKvLtXHWxpeIKWJwAcgEmMvel/o3JPCqiWIgkyCVeFCdgewRc8/l1aDPW6q6iS2EyaoIAXyY7TmdVX48rhLLuzfiKI4QB0zmc+Ot2gEYM5gvIm1nYqjO5XpvGENjobW7LLGUsVInSVIzhKZeZ7o99mGwsyZIQAGgibdI8uH1KlxMQ9aUrMR9HSGS+LSmWP7LchLIvpd3fnHZjN5m/2jLh3bbAqOuxoZd/cA3EX+oolz9jvLqzFem79QL7sTk9oRS0fXGU6lhOVkoSxh/XF+6wO1+oyld+TfEQje1DhFyH96LIthQCX1AK/qskbgJ0eW565eMDQD2s23wO6RmYn0D0EQRh+f9Zf6Ob9IxZHzD0MbAsAbkhMMJO/YjiJxgbFubegyedn46Rplysi3R3ojEoALd+yRs9pNzTYBY0GhDR0/hwEeYL/04RkFAvEPXvv5IFkpcv9eLfnaePbg2KgeZRDrTwcBbi+/EaJNDWzcJw1c0DjT0SPues/PwEJmC5tYUXeIw3hM4x3On48y0fZF705Jc8xFSNteuoh0oaDpszbYYHl/WoA8WS7HSg1hCqcLsjVXWAqonEDW/cDhs+YibYOpJuMQSr1RTIeC8o+0jrPRfZ/bpX6GBT2QUEzJEaXjAa9AG4Iv98INF0X1VP0oyUuOb1mbjMYXB5qwnwDhkiCWLnlEW7E6KIg6safhoyARVQRpMRWAgV0lCHTFUyF69Eb6PvQ/F5wLXZnsi+RioOuzCmjmBBE0saLJWV7NOQMHviia8iSmNIiL4TE1AFbDa+xlfHQWcNaLZcX6H4dMcfYzaKh93/7QBDS3r4G6YdFblrIjlMhi5kkmoLZxSj+uFsAJ5NsGG8QeFLO7D4vrAqHp7UNISMCH7TCwB7 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: w0Qi/K7NRnmMxnZKhKIeCltXHkxvknhjHHR608UCxDl+cBpKlNodNlBIXHQypiMu4708t0hRJ4qscDmQJSexc+ish8TT+dlm+MShi1wxKSsh0xIxjbOhYmkI7tOOuc/soRd0UFCm5xaw29OV0MQCbG356U7LV3YwlhPujhPrj9YEjplbFBxPOT9wxFD/5YJb5icV+aY1kc4gyV6Q8e866BNI3+8Dj3gdh33n16HdhgEvJUxNSUbL+iw/mXWsbtISu85msXbq8xivTeDEWn02z5kx05H7bcUrZdyHzq4AtiWF2rmoMmztrvWq8J0yt5EyF09P44nJNCrm2YwAAwlqlBMSWs7lMJVtO2zgppsTffS6/F7SmOPj2qP3yudQS+SBvoHvnd5xROdYgyQtSJz86gwWVuWJfJOUvC3DLvejTwvrZ8NwEdvp3LEIPBd7J3s7hu9hZ8BhvaIZzL8wKqSPmxc5v0uMtJozyJdwurzPtKXUQMUy0UV7KMuU3G9n0FQ09TdUbedkykvNC6weKYLBXdzPCJyVkUTVIQ6xK6DIU0Mzwc5f3d0lD3HQyYDVTuY7eyjqofiV9Wv9zR0jrl/e1mV4poaJQpU422dHQT5Ki4R822+WIeYrcBpExLOyMxx/bL2LBBmZadSUDmc9uQo19ZhPQ5OWKlB3owSXIe0yju8e1VqokXs88O8QE5HhR+98MlEAs5UKkxoHcZS4fiF5CcT1Jb7KxbAiojMCg3V8TmcCHS562zQG04rc5sNPJcVWQH+SBgOn4Jdev10xxvccEpaC9evYtuZwAr0bO8yJkrddNhjhBY11liUMnRXFD4FANo8pCrt1bDFz1TSaPbFKLmrHWQXGOCmBMB7ie2x2u3skpanIeM+JmBHUJxSRaiGZ5xHtLjkRWjlzvnV7kfFXdt8fQoM1wYsFlCv7x9QAEJHxgGodFVbtx6YP9ph2gzmZBE3qfQE6hOyHSqpWCZBV5IGPxsq+JLwOX8i0GdtY2lTDjac0lb0Wq6kuktIBbdBk64nAWVPWcftTMhxr9v1m/4WpTMGhOl+/jgUunJOqWU2s75kQsO9PA8jXo5FIaG7W51hI1xSDcLcwgQtJsBtx6+hIoDFXvjHWLtcjkC2MO5ZSlb1yL7k7r8F1IQVxQnQipm9njeq3ox/ixXQ8cDmhAZ9QAn06GbIJ3b2ieWyqyEl8yTFMaP1Obo5P04wrfy11HqALNBGOGD8QVtY35ULUpbXJAwjH2djaosShyVlu+JrNKrIn0IaBAzsKoMLe6TmAY4flNbp0rd9CAwtGu5tAP6S/2VKzgEuuyx9YFdCzsiubG4ZCeYBsYqGORwMGY8eyBTgDEy5OHmBxoUaX4O9C4YE4DOquDqm229zUUS/0knJCMu6eyde6nsoj50aQuc834ovGbXUG/ktb4JZDNDIIoPBtCqCvWuYSzUF+cp5YuvyO0fLy/dnuRkMjmsKCjfoR+akkHvpNIcoTdb9zP+F8YxprqXeL1Hez8l98617HwfzLR22E1EKfebrrRHnsIC1xd4JNwG6QuxgBJAM0D+NB9uLQ1BZxd3WBxfAxqVQ0tNbjK0qisPV+ePwLf4f5bBstbkQFopykvNiyobVIf+fvVw== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 845f7e4b-86a3-456a-8bbd-08dd4d0f6160 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:51:01.1027 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x0wOq4oFsT/psb6KaJi//794Y7HaM5RSMEL4t+QF5yJcod68HrvLYsbhfqsXaAKx9vxfRkeen7WDQnad3pLatw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR10MB4081 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 14 Feb 2025 15:51:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17223 When `ud.shallow_fast == 1`: - Prefer an initial shallow clone over an initial full bare clone, while still utilizing any already existing full bare clones. - `ud.shallow` will be automatically set to "1". This improves: - Resolves timeout issues during initial clones on slow internet connections by reducing the amount of data transferred. - Eliminates the need to use an HTTPS tarball `SRC_URI` to reduce data transfer. - Allows SSH-based authentication (e.g. cert and agent-based) when using non-public repos, so additional HTTPS tokens may not be required. Signed-off-by: Stefan Koch --- lib/bb/fetch2/git.py | 96 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 75 insertions(+), 21 deletions(-) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 6badda597..a38f6bf0b 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -207,7 +207,8 @@ class Git(FetchMethod): if ud.bareclone: ud.cloneflags += " --mirror" - ud.shallow = d.getVar("BB_GIT_SHALLOW") == "1" + ud.shallow_fast = d.getVar("BB_GIT_SHALLOW_FAST") == "1" + ud.shallow = d.getVar("BB_GIT_SHALLOW") == "1" or ud.shallow_fast ud.shallow_extra_refs = (d.getVar("BB_GIT_SHALLOW_EXTRA_REFS") or "").split() depth_default = d.getVar("BB_GIT_SHALLOW_DEPTH") @@ -253,6 +254,7 @@ class Git(FetchMethod): all(ud.shallow_depths[n] == 0 for n in ud.names)): # Shallow disabled for this URL ud.shallow = False + ud.shallow_fast = False if ud.usehead: # When usehead is set let's associate 'HEAD' with the unresolved @@ -366,6 +368,33 @@ class Git(FetchMethod): def tarball_need_update(self, ud): return ud.write_tarballs and not os.path.exists(ud.fullmirror) + def lfs_fetch(self, ud, d, clonedir, revision, fetchall=False, progresshandler=None): + """Helper method for fetching Git LFS data""" + try: + if self._need_lfs(ud) and self._contains_lfs(ud, d, clonedir) and self._find_git_lfs(d) and len(revision): + # Using worktree with the revision because .lfsconfig may exists + worktree_add_cmd = "%s worktree add wt %s" % (ud.basecmd, revision) + runfetchcmd(worktree_add_cmd, d, log=progresshandler, workdir=clonedir) + lfs_fetch_cmd = "%s lfs fetch %s" % (ud.basecmd, "--all" if fetchall else "") + runfetchcmd(lfs_fetch_cmd, d, log=progresshandler, workdir=(clonedir + "/wt")) + worktree_rem_cmd = "%s worktree remove -f wt" % ud.basecmd + runfetchcmd(worktree_rem_cmd, d, log=progresshandler, workdir=clonedir) + except: + logger.warning("Fetching LFS did not succeed.") + + @contextmanager + def create_atomic(self, filename): + """Create as a temp file and move atomically into position to avoid races""" + fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) + try: + yield tfile + umask = os.umask(0o666) + os.umask(umask) + os.chmod(tfile, (0o666 & ~umask)) + os.rename(tfile, filename) + finally: + os.close(fd) + def try_premirror(self, ud, d): # If we don't do this, updating an existing checkout with only premirrors # is not possible @@ -446,7 +475,40 @@ class Git(FetchMethod): if ud.proto.lower() != 'file': bb.fetch2.check_network_access(d, clone_cmd, ud.url) progresshandler = GitProgressHandler(d) - runfetchcmd(clone_cmd, d, log=progresshandler) + + # When ud.shallow_fast is enabled: + # Try creating an initial shallow clone + shallow_fast_state = False + if ud.shallow_fast: + tempdir = tempfile.mkdtemp(dir=d.getVar('DL_DIR')) + shallowclone = os.path.join(tempdir, 'git') + try: + self.clone_shallow_local(ud, shallowclone, d) + shallow_fast_state = True + except: + logger.warning("Creating initial shallow clone failed, try regular clone now.") + + # When the shallow clone has succeeded: + # Create shallow tarball + if shallow_fast_state: + logger.info("Creating tarball of git repository") + with self.create_atomic(ud.fullshallow) as tfile: + runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) + runfetchcmd("touch %s.done" % ud.fullshallow, d) + + # Always cleanup tempdir + bb.utils.remove(tempdir, recurse=True) + + # When the shallow clone has succeeded: + # Use shallow tarball + if shallow_fast_state: + ud.localpath = ud.fullshallow + return + + # When ud.shallow_fast is disabled or the shallow clone failed: + # Create an initial regular clone + if not shallow_fast_state: + runfetchcmd(clone_cmd, d, log=progresshandler) # Update the checkout if needed if self.clonedir_need_update(ud, d): @@ -509,20 +571,6 @@ class Git(FetchMethod): runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.clonedir, d, workdir="%s/.git" % ud.destdir) def build_mirror_data(self, ud, d): - - # Create as a temp file and move atomically into position to avoid races - @contextmanager - def create_atomic(filename): - fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) - try: - yield tfile - umask = os.umask(0o666) - os.umask(umask) - os.chmod(tfile, (0o666 & ~umask)) - os.rename(tfile, filename) - finally: - os.close(fd) - if ud.shallow and ud.write_shallow_tarballs: if not os.path.exists(ud.fullshallow): if os.path.islink(ud.fullshallow): @@ -533,7 +581,7 @@ class Git(FetchMethod): self.clone_shallow_local(ud, shallowclone, d) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullshallow) as tfile: + with self.create_atomic(ud.fullshallow) as tfile: runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) runfetchcmd("touch %s.done" % ud.fullshallow, d) finally: @@ -543,7 +591,7 @@ class Git(FetchMethod): os.unlink(ud.fullmirror) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullmirror) as tfile: + with self.create_atomic(ud.fullmirror) as tfile: mtime = runfetchcmd("{} log --all -1 --format=%cD".format(ud.basecmd), d, quiet=True, workdir=ud.clonedir) runfetchcmd("tar -czf %s --owner oe:0 --group oe:0 --mtime \"%s\" ." @@ -557,12 +605,15 @@ class Git(FetchMethod): - For BB_GIT_SHALLOW_REVS: git fetch --shallow-exclude= rev """ + progresshandler = GitProgressHandler(d) + repourl = self._get_repo_url(ud) bb.utils.mkdirhier(dest) init_cmd = "%s init -q" % ud.basecmd if ud.bareclone: init_cmd += " --bare" runfetchcmd(init_cmd, d, workdir=dest) - runfetchcmd("%s remote add origin %s" % (ud.basecmd, ud.clonedir), d, workdir=dest) + # Use repourl when creating a fast initial shallow clone + runfetchcmd("%s remote add origin %s" % (ud.basecmd, shlex.quote(repourl) if ud.shallow_fast and not os.path.exists(ud.clonedir) else ud.clonedir), d, workdir=dest) # Check the histories which should be excluded shallow_exclude = '' @@ -600,10 +651,14 @@ class Git(FetchMethod): # The ud.clonedir is a local temporary dir, will be removed when # fetch is done, so we can do anything on it. adv_cmd = 'git branch -f advertise-%s %s' % (revision, revision) - runfetchcmd(adv_cmd, d, workdir=ud.clonedir) + if not ud.shallow_fast: + runfetchcmd(adv_cmd, d, workdir=ud.clonedir) runfetchcmd(fetch_cmd, d, workdir=dest) runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) + # Fetch Git LFS data for fast shallow clones + if ud.shallow_fast: + self.lfs_fetch(ud, d, dest, ud.revisions[ud.names[0]]) # Apply extra ref wildcards all_refs_remote = runfetchcmd("%s ls-remote origin 'refs/*'" % ud.basecmd, \ @@ -629,7 +684,6 @@ class Git(FetchMethod): runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) # The url is local ud.clonedir, set it to upstream one - repourl = self._get_repo_url(ud) runfetchcmd("%s remote set-url origin %s" % (ud.basecmd, shlex.quote(repourl)), d, workdir=dest) def unpack(self, ud, destdir, d): From patchwork Fri Feb 14 15:50:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F78FC02198 for ; Fri, 14 Feb 2025 15:51:20 +0000 (UTC) Received: from AS8PR04CU009.outbound.protection.outlook.com (AS8PR04CU009.outbound.protection.outlook.com [52.101.70.23]) by mx.groups.io with SMTP id smtpd.web11.24156.1739548268282327099 for ; Fri, 14 Feb 2025 07:51:11 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=yUAd1BV6; spf=pass (domain: siemens.com, ip: 52.101.70.23, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KM+lVIJB66LSGAV2CCYOnSuKxtjK8k0j2Ef1l/dsGINFGZjaiOYm/pkCMROt2pweobygCKFpbtbd8ld0/S2bWjM/Ne/izrG/CbTAwfyMgixOZGkwxZVPrK86TOgRfrsU3EE2dCc+6R8zV2oA20+hEe4jsdgZ0tWeVFACBG6+bsKLXVx4juEBs1N3K+dQDUEV1pbUr+CBIeOoD0O5avI8x4dI5lZUDALY1Glvqcjt/m4891jRESDUZO26JmwA31fwvI7vgG+uxUdTTw4JXA+uJP5drF+d92cWIIunM8vqtMEEY0O2E3/Ki2jQdX/deihcs4FmeKPQt1fy4YxBUEruwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yMHZEUs9RjSe0xLnP5YSeRYDmEwtQzadq/I9IQzHi+8=; b=YTDFsmUCVgiPtsmTzHT5qzRvPm0Wt1asiXN1wlu1x5Zlt08zI/E34DeYRNQQJH8svkqYH81v0ZWjjnLchLO0NLKxO6ADYe+uc8LLje9oIrd+iDmJ2ruj7YbkBs46cyDzq5djlsll+vLSlgD4dJrZxAX4cXk3lSVLF5DHUyjoz7GlKtUdrTKqheGUgogOy06dn3pGLCZPwBY7tylnJXkaHWHZMo1AMdjV7lga4AVXJ/NLuCy2BGg3DtPrNr+jahEwAOkUKl4dWoFlr64hWG9nbnWeunLLLYxVb4UEJ6I/X1iCyA7AG98YPO11Mkrqj2NMUi1jBG7xu6m5uhImFWyk3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yMHZEUs9RjSe0xLnP5YSeRYDmEwtQzadq/I9IQzHi+8=; b=yUAd1BV6WwOoaaSwNg/VNCLgXjm6T1zkAEFqiGY9fK1D6987scHlZBGL82BpQNjpeXiZzPVWP6kNrbiZhiwolkkA6cO16ZAOf150InKVED5BHICGKajhJM83A3AQsbYeTQY1Y2zEVh6s0x25nk2li1d1ugnIfGSKhppx4//kfPc+dyAh4pfiN/bcXiILqVJJHERjXu/i+QF+msvyR/5g70dxPInA8hZq+l/nBDZ4SseqDCC4MAvsV1StcKRQ5WQZ85uSsmpxYMQyGQ7+8ZYsdlZLiivRR9yFiuiqq8LUBElMKN6kgmf+r5LHqQH1kKrf3Q4CVBj7KiK2pS07PjSx1w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by AM8PR10MB4081.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:1ca::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.5; Fri, 14 Feb 2025 15:51:01 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.009; Fri, 14 Feb 2025 15:51:01 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com Subject: [PATCH v2 2/4] bitbake-user-manual: Add documentation for BB_GIT_SHALLOW_FAST Date: Fri, 14 Feb 2025 16:50:55 +0100 Message-ID: <20250214155057.1748020-2-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250214155057.1748020-1-stefan-koch@siemens.com> References: <20250214155057.1748020-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR2P281CA0157.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:99::18) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|AM8PR10MB4081:EE_ X-MS-Office365-Filtering-Correlation-Id: 9424f15c-0543-4e44-d60a-08dd4d0f61d5 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: anWh+9DrwwmOry4jhTg3eLZjmmFeESnd749vL9pFYDwNv3Sxy06kgOadDQvIwvOPpVrpf+QY8P9uoo+lH+BYprAAZFGvI/KmmX+m2GmLhTOZvpiJ9SgIkK7B4ZXdZuqMFpdmJ5doMy5GfzzffSHjr3GAjWJM7jsY6Xayw1QPY10vJRupgUCVu0VmZAurFwE8c0i0FohuiMtE6UMTdG9V/ZUqgpZypd7eQabgupKdO1FvHvBsMDMTtVKM89Auiicme+ViDHpneXMpkILKD8azJWPT+dOGhiN8D6xhTdjlU+/EkelAjKWAyBmtWYSiHhgDOkL9kVEJECofdNFFnbDkhVe7HeSvGyEAAkdSsSy02GNoGyKPp31EbMqZ4hMfTHTLuT6aN2hQZzY1wFIdqUNf7hjpOGABrTXWlP9mw2DX9XIPS1+Eao3M2/NS7oiHOMkle/irhJ94Vz8X4mCizmS36azj/aRZo9Sxt6t2A2bMhVCbCcHPT/kS2BhYh9Yk/7akfMVL7mC7ETTzpA2zfdHRVcPgd3+5ITxr3dmOIWurI+zKcBHyGQ9V7iVwpGjsm5HUcQY+yXMrKJ2LEK8nRs8O2q2EAFrXe/BKQAA1Nq0LoT/3cC7Ts68MVeJBR0J845xsM793vJa1rLHLkmle8Onm4jR38ai/Sp2eSg8P7O+OR5PPVZEm4jTh88lhrXqMykfd4Vw653vxmGE90Y9PJwQdbjfocWvlAXZCFYcg5wPkm519PWhWZDj+H7gi12Gab+Ggocy2vhIUcm5ilT8tplynTYBbM9gObCRMcFp1uH1tx0jKtkEJlNagzmnSXUZC7FiPUQSWNlT2Sv+r2LsZEp89NJNLbHGeJho2ToK0KqRYTfic6kxosAgtPf64/accdnbYE2i+uwPyFkR2KAhgqSI5cOMQV2/VB0ewSRRku15w18QaTopQqc55/z7OYvv5XyVCreGBhWLJztvTEyHaGXRn5DjfZw//zfsutXdOqK2VVS80ZxYJPJwhZop91xSk+GIVUDc+Qxb8GRQQ8SJJRoNX4Dmxn+0hh6KYOw9SIeaac0I3L9TDbZF6RUZ+rtJC90nZ/U5ARyStz9OnE7KF1+XRPGRjZQQOngVb7rntAxpTIWtQ4IBNrKQd+RJHM9vcuDjABoCLJF76qR+RqBxBi7cWH/hovpo9lXVQrPywBqQW+C1KCu2XgIG74Ld6O9x24tKwIoJfNEU39HYa2FGNS3HNKHYrKHY/CUlJ5ejqcmajbhjYGznKBW19yKsFh+Y66AAjTDR8CriTGkfN3YSB44RQ2ux7eY0extPJVVj84mHwqSwQ2M4qhdwn4Z14jtEBkUBf2Wv2BgLJbKIG6U61rRF71oVq91MAMy7sLiUPAVbsyx2QZe+mA5WeHiC82D5yf9/Y X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IZcM0ljx+MENBOJSwxx04TV/B32bemOQQv6ptY0OSos+PFBpQy6CC1/uLrBRk9p0hBcd26CQ5UAMsbrb1g15l/qb+k6FE/LZeiq6wZW2sHSlgpAzoBwqMlFPGBvkjei+qWOmomIHqc/ZevDPQC9e9NTMTyC0Sd5LPDk+39IjqHcsOa+8dOAmgn5+ECXgtgxmJBYn+8IFtc5zTLyoTZPC68oJvmj8MICxd0VnqOXCI7nAPmhEgnbhAHiwMY3LU4/bH9nwV+H1qMr5G5pRw2Za/P3DZ4AiDmGpORo/SSW/yrlObX68UJWQbbyEOEPF/K4gxGXI0CD+ntOIyUnaPulIUmGPlcDHT/oXPKCQDh00e+vPn4cjZlSmQ8gvL1z28P5yBpDpdDsFntdi0NoYT5HijNCVdDsrQYd9ZIvqnlK9Uv6hrHPHJ+otyo7UMzIcDupHAv02k9M03+mc0cu7L7elanRzvkILObkUrL4+GeD7ytnphr6Tjbh7CDXDZY0OGo1J/OczkG3s/OSeWs+GhpqP0hmjELKInRoIBPKHIPoUKF/dDrk0zrQyOxO4wn1PD3N7OOQB8ZsEmCrCdJP0BXnPeJcj0YGxxAPd5QQtxo41WMXFHVcXtHf8qsIhfm1j7DhGyx5WPWSFvbjTZpthXEJNzZ6Rh5Ns6tR07IcEdL6NiCfBcdceiKJjAJ7kYAcnmw4osjEs8vW9Qs8eCTBqlZTaUol+mBycvf356S6Pe6UP7oO34L4PdSMq7OSquLHvdAOVykNsiHMjE9r7vfNWcoGNXqDhso61i2Vovbohe3W+bP9xa0kbYJCN3OKl7aDFmGbJrB9PGbvJi3IigYuvxAzcW8l7Z6+gx7g0jc0kAdp0pBg5Tmng0ipDd0ISYpMZkmXTLdxQEpX/K6CvaWaMaDPvNc3ZtK4B/5sBQaC09qT2un7SXF1AgZqdImnJnokgrLqvVzLzMuLokbbXvjYK0Eqy6J0qLSiMlpUHikPI7f8RTmkOfO+2fykLTpAkxo92sW8GW+ifsAYvNafZa7HDSCKvSPpcte89p9I0pN6Jd3IZL278oy6FKjHFTVpzuR9LqAr2GklAbxpeYGEr40n+uO5MKArZm8l1XAPPQjrEaD8g8eZj4wZ9TLATKKLIX5ZbOjvQI/OSyUTarLDgZGO5EX3RPZjKF0KTZ1xrqce/bgtnSmQz55C2miJobvByfEGOPBSikwxCj8VKTHsJ/m41ovF45ra1S6HM1GTwWQpDAVI5HXBsF+YOyI0Rv50XtTODwRJy9e1z1huWkDcxgAM/DLS3hw9Oj3y2K8zWM0ismB6MqZa6RvMO/6tNEOLUqsDJgFfsL0b7/Eeg8HwH1k7w3yxJSMwYoxfIZJgCNWM54AGT/65gjJAWjp2y+JY0mz4QkD8msLu+0sbBePO4qYFcx7OvsAwa5cutfXaugQdIofuZr/iyWHeoP/WS8HGYGOono3f2Orj7kudopuwCSYfdhtnS9Zye4222jkXzBnBOjABPOmEo0uqMNnCwNPx6q9hEMAC2RbzVw5tT9mb9Sfe4gNUDe6MdtsWVNenmg8UQdAmdlnv6Ncal7NAv9BRoAIOrLSIXZnelV8i+jtpSeEYw5b4+sQ== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9424f15c-0543-4e44-d60a-08dd4d0f61d5 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:51:01.8828 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VS477Uw19jNpeVAWw5s3vfPxx1dKKcDgUTLnCvBuw2yZ06r18x4duZaiyz6h3DK/kfx8PuR7ySZG1ofe83KyoA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR10MB4081 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 14 Feb 2025 15:51:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17226 Signed-off-by: Stefan Koch --- .../bitbake-user-manual-ref-variables.rst | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst index ad219b531..bba6cbcd9 100644 --- a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +++ b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst @@ -347,6 +347,23 @@ overview of their function and contents. For example usage, see :term:`BB_GIT_SHALLOW`. + :term:`BB_GIT_SHALLOW_FAST` + Set to "1" for a fast initial shallow fetch. + This setting prefers an initial shallow clone + over an initial full bare clone. + The amount of data transferred during the initial clone will be reduced. + `BB_GIT_SHALLOW` does not need to be set additionally, + as `BB_GIT_SHALLOW_FAST` takes precedence over it. + + However, for updates, the data transfer may be significantly + higher due to fully new shallow clones. + Over time, numerous shallow clones may cumulatively + transfer the same amount of data as the initial full bare clone. + This is especially the case with very large repositories. + + Existing initial full bare clones, created without this fast option, + will always be utilized. + :term:`BB_GLOBAL_PYMODULES` Specifies the list of Python modules to place in the global namespace. It is intended that only the core layer should set this and it is meant From patchwork Fri Feb 14 15:50:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E4C2C02198 for ; Fri, 14 Feb 2025 15:51:10 +0000 (UTC) Received: from AS8PR04CU009.outbound.protection.outlook.com (AS8PR04CU009.outbound.protection.outlook.com [52.101.70.23]) by mx.groups.io with SMTP id smtpd.web11.24156.1739548268282327099 for ; Fri, 14 Feb 2025 07:51:08 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=kKt0IjAu; spf=pass (domain: siemens.com, ip: 52.101.70.23, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TvpE90aG+dKd8zHhUF+6UhaHuB7WFK0TmoEkX1GGxrUAIIEFATC0vcDl+xrH1wMys7n6KKaaIgyb7nSgHcRoV0b1kADFCWQOn0jfoK6lVbRCsRJ15oeBGnldXoCjrGkG882kjrkwRhOINYEGmc+SssajUHhrB8i1N/PB3KectN7X1ArotC6T1uHvzrgz/MawhFqzk1m0B6C2sBfB8YZ//a0N8UPrN+KD2f9JcBLA5DKi/+K+gghuKDhcFwfRD8NyQvKW8kw5moZ6NIOFOy2E9Xr7oDXcGub+WFSNtTMFQ5w5/ibKFANdlic/4e+yhSWnR+FPwLs6lG00aUeD9yFUhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yiyXOuX20uy+LH207lz6VlG7K2RapUL0oeXFrYdf9AM=; b=UD+HcD2spSZO3l2UfpPjPtjgtkMP8BNBCfaectS3qfVXH+QJe9/wdo4BmW7yfVWIxTVNkqN3dR4xF+u/WRRQz5VsuHCyrPZXydQA6vXWmdEqOlJ5AahQUT4addlsMySI+i3MuoRKzlAth/UNmGhzfxdgFvZyzJ/2X7V2PDOm6oCXk7x8KTNHJi18vIeugi4yA6qLc7oATSOW5SbEgjoLCd+pHGT5f6a5wUWwbOUnjEw8yPVrsFqgP7HlLRum2cstKcvUyNUXujduqgLRHVr4luJqkhyQj734hur1MtuwhBPwcErna9WUAHW/tbSLtKdI3dTs2qbKjITBdrPXZWAIhw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yiyXOuX20uy+LH207lz6VlG7K2RapUL0oeXFrYdf9AM=; b=kKt0IjAuZUCsMU5tRhuhAV1rrgfjqbkvTlb+nW8oeYmGCNL/WrAmiljcyQuPYPlLa6WC5fQ+8/ybAs19bX4mbJNUeXM4nDB0F3s1us+qc1gmwYkIk4J1+rNO3oR6t7I/DzCc+9LgDwXoDiGPgngZEBXL1LD7Bv5OZXSUjk/bM++WviFIIeD42eYpO3gwreIirG8k8D4r/YopTE/BPeidaJTSCk/ST3ltH7jIh2/tbqmryVk5S1DtwakNUbFcJyoYNziM9S8zw0o7Y7Zde2BtHR4rO23GBmqXKnpxaGK3z8ajt3jdOfmjpMj8g01QWX9TS+YAp2llv+tDWqVAF6+uSA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by AM8PR10MB4081.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:1ca::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.5; Fri, 14 Feb 2025 15:51:03 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.009; Fri, 14 Feb 2025 15:51:03 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com Subject: [PATCH v2 3/4] tests/fetch: Separate test classes for default shallow and fast shallow modes Date: Fri, 14 Feb 2025 16:50:56 +0100 Message-ID: <20250214155057.1748020-3-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250214155057.1748020-1-stefan-koch@siemens.com> References: <20250214155057.1748020-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR2P281CA0158.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:99::11) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|AM8PR10MB4081:EE_ X-MS-Office365-Filtering-Correlation-Id: 4df7f69f-5ac3-42fe-82b0-08dd4d0f62bd X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: nK4V+vxPly9LEKUSzfwOguCIagQ6WwbX6aeTYTsYJhHt15Nt8cvWriYHYEQgfeAV/WadNp+qI98xdsvnewojBZ0/+/C6nth6ROkV6+teFDdomV+nagTw/xyyyqGT5dMeRjpGrUD1xVAgrUh73z1mrdUzaXbOMw9v82B/jtMK8WEE1F+3qRulhhwoKIhxyGg6z2lrJ7glTXrViEk1m/1WStNjfLw96IZMBlKnILY23bHwQbw0UABU22f7HMom2P08/iXbykdV9+HLthd4pgbFaUN5dTUB/O9HhbGlpYwDLdNkkBzAtYF4+uSnKroVn3IvPIUi9k/oUg6av+JC02nAqoUcI1eVlvlwr+9B0gY1xQ7cN1JOZNM5XE+c5fxGGB1lAOEbDqMrTUx6zxjOwCQSqVfN4a6GqE48iyOCTohxy3h9ZhjDBozpsXUuMsDYsLU7tQyqpiLP5iy6CGL0HMGmdX2n0ilvW+24HejGLvlIXn4TpvHiwsZmtqqu1o3Z3h6NB23zxicEyll9RnAKLAbu7rV75WPtJl0YaNyekUw2WzwBSLM1VY+2eRoJtWOegq/QlObqclAeyLdpnlwKdVTjjHhpSrnt3a2+B0GDf1U+ZWDm9oB4CeTPRqz7C0YIVpxC71oDrmom9tHTOJPA1HwovaIKNiV5jLOb+IUZ0q986jDrQPU26g1AeV9iQki6XV2k9cavKMeJHTtnqwqtfW997tS7+gIXZnIVUVsYFuFrWBVfq+CGEeaz9nMTd9JRGgynxvfPDPsE2B7+/m8uyy0qu4KDO49s/G+aZSb602EgqtBo+hInxBIxhO11jdyVhdxZ+6ruuMw2Rz8puqJ4CUinj2arZRqPIzCn3F0EPzR7ChkLzTDDLRDp8rhkwcA0vMavbGk93Shu+25FkOKdUpn6zGZ4oUF0Z6IKkIi6qQizdewc9Xfc+wcuBFmrHZaUdjNat71/PxslPn8qCiK0WSmZxg/cTZkikIagBAggTWme9BR344r8Qkgnnc5D4HK7pJHF3qk4igztrCY4tWpIpy5m7YvDIGmt/He8YvWnbN7vXM4nG3lkY9kRT37ZoCWAtqXY3qm4qKAEsdEyo8zXr/u3kt+2KUCCdeRExRDR9gqMx0lpCQLjC/qZmTLHgBnNmEKwYi/LgUl5M3Hs+OgKD0+4C9h3TU5smEdFsFBx+ATimTy41m365cM899NNj/j4hWa+ePKRgwi7/DrlNmSNUk6V3eoSfhsDCV6DoVKY2yNj03fah47l5bueMt7nVtANSL/W3SabT7dvc/Qso1+sk2Cglo5F00ciNM49N95FjgoQi87HpWR+j52rYzsUXtx4xC4DkFilu5O8rWTexKoMQgEgN57d5ABD0xgR75bINkbwmpEdcQIPNTQPIsUpV8m+XGzI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: D+GJlfkYmt+14kqhewEIhFTpoxdexzFEOZZyn0J1TUk27jpIIIVZO4cUfLqza29jWJZLY87WwGxBJfaqpi6bBcwQeuFNdtYP2b2bjMzpSw/PyAifPuZKGAoHapE7FymVBdljncCL0V29rDNrkGLE59g24BUsdOVfo5PihdMuHi2jabgJz4DBYOlWmyfs4Rm8t3hhKcw3ES+IsNXCjMkC7Zqtg8QwQZjBf6/l5iYIRj6XiZr9r8YIJkzdcoggFC461LxM2hJx4Aw/edjaowdBTN3o+Wt+edtDu8oC8ojnjL5my88ZoiZ6sos7AhqXOgqD8U7jU7DPuncmlE5ywkcG9gYZ3zEFXDNBRkacbNk2Ii7dRsBPo/vBjJYOhrJZEaAWH45yyVzryo1glJHeWhZOAIoEpVwGssUOPLPvvi3V7CSceaBSXTMNa5OeruVeBW5yfRqfvnHTujeOFU/w1+qs6FbL/fPunWfFEK/SeCdHRnMu7y1tL1bn1My8Emzp83DgSkEHAmCvTGK+SYsHA58HqgPClwgo461CLbzfDdEh2EJ4ZzawTgUeo7liV/KAYceo7Pea9gYERznBkDS7doe6o6nXd454S1m03Pp1xPMont7Kk+AgYJBCPuafybNkyAdbfqv2/sPRsnmF5sJXHu+l5D/YmN8vTa5mq1FqVZqBir7vsvSTgT8QWP8EN0CmPbWE6KipLfcpbFD7b3shweD3y1tqaXdgXHzAKRljfvCrdMfZh7CvpH1E/qKXKK3SKFfmYbcTHFHW//J+BzmJmC10/ByYthC62Gos3p57Q+EmZaJXTHWNWjGl63tgx7G9DDgRXpgKVUbs9isqmX6Vn3gz+e3hLrdOQqgXXdzO67CB4jbM+ITb0MSrLA02qb9LqgczypJT1ZbNzSp2PTXuvqRmVr2UNF68PEjYDrOoahoiATovz9Tu8EXm3jc5nX/LSw9d0rOm1mBBzl12DFaUMO9gZA6fftMLdQh+itt0VE5mh6syL51yzvKLxUVyjJEgyAJe23Qu+ZVWVuum0E0irxxlEM/oifdrvjB/nV8WXtFQsi1kpF9lKGfQS1G0/OWGGFLPFlKiXhB7wz/R8O05LGM3eG8knl1/QJB+B/xpqOkxtTTgNdtPGKfTfxabh+2LTjYUdCR7yiy/oh+g1uNhx1bNAnkFByg6DakrZmCzyc2nIU2k42IxGB0MqoyBTFpkC7in10HwN7+PtIjDjlDtSdRxXYE4SJdp5UXHHi0UWRXR0YhuU5qPCZhF8n5DRMR3HXBw/FFcqbZW7eHA7PnzAvvF8yRBh59EUUSffFSEsw8yqczrszEjwGEJS2jiL22BnZ/dTheCz8Twg1qvE26qsL8X3VX2L3sDHrpQbmtPx8nqE8ubM8VqTMk3viD+VhRNGulYq7Z9LXCj7qB26Lm7gLmGRsXa8+OcXeWmwJQkMLsP0OJlOoirP4Z3JjCeaPF9ZcgYleav1Snr9kTADuBbokjBcy7fMO8hmvZ2qs4g+wCZ7oe4/ghUrcL7w5HOucrKyQn9RE4x6pYzsUHHVA+kwUo7/fWR1DBUeeaEvQeQ60pf+3kGvAO67i+KzuM6m7OEIyFR8upXsoSFKIdg1Ld2WALehQ== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4df7f69f-5ac3-42fe-82b0-08dd4d0f62bd X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:51:03.3694 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SXC5O8HfQoUms6J/TYUT8IBdYeFftsiGXLOIv+3KG6VqbP4ayg+NbyQOvCg/EHKsEHd2rtNUFEWMoBNlsfXF0A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR10MB4081 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 14 Feb 2025 15:51:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17224 - The class GitShallowTest contains all test cases used with `ud.shallow_fast == 0` (disabled) - The class GitShallowFastTest contains all test cases used with `ud.shallow_fast == 1` (enabled) - The base class GitShallowBaseTest contains all test cases that are used with both default shallow and fast shallow modes Signed-off-by: Stefan Koch --- lib/bb/tests/fetch.py | 119 ++++++++++++++++++++++++++++-------------- 1 file changed, 80 insertions(+), 39 deletions(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 48b9e4af1..2891cf9e5 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -1739,9 +1739,15 @@ class GitMakeShallowTest(FetcherTest): self.make_shallow(['refs/tags/1.10.0']) self.assertRevCount(orig_revs - 1746, ['--all']) -class GitShallowTest(FetcherTest): +class GitShallowBaseTest(): + """ + Test cases for use with default shallow and fast shallow modes. + Test cases must succeed with `ud.shallow_fast` set to either `0` (disabled) or `1` (enabled). + The class GitShallowTest contains all test cases used with `ud.shallow_fast == 0` (disabled). + The class GitShallowFastTest contains all test cases used with `ud.shallow_fast == 1` (enabled). + """ + def setUp(self): - FetcherTest.setUp(self) self.gitdir = os.path.join(self.tempdir, 'git') self.srcdir = os.path.join(self.tempdir, 'gitsource') @@ -1814,7 +1820,8 @@ class GitShallowTest(FetcherTest): def fetch_shallow(self, uri=None, disabled=False, keepclone=False): """Fetch a uri, generating a shallow tarball, then unpack using it""" fetcher, ud = self.fetch_and_unpack(uri) - assert os.path.exists(ud.clonedir), 'Git clone in DLDIR (%s) does not exist for uri %s' % (ud.clonedir, uri) + if not ud.shallow_fast: + assert os.path.exists(ud.clonedir), 'Git clone in DLDIR (%s) does not exist for uri %s' % (ud.clonedir, uri) # Confirm that the unpacked repo is unshallow if not disabled: @@ -1822,9 +1829,10 @@ class GitShallowTest(FetcherTest): # fetch and unpack, from the shallow tarball bb.utils.remove(self.gitdir, recurse=True) - bb.process.run('chmod u+w -R "%s"' % ud.clonedir) - bb.utils.remove(ud.clonedir, recurse=True) - bb.utils.remove(ud.clonedir.replace('gitsource', 'gitsubmodule'), recurse=True) + if not ud.shallow_fast: + bb.process.run('chmod u+w -R "%s"' % ud.clonedir) + bb.utils.remove(ud.clonedir, recurse=True) + bb.utils.remove(ud.clonedir.replace('gitsource', 'gitsubmodule'), recurse=True) # confirm that the unpacked repo is used when no git clone or git # mirror tarball is available @@ -1841,6 +1849,7 @@ class GitShallowTest(FetcherTest): self.assertRevCount(2, cwd=self.srcdir) self.d.setVar('BB_GIT_SHALLOW', '0') + self.d.setVar('BB_GIT_SHALLOW_FAST', '0') self.fetch_shallow(disabled=True) self.assertRevCount(2) @@ -1900,26 +1909,6 @@ class GitShallowTest(FetcherTest): self.fetch_shallow() self.assertRevCount(2) - def test_current_shallow_out_of_date_clone(self): - # Create initial git repo - self.add_empty_file('a') - self.add_empty_file('b') - self.add_empty_file('c') - self.assertRevCount(3, cwd=self.srcdir) - - # Clone and generate mirror tarball - fetcher, ud = self.fetch() - - # Ensure we have a current mirror tarball, but an out of date clone - self.git('update-ref refs/heads/master refs/heads/master~1', cwd=ud.clonedir) - self.assertRevCount(2, cwd=ud.clonedir) - - # Fetch and unpack, from the current tarball, not the out of date clone - bb.utils.remove(self.gitdir, recurse=True) - fetcher, ud = self.fetch() - fetcher.unpack(self.d.getVar('WORKDIR')) - self.assertRevCount(1) - def test_shallow_single_branch_no_merge(self): self.add_empty_file('a') self.add_empty_file('b') @@ -2111,16 +2100,20 @@ class GitShallowTest(FetcherTest): self.assertRevCount(4, ['--all']) self.assertRefs(['master', 'origin/master', 'origin/a_branch']) - def test_shallow_clone_preferred_over_shallow(self): + def test_shallow_clone_preferred_over_shallow(self, fast): self.add_empty_file('a') self.add_empty_file('b') # Fetch once to generate the shallow tarball + if fast: + self.d.setVar('BB_GIT_SHALLOW_FAST', '0') fetcher, ud = self.fetch() assert os.path.exists(os.path.join(self.dldir, ud.mirrortarballs[0])) # Fetch and unpack with both the clonedir and shallow tarball available bb.utils.remove(self.gitdir, recurse=True) + if fast: + self.d.setVar('BB_GIT_SHALLOW_FAST', '1') fetcher, ud = self.fetch_and_unpack() # The unpacked tree should *not* be shallow @@ -2240,18 +2233,6 @@ class GitShallowTest(FetcherTest): with self.assertRaises(bb.fetch2.FetchError): self.fetch() - def test_shallow_fetch_missing_revs(self): - self.add_empty_file('a') - self.add_empty_file('b') - fetcher, ud = self.fetch(self.d.getVar('SRC_URI')) - self.git('tag v0.0 master', cwd=self.srcdir) - self.d.setVar('BB_GIT_SHALLOW_DEPTH', '0') - self.d.setVar('BB_GIT_SHALLOW_REVS', 'v0.0') - - with self.assertRaises(bb.fetch2.FetchError), self.assertLogs("BitBake.Fetcher", level="ERROR") as cm: - self.fetch_shallow() - self.assertIn("fatal: no commits selected for shallow requests", cm.output[0]) - def test_shallow_fetch_missing_revs_fails(self): self.add_empty_file('a') self.add_empty_file('b') @@ -2295,6 +2276,51 @@ class GitShallowTest(FetcherTest): self.assertIn("No up to date source found", context.exception.msg) self.assertIn("clone directory not available or not up to date", context.exception.msg) +class GitShallowTest(GitShallowBaseTest, FetcherTest): + """ + Test cases for use with default shallow mode. + The class contains all test cases used with disabled `ud.shallow_fast == 0` only. + """ + + def setUp(self): + FetcherTest.setUp(self) + GitShallowBaseTest.setUp(self) + + def test_shallow_clone_preferred_over_shallow(self): + super().test_shallow_clone_preferred_over_shallow(fast=False) + + def test_current_shallow_out_of_date_clone(self): + # Create initial git repo + self.add_empty_file('a') + self.add_empty_file('b') + self.add_empty_file('c') + self.assertRevCount(3, cwd=self.srcdir) + + # Clone and generate mirror tarball + fetcher, ud = self.fetch() + + # Ensure we have a current mirror tarball, but an out of date clone + self.git('update-ref refs/heads/master refs/heads/master~1', cwd=ud.clonedir) + self.assertRevCount(2, cwd=ud.clonedir) + + # Fetch and unpack, from the current tarball, not the out of date clone + bb.utils.remove(self.gitdir, recurse=True) + fetcher, ud = self.fetch() + fetcher.unpack(self.d.getVar('WORKDIR')) + self.assertRevCount(1) + + def test_shallow_fetch_missing_revs(self): + self.add_empty_file('a') + self.add_empty_file('b') + fetcher, ud = self.fetch(self.d.getVar('SRC_URI')) + self.git('tag v0.0 master', cwd=self.srcdir) + self.d.setVar('BB_GIT_SHALLOW_DEPTH', '0') + self.d.setVar('BB_GIT_SHALLOW_REVS', 'v0.0') + + with self.assertRaises(bb.fetch2.FetchError), self.assertLogs("BitBake.Fetcher", level="ERROR") as cm: + self.fetch_shallow() + self.assertIn("fatal: no commits selected for shallow requests", cm.output[0]) + @skipIfNoNetwork() def test_that_unpack_does_work_when_using_git_shallow_tarball_but_tarball_is_not_available(self): self.d.setVar('SRCREV', 'e5939ff608b95cdd4d0ab0e1935781ab9a276ac0') @@ -2309,6 +2335,21 @@ class GitShallowTest(FetcherTest): dir = os.listdir(self.unpackdir + "/git/") self.assertIn("fstests.doap", dir) +class GitShallowFastTest(GitShallowBaseTest, FetcherTest): + """ + Test cases for use with fast shallow mode. + Test cases must succeed with `ud.shallow_fast` set to either `0` (disabled) or `1` (enabled). + The class contains all test cases used with `ud.shallow_fast == 1` (enabled) only. + """ + + def setUp(self): + FetcherTest.setUp(self) + GitShallowBaseTest.setUp(self) + self.d.setVar('BB_GIT_SHALLOW_FAST', '1') + + def test_shallow_clone_preferred_over_shallow(self): + super().test_shallow_clone_preferred_over_shallow(fast=True) + class GitLfsTest(FetcherTest): def skipIfNoGitLFS(): import shutil From patchwork Fri Feb 14 15:50:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 57338 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 249ACC021A7 for ; Fri, 14 Feb 2025 15:51:10 +0000 (UTC) Received: from AS8PR04CU009.outbound.protection.outlook.com (AS8PR04CU009.outbound.protection.outlook.com [52.101.70.23]) by mx.groups.io with SMTP id smtpd.web11.24156.1739548268282327099 for ; Fri, 14 Feb 2025 07:51:09 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=y/lyZYEb; spf=pass (domain: siemens.com, ip: 52.101.70.23, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ujeHvtXuDsg97TXWmPqZGb0UO+/E9fKI5TrqSCzfMO8TQSjTfQJEyJ+wVhhTZ7ZUAk+ULNmQwDObyyrGOZCgUmGDfFgFzRf3C8LlklVl49j22Spr7iRke8tKGQK7K2FX5XZXI1dOaS3OqYnr6zBqqvfOHXmVta4/3Pjct2vxs4EE7FUDfI6bbEZIZiriDexazMPMfvx7Rn0eoQN2YOzHCALdJuCcMRr/VagCHipyPi6N0seubPMdQTrODabEKe2wc6u5mOyrnRJt90hnkNdLNbuv1kA+uvGZydNEH6qtQhCGx8YYofzI0rVLt4q+5auEaVmR0mNiRFPdylqBh1wf4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NGjVzQ3yPj/sI2xP3igcwiTRMFzFxuDFmYPGu62AyG8=; b=PmFG0WRprr/2H+JyIc2l7R7eG69DIoDgLUgV0nk8xQIW3F4c7u6KiWL5TNtK5xKexvEJIj/n2yVAP7SJaRqnvBxX2LRGmrDdWywGcCOMLdV3gDfpjxjVJnHC7Uctw7Q6rRyqZRiQ1uAT+yhE7PIC1TppBlolZP029TYzbjvkHwQU1ia2kl4jzDZIRNYOEI6OUnC0nFxSkicroaI6djkMvhiKhIzGTJUuauHs68RZCn64fh+B4DjBPIFnfmIQNZq97TvXYExafcFZmqGNpT5TT7YspI+IlfDlIKXwWOlmCFfg9wEQvMwkp2N/F0nX33g5tnpwk9cMml+IqXeKPUWpDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NGjVzQ3yPj/sI2xP3igcwiTRMFzFxuDFmYPGu62AyG8=; b=y/lyZYEbSoBSUi+3r51Rcg0JM+Urf65FnDmw1nQPXrCP9uGwoOLgvrvDjFYdQOO/Gglp17VdZGh7jK5xkkiv2Yljolit+0CP/+SMU4G25WsRmfvVcVjQqnIp0oqA/qTT6gStimWUQM2+nZg4jWhtYfs+gMFhjd6IKvuLgSyWJSXI/mL+4gaB7W87x/v+WKo5jKssIB3A9H490YMg8ZdwpjdYxW27Obv90gJj33qW3mKpRm4z/lDLR+ms3wAp6OECVfzKvlGTIjn+xMMTl7/hrf3HJoN+Na5mm3CAbbGyew0POo7Nz0tDPhOLSWq/ESG7h8YMJRXLGWST9WG1ALjFAA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by AM8PR10MB4081.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:1ca::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.5; Fri, 14 Feb 2025 15:51:05 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8466.009; Fri, 14 Feb 2025 15:51:04 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com, alex.kanavin@gmail.com Subject: [PATCH v2 4/4] tests/fetch: Add additional test cases to check whether the fast fetch is shallow Date: Fri, 14 Feb 2025 16:50:57 +0100 Message-ID: <20250214155057.1748020-4-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250214155057.1748020-1-stefan-koch@siemens.com> References: <20250214155057.1748020-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR4P281CA0222.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:e4::8) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|AM8PR10MB4081:EE_ X-MS-Office365-Filtering-Correlation-Id: 65af83ad-b163-41ab-0bb7-08dd4d0f6351 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: Lifm2CjY6MW/z+Z2MmYzc23djG1HncgVIknTVikfckX16uyj54fjzFw0b0WYMQsHJbNWL/YWFh4Wc7ekhutA4gbJ5/OCtTGQcHJRq+f4grDfZQKUDTwcKQvwRznGJ98EkUI9I5pYaVy2FAFmj5x1db9IgaxOXu+AEXFr5z5PUr35fXZcpUi8FqXeX4RQMVzIzslJI9hcwty+b4LDd1LXD89wDXgWHgnpkRswEEszVnp2uHNy43Rtn0beaMZUpc6Gc6E38sk6dC93hM1ABn2zSDK8E9YBVrpGRSOh+FtInWluCdBD8lbKTjvFYL5uiU8b5UDRQoL8BWZTxFvP2M+y0QWS3gYOJ//0vf7ywx4YdnSgPM9MMgRIVXuAWhaM1IKR2iIuz+WHsPUfJCWT86zHSZh98pddpo8W++AFRuII2GvkCKbXDFkY0TQhW0W5sLTvilklMVlY7AFKAhuzypAadGBylmYHXwsCwn0H2Gvj3eGdrOoEwcwkkksQaaTRoxXU1Ov4ekND40RdRaT2fQwqvMg+R6T9zhkH/3VNzUT5JweXkppxiGJlDPt4tFkM5JnfAgF5xDzTZ7+NUlqxPcpNCtcef+5nZWxtC0TS7NOquqraRbBc7ybbbeXbZD4Xj7pnpenE7W/aoB1EvbknFHWaSxcq6f94nWGPMXfqNp5yeMi1H2wE2L46PPk/lrhA6VLucwLGLVXeADhyYaI1E2TfY0MuCZM7ve2R8a8Fda3iRmbnSfbGmvgWuR2pFvokQ++x10sOTcr4BT3NpeiWYnJ/Ehsk/IKMuc2fnl8n34FpMmBPY9an6QJiDKRJxc6wcPIWULu/pXT0pdjTAsWjyQEohy7wO4HwXc677WyJJRSDYnDqszDKWq0TQRXSEKeVE1VhHbtCkwrsoTJt1mKA/mFGCHcdHBY+LzsszV8BxtO2oU5WFPZdhu7Nd3tisfGeRFWYJI+6zzN5tYcNWumD4ywYnhiuNle+kcVaWr3GM4QCaU4DNW91Yo1Nm/GV+KQ/kY7ZaO2+STLAE/xCaJ+gtuQzx76Ga3fltaJ23zDzsr/3ZI8ug5T7Iy3j7WJPL7BaGktjVigmjdF3IhMyiyt3L8xsvoonrsW1oRGmCsys9OmjytywtxNgKWBMZlgHljNdi7noVs3MYOq87+zb6mKrsqw+Arz7O6jxDSS/u75ZLrAkTf2G3mOdimfBb9f30K0QMOmo1AgAPqj8SrZIg35jUEv0ezD/ZEREoSNlBB5SPIK28ZCiz1pUDY6wwHc6IBekZzuJpxmw9h9JtudHofn1Ae5X1g3pdZiir/EvlY9nc1oVNjryiTmLLVhDu9mJJeWtuyKprpTSONA6pxa7ZgJc9GBeDmqaLmpdf/t1HXTN5+Pc6QVul4pDNsFMzlmSofrGEndQ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: d5fM7eixHSaDoEkdbgDHg1gvsrSGp99KoOWooU/cUfM51z9QGFMEeEdXEjPTxj0rLgcDrtLW+i1OUN2cHnvJnw9QC7eHcCXPwDYMA1QOE3kt6ztW3r1KeWDNRN/R8A6B2WmurI+nRJzp+azaY/PfuNgAhjGMFJH2QOfbG0wohRo4l2J+kTr6sOn8XKOoVzWOvjK2z6PdSCdF9ZgVvW/q3jdS4X8LMJLMC0D63/ANF+wTwc2OTKZAGlVyD74YhqgiLRkRtp7thnYDdxvZpzmAUJliAZLGjOWNZ2djvmwXSStKWM4zcZNo6yMRbb82hOWe4gwr2hunjOJozNOcswKgkXpQ+4nB+ZO0CB2QRz86nO41LppXdLZ0wRM1r8++rQtTRbnffzD6zunhrTgEHcuCBsYPKIFRu1KNt9AKvEs0HZ0/OjkiAclob9gLW36p384w7adxidAmQaAcSB7o8uAqKAkXbK1ZpSCRCuuIPQQ80c0YZ7NFaPd7D6wpjGiP91WKHS73SiDYqkhugVHb+PkgDH+qOGReGB4D5xc6+y/SUUtWQTqWCuxQKoFhxLeqP9gG7e4laBNiQ8ik9OnICiIsB4RBHEqE84vLFxPl0zzqYXoFEZVH8xjsl6QEImrFbTgZZRbGYs7b+p0RH99/WCVbPED7O9WWlF3q8wu/MPnN5ELGuMgggSL/juSbHt4qUGoyFSpyPi/GZ4zAL4oDdaV5k8vtzKkDDNFrcz5NtrKlBvNybIbL3+Z6O+l3fHv3DHLwnZJYL7rNJdRCPc16lQB85fuXks/jlMICn2gKjZWIj7xbmHKPZX9r7GPdYj3aTV1m8a40GAxApn6IQdahuvQvBx96AhZQMf8KsmmT+vPIgrGGAXkKccpjpgAp/0Jrqlt6h2jVwEzGIuFd4wYdBWQKIA0pk/ligkyUWO57fC6d5DhjiuPzXVAW5hXZ05E6KyZWD5gL7zws752eSOw0olzjtRiTLa8TsXmUvNfwJadMol69oZL0qBrPpDYIilobzDJNmozm2UTNptkNtvZLQpCkDSOKgMTUjykfjueoQKGoH3TVGqz0n3ghGTrjoRh0j9x6FPy7zDX2OmsWQdQx8fJsxrRiD68JWwoHNwuXaclUBSNQRYMrSlS5cuCzlGSmme3eojeJlGZS3VjvWKtXUy31e1nJFMx7MzCQFdQuHzQaernwkywjtkC6IR9BW+KwtEbIwDkyDU2L0MxvSN3RnAuu6cBNWhyN1W8Zo233yAgCTWPmfqoFzJGOuQc3aWMJOpWUVTzFCCWScVKywAFXymXMr77Jy5lHPu9xJidbFyQJDAaP/BiVjyN2Azu7rtV0EuPiAAZ73bZQHSmPIDDSE8tRpv5FQvvO5q+eaDnLF7yZTQxyFJwZLGCuIcIJiAHz63a3ORrh+/p2rxlPQqG0kfGFIzig8aXRH9WB84WL6HxK1U1uSvTCsprJzkq1mlZ0Ss69kP9YJj2cedRa4r6CMOHQ2ga0nGSPW4AXq4EYVirfCFB/KJChN8JXggQe+z9UwdPE9Irr/TT4tGwfMdk5HG5YgrawtrStLq/81HhVYHOTCbG5N4W53D8gzSV25wRAzjh9Z4CE+FQVhndQ3C3eV9li3Q== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65af83ad-b163-41ab-0bb7-08dd4d0f6351 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:51:04.4004 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oRzkKl60MPZM63xCMVWLwveo+2CkuO5fcPbo5VKvHJeCSLIIBrpv43PzaMBMV78tFDAwE/2Ml9vXDXZxIREt0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR10MB4081 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 14 Feb 2025 15:51:10 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17225 Signed-off-by: Stefan Koch --- lib/bb/tests/fetch.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 2891cf9e5..a1d9f4e61 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -2350,6 +2350,18 @@ class GitShallowFastTest(GitShallowBaseTest, FetcherTest): def test_shallow_clone_preferred_over_shallow(self): super().test_shallow_clone_preferred_over_shallow(fast=True) + def test_shallow_fast_check_is_shallow(self): + self.add_empty_file('a') + self.add_empty_file('b') + + # Fetch and unpack without the clonedir and *only* shallow tarball available + bb.utils.remove(self.gitdir, recurse=True) + fetcher, ud = self.fetch_and_unpack() + + # The unpacked tree *should* be shallow + self.assertRevCount(1) + assert os.path.exists(os.path.join(self.gitdir, '.git', 'shallow')) + class GitLfsTest(FetcherTest): def skipIfNoGitLFS(): import shutil