From patchwork Wed Jan 29 11:24:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 56214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB9C3C0218D for ; Wed, 29 Jan 2025 11:24:35 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.66.35]) by mx.groups.io with SMTP id smtpd.web10.10469.1738149871051466691 for ; Wed, 29 Jan 2025 03:24:32 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=ksH/+KIh; spf=pass (domain: siemens.com, ip: 52.101.66.35, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EL5AlX55eHDLr64VC4JmLfI3TB2AZkOWFtkCgxvX0kbGkKIGOuif2y7NKev9xCAUX+o1pp99A76QOkZe9sY+3tEo/GjUCoBcP+RLKLqizJsiSOThXO4h5w0+jBUz7DMxuQqHyIwlEDNdpOFIl0oC/SgzKrDCIddWT8GgGwGUsyxFL+kZhB8I2MUYxeaHXw33aBduZ27g3DnlQI9ULQ/FcpG3pzOVgZWD2aeowt4+kGZiX1k2LNQMW1wdEdgq/XuElP2lLr5ATYttRahIJyf96c6+KoVXp8ux66h+l7ZS1Zeo/8lHb0aHQlGnK31R/NEV7UdWrEj+yu/qzEaTI4zmew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8kGhT4BSXuTPcAZUwNVEXPa0SNfWiTmgihk9nD7aDuE=; b=joZFZ5RMTeBO+0ocKWoF4kKt1r/D+z2Ulk0F9V4AO73irO/MXOIl6NEupdP1SaAP4rXs9HfQhPoVJZ4Fpdm+54z2X3LEf33h5+5NZ2KQRb/k+1lV07t/unuCVXf73jaOKIZMW26UApMCyj6751lVr2Pqh6oqPWd/09szB9wBn1KFjzlYB1G9nOFBLiPwWLADk5EVL7kHizOWs7MQyU0qCYIHito747A1ASRyvlbCV8u9CMRVibOY9PgsvmlpHt8oZyrvggtny1stBEKrMOebWA/EDzGVj1W8dq0vgJkU9ZhuDwnqErqhXPT6CgjEctZPkDGiDzZM3uK188fPSBAyTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8kGhT4BSXuTPcAZUwNVEXPa0SNfWiTmgihk9nD7aDuE=; b=ksH/+KIh0kgawjo7Jc4rYnXIwuGi4cnt9Y+qETeGuDSZHMDPkP5l/zXBj6aDAgbOAL4gjqP3/zVUNSDe/WWb6e2s9WhoMhbe+NKiymYkcrJTbdiVNp0EitI/JZNGeceuggsfU13rm3ep69Bw0TVaJx6AK8/VkezeaZI9+Bd10jxNMY9Y4+pvM14Uk/qIXEw8OUOCfScE8WCsNGxXKhjKkaJeuv8Ob2CuxmY79/lifNZ+veiA2n4hGiRCvMCkXDV8lGXw+PFfu8b1NHgQCHMq0VePBCKlSKEXu+ZIF/JBPhaRc37x/RKGwcsWv52jZv59jmEPRzraJy2WEIMBDDhxbw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DB9PR10MB6421.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:3db::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.15; Wed, 29 Jan 2025 11:24:28 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8398.017; Wed, 29 Jan 2025 11:24:28 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com Subject: [PATCH 1/3] fetch2/git: Add support for fast initial shallow fetch Date: Wed, 29 Jan 2025 12:24:04 +0100 Message-ID: <20250129112406.1660522-2-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250129112406.1660522-1-stefan-koch@siemens.com> References: <20250129112406.1660522-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR0P281CA0143.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:96::17) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DB9PR10MB6421:EE_ X-MS-Office365-Filtering-Correlation-Id: f7f95fc0-83c1-4f87-e1a7-08dd40577e3c X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: h5fj9KbyApz+ZG9SUuQz6BRQBip9YaWe7zAZNrs4IO/Rog8vhTmofdUhrHZ6jBLMQxk0YDEQOQXlyzr1squwER2DR/WPa9QyEfY3R9pjKrofaXNYXYWm4W7TnmBNaQ6HOOxDn2z8PbTi/W98e1+E/joyR0JTVihu1feMcSEazRPgn+wgMoBKRRo7SSALZRtX3LwtARLhTu1gjvYQBk05PZ7/YFgaDyWRM9ihMa3cbAQw0znwkSdo1GqaggzEozC8hgf1+DeH3ysg4INx66f3oCFJjqiDZ32NorFMHxFJ7UTsIc30xhHU3L3cOzUDZNW+nxIvW603NshiHD136bm3WyyglYk4yX1Hc/cTrS7uqVHFesF2pshWPOjcAG/Sgtuu0Yd9b0FMoSgZ5ZQSjWfS6agetBGhWwJMMYx5BHdFpuZXHg8pzEfohvlnUlhBV1OYJuxfxztjzGUveQ0t8H8sXuGBehk2s30igFih3HBoWdILLxeEwdfVwALuXN5bi89chikaY5plFHNAiLYMlfu5saWAF9ERl1JWebOXNuTUBzylSPfhoO+dHGkyl9QD8VfYDDFtvz20Cv5uKhcakrGkBVPsRE/ZgA+iLllaQmVPqSe1ah4g8BfSY4t+y+seZGgm9qr1W8HsblY6YOQ+b7qH71oNJKPYy8W8sBhVJetw89vb/b0NCHXxTegrQvh12G/mOTEF1rmvDD+VFUZMctpKLnrgUVG4PEWtFGT4VgSzu9b6xGy70bia/2Bh4fL8NntyK/9nmQ12BwGA7ID7wkH2qx59LwdgterLilsl+KbSv8+dEmX1AJhZiN20W393w1Du8pcAlQAk9lwMvk3SRTrNTbwVDL68zRWjTC0Q0gVUi5W7S+hqRn1YL1pya+BlcXFNmycen45pdAsNTBuWSktzS01vQuCRAXwgpYxQwT6lGfUYf+48Ol/eFQeNdMx4PMIbuK45d8JWdeKkrxhyFgGxsQOVbLnxFXH3sPXBVQZliNUfs3yfAVZL9PxwDKsbyXjUqxjbCxPNZjlnY0VqwJjSdhenmk+KlfGyOlS9NjHg3J1xBK0IaggKQEJh5OVCmHBZjp9P1cxw2xKqZw0neZcItB1AgDDEf3MDpblh60oDXX2XyaOQzGE7iPCneHDK7x68GUfrZhTZZLFhri6XoH7xUooj+SZddvp8ZUrhAySX4UeaanQH2z6lt6YaO4Ap8Y/r7Qn6EJgKo5n59DPBICMX+LtWNowNpK317K+YsYNREzo+GFzWE+2Veqy/3MN3ZUI0EQXBGRmxrjD26Gbw1OdVNrq+WDtQujBQgJi4yx5U99DRc/XACcPiIRk/5MYDAacJYs6UQwVn5h+jfKznMuAnwrjWWJt+aehx6B9U0Xswjsz5BJJMowhH5SlY5KNmxGBK X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ktqZZI3hdfv7KwEUThLQcl2r0whYVbIRscBb4/qxdSSXet+DhYqkeD8z30PMPSqoP2Z4MlWI+d/2bZ1vXE9QvG4558fRqtF3t3eBLlfcEdf4+dTVBqcJaF01SDG5kUiyIGlWm0j4vk6m/bkvHo+FtjD9CSvP0s56xnq7au2dVgPIbuJ2Sr5EfeCB2ecUX7FS8Z6mmKORRwCUHsmt68CTEEjAL1a3XEzFF1vPGWO9tXE8eMiNh8b6Hf8j56JfGvps44dKMkmLLvAbCkCdk3+qvSCGXtM0wiMsgYLcrdsBSqh2rGuqYpsgUpiOxP4NlvejgykdQCtNsNRIsjcjdJ5Ms77JrZ4FeBvoAZ8G3eKPm/ixlakHseyRAiBfcAnaAbU+LPoAOLXlQYniOMl6qBsFSM1cUJUIFjpFNBXIOQQv0+bu3jgrhWR4mtpoPst73dSMM+Q1VBacqwcd/dY5sce1bqiqXXyoC9nNzIVbkJsJcy5Cn1G7GA+mqMFuwEQZfBb8SBKauQ4ISBLQ7hqUG+SouOh7M8ksPms1LXHGR3Ofc+xVqKZARvYv6PtLcC4nlY1DLM0LuUMPC2VPlFN4j4JkxctUgdmyYLK6XqjoALQwgQI0aQFVv4woXbxGWBZu8sUCoWqPi/wQi7f5ZmhLBph45WoH8NmKSn+Q611DCFr9le2fL7rlfDLrkbe49LoQPXLUVveM3QbNsvbtDcubqdBGmHQxOBvqLh7BM22diEs3z34cnrorqeHyEQquPcCwtbfjoK0WozoW55yoBriMmmv3NjBWWaCDxSzHg+qKdd6wGB61wwUArakKT2to/n5WoPXpvDIauqJA5bgEDl72pN54U5ThLO4Jf8au9F9A+yoxuodjCpB4JQ54x5IBFOeoFnkM8ZuCCT5OR/gyPRADX2qUcNcMNw09Lue7Bodfy7lVvjmkkDucyFxEdKPoBdMPhj078omYp3+tNKO8v9AspSsJ4f5LD6n7oCX6IJmxo4ANvwB+r/Ix8+1rYnxdx0cDWueYHorkn0jYlsTglbl1fcEJA1JwPFDm468hs00nN77dtLw9IkMK3csMbZZbCi3V1ujLKZrPZu5t+PPCQJeDZwhZPw0civp5gbG7mDfot7YLtRo+AB4CB76VhhqDMilzTuv5YrhW02kFqbFRMO9XwmIWohRnP0/twFKK8gYD9QJDovjdrOGA/H1IAIJHEpNDgwXt9V62ZhroOU9v7oATuxx/IXDIiSdYM40ErO6ncBV+2v6SEq8Cn8GBE4DA9R93+9LKEd2IMqkkdu6J7LCQrvj9yI90irZQKGzMjviHDZny/O1P5BYhHpkCpk8ZfU56zEkHtKhpL8sgOM48dAEU+hKMsoCWOcliNBAf4dzKpOO9MzwhneADC45QDF2z4HAi/yI5nfy1bbH7jXQ+3vJvXeTllqjm2248LNwAF1a4Y2zo3VmpOEY1XbB2KQ3jf2nvZMsms2/fPKdL/KYSDwIuQH9Fi7436vRSDi8CO6EmXXfl9Rh3NsYUuMOhOLTI5dru8B4SaYkbpPbLzfjwVXOySa0A9qKnt1SK5WG4zj1hlorS6DRSW9yWtT0AcHsoIEVsjF9hlROgBbFzNPhKaU4f4rAmRw== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: f7f95fc0-83c1-4f87-e1a7-08dd40577e3c X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2025 11:24:28.1941 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7e9mwUqyXr9F/0QJ1sLR5bZbavl2pVlEP+iQ/rvgX410YrfF3NFsa9xkbWNTmpcxcHbIRWMr0yhyrEpVqYtIsQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB6421 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 29 Jan 2025 11:24:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17109 When `ud.shallow == 1`: - Prefer an initial shallow clone over an initial bare clone, while still utilizing any already existing bare clones. This improves: - Solves timeout issues during initial clones on slow internet connections by reducing the amount of data transferred. - Eliminates the need to use a HTTPS tarball SRC_URI to reduce data transfer. - Allows SSH-based authentication (e.g. cert and agent-based) when using non-public repos, so additional HTTPS tokens may not be required. Signed-off-by: Stefan Koch --- lib/bb/fetch2/git.py | 92 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 71 insertions(+), 21 deletions(-) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 6badda597..6d87c2f18 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -366,6 +366,33 @@ class Git(FetchMethod): def tarball_need_update(self, ud): return ud.write_tarballs and not os.path.exists(ud.fullmirror) + # Helper method for fetching Git LFS data + def lfs_fetch(self, ud, d, clonedir, revision, progresshandler, fetchall=False): + try: + if self._need_lfs(ud) and self._contains_lfs(ud, d, clonedir) and self._find_git_lfs(d) and len(revision): + # Using worktree with the revision because .lfsconfig may exists + worktree_add_cmd = "%s worktree add wt %s" % (ud.basecmd, revision) + runfetchcmd(worktree_add_cmd, d, log=progresshandler, workdir=clonedir) + lfs_fetch_cmd = "%s lfs fetch %s" % (ud.basecmd, "--all" if fetchall else "") + runfetchcmd(lfs_fetch_cmd, d, log=progresshandler, workdir=(clonedir + "/wt")) + worktree_rem_cmd = "%s worktree remove -f wt" % ud.basecmd + runfetchcmd(worktree_rem_cmd, d, log=progresshandler, workdir=clonedir) + except: + logger.warning("Fetching LFS did not succeed.") + + # Create as a temp file and move atomically into position to avoid races + @contextmanager + def create_atomic(self, filename): + fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) + try: + yield tfile + umask = os.umask(0o666) + os.umask(umask) + os.chmod(tfile, (0o666 & ~umask)) + os.rename(tfile, filename) + finally: + os.close(fd) + def try_premirror(self, ud, d): # If we don't do this, updating an existing checkout with only premirrors # is not possible @@ -446,7 +473,40 @@ class Git(FetchMethod): if ud.proto.lower() != 'file': bb.fetch2.check_network_access(d, clone_cmd, ud.url) progresshandler = GitProgressHandler(d) - runfetchcmd(clone_cmd, d, log=progresshandler) + + # When ud.shallow is enabled: + # Try creating an initial shallow clone + shallowstate = False + if ud.shallow: + tempdir = tempfile.mkdtemp(dir=d.getVar('DL_DIR')) + shallowclone = os.path.join(tempdir, 'git') + try: + self.clone_shallow_local(ud, shallowclone, d) + shallowstate = True + except: + logger.warning("Creating initial shallow clone failed, try regular clone now.") + + # When the shallow clone has succeeded: + # Create shallow tarball + if shallowstate: + logger.info("Creating tarball of git repository") + with self.create_atomic(ud.fullshallow) as tfile: + runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) + runfetchcmd("touch %s.done" % ud.fullshallow, d) + + # Always cleanup tempdir + bb.utils.remove(tempdir, recurse=True) + + # When the shallow clone has succeeded: + # Use shallow tarball + if shallowstate: + ud.localpath = ud.fullshallow + return + + # When ud.shallow is disabled or the shallow clone failed: + # Create an initial regular clone + if not shallowstate: + runfetchcmd(clone_cmd, d, log=progresshandler) # Update the checkout if needed if self.clonedir_need_update(ud, d): @@ -509,20 +569,6 @@ class Git(FetchMethod): runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.clonedir, d, workdir="%s/.git" % ud.destdir) def build_mirror_data(self, ud, d): - - # Create as a temp file and move atomically into position to avoid races - @contextmanager - def create_atomic(filename): - fd, tfile = tempfile.mkstemp(dir=os.path.dirname(filename)) - try: - yield tfile - umask = os.umask(0o666) - os.umask(umask) - os.chmod(tfile, (0o666 & ~umask)) - os.rename(tfile, filename) - finally: - os.close(fd) - if ud.shallow and ud.write_shallow_tarballs: if not os.path.exists(ud.fullshallow): if os.path.islink(ud.fullshallow): @@ -533,7 +579,7 @@ class Git(FetchMethod): self.clone_shallow_local(ud, shallowclone, d) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullshallow) as tfile: + with self.create_atomic(ud.fullshallow) as tfile: runfetchcmd("tar -czf %s ." % tfile, d, workdir=shallowclone) runfetchcmd("touch %s.done" % ud.fullshallow, d) finally: @@ -543,7 +589,7 @@ class Git(FetchMethod): os.unlink(ud.fullmirror) logger.info("Creating tarball of git repository") - with create_atomic(ud.fullmirror) as tfile: + with self.create_atomic(ud.fullmirror) as tfile: mtime = runfetchcmd("{} log --all -1 --format=%cD".format(ud.basecmd), d, quiet=True, workdir=ud.clonedir) runfetchcmd("tar -czf %s --owner oe:0 --group oe:0 --mtime \"%s\" ." @@ -557,12 +603,15 @@ class Git(FetchMethod): - For BB_GIT_SHALLOW_REVS: git fetch --shallow-exclude= rev """ + progresshandler = GitProgressHandler(d) + repourl = self._get_repo_url(ud) bb.utils.mkdirhier(dest) init_cmd = "%s init -q" % ud.basecmd if ud.bareclone: init_cmd += " --bare" runfetchcmd(init_cmd, d, workdir=dest) - runfetchcmd("%s remote add origin %s" % (ud.basecmd, ud.clonedir), d, workdir=dest) + # Use repourl when creating the initial shallow clone + runfetchcmd("%s remote add origin %s" % (ud.basecmd, shlex.quote(repourl) if ud.shallow and not os.path.exists(ud.clonedir) else ud.clonedir), d, workdir=dest) # Check the histories which should be excluded shallow_exclude = '' @@ -600,10 +649,12 @@ class Git(FetchMethod): # The ud.clonedir is a local temporary dir, will be removed when # fetch is done, so we can do anything on it. adv_cmd = 'git branch -f advertise-%s %s' % (revision, revision) - runfetchcmd(adv_cmd, d, workdir=ud.clonedir) + if not ud.shallow: + runfetchcmd(adv_cmd, d, workdir=ud.clonedir) - runfetchcmd(fetch_cmd, d, workdir=dest) + runfetchcmd(fetch_cmd, d, log=progresshandler, workdir=dest) runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) + self.lfs_fetch(ud, d, dest, ud.revisions[ud.names[0]], progresshandler) # Apply extra ref wildcards all_refs_remote = runfetchcmd("%s ls-remote origin 'refs/*'" % ud.basecmd, \ @@ -629,7 +680,6 @@ class Git(FetchMethod): runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest) # The url is local ud.clonedir, set it to upstream one - repourl = self._get_repo_url(ud) runfetchcmd("%s remote set-url origin %s" % (ud.basecmd, shlex.quote(repourl)), d, workdir=dest) def unpack(self, ud, destdir, d): From patchwork Wed Jan 29 11:24:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 56213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A06FC02190 for ; Wed, 29 Jan 2025 11:24:36 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.66.35]) by mx.groups.io with SMTP id smtpd.web10.10469.1738149871051466691 for ; Wed, 29 Jan 2025 03:24:32 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=rM4Qz3ZP; spf=pass (domain: siemens.com, ip: 52.101.66.35, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=q4Ynswg+2ratTHKsgyljI2iBLS4Rcc+3F9IBV/p9h5/n6swiHy196jOe96NIyJKFyTizVvKQqDD4HImztGgJnIV1H7oJoHCZrAFBzwNL8A5dR0Qpr6zFomsmlEPH9V24S+NsgB+KUdxZIKcMQiqEm3Gf0fRpu9bOtFLDCl0/qSCxA5TA0BWngixuWGdxQ/jdOhVhzlZ68k5rPN11ENfl2iP4WgCFLe88HYVCoLJQCC6sICTfpq1v5znIYE1SEoP/+RaXEVCl+wKFlZlyIUF7Joe5MVazEyYWf2ukuG3Ws3UO8jkJOLE9utoM2dESj0itlhKJr5sSOeD4aNE56mgq0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R1Z4z5HArxnHexzJ86g2cNS47HF2Q61tRaEO2up+sFU=; b=SoH7Oji8FnHs/1bhKMyHj8bwZM9WuL5FyEJwtMn1uEmRHyUt05NvQCRdeWUKyzYmc1SKOrStPgFr5yPV6yZDrPkvArz4ltZNPW5kjxJ7aKTTG3ZaxRM5q4VbfMaM4RXPt7AnZsE4unPDLrF46bPNm3tYbzlamTlQpYUuE68+WwyrDmNUnHE2vhTPqolxMVzum7WPkfPIYr6xBWa9AxcnTuCwG1Q6P8RPC/MBu/S897oV6qZTyywiSyExaYDiWhWD6CGPBySgIUlLJos7+PfQkxy5wZo02w77DTrzRjnN5xuYe/wwi1uDbwTOA7c5CScx3mS7U5gcgjjWxrcpQlo5kw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R1Z4z5HArxnHexzJ86g2cNS47HF2Q61tRaEO2up+sFU=; b=rM4Qz3ZPLOVxknyQiAzZYQtrRMdpmwAQfrSkk3bAfZHSCp/siOP3GseEzxa+MR/VrVDjFvxAS0IqQJu4COx6lbzwUnIDAahjFqfB+g6oTNUAki4u1tbJ/uwiY+t9+PpccrPIWeDx094rc0ia1t6Ac4Qh9yDrGfZ8zC/gikH55cDCBL8TvE+BF8Uwwzm86GYtsC/UFRujJ3BG7FFvlw04Gs4ND3E8uNEVPyS+bCfabyi/Tg27bj3Hfvugz2l7BSpCBm8GDpCiXb6VpqMaM3+hG/shz+yiTlKdKl6lw7Ma2NanODpGDHMd+FqnWeXefGcZVzOgraHT7yBK8ssz7X5HdQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by DB9PR10MB6421.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:3db::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.15; Wed, 29 Jan 2025 11:24:29 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8398.017; Wed, 29 Jan 2025 11:24:29 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com Subject: [PATCH 2/3] fetch2/git: Add offline build support for tags Date: Wed, 29 Jan 2025 12:24:05 +0100 Message-ID: <20250129112406.1660522-3-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250129112406.1660522-1-stefan-koch@siemens.com> References: <20250129112406.1660522-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR0P281CA0144.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:96::19) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|DB9PR10MB6421:EE_ X-MS-Office365-Filtering-Correlation-Id: 502c8ffa-4152-4d8d-e492-08dd40577eba X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: 94sooAawIrWvXACFqPKcnipb7csk+wB/7JRHoduO+eJbpEIFnVOdSyGxmVtLzeLDAug7MtkqZ1wsRapjIOBA5AI3yvoGoOrrKNV8VpgTFiKgm7qLRf23NvLhumbRiPkmzv/EQpcShVPVjuV16X/+THUAnWkPALegRHMKKsksW8MmIjTEPyyUpkUCTnJBUA/LadRYyO+g1LkgXbntC5I8keOmdrNbqDjXajQbZB/Crf33nhDvI3iljhjo146ZsPDoxBWn8bdYp1vCV3ZECWzjfA0ykEh+noxA7A2+jZyWuuFQqjUgYEggZk56gWs8N/2J3oHdbcFiJwn7G2svBNTKKmqpXklEcGGIQzGoyHlbrQrXjFBQ32uZx00iP1sEqh/zB+HFX/DNzWLfmdhTpcAH0k7caNeL/gTLca4hGOf8zwBNRKK+TvHdMJaR/UMpGhzNfxvY7tBkrECRTXhESGmQSlTTlmIhhbm27XGbT51WMn0WJ04kh2mU3RT5E+Th+BrW/XfRXwgfadgm/mAYrwM1LHE8JcKx+4Tg9+IMFOq1DYwLdqaQ1kP6PeBAtj95Ije1LYk3r/LV/I6B8PmMPhOYf08IxXE2pVVRIsD3kInQOzuIlY0WsGlMukba1vQqkXLWbfihCmGdMJ1NXf/M/ZYvD+AscHtKk1n/dKjRmr5r6rraZNcUJ9PKG6IIDkNqIKwMJuhIbbwqDR1OJjD9gIL4/hNO+1C3UjdIPyjzhSNIbMPygVEfNkjEJ3B/mDA+MenTkiLXNDD6B99rBg8cMKRTavqEa4GRAt57pPXfTcIjsPoTSdEC+Ae1fTAkUGccBTTOxw5xuWZRm41LuNrurKQEN5C29LiX6AovJDl+m4JJlKBiyn1Mc4q6ErRRJA8uRaz4equfRl2xtMabT0xLA57W4lZ+YtAoD/xb5jurjuIA3JyJgzmMZttnovDvzVLWNQHDSj1of4EtuZmgFJZb4vvkLF+LjVIAyQq2VXdfKGYs/q01FkFVru2f1omODsQyT8hnUuh4nuL+k71cfkxapqk2HVsVH4tBhqOMVDPsjb98ARiP7UfvDeIqGjU7mwvLk3onxmesG4fy6KmwenVNvMhH2QygySc3rPtz6s36a6bOYN38JSpfkv89S+PjqHYht0NlGDLG+gXu+/nwlFZdTb3jsfKLY3St+kIKbIaT6VU9qCIRcIJ+vpTudSdL9LKbtq6l9Y3ras0lBW27AhPmwbqb2y4lQaiBPH/pUhbNj2BhqMOkWrm4FCLaKJM97vNXPGaTJd8RyJugGyBr9oIZarP+1P03TAmNah7ASYbbOzSfThpajcpf+W4RbimwateiwZFkvaW3KI77aY7+VPnewtJ3KBBqnAMmToEq9DQsQY6SHV4RwO1BdvuPAG1qoq/zhU/6 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: e4nGITHBIBMnVXy4qif1kFaJDk2t8HG8LF0DOPzA5ulWptwTRi6l3Q10mUA6QMf9hu5IA0ikgGV0uudNG5jQUcc5XkN2O/NBbx6jlWl5a/lfuKPECRo6WwfdBn8p0U0CyxlZQAEl8hmVNTJF1AmbQ+6SREjqZ6nDRU2zi2IUp5RyNV5Q5LltG3D4KVtHTpSO5u2BT1gb6so+ZUm6iOvtggvFz9+bX+JpnwiiYU3AUqj6QqaKmHNDTFaYjlP5zvcAVoJKjty9Uu6T41LsbgR3TkPIpW9PN8GvKgdP13PM1s0cb+Kb27R0LLSS5RfQl603+Ic0MFnbjXh8aXO/fNU5hlZyPusnSVRKrO3UY/WdHx4yTkBpHUlWMon79y7SPxIpx1qhaVrcHuyI5r8ZwzfZkLf1r2b5bK219kzcwBIq74986scfyJXJSiyaM3gGHy8TnguRgNcwXhdFzjllnaYfWaBWi4eeQV4trtbyTo9763cX3WbInh9eeSiLghF97J2wOv+2VchcN8hmip3Ag/SFEv2y9FKnOQJ7zn8QbARdi2JAq9gKt+e2SHaXWxwndr4qlCiNroEyM/c9VsspwmohO82c/DljziE3Bil+JBE3JAhMHe4RjdxuFYiIQB5eyQ27yTarXpFDqHsiEgz/kFJl0oQqLdp9jcflV17L0+jHPfUbF5b95sNwbQj4MfZM5KgtDAsl1aJRVuYXEBOg+XgGyaGr3Z5AEWnfc9QNTSv8nV/e15xA7Wc9lyLaw9cMafHxnU91YIgFSGlp9iTt5TpBvcm3I3BNAZNXsZqonnBzxm/hxBFUbSAfmiBJOhgLDSOTylbHMyvIr6mRW2lJMk7EY7w/HyofSXDlAStLiE+EWcYgHo4tCst9BZC0t0FakNoAoApKK9jgjmKwVsP+rqjL1OmMCYcOPYHS3hDcCT2woA60oyVTvDeuD2x2UTPMxn3CYTUq1taYjhH/P7alOvZC1ppaeGkEcbKmF0hj2DrQZlX9//cUZEmkcG/I+DiZn8HNcThJ+3bwZy4A8imOJQVP/ji/fCvtZ2laUoLemZUL3e4r0EvRk80pFRfvdsTev8EpRqfHRsUObPexKJ7JCRpnhoWfw9HtJ0WxIXT9gFgNKNhQPBHYAquOB8f49n0fSHMmFqM4Dkc7mZjGh4QkkoAkIRMs3tZlW+wUrQxu9b9rmsfP766MUTkgyxMvetK47zbACxtKr4aJar0cpR6+SL9Oa0511Amp8pUEO2gCHPDC269tHKUGteUPghCDBIuh7R5n/9+ZURtSSOWUJdbFVoFP7Io0tE28/whCFoeTGJLkP6cf6kRO6j5jD2Giu0H9RsddD6LvK7gvF3Y9qxa7TQTCvi3NB+6I7jUGYx6cGftHC4XVDIcgEaAJDBWdc6qAdECaVgME6EJr381Zswpkcab6O2mI/RFZSiAUjjFgB3nxVlEaaE+SSdMMeIwMoekFIdHIsmmNkSUxVt0QCFm+FmlEI7IxDXEbu0kKGvW95tE5+Dh7Z0njPKSRE/H70DCN04190T2+KytNin4of4AxW8eHfhNFJBH50cbDLDgXL42wiM1X7V8ND3Cb1GdSxOY8he6gMgNul54wozQVOEaWXA6zQg== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 502c8ffa-4152-4d8d-e492-08dd40577eba X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2025 11:24:29.0032 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QqgNSvfglSAK77dxHX7lusHMuZwCRWnW3aISoDdMeZHedIWPiKXrV/SZaIOFPi4cQ23o7y1pnfUzydFVqrkEnw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB6421 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 29 Jan 2025 11:24:36 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17110 This adds support for offline builds when tags are used within `SRCREV`. Offline building is enabled when `BB_NO_NETWORK = "1"` is set. Signed-off-by: Stefan Koch --- lib/bb/fetch2/git.py | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 6d87c2f18..7ac690bbe 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -270,13 +270,6 @@ class Git(FetchMethod): ud.setup_revisions(d) - for name in ud.names: - # Ensure any revision that doesn't look like a SHA-1 is translated into one - if not sha1_re.match(ud.revisions[name] or ''): - if ud.revisions[name]: - ud.unresolvedrev[name] = ud.revisions[name] - ud.revisions[name] = self.latest_revision(ud, d, name) - gitsrcname = '%s%s' % (ud.host.replace(':', '.'), ud.path.replace('/', '.').replace('*', '.').replace(' ','_').replace('(', '_').replace(')', '_')) if gitsrcname.startswith('.'): gitsrcname = gitsrcname[1:] @@ -294,6 +287,13 @@ class Git(FetchMethod): ud.clonedir = os.path.join(gitdir, gitsrcname) ud.localfile = ud.clonedir + for name in ud.names: + # Ensure anything that doesn't look like a sha256 checksum/revision is translated into one + if not ud.revisions[name] or len(ud.revisions[name]) != 40 or (False in [c in "abcdef0123456789" for c in ud.revisions[name]]): + if ud.revisions[name]: + ud.unresolvedrev[name] = ud.revisions[name] + ud.revisions[name] = self.latest_revision(ud, d, name) + mirrortarball = 'git2_%s.tar.gz' % gitsrcname ud.fullmirror = os.path.join(dl_dir, mirrortarball) ud.mirrortarballs = [mirrortarball] @@ -909,7 +909,11 @@ class Git(FetchMethod): repourl = self._get_repo_url(ud) cmd = "%s ls-remote %s %s" % \ (ud.basecmd, shlex.quote(repourl), search) - if ud.proto.lower() != 'file': + + if bb.utils.to_boolean(d.getVar("BB_NO_NETWORK")): + cmd = "%s ls-remote file://%s %s" % \ + (ud.basecmd, shlex.quote(ud.clonedir), search) + elif ud.proto.lower() != 'file': bb.fetch2.check_network_access(d, cmd, repourl) output = runfetchcmd(cmd, d, True) if not output: From patchwork Wed Jan 29 11:24:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Koch X-Patchwork-Id: 56212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2F1BC02193 for ; Wed, 29 Jan 2025 11:24:35 +0000 (UTC) Received: from AS8PR04CU009.outbound.protection.outlook.com (AS8PR04CU009.outbound.protection.outlook.com [52.101.70.31]) by mx.groups.io with SMTP id smtpd.web11.10287.1738149872854193926 for ; Wed, 29 Jan 2025 03:24:33 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@siemens.com header.s=selector2 header.b=wGk2yFQf; spf=pass (domain: siemens.com, ip: 52.101.70.31, mailfrom: stefan-koch@siemens.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ALjiGCALJlq8JmFdNqvAZ0zO1pe0q5s6jmyzKv62kARx2eAPkKA0i10hTCm2fXOcY1bjZOZc6hiSvth6meRxudGTMNlZJAjlSVRFW7kE8Cq8WMhxHaOm/e6LOUGeWo6y7OtrJ5O6m5XXslPQY10kqDHnveF2OJ3bFlL6XgV9PdFueuZFrvLg/g6X8F0oY+OptuDsqTuErgqgtMjdFjRpWVUlUoTDHzZ2J0Q1dvLvlUUd51gEZltRTyX/qTCLQImnMfms/bvMziFLId0qcWyT0Z8ibwzSxakH5LeHuJQYHVuZALTFt9CeUvCSqsFFBtYhkMnJiSRaA0HEbFh2i3ECwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9tNlCAi9Lby90d0H2KjFaOq0euYL1mpv4fN27HDAcsk=; b=vVMcSQ9cuSZcvR503F1PimwF6cagzjEZqMWTHBHR2OmkEYLYCV1HnfxdhUGf+iYWkfXKap5Mnjsn1q0NYgF+GR7s2uf2Kir1y5/hlq22RS/UnQwwScaLAXv0UXunum48j+zov/bTBwQ7idqfbQAtANO7wefjXUwJLB2z5K3wtCWffIVeV4jnKamASA8mt3TAJWWpL3Gh3HPX51WO3fJ5N4yaZQ3nGay9EXFpDVdA1gwwNYiGUVZW2IENqCTFRvIBegLupWvDdpLekGmIbGr50SO2em9+jBGZaLkM0+SBtYC9sy3UZkLfCZh8gzlH1BQPpcf01CamVfz0LBJ+2QyKiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9tNlCAi9Lby90d0H2KjFaOq0euYL1mpv4fN27HDAcsk=; b=wGk2yFQfA7ZpQwUFOktBlB2oD/OQIQIMb2Gdf9kNwW8veikTLsi4Ew0FRBjRZ1K/iR3FXo7NVyoUSI8I2pi8X4W4TRpGBVV3/TLiY/f5pjvl4BhCC3/3LLEhwJdTerwZyEZoxzR1M8ay/BPccnHaAR7WcEGabtfpi0RZ/XS+dmxXkUy/4sbrXXuXybXA5jEObzPYEGsTzEt8Akrn7Tp1EpJmi+IaYd+4ifuE5a0xeb1SDzHZu4s0uwFtvvHf7yO5UZJSUQ2tB+2t6qaQR2VpQdutoSypAQUdBwwGGrem/2/ql1q9MVFxrkjWPBuBraT0JZDEQNBQLLDQVluantwVQA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) by GV2PR10MB6233.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:150:78::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.18; Wed, 29 Jan 2025 11:24:29 +0000 Received: from AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec]) by AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM ([fe80::71d7:e998:3abf:a1ec%6]) with mapi id 15.20.8398.017; Wed, 29 Jan 2025 11:24:29 +0000 From: Stefan Koch To: bitbake-devel@lists.openembedded.org CC: stefan-koch@siemens.com, simon.sudler@siemens.com, jan.kiszka@siemens.com Subject: [PATCH 3/3] fetch2/git: Add offline build support for LFS Date: Wed, 29 Jan 2025 12:24:06 +0100 Message-ID: <20250129112406.1660522-4-stefan-koch@siemens.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250129112406.1660522-1-stefan-koch@siemens.com> References: <20250129112406.1660522-1-stefan-koch@siemens.com> X-ClientProxiedBy: FR0P281CA0213.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:ac::7) To AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41e::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR10MB4959:EE_|GV2PR10MB6233:EE_ X-MS-Office365-Filtering-Correlation-Id: c339a6b2-0cc3-40c2-03f5-08dd40577f27 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: UlusUnhT3QnKuFZzXtQODlljdCoEuyujH5B15OOZDrsc1qBqt1vavj8Ych089DQWCA2A2SjwBtg0njxqqMgeLlbXkC4VO3Gmf/sf16uLtMCJ1O1WeEkIn5XhHNfuwIGEolopSxePLcnb1bfofuhe62EvAy0IQZ9NKP9qSvs6vMwkiMsqr+72qIoAsyqDkRxxzS4t38O/fsBguDQHVeg0UHC5KrgD+LPAWny8BHEmD8B2lICDLNBAN9fChCYUejDl5SJIjPTMC8i6T6683bpqoC3iGiZcfsRL0RO29/44EvsyVOmB1U4XlbDWGpZE4sNqeHobXCewyLnDhLIYYifkhVWQ7BV0iedzDHv8PukScpL3JzX+HYTCj6KI/XrC6NL7/j4MK2zl3G8PyvJTMS+1/QCSSkVNRFWb5rINExpBx281D07SPHa0VmvjbIFZUgW5z8o6Kqrp3hoGStaYfg1rsqOh8cYkytzG25PqB81qGWASKKYVujLtUrCCvs+TaUyD73QQwDdW7gGhpkOeX3LPCIVak1DFKmP29v2n2IUmPW3/WY3QGoreyaO2T5R14VdYLart6nQ6Y9fFJ/7ScGQGmkV93PRYDWjtNLJCFk1DiWFGmhxZ3SsL31AO30XvjmlqeOurwJpWTSQmC97iQXpJ/ay2PQnPeXgTc4fQi0i8Tw9R9HNprPd5WTufgVldiAP7M99OmTHmeuBAEOmq1QK5amfKmdd4miDi389z9maFtepFuwwqQRo5mlhgmhRrq3DPyPIKC5ugUk7VeGLFOUls4EWS6ng1StAe5+W/HOR+1JtVNEST3PaLN3nYj50p4nY2ImBXlNigXLqfqyXZIKlSW3qj8jEZeiw7rUQMfexJVdqu6edca42JgLyY2+ljTjkZSwaHTRhP51cx+/VFZK1k7hkPt6c8GcB0YVDFw8cj7DOWcZOW7MImqmQFTFBevhCWYMJt2VKTl2quyx0c89ASwUERa6RxGG6b/CEDbYJTrYl5kte6rlUZgXI+dH4ZYVRdi7rDcm86utY12s0sXFi/6p0yiR4N3GkZFoUaXWIBfQhH+Ni8v625NKQ8f/r8CCwvIpnYXP/8bOsXmkPrzajONuzIlMkKK8r9IAVqFJkt4BimZvYGFmSvjvUxdlelkezy2Eysz05f3y46xw1O5yC7mIDIGeeyxjaV0YsJav5ymydwkoZK9iJREgGPjquSAai1lFnaFs3x1KpIiuhmZA3BBSM0GtbWgkfgizcJtjq2gBqnU+vBuTk0aaURMYoXjRev82CTsuu6E7r79B1Lf7OmMmyCJLrqgQ9t/voiNKyPVHMoT/ptB/QTG6kTRyvadYN1h9B/7+SvCKTkLODSbFQpTyWI0x4lmaY09+vPFfmm39MU7tamMfOYtGsRtsrYABI7 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: t5Y852fHB+wwTIjxI5tMCL6VLOE7tT0YG4meehv/aSX4lGMSTKXA6Ju/oBAMv4m0Gd49S45IU9i/oyn6yQIhiLY/x0jtDnjfvVwRUDm5h2nBXmDonIJC7v1E7F9fyzZX8kZzXa79Gt5Up+LEv7FbaakYKhOU7b/JLjQi2qlOHsJ5WXpmD+bz6QiRFdjjHyBXQ4Ihi5oFq2U0a5qOnT2yVuBCtUMMdxt4orLbaLUcTNqUaTJtTg2nImEBZmUz9vz165r++ul0KKhNQcY37DB8l/TNXJWyJzXPAyJmV32d/aLyvMiHw6XcE6MrWBlHA90IvTfNDWeb94OLAUiua9gJz2qDuCj0yzmuw35ZIhVRQ6oZLid//Y7BLC3gUw7CoE4QDJ6Ik3LlbODCD+GAFFuPHKDiN5rJyveEglyzY6psBYVpnrPB/wyA7ZRFHdseDM3qDlkdBxL5Dy1c6woR0Qpw1aMIOEonTuzykVn9ZlvxcpumbTrdhUQ2Mkb2X6zUciC3EzmcMMfWIzAr9vAIiyQ1/xwfdrpGB7a9njNtAJZ1bFfDhonQL9wjtrUPIY6kFanTJjaeeMbwfBAI5s6C6TfnbUfSbI7UkAyJqEaDR9PRIzkeVjdl53SHpFjQf1TpFxvAFS6ul+mvY3HmfiFCad04vftq9l4J7S5KaxIzvDp0ozpmPEXt00nWrInkB4spLRqe1REHOHkcd97qOyHzuoIrhm1+OoPl6/xayARw/HUMwV3b5P8XFuoHxLgW28Z01BgjrsmQ+zN4DeZgh9dQb1q0ysOQhbD3sppijzcFqFEQS/XQXSx3lTej2Z10TE1ARQGInyQUzl0ZOrYqGZh3uEzFAm7hzcQaJJwJaaiS9it6XbgHwj8TEvQ4jyRLnNo4yN8TwEeMt9KqECK2wKZ8fot+5eZDDSQTGNQ3mlX8MA20LrAWpg+LC5/QOpAlX7oH+XpqHWf+Gyoe2hjTXPhJO5NfC9vYVb7Yx/w+cb0mDg0+vlfoeqHAlJWsKfxDMkHkJdWR1nd1riRzl5utIf79DtW+6i4nOgtdZFqIXJe2xTGI+YRcvJp1856PhPHhrBg9mrtFJ8zDnM4wwQAtblF6ANDhzhj5428pli9G/tRZbdX7v+kThIrpIUW0asx+eu6G3tm6i26MeYEY/NvlJ3Jg+lGize56zHD9hgXnI3Z0jYRhiAsp1NYc5l8Ze+AfxaQ2i8XoHZGiJUGwmkLKOeZYBx2/gM/Nx1AAe9c94DxJDEAkPg6s3BtaLCgri5SzCrL/31h4iCdJ9kvON5u5Nbt2t5P8YDNxyFu9rf6iL+rgcw3j6FpyUqD1/4Xu5x5upPGLB6DH0n+aMLn14i94ibQT/QSvrYXTwBumRN6ydaC/iU6A+B/bMsbRieiDazJqAJ4fdyq1tMBloY3XcNjnZSc2CbOtpq+zUu0CgBy6EgnAF1+rb6w3FmJV0s48ms79+2x5N/Vl7wrl1juFXU39jTeLrrGo5T37+I7Oe0ys1N+NE5bqped5rfMQO0V0AJsJ5YWSwFJMgBlyeuaGT+gKc0NbK6/TxUnjOouUccdDtMtWZRw9oW9xqKsH0hiFYCNhphZPcYCLpIJH/qL4bP9gZC3MtUtK3Q== X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: c339a6b2-0cc3-40c2-03f5-08dd40577f27 X-MS-Exchange-CrossTenant-AuthSource: AM9PR10MB4959.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2025 11:24:29.7221 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Mn+dCKcgx+4Y16UckC/u1RuIW2kbs5ZMaIUmHSsU6f0Om9hxJrythXuYlubdkK7vg2iu+hS0stQE7/0XcIYefQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR10MB6233 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 29 Jan 2025 11:24:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17111 This adds support for offline builds for repos that use git-lfs. Offline building is enabled when `BB_NO_NETWORK = "1"` is set. Signed-off-by: Stefan Koch --- lib/bb/fetch2/git.py | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 7ac690bbe..b7d36c3f3 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -507,6 +507,7 @@ class Git(FetchMethod): # Create an initial regular clone if not shallowstate: runfetchcmd(clone_cmd, d, log=progresshandler) + self.lfs_fetch(ud, d, ud.clonedir, ud.revisions[ud.names[0]], progresshandler, True) # Update the checkout if needed if self.clonedir_need_update(ud, d):