From patchwork Fri Sep 5 06:20:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 69719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CC64CAC586 for ; Fri, 5 Sep 2025 06:21:24 +0000 (UTC) Received: from OSPPR02CU001.outbound.protection.outlook.com (OSPPR02CU001.outbound.protection.outlook.com [40.107.159.75]) by mx.groups.io with SMTP id smtpd.web11.14988.1757053273963266587 for ; Thu, 04 Sep 2025 23:21:16 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@weidmueller.com header.s=selector2 header.b=m96DEGBc; spf=pass (domain: weidmueller.com, ip: 40.107.159.75, mailfrom: stefan.herbrechtsmeier-oss@weidmueller.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NuKs5pYPR0TwD+ZFadE9RetokotzPgaBZWEjHg9RkJDw7skB8HxoEVBAQM7D7gsMk53ICk8ms2HjBcWFRNVt6uSdDQgL7tiJpvTQdyOJCPoP7K6l4rePKlwQ8MLyFYfYjepZ88/2JU3EMCtT0/3D1WH4atBbMzoUfnqX1zIQhLdfKkiREVY+2wftgBY69Qy48gtnzDS84rTNpbxP78JkUEMZsP34mTBR32OHQhHDSy7m90/Y3wAE7HzFw2Tf1WczkwtrnqXY3mUARwDPv0uUUYSC4WW2VLqrSrtUgHkAE2ogTnYuSnP4BwZYNi1PIL5xGyI1qW2GOHPnd3mh3hkVcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k2/DaQgm8ceDMMNT6cw1z+Y4j1nLtSWvhhU2pOP+lmE=; b=OtTk0VomJZ0GYYfBXVR1K9IQHUdxWw/IHBiphSrChnCjKxVvifOLrttB+wT1Q+l//ND+BmumsI+dBUG6DXOw/Rt3QVZXPfbKH/9bhSPbbtjMUYag2CJbwHizbO46xYXNGEtnH2cGdlFnov1hunbHftMsAtxCIDTmTDunXypRToBFsYNQrht21KyqLf/iKu2AiG/NIEc29tlsmPpNqIGW548q4R2sW5vmGuT2CS90MiwvI55JgkD1K0edQ9FuhBYe2dfZYeAXpFDLBiDfHruC+8dInRs+lAE/sH47aJih36UKkyaF3e8n1nDuQ6OOubGlT6lK33pJGC+zDcnGF/r0oA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k2/DaQgm8ceDMMNT6cw1z+Y4j1nLtSWvhhU2pOP+lmE=; b=m96DEGBcO+b0perIfA6jOSt1Bpo9NC194FnDfltK626T47OvgepJD9w828+tEwRWRI9jI3Jyy7rVDSGGYxEYS88TQ9kSaPoRS1PsZWnfJSICZF/SDsKZDalurwOTxKwhQCy7RnrDlWL4wHmjyXv+9sNlb8fTPSd9wTFc+Xv5/v94jeK2QrrHORHjAZXGNaE+IG9iU5vVJLk5eiwN1zs7Oo7hSTElyglykH/2BYdZo7F9v5McS1MBk2s2/uAz45bpdZ7l8BfJXsTHwfzkqvU64tJ/cvIQkgWMpjGxe+s/5ztmmgIujzAS4wA9222W6uq4khK+i6OnCaxitiZxy6yAqw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from DU0PR08MB8422.eurprd08.prod.outlook.com (2603:10a6:10:406::6) by DU0PR08MB9632.eurprd08.prod.outlook.com (2603:10a6:10:447::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9094.17; Fri, 5 Sep 2025 06:21:10 +0000 Received: from DU0PR08MB8422.eurprd08.prod.outlook.com ([fe80::4ac7:a794:5e4b:fc25]) by DU0PR08MB8422.eurprd08.prod.outlook.com ([fe80::4ac7:a794:5e4b:fc25%4]) with mapi id 15.20.9094.017; Fri, 5 Sep 2025 06:21:10 +0000 From: Stefan Herbrechtsmeier To: bitbake-devel@lists.openembedded.org CC: Stefan Herbrechtsmeier Subject: [PATCH v2 03/15] fetch2: git: replace destdir variable with function Date: Fri, 5 Sep 2025 08:20:46 +0200 Message-ID: <20250905062058.2563806-4-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250905062058.2563806-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20250905062058.2563806-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: FR0P281CA0130.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:97::17) To DU0PR08MB8422.eurprd08.prod.outlook.com (2603:10a6:10:406::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU0PR08MB8422:EE_|DU0PR08MB9632:EE_ X-MS-Office365-Filtering-Correlation-Id: acd933b2-818f-413f-0365-08ddec44682c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|52116014|38350700014; X-Microsoft-Antispam-Message-Info: mqmeBlQNe10dFanSl3B8DFrUPr2mebnWF5kf1H5jlurTb0POvS8Q+pamPMfNMcltWBodpjdY7F8JpxRf2Qnw6r+BRegiXJKDJuIq9/oS6sqEGSvHpYR+u09tbHhShc6CYjnaBRmoXvujrikIiU8PbZqyGBuLPaBQhsXuuHDyINRhi2zARNOVaWspRU+1GP6iMjYHTOeHeNjB1zeVFIiaiDUAKvKJsMM0SnaiFMKKGfgcUD+5R7x7fSydkKBkiSAaJnPx6gcJlqqQMFGV5wlK8/KFXt57IQlmxpJ7RO/2R6AmhKN/qgd8D+dGPgX3m5r172P+f93cUVfP2MwF/tqFl9u6x+Odtbjx7SNvtwqrKEWJ1R9dVuE0pXBSf0HkC8JDQp6Uj4cJsoH+bv7KFpLlkWKFaR9MRzX9rAQDea95vBt6EnQGViWKksZLDgSToLb8bT9Dzb9md/IGGohA/pp7jc1DjTeVTg/BDh6Zofb2aGn30WxVBv1D3cRW2M2GwwctELUXvjHr9J+nZCcq6oHQe23U2KzdZ1c5JFCkEEuLKgNrNLpCCzfgvns6Vh3C66nASx77r6MIJHCEQ9QTD2lHx6DIWJkCuvlyBvzkpb2ORJ9J58USmulE/w2Y4PkxxITpUZNq0OWDZQ+zC40JOnvVsrdh6kvqTn+e2pIEqmIWCbwxeF0gzCo17JfTjZD+PLTVzqEGQOnL3w+EsA/HCxPAFZYIcmau7bJufYuurOYaAk6hlfIAWts5MemK1pqg6c2EuAtxkmC6lhF0yrxT+CFJTnglmQ2FzaVVgKk2x3gXkEJIUqgKTAxHU+uku32GJ325h8fe8l1+8NLo0A7HNf6U5LU7HQzfgL+RMTyqk8omwFH8vJyp670f3qbjB2DhCAH6wIOkivVd0gpCTXeYmwVao+XNTPdXT+t0g08R6t5hai3fJnRMboI4neCioHKe6S+baLQMn4wYeODp1JNIPUuBmHdTajZdMX5kQHJf4mh5FrscvC7BdR2a4BFrAi2Xpc7NIUlq+YNZSQLR9Bg0k+zFg3s59O1cznuWG4M1pZp/4m8qYcuP9+ncLEME2J9VbDFUVW01aAwp/M4oD5/P74MQy4JwrY08zTffWs+Z5z5UECbNeT20W5ukl4eUylNlW/RWHonFxY6HC1mJdF/UUdwcoI+Fw0H9jm+Lp2EavgLlV83VD0CDLCti2gPU1aWcybpvl+yXkGQv8ArpiMCUK4XqLOES8+w4e/TMrSU8D5+3/KNGBWPpzsBiErWWUM4Xu8tEx58CHvnh4wswzKjqea7FGgGAgpLZuLG54vpSO9glf/tDOS5ZB0ZFGBq4znk5aZ6Apzm8o5TGivSStwO/V9+lJ/mw9G5oELBxuJ+AZiP0qeIMtxZXowSvcY3a/HJAcUJng/MhXE29FLGAadlbmL/PFq5eF8IDZp6OsqDKhXzSB4D3kDCOC0wlHUgRbzJfuIm3dZcHqdzjs7ZN3B9cemC0IA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR08MB8422.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1OaEiZwcBoUwywjuVWehg13cV5g9iCOegA2/i7RtWaXUX/9m1kkZtWrQzKcl7o5KD0Rsk24Q/UZP2VNOclSZgwq4yhmsQGoLV6AGSlH08LhALjBOZj+AI1aoCx7BtkyIfzM4M9BQhfxUt/IRvJE5w9EAfPJzJ6OmX0jjfXBHmR3lau0e+tUIsQP2iRe8AL15FYkt9GXc64+Y4OeN/PkrD80d6bw7zKrinwMbfIUcHkgiHHGeuIxwe7OePSBdtovd//Ot5ZAf+2Nfwtl6/2HdV1PcrPFVd1ofmajRfZYw0qGl+QbcfNiZoPkR9Fr7LMI61BaVzvFoFujxgNcZgQinmShEAOT+Z0DIimfjzw4XdTyBvCbQjEHhdmqaGpaIvKfIzRBr4oa7auyhkaKxa7D62KH5rRK3W4e823UWL6zgUXqc2jhlAAWC4BtVDO0S8BA+QISDASHbYbsZ0+48sWaweN6RmG/6CUDkDQyXD4yo7jKLs/5n6Z3Xzh5EzAXoadt/DswpomvdX56vwN2HsC/HcPePJwAXyaisls2sgGHnMFXDDgEdtu6gS8/GwQKisiUz1q57pp29cJgLZ65RMz96i+nNZx6YtQ3RKJsnfcSvjJN5XnpAn+bHhfjTpr5fA3cV2mKDuRxHpmZdoXeTufkeBlAyU2brEBB7fQK4QzornvdpTBiMhORlOdHOlVoKuPPSEJ2xsEUmoGur3U/lieXCzH+MmcybGTEEBkVI/KTBi0NvifdW8WQdcAqOXdfbXaVpWfSRQKKVHj4juduPRmwR2zo3Kwz+FUtK7rWpdhj/BGkL4OX8PS7DWqqoY2V5qstMlFxu8Z3oS8Xkw5CJZ1pp2pZNLUJixi6wHRftrvNP1PRvURehD3jqzefIlG1iTgg8pRWp8kjZ6gSO/y2KHcmNTRe9v+k2tiEIz9axAcfC5wpyVKPDOBgcRNBrVS1wB+DqsEVBA3HgI6tAB6BnzzXZY1JjC/sW++XAwE1R9GSRIxS8lIcaAMwW/iIvXysdgU/wqJbDg0IM6WGZ7mITUvjGDo27OPN9hYittiD/70SEE34i3e0LgFNfmBUs9FjRcmUUoJQ2vHHCxLUiaygeM9janfVYhqNBmjHbb1E4+U8+ROpeUfeAqQsjtR+0SggjBGpwzVs6g9EihCGR33ffCp9DEjo3ihOx3gQBO61IA4SO0jvz/zAF8ukmKg5v2z0o+cBOazwPsPGC1qHy7AYRCAoYpcJN8O75p4EcYaP0o6t3BJPDW74ZA92mYk2qytTEO6tJiEEOw4MDzmeZzsDyOVTmmYveD0gQLBQJoITkBFS97/cNf7v6ZbuSaXe/9B2wjcTJqxxPri0trfGCBDFTIxUEZfWZqV37e6RPWsnj6hU9yAuZZORD7scBVqIV8IE4pp50nIy+T9f5pCK4DURW8YUzPJ4UdL7rj3fG4IVGQ6ycgWsuFrb3coGfm+lmi0oJ2GQV+RiMqrEkAczLsYyYVZ5khsaLHJDcxEQSvsp7IveXwaZuAK7riAt798zXh+YRRs3nxWXyaooL3G70hl1n9p0iTf9ZAMgADvRbldd3ReBrF/aIZVhRSF2V16samfto6wuxKZ+hI1Pl/V2RcAuvkuKAxg== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: acd933b2-818f-413f-0365-08ddec44682c X-MS-Exchange-CrossTenant-AuthSource: DU0PR08MB8422.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Sep 2025 06:21:10.7726 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /3zfSgmsslVJCRw4XsnFDXCsXV/osigGoQ6j+nAjfH9z2ltc+/TMSzGTRMr1Kz3JgvXecAaYkHrlQcFWKtU/zQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9632 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Sep 2025 06:21:24 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/17960 From: Stefan Herbrechtsmeier The destdir variable of FetchData contains the volatile value of the destdir parameter of the unpack function and is only valid after the first call of unpack. Replace the variable with a function to make it independent of the unpack function. Signed-off-by: Stefan Herbrechtsmeier --- (no changes since v1) lib/bb/fetch2/git.py | 25 ++++++++++++++++--------- lib/bb/fetch2/gitannex.py | 10 ++++++---- lib/bb/fetch2/gitsm.py | 20 +++++++++++--------- 3 files changed, 33 insertions(+), 22 deletions(-) diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py index 52fffe21d..6d3bc3447 100644 --- a/lib/bb/fetch2/git.py +++ b/lib/bb/fetch2/git.py @@ -656,30 +656,37 @@ class Git(FetchMethod): # The url is local ud.clonedir, set it to upstream one runfetchcmd("%s remote set-url origin %s" % (ud.basecmd, shlex.quote(repourl)), d, workdir=dest) - def unpack(self, ud, destdir, d): - """ unpack the downloaded src to destdir""" - + def destdir(self, ud, destdir, d): subdir = ud.parm.get("subdir") subpath = ud.parm.get("subpath") - readpathspec = "" def_destsuffix = (d.getVar("BB_GIT_DEFAULT_DESTSUFFIX") or "git") + "/" if subpath: - readpathspec = ":%s" % subpath def_destsuffix = "%s/" % os.path.basename(subpath.rstrip('/')) if subdir: - # If 'subdir' param exists, create a dir and use it as destination for unpack cmd if os.path.isabs(subdir): - if not os.path.realpath(subdir).startswith(os.path.realpath(destdir)): - raise bb.fetch2.UnpackError("subdir argument isn't a subdirectory of unpack root %s" % destdir, ud.url) destdir = subdir else: destdir = os.path.join(destdir, subdir) def_destsuffix = "" destsuffix = ud.parm.get("destsuffix", def_destsuffix) - destdir = ud.destdir = os.path.join(destdir, destsuffix) + return os.path.join(destdir, destsuffix) + + def unpack(self, ud, destdir, d): + """ unpack the downloaded src to destdir""" + + subpath = ud.parm.get("subpath") + readpathspec = "" + if subpath: + readpathspec = ":%s" % subpath + + root = destdir + destdir = self.destdir(ud, root, d) + if not os.path.realpath(destdir).startswith(os.path.realpath(root)): + raise bb.fetch2.UnpackError("subdir argument isn't a subdirectory of unpack root %s" % root, ud.url) + if os.path.exists(destdir): bb.utils.prunedir(destdir) if not ud.bareclone: diff --git a/lib/bb/fetch2/gitannex.py b/lib/bb/fetch2/gitannex.py index 80a808d88..dbb74e35d 100644 --- a/lib/bb/fetch2/gitannex.py +++ b/lib/bb/fetch2/gitannex.py @@ -65,13 +65,15 @@ class GitANNEX(Git): def unpack(self, ud, destdir, d): Git.unpack(self, ud, destdir, d) + destdir = self.destdir(ud, destdir, d) + try: - runfetchcmd("%s annex init" % (ud.basecmd), d, workdir=ud.destdir) + runfetchcmd("%s annex init" % (ud.basecmd), d, workdir=destdir) except bb.fetch.FetchError: pass - annex = self.uses_annex(ud, d, ud.destdir) + annex = self.uses_annex(ud, d, destdir) if annex: - runfetchcmd("%s annex get" % (ud.basecmd), d, workdir=ud.destdir) - runfetchcmd("chmod u+w -R %s/.git/annex" % (ud.destdir), d, quiet=True, workdir=ud.destdir) + runfetchcmd("%s annex get" % (ud.basecmd), d, workdir=destdir) + runfetchcmd("chmod u+w -R %s/.git/annex" % (destdir), d, quiet=True, workdir=destdir) diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py index 5869e1b99..d9ebe8d18 100644 --- a/lib/bb/fetch2/gitsm.py +++ b/lib/bb/fetch2/gitsm.py @@ -205,14 +205,16 @@ class GitSM(Git): self.call_process_submodules(ud, d, self.need_update(ud, d), download_submodule) def unpack(self, ud, destdir, d): + subdestdir = self.destdir(ud, destdir, d) + def unpack_submodules(ud, url, module, modpath, workdir, d): url += ";bareclone=1;nobranch=1" # Figure out where we clone over the bare submodules... if ud.bareclone: - repo_conf = ud.destdir + repo_conf = '' else: - repo_conf = os.path.join(ud.destdir, '.git') + repo_conf = '.git' try: newfetch = Fetch([url], d, cache=False) @@ -220,7 +222,7 @@ class GitSM(Git): # checkout dir new_ud = newfetch.ud[url] new_ud.modpath = modpath - newfetch.unpack(root=os.path.dirname(os.path.join(repo_conf, 'modules', module))) + newfetch.unpack(root=os.path.dirname(os.path.join(subdestdir, repo_conf, 'modules', module))) except Exception as e: logger.error('gitsm: submodule unpack failed: %s %s' % (type(e).__name__, str(e))) raise @@ -228,28 +230,28 @@ class GitSM(Git): local_path = newfetch.localpath(url) # Correct the submodule references to the local download version... - runfetchcmd("%(basecmd)s config submodule.%(module)s.url %(url)s" % {'basecmd': ud.basecmd, 'module': module, 'url' : local_path}, d, workdir=ud.destdir) + runfetchcmd("%(basecmd)s config submodule.%(module)s.url %(url)s" % {'basecmd': ud.basecmd, 'module': module, 'url' : local_path}, d, workdir=subdestdir) if ud.shallow: - runfetchcmd("%(basecmd)s config submodule.%(module)s.shallow true" % {'basecmd': ud.basecmd, 'module': module}, d, workdir=ud.destdir) + runfetchcmd("%(basecmd)s config submodule.%(module)s.shallow true" % {'basecmd': ud.basecmd, 'module': module}, d, workdir=subdestdir) # Ensure the submodule repository is NOT set to bare, since we're checking it out... try: - runfetchcmd("%s config core.bare false" % (ud.basecmd), d, quiet=True, workdir=os.path.join(repo_conf, 'modules', module)) + runfetchcmd("%s config core.bare false" % (ud.basecmd), d, quiet=True, workdir=os.path.join(subdestdir, repo_conf, 'modules', module)) except: - logger.error("Unable to set git config core.bare to false for %s" % os.path.join(repo_conf, 'modules', module)) + logger.error("Unable to set git config core.bare to false for %s" % os.path.join(subdestdir, repo_conf, 'modules', module)) raise Git.unpack(self, ud, destdir, d) - ret = self.process_submodules(ud, ud.destdir, unpack_submodules, d) + ret = self.process_submodules(ud, subdestdir, unpack_submodules, d) if not ud.bareclone and ret: cmdprefix = "" # Avoid LFS smudging (replacing the LFS pointers with the actual content) when LFS shouldn't be used but git-lfs is installed. if not self._need_lfs(ud): cmdprefix = "GIT_LFS_SKIP_SMUDGE=1 " - runfetchcmd("%s%s submodule update --recursive --no-fetch" % (cmdprefix, ud.basecmd), d, quiet=True, workdir=ud.destdir) + runfetchcmd("%s%s submodule update --recursive --no-fetch" % (cmdprefix, ud.basecmd), d, quiet=True, workdir=subdestdir) def clean(self, ud, d): def clean_submodule(ud, url, module, modpath, workdir, d): url += ";bareclone=1;nobranch=1"