From patchwork Thu Jul 23 08:11:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jamin Lin X-Patchwork-Id: 93313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDE21C531C7 for ; Thu, 23 Jul 2026 08:11:38 +0000 (UTC) Received: from TYPPR03CU001.outbound.protection.outlook.com (TYPPR03CU001.outbound.protection.outlook.com [52.101.126.87]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.18795.1784794286963932835 for ; Thu, 23 Jul 2026 01:11:29 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@aspeedtech.com header.s=selector1 header.b=O4dWAYzV; spf=pass (domain: aspeedtech.com, ip: 52.101.126.87, mailfrom: jamin_lin@aspeedtech.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oYPzWAkVSZP4Nzsef9VM25hN1exHGQUkC/526GjiyAsoangt7nkaxshYfyNiXQFRfBmHampGmxb9D+EsSGq/MdE/uAnDx5JTJ5SjJ2cQSL/3Minql1+HfBP6ViPejabCitluIHHYRROEqga76tzAAIxX+b/fkUFCiwE+eymrdA2mX7CQY/kP12UaISyfLrtx3h56LacI8NAuTmsvh20GiUzYqvLTzZuFKWLpZTkYybpe27sdKR7JZuQWNZzbDaBUTxl0/GdBsbKuRdeIMYrmel5/rRLeNxDIN6Yq3W5M0x/oSZ7GKtr6Wnf82Ycfr1ngIaFv3p/S9FcNv79fgr9VVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ehwGFgobKnuK1NsPI8i7DPrqoWYo3pkos9f3oUkWiSM=; b=IHDhgE76Hbc2awLlvGX4kwhNmWHU4XIjNnekx8UjkcC6duHnfmTeLh2Rm7e0lMHKz8oeOrEAPIhHi73QcymcG+UymexHGtWu53Iw/XwdLMFNh3yLqqqiU6yl8Fk5FHkP4KEdfmnW45Etn3qhF3NyR6gJG9pEGoqeUb2kOdR4lfg9BX/wZx9C5bAg+q8XgD5MFgwSb6ke53NFvmIsy7NucEj9Kja85RhD9ibkqB4jC5UcV2QYTJZNe/+OwkwgX825HIWhtEvVzk0iaofEGm34DrR2+QvGwBr7Jsj9Sv8aWnOCQLKcbNjHlQT0nOrAxRQhSalnzmDVq3wQJLknIlRAcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=aspeedtech.com; dmarc=pass action=none header.from=aspeedtech.com; dkim=pass header.d=aspeedtech.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aspeedtech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ehwGFgobKnuK1NsPI8i7DPrqoWYo3pkos9f3oUkWiSM=; b=O4dWAYzVZebc4xX725wjs3YtVDu9HazxZQBikNPj1lfneOnk5kElf7UtbWa8wApkw2plZdswnpe/nvEu4o8Pk4tzXDtkMtV9V7lz/k6fNGRmtC6lCklGo1iH2Ja9I1AfvsPDxUyaWOQtqrwLQ65ngN8tWyu0kHVEP7sa+QRD05roZA7Sw2K4LS4ntkSyc+bHWJc+NBnteusck7RCjYljRFpOfbUlilfpvy1HiL5rxbWux2ZSrPDHMWb+sjHcIX/vwJDewqjOPAsnxM0LYlfTu3Z/UB7+vY9W8dyT6hzTsdXMNZcGKe+evoL/YK3OWuMQXVXyGjIDKMoSSkqA/Eaatw== Received: from TYZPR06MB4980.apcprd06.prod.outlook.com (2603:1096:400:1cc::10) by SEZPR06MB5022.apcprd06.prod.outlook.com (2603:1096:101:48::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.245.10; Thu, 23 Jul 2026 08:11:23 +0000 Received: from TYZPR06MB4980.apcprd06.prod.outlook.com ([fe80::ea8a:7cb7:4822:2fb3]) by TYZPR06MB4980.apcprd06.prod.outlook.com ([fe80::ea8a:7cb7:4822:2fb3%6]) with mapi id 15.21.0245.009; Thu, 23 Jul 2026 08:11:23 +0000 From: Jamin Lin To: "openembedded-core@lists.openembedded.org" , "alex.kanavin@gmail.com" , "paul@pbarker.dev" CC: Troy Lee , Jamin Lin Subject: [PATCH v3 3/5] devtool-source: Convert nested git SRC_URI destsuffix repos to standalone clones Thread-Topic: [PATCH v3 3/5] devtool-source: Convert nested git SRC_URI destsuffix repos to standalone clones Thread-Index: AQHdGnrZtiIoNhwZ3EODfKkGkiR5+g== Date: Thu, 23 Jul 2026 08:11:23 +0000 Message-ID: <20260723081118.1558249-4-jamin_lin@aspeedtech.com> References: <20260723081118.1558249-1-jamin_lin@aspeedtech.com> In-Reply-To: <20260723081118.1558249-1-jamin_lin@aspeedtech.com> Accept-Language: zh-TW, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=aspeedtech.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: TYZPR06MB4980:EE_|SEZPR06MB5022:EE_ x-ms-office365-filtering-correlation-id: b72a21aa-ab71-4bd8-37f2-08dee891fc45 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|376014|23010399003|1800799024|366016|22082099003|18002099003|38070700021|10067099003|56012099006; x-microsoft-antispam-message-info: 0ALcLFD+JIdyws1Ww1ZXbUBi42qHS60+x+962vHiXoeAx0jcpdSBk64EcTCZZvHUR3RfLuM03eX5OEps3HggC5+IVzlEFaMuA6lKWfAOmBrKyfVk+XiNC7P/CHzO6To7v9ZGB0f2jK16e47pWQt7rRpOpLRjbXrtKDEtgvKCvc7WzfJ1CdlxH5FARHZFQAlauVP4AcQE+JRmuKL0v7l/7/8tGrwWajox1hAGZf2T3wgm/E5uOArdoatcPDl8V2iga2dg1TLL49tvYzUCxynKmPfXTZDuFmzzbptMvf6C6Nsuqbv9NPUsEUvS3Le5uznUBPPi21ipCN5kfYGGTnO2SBLs3+Of+uYu2kFBsSKWX9w3W5pzjkGh6Jno2a8/9T3BCoTK2tmEISgXuB8c5ivS/F2V8BKQURSl7dBdEv1hDGdYRc8FzMQfzbWPHZ+w/lOz+5k4WCupNvVJRowUFb4xcsd7nJ1KL2CV+vw/2wXlWl7+h8ntNIJbsT7y8YQB8elXSWnGWj61mOJwpvVbHMEwxx4bfeS+3LI9raBVaEuLe5H4t1YlB9P2XqFs/Kq3VcZAI/9CLdusuL7m27Z8x2m6C4nRN+UzcxqiumuR1s/o0dSmO1kVY/PBtZrBETq4gv0+6PPqIyMkQrULX0ZYix1TaVK2aNr26Y714nzZlyjSBHGKSa4rUnZlSmh5+kpA+VHtxJ7BLNbzonjE+RY7ce5X9NG0cbgtbz2enyjFwIjUL3o= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYZPR06MB4980.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(23010399003)(1800799024)(366016)(22082099003)(18002099003)(38070700021)(10067099003)(56012099006);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: QrT8sMpJNp/LprH2RW7Udo5EWWNQFm9gBlWYA6OMtWUoS5tSlnh+Jy24tyE7lCXMXBoKX4zrpdBC/Xd1InFkHmHf/+awYAX/3O6JeYmkL7K70G+MjMfR7cT66CPfGwty46obadtNPSn3MFSfyWevL92JQgKzfAiay4iJnALiL+H7z4ROZVSw01+vYNurF1DvPpyO+QruYhuZd43bjEl9bDuVr63ManpdXsTn13G8UgvuDX0MvcT2fbFNVRxiv8PpKcsOizvpHYpslU3muJOk40O0faznrCNFCqq7FICXk5efJMpk+A6j6RRp/pb+s9IOO8CAiVVwypDPkU9Yiv9G6MJB28DD3WZXESMxcWw4sz8aEFmyVHDvSpZKiUGZyfnhemdWxlyvtJz82q7YpfRenygQhHRe/MnSf2qzLy7+nEZS9o9yfR3qyTERFXbCDGLOOmkBs8mxvI0N8xiACV4tpK+JsliZ+6JOEKB0LodIgHbTZmJuEHb3ewKnFOYIkkBNKrBYT9Q5C6WpNUnn9s9fwB2ojOX47ABoMDlWNh85jbw4fwIFw1a9XnsT78Fkgrtnwf2B3Wsh/oyQ1WgSnVv1fnq3Z5eFQNarX7/KGrvcegT+1Vv0LXKi781n5mkaFR+vk7ED1RoEnNhiDqBsIjOH4y1EQbzqUngfCoWlaFkUBwfucUcdoPFXoUCs4QuUyZ533PVY0y2kgtIqMokRfJc18A2xDPW+FQfullyrLDF5oE9znkH6ubMo2FWZwqB8ktJY0QFAVCE2c3Vx0h0ZcDTrEnUm+y3mT8AlMv7PuKr2iFx03lkbKVG0vziR9LQoY/Cp8zidyd3H2Ibh09IfFx0FZXebuzEPnRVZBP+MsyO2c4a4r928ptp6Th7+bAjZme7jvQpyvXVEyoeRMKOGinvS20xzCOGekbL7K+LdLo7Vo363nJ6EIBaUm9sa5d4euZT8K/7ZPv9rXlg9NM0czZAVwJdmgUYG75cbfLszbaG8SBkWppu7FfM6kk2DpfrFDTrvc6oY6dynVZyBm8DWDH+Tnd9P1kOYuBobpxNZS4yuZlbUP6q7MKHGBDfWmr+PajOxuiwDmvPC1PCmAzDI3Uy2afkY1kIj8mZD4BSp8sgg97kqAtPuYaEkLTkdUsXR6NhotMoX89CS/OzclClFxk0eJf8qWVfpZf4vLMmPiF6kU1C662I6lBdJ1AQp61A+dCzIOvaStCwShl7WTU/LtkhKp+2dmQht4twCOUNbD4MXDtmZvBKc2Hp4iNdRpnakOdqTY15bRXkUXi9BJmciyACoH2dDtultZCTwckQQMovVtZZSn4IS3iM/GPKnDTUZUqa/v6Ne+w6ZZSLrv9RPT2NKXaIPnrisqpfulnBnfK9aLbu+Td/s4OhqWOsPVOUmX8I1ECin4czNTkFpsJ966Whmgqsqru61GO0IqfDQ7iOtBO5nvsIG/m4N+jdpwxJDi/dB3KkZv44jrwOzO143LckcGUet3WVNmdCmsTcrxvzJQ3acyyha9UNQdUDUXCXj0z4g6Tohx05aOZ8/BlC5T+sF5BbnHqtacWm8mCASyFs1syLOApRbCzHcmZQEFJar8wBZ1UkxeIC2qAHg3wQKFAN+cc/IuLy3+PijRkDLh1SmFpW3OxQreXRdaPWKwdrlON1UQuX2Xh4GM1YqTuKSGZMBhPs3FOhGhJEBlcK5zlaYILQOP2A3jfgSFCBRyiNA/ZskCmc1EEbk0iQJxH/2BDEoUA== MIME-Version: 1.0 X-Exchange-RoutingPolicyChecked: UYSBGf1mmpGFWnFPX5WUJ+UeGz+9VMJAXGYZapdBImeTCtb1CrzL+yQ9drnBi38e7fy2vU7Z5HDQueoFp25KzMhfeWr7/ZLwx1deVilEmSa1MMZpz4APsukKt8eqXLFWSAeeorABy1EPw+xPzck/pNeGRZVMLkHwdeUnyN6RVeoEh4gjW3ITa/JEqjPgutvgQz9xgUt7rGe2EcxMKlSzmOVh0dwnkDLGmZeZc7CsoxchQmmunlgu+Wy6sz++GwymXCYSvXm22nNnI8AHOY4ECXhFTcaC5eKNQB/VMxkyXPEgKDrZkhfLwtAfPwcDdKF48YUOBO1/3wCwdHctz/1Aag== X-OriginatorOrg: aspeedtech.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYZPR06MB4980.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b72a21aa-ab71-4bd8-37f2-08dee891fc45 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jul 2026 08:11:23.2634 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43d4aa98-e35b-4575-8939-080e90d5a249 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tQjbgjMIbXPjUywX5IcFUve1AjQB9fxKqRSVpGAFjizngxi4CF3bbAy4rnvPQkSXjPEG+WgrGamHRI70jfDC+OKw0Zh1SAt2nFxSAO44WNg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB5022 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 23 Jul 2026 08:11:38 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/241786 When a recipe uses multiple git SRC_URI entries with different destsuffix values (e.g. recipes with separate repositories for the kernel, modules and application), do_unpack clones each source tree with 'git clone -n -s'. The -s flag uses git's shared-object mechanism: instead of copying objects locally it writes a .git/objects/info/alternates file pointing back to the bare repository under the downloads directory (DL_DIR/git2/). scriptutils.git_convert_standalone_clone() is called by devtool_post_unpack to make the top-level source directory standalone: it runs 'git repack -a' to copy all objects into the local object store and then removes the alternates file. However it only processes the top-level source directory. Each nested git repo created by a separate SRC_URI entry retains its own alternates file still pointing into downloads/. Steps to reproduce: 1. devtool modify 2. bitbake -c cleanall 3. bitbake At step 2, 'bitbake -c cleanall' calls fetcher.clean() which deletes the bare repositories from downloads/git2/. The top-level workspace repo is standalone (alternates already removed by the original code), but the nested repos still hold alternates pointing to the now-deleted paths. At step 3, srctree_hash_files() runs 'git add -A .' with a custom GIT_INDEX_FILE. Git internally calls 'git status --porcelain=2' on each nested repo to check for changes; this fails with exit 128 because the nested alternates are broken: error: unable to normalize alternate object path: .../downloads/git2/github.com.example.module//objects fatal: bad object HEAD fatal: 'git status --porcelain=2' failed in submodule modules/lib/module This halts the BitBake parse phase with a CalledProcessError and leaves the workspace in an unrecoverable state without manual intervention. Fix by having devtool_post_unpack() look at the recipe's SRC_URI directly: any git entry with an explicit destsuffix param names an additional checkout nested under the source tree, so convert each of those to a standalone clone the same way as the top-level tree. This is deliberately metadata-driven rather than walking the unpacked source tree looking for '.git' directories: a directory walk has no way to tell a nested checkout from an ordinary subdirectory of one, so it either has to stop at the first git repo it finds - which then misses a destsuffix repo nested inside another repo's own working tree - or keep walking into every repo's contents, which is wasted work for large trees. Reading SRC_URI instead gives the exact, authoritative set of paths that need converting, regardless of how they happen to be nested on disk. Only entries with an explicit destsuffix are handled, since that is the only way a recipe ends up with more than one git checkout under S; this also avoids having to duplicate the git fetcher's internal logic for computing an implicit default destsuffix (which depends on the subdir/ subpath params and BB_GIT_DEFAULT_DESTSUFFIX). Signed-off-by: Jamin Lin --- meta/classes/devtool-source.bbclass | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass index f29f40588f..940cdedbac 100644 --- a/meta/classes/devtool-source.bbclass +++ b/meta/classes/devtool-source.bbclass @@ -97,6 +97,22 @@ python devtool_post_unpack() { scriptutils.git_convert_standalone_clone(srcsubdir) + # Recipes can use multiple git SRC_URI entries with an explicit destsuffix to + # unpack several repositories as nested subdirectories of the source tree + # (e.g. recipes with separate repos for the kernel, modules and + # application). Each such entry is unpacked as its own 'git clone -s' and + # needs the same standalone conversion as srcsubdir above, otherwise it keeps + # referencing objects in the downloads dir that 'bitbake -c cleanall' removes. + # We only look at entries with an explicit destsuffix param, since that's the + # only way a recipe ends up with more than one git checkout under S - this + # avoids having to duplicate the fetcher's internal default-destsuffix logic. + import bb.fetch2 + fetch = bb.fetch2.Fetch(d.getVar('SRC_URI').split(), d) + for url in fetch.urls: + ud = fetch.ud[url] + if ud.type == 'git' and ud.parm.get('destsuffix'): + scriptutils.git_convert_standalone_clone(os.path.join(unpackdir, ud.parm['destsuffix'])) + # Make sure that srcsubdir exists bb.utils.mkdirhier(srcsubdir) if not os.listdir(srcsubdir):