From patchwork Fri Apr 24 11:45:55 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9495BFE5211 for ; Fri, 24 Apr 2026 11:46:24 +0000 (UTC) Received: from MRWPR03CU001.outbound.protection.outlook.com (MRWPR03CU001.outbound.protection.outlook.com [40.107.130.23]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.19021.1777031176774580837 for ; Fri, 24 Apr 2026 04:46:17 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=it3uF/3a; spf=pass (domain: ericsson.com, ip: 40.107.130.23, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JY48y5rEVPnHAbmkdmPUcOEpymZys8aVM9JCZiYQPkKO5mLgBSh84vUPLbRMR4MuekoW//lk8EFKYYcV/ra6KSJxxHcAZWKO3GxB2WQsiz/4WfFvVHVsQuHYK/vxHfPDwCG35ZB6394+sr7nW4AJRp5Y/sqigRU/Y7ROExogEKDqZbTCqSMy+A10e9RtZ3/Lohw/YnzxlWngnYgoJ/F2GvfvuwqHj3A3k7F88bf/Ir68fqEECoLIE4n5VOgL6YVT+c+3XBJSRoJiXY8TxjgTczwWKXqcVfhWUxXgwzFX5YYF+RFpl3XUP4Gbu8hPpGxyBaNAKI05K6/1JhMzjv/qOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5cNmn91x2R5sIBlD780C6jYjWPC3jDK6GwPBFNMI+Ow=; b=avc1hd5C40pOzCs3OVJhn2DnS4LphSTf3xKr36lXvxCCbSIcRTZAzBEpugnNnmeO+0dUMINjzBz0+BsMb0hnENxooN5ofEaC3/YuzO3iFyFanTUJIo2+IE8TfUMm5mxQwB+s8nIfi3mtxBl4HRmu0/jeul+QDNPtAM6s4bgFeiMJaNZUszTHBzJ1pWnJsQ3n+zLTuLNpLqQQI7KjdMC8a/4l9U+pAdKJ9Mvf/VIHCwxq1bfNddnqUb7CbtC0OSvw3e0ZAce0XmSTg8AgeeBtdKRrmvkHIXxVlqu/tZv0MMc1zC+8R9t5+qPoW9YaOL6fohsFbgbVwH4lM+QOzuxRVw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5cNmn91x2R5sIBlD780C6jYjWPC3jDK6GwPBFNMI+Ow=; b=it3uF/3aOyCTNjfQQJS6n+nkhZDLaVS4wk/IPUOWdLAf7gqrKqVbQqw4Umyh9/jQFH/DJiqZ+1e8bNXPFQGYhgRJigzSX9WNfooA28bYfesot6gkdAfOzU/5bTjXb3qbvYXD2cnKcFJXJV3NtwTtUogEoi8Y/RZUDlG0hE9VaIm4EH48vhATYDATaoJbeCjPHQJbXoPzcV3rt2Ad9gMmBbrrGAM9UwMKd9DBl4EEDV4OVU4WHUJt1RedXuGwgk4+AX8Ws9SJLjLkDXactyOTxETkhkOIoVlKojYRzHZdQpYz451ZlumGBXOS0DLDX1dRun6GHtjcpPde9bdEwAdZAQ== Received: from CWLP265CA0453.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1b7::22) by MRWPR07MB11824.eurprd07.prod.outlook.com (2603:10a6:501:8b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Fri, 24 Apr 2026 11:46:11 +0000 Received: from AMS0EPF000001A3.eurprd05.prod.outlook.com (2603:10a6:400:1b7:cafe::5e) by CWLP265CA0453.outlook.office365.com (2603:10a6:400:1b7::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AMS0EPF000001A3.mail.protection.outlook.com (10.167.16.228) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:10 +0000 Received: from seroius18815.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.65) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18815.sero.gic.ericsson.se (Postfix) with ESMTP id E7CC94020663; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id D3F26700DB92; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 1/9] upgrade-helper.py: Add compatibility with Yocto scarthgap Date: Fri, 24 Apr 2026 13:45:55 +0200 Message-ID: <20260424114603.2444938-2-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF000001A3:EE_|MRWPR07MB11824:EE_ X-MS-Office365-Filtering-Correlation-Id: 15fec10e-a644-4bbd-62da-08dea1f714c3 X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|82310400026|36860700016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: vauywhRmelElBFUU6OGiovgn59WhZFQJWqB2FadRorPw/C5rDWQnvAhzFl+NR0YFj6XUhx7nfwtSlfcynKZjvQLDFrcXx1cdqvwWVHPJ3faD/gPkM+534aCT57WNxbAmhap9t9HsgAO94dJATkPpTaPKca8F41UfDSfwH2EAvJe2izYnljcbSlqUyP5g1y/63t0ysthi2192xI3eeDupsgjxpohqb8QuEKEwwbI1H4ybsrIO6VIGfPKG0YNTbRFyNop8E37ZJKtz/v2HoEkHJlgdva4m1ogKd2fh6a1RwqTtvZOgynspr7eg0ZDHTOpxk30xR51QCFu2j8HHAfkB0SqPw9Jk4XWGYfsr1zoN+n9x/ZNSGVDWOs6+5r4vRR4doUkmD0KeY+m6FXdObSjPe5zFZrjDA3iWWaiEOKRHE+nHaeOYUqtPgrUWHv/+VP21Q1f/aZbboVUZyOYFFjGhongohCqBl6geQk1cYKqoAzcbSuduBbb+5lUYc5cQckOQXclFp/JJz8T69j3CTKueBqMyNSCwAtz2QVT90gi/yrEa9ezx/UlA+MOoBTpIhoxqGQKAD56qasad0O0jQLeufHGmMwfmZYr9/E0GWZDRNKlBT0JtJ8d7AGbEjuUZxoQZNoFcPN+sjIaKIVSN9Fxw/l8a2sgQYexSEAIyFLGMJN8nOOuGIGBhnvWeRQapM532rYFcOvVKoTxHwH2bpePWevid0u+xkDo7dD1uAXODyKiaxV0abtLmwuyP8DkMb/+Apq+dui2S2zzqvMTcWaaHtg== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(376014)(1800799024)(82310400026)(36860700016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oN2TUv3otPWTBEQinNLAvzGHVlyXCSt5WPt8hijhbbD1eCsvc0kAyfsfLVtiRwul3f6ZzMEFx2mMmiLFIJylTqUrB9z4/5sLeNW11/xr2SKHh80zfVYGOWHW5t0pRySXdES8PjrpAp7VsG2ZAytVawzC9U5NFa0zIPZGSzizNta8b+EOO6g7p4+b4edqXHXpJj0Ctl+dU3Qatrtw/FBmRrWRPNvrGNqZgYlyau83O9psAjJU1WQG9a9r9uET1XCW2+g/3dOry7vUJc3u89IyK7zZmrZsluABGg2MBLq3gHpMXRogqDAGIWw5h9OwkwQu0224dvyinlvDm2E/amFT+HwxgdqurFb3IGv5tTCNjhCLqqbqoNBJQaF8DVXQn+WAVoh7VXqjxJpZEN37ngg0UtwuA78D2fXN2i/zRCkfIl+MbsVzYH8HpQsu8tE40EBJ X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:10.9890 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 15fec10e-a644-4bbd-62da-08dea1f714c3 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A3.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MRWPR07MB11824 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:24 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3796 From: Daniel Turull - Handle flat tuple format from get_recipe_upgrade_status (scarthgap) in addition to the list-of-dicts format (master) - Fallback to MACHINE env var when bitbake-config-build is not available - Resolve symlinks in recipe_dir with os.path.realpath() Signed-off-by: Daniel Turull Assisted-by: Claude, Anthropic --- modules/steps.py | 2 +- modules/utils/bitbake.py | 8 ++++++-- upgrade-helper.py | 9 +++++++++ 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/modules/steps.py b/modules/steps.py index a6ec341..b3ec61c 100644 --- a/modules/steps.py +++ b/modules/steps.py @@ -36,7 +36,7 @@ def load_env(devtool, bb, git, opts, group): os.mkdir(group['workdir']) for pkg_ctx in group['pkgs']: pkg_ctx['env'] = bb.env(pkg_ctx['PN']) - pkg_ctx['recipe_dir'] = os.path.dirname(pkg_ctx['env']['FILE']) + pkg_ctx['recipe_dir'] = os.path.realpath(os.path.dirname(pkg_ctx['env']['FILE'])) def buildhistory_init(devtool, bb, git, opts, group): if not opts['buildhistory']: diff --git a/modules/utils/bitbake.py b/modules/utils/bitbake.py index 5514c98..f6821bc 100644 --- a/modules/utils/bitbake.py +++ b/modules/utils/bitbake.py @@ -123,8 +123,12 @@ class Bitbake(object): env = "TCLIBC={}".format(libc) else: env = "" - bb.process.run("bitbake-config-build enable-fragment machine/{}".format(machine)) - return self._cmd(recipe, env_var=env) + try: + bb.process.run("bitbake-config-build enable-fragment machine/{}".format(machine)) + except bb.process.ExecutionError: + # bitbake-config-build not available (e.g. scarthgap), use MACHINE env var + env = "MACHINE={} {}".format(machine, env).strip() + return self._cmd(recipe, env_var=env if env else None) def dependency_graph(self, package_list): return self._cmd(package_list, "-g") diff --git a/upgrade-helper.py b/upgrade-helper.py index 40f31c4..aef7ed1 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -714,6 +714,15 @@ class UniverseUpdater(Updater): pkggroups = oe.recipeutils.get_recipe_upgrade_status(layer_recipes) for group in pkggroups: + + # Scarthgap returns flat tuples; normalize to list-of-dicts + # so the existing loop handles both formats. + if not isinstance(group, (list, tuple)) or not group or not isinstance(group[0], dict): + pn, status, cur_ver, next_ver, maintainer, revision, no_upgrade_reason = group + group = [{'pn': pn, 'status': status, 'cur_ver': cur_ver, + 'next_ver': next_ver, 'maintainer': maintainer, + 'revision': revision, 'no_upgrade_reason': no_upgrade_reason}] + upgrade_group = [] for pkg in group: maintainer = pkg['maintainer'] From patchwork Fri Apr 24 11:45:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86832 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F106FE5218 for ; Fri, 24 Apr 2026 11:46:26 +0000 (UTC) Received: from MRWPR03CU001.outbound.protection.outlook.com (MRWPR03CU001.outbound.protection.outlook.com [40.107.130.24]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.19332.1777031177557329159 for ; Fri, 24 Apr 2026 04:46:17 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=nb7B9yYZ; spf=pass (domain: ericsson.com, ip: 40.107.130.24, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VBPSXyh21hXVEEte3u89emwUvrY0Od9hb/C7FIHKOOSRU5s3JxDeguTSRaVifNGkSNF5fYvxcYW7KLhCS6/JU2KRGSctTVq8RFv7X7I0l2DPwgE2OoXmxuLTb2JrlupZ+yARZ+VKu5D5e2Bw3UKK0nufri1i/B4UE/i4mKZpVMwD3+f93Attdor0y6Q3n/13BlxgMhqU232Jtjmjd77CT//nsimXKFKJxWLylVQ6yyBU5n8wAtBbO7SQDCRAW7tOVKEiDnmzSRPfYuBOfVywiuRtliXYkE/pmNkbQd+QWBcAnF+4xmEie9rVhfzB01MOewvQcRbCZjX3XzTAMNNibQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GzL9ZjgRWl4nlPSudBsrTN5W0L9NwmSrSNo6GX9rSic=; b=MnlQaCMCtU3inRTVSJAFBCUIpin82l04RUB7IMd50ABTRaZ+td2imdAWXaCfcBIWh0ucJvrKQG0BMX7xV/61thKn2G8g4Y9BeB3DE8K/AgMsck5koQYkXAhUr+PNKZSZDUSEq6Jm9XFCwlTaXjSebESYIgP7ONLcfDcfIOqmJYjW5Kz2A4wunauZ5wc0cIW0iImxSYZv/Lih97EcRMCoUhGDd7eAtSRpkrptXWZRfnfvzK5ejKEWHZOzzgiM6S32MEPiVpL0vK+joqLBOTb4xSC6eWzU2WenoFTqUxXqrP1TyUlM1kLX+J2fYrPvJpwxF7RQLFGApMErgij6E4GOWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GzL9ZjgRWl4nlPSudBsrTN5W0L9NwmSrSNo6GX9rSic=; b=nb7B9yYZTBH3q6ewyYG9S9/MzhsGKnOmiBI5etDq8lwmr1y8rKgT26giksCMGOpwqy5X5nvrztrP1BmltUDRwRxwqzcNkSfW/hDFUioMrbSo1bI3NYV6iN6xqNZYt0jMDXF8e3ItgTUL3NTdrLkhkvbyxwoRcne3dap/g0miQjOEgO5/+A7AK4yLa2Ml4NcWeQy5ThyxtSf7CwyBFLHyi6L1JVbuFur5/KGNnH/XxLDS/ge6XvjVD2TwPU1Aid1IEGnL5arTBuNVY0sTUSB3biAcsOqvUt6kaqeV1D06fpbBVK77HP63N1STbBqfA1wg7dp+8oaMOfUA8nQeAJssRA== Received: from AM0PR10CA0047.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:150::27) by AM7PR07MB6964.eurprd07.prod.outlook.com (2603:10a6:20b:1be::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18; Fri, 24 Apr 2026 11:46:10 +0000 Received: from AM4PEPF00027A6B.eurprd04.prod.outlook.com (2603:10a6:20b:150:cafe::c4) by AM0PR10CA0047.outlook.office365.com (2603:10a6:20b:150::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00027A6B.mail.protection.outlook.com (10.167.16.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:10 +0000 Received: from seroius18813.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18813.sero.gic.ericsson.se (Postfix) with ESMTP id EE8299569F; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id D8652700DB93; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 2/9] upgrade-helper.py: Handle fetch errors gracefully during version scan Date: Fri, 24 Apr 2026 13:45:56 +0200 Message-ID: <20260424114603.2444938-3-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00027A6B:EE_|AM7PR07MB6964:EE_ X-MS-Office365-Filtering-Correlation-Id: 0df55fc5-c85d-4136-5ff1-08dea1f714ac X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|36860700016|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: VJ/wnDegqMtpOyzBFQLA+3TcNQlSUH1Gr9+q444dNcuqwwEnP860QxbrW9zP0qkZY7oIu/7+IJCRT/1A8gg0eyeP3SuKmr9f8+Xks9N2yeHgazllED/Xn9PsZtUjr6gu33zSVA3pjD8RpLwXWXHJQPerkMMv8cqAbN+pG5lHKx7EGsE9TD7OEfy1BGC6TUNXMTM9584nA6AR/oD1cufnrGehBn75mX2L0wJqZrBQFe30izNwUVGCsHddDrZmaWem33U8jXvcaA/9+HI2DiWbLUcE/cNxSzak7YNPmx5G84v6VFAMot2A29rtzw4K7Inf+NsTcPanzRp3qlhHnGWBXBzfrf5Q7Rgqk1t4j1nbVln6/ndcuKP3/5JogyOtmVvx5YT+xAWa1zSRzGq2ClC6+LGVhxAqNQkLZfz9J7vplVXAaEyk3PXG1j05p/2jYnVwn4kY6ua02z4kK/sUaoCvdrW8d9B6j72aFO8HT5Shcqwpcu+OVboiZnJpNkFDRGmvlbfGatY7cvandHhIIOrG19VYC8AGVhwxBpsw8H0Zc6sYhAG8VG+pkHjtkJctmdbxs1lm8QBg0bQNJlePdaReCcv+lk8nl6+47YBiUKzlb1XlkFzfjNu38ydaQ75basi1eaJA/rE7VlVknV78NH+b8quNgSWeGSi1tkjWTfIMWIB2Ek7inyYrrcTfVaeD8nbnTSB6SpKCVGxFVZGxkyJP3NzjBoSE3uTgpwhHbJCErbCxylnXwYoCC1tDMAfqt4DDwX6sE3Ka89awhhoBO9EbqA== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(36860700016)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 030T7HG48xBZEXnpuozvpf1ekkwC4dEIwiz0asUPjgxT6356Ofw4Tk+MhadtiV9drIX37Y7MpBH2CUBTEWhJ7VolvHOGaJ+f+hrxLQ/lzzXbu+KJQxW1mpFlBHwaxx/Wq9of598unu3soTHiAv8FUC/Ee6cZg05WpPnWYb9AJCeNtq4cI3+svXZIVtx8645N6M+gZdCDC8QrxPjUjDV/a6HWjr0G3mBbrh91/YqOGyzfzr5IRC/6Ja/i2l7MfYLOpYGVjH7+BGN7A1sz30wRNchZJoS8ZDfo9AsELx0uwzGjjQuz2n5NhK2gnOZl2g1+01QYqzFNFf4mpvBIrdJ0N/+KbbCk3n2jstg3e+2k73L4i9yhupBP9cCBu7ieHqGx2g1FzkCGrf1bVwUdzSeQxWXLIaX8eh/5a9zKE1b/FwPTEd2lZyCcve7x6p3hldWs X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:10.8376 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0df55fc5-c85d-4136-5ff1-08dea1f714ac X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A6B.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6964 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:26 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3798 From: Daniel Turull Materialise the iterator from get_recipe_upgrade_status into a list, catching any exception that terminates the iterator early. A single unreachable upstream (e.g. git.savannah.gnu.org down) no longer aborts the entire scan — the recipes collected so far are still processed. Signed-off-by: Daniel Turull Assisted-by: Claude, Anthropic --- upgrade-helper.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/upgrade-helper.py b/upgrade-helper.py index aef7ed1..98391f3 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -711,7 +711,18 @@ class UniverseUpdater(Updater): upgrade_pkggroups = [] for layer_name, layer_recipes in self.recipes: - pkggroups = oe.recipeutils.get_recipe_upgrade_status(layer_recipes) + pkggroups_iter = oe.recipeutils.get_recipe_upgrade_status(layer_recipes) + + # Collect results, logging any fetch error that terminates + # the iterator early (oe-core raises inside executor.map, + # which cannot be resumed after an exception). + pkggroups = [] + try: + for group in pkggroups_iter: + pkggroups.append(group) + except Exception as e: + W(" Fetch error terminated recipe scan for layer %s: %s" % + (layer_name, str(e))) for group in pkggroups: From patchwork Fri Apr 24 11:45:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1CE6FE5213 for ; Fri, 24 Apr 2026 11:46:24 +0000 (UTC) Received: from PA4PR04CU001.outbound.protection.outlook.com (PA4PR04CU001.outbound.protection.outlook.com [40.107.162.58]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.19020.1777031175964861904 for ; Fri, 24 Apr 2026 04:46:16 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=rp09l2yD; spf=pass (domain: ericsson.com, ip: 40.107.162.58, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Waea+o2BUQ0L8juvYOB/taGs2onCRXP4KjGcirwTo+9nRLDiH1G49RFyvBxmaudG5FRHsq2KkUg/oyEhB/AbaTN9nFftvtCcx9NfyV/eKxvbB8252P/VDgATyCiT8/dr7kyyYXoa3jPnGpFpHB979SYJlKI5sw43jLgjd4pJmgcCuRnermUA7O4OxulisPEH77lK/A4IkWk+lJkrK0XfjCPyjyl3vzCQrNKs2nsxaLtnRjIvO/pKw2IUlAFB5c5Zd7ChD/B+bC81fmvldO6Xis/2yBGuVzSVtlXP+T+0vP70vgFxAGqQEgvofX2WKZ7JC3BFd9o5RzRb1G1oiW+IwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0eCPw5bcmjGwT94EaDKbTMYTwSLJgOKTQdVyzw6kZUI=; b=PtIvAVWjH/nh4JWq8h9Yr5u4FI/S9nuW1VMw3bpCBMV6d460hg/cQy/+VQ5dgO5Eu0JjSrDm0nOGTTNa+PlOmFhlMsPx6rfGEXSoLKD6lveL1Fa1cWpR7q+vwg0LpxDRKFFZDsBrjGNeiLWfVIhfKJLd6p0/eP8SXWrpQCPKEi47gWzKLairojK7W2e2Lk5kLnNd/cEr53+eY8FMLbtFbcvlbxq3iK0510mTEoWqQRfQXlIRde0WGL+Fa5ayvVh7v7PpoLAOzeFFS2vT5S5M6e/Y0hBFeUoorWoFAlhXLSGynGp7qD9/6cPA77AUYsVQK3hw/OOdkfm6FdXm1jU6Vg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0eCPw5bcmjGwT94EaDKbTMYTwSLJgOKTQdVyzw6kZUI=; b=rp09l2yDKBkAF6DsXB59ogdpbwtVp2sfzx9MJuMEUbU4Y9o1mTQyiS2TGVdrLHZBiZ3mSdzFGfKdSY4lX9BjIe0WTrhWj8DnkwevDeCdHcsYx3v1eD/uBK6pUyCKwTCgo9rhzGlManEBN1+Wqa80RXzze5g2vTwls7j8yyAcfAT9PMJPNvthk/+OOOePts0yHZ37814K8cN3XBfPvIzincC7Dg45h/38j/AhvTJ1QdV7Gtz5cG8mE/m6/qsxuTxSuXckRTM1nRwJk+u4+pKgcleTVn3lLwAmAafAeGlHEo0QqnlEUHzjSuEUcTQOzlp4F82IMWTI6wLl8teTvU2kxw== Received: from AS4P192CA0002.EURP192.PROD.OUTLOOK.COM (2603:10a6:20b:5da::9) by PR3PR07MB8209.eurprd07.prod.outlook.com (2603:10a6:102:174::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Fri, 24 Apr 2026 11:46:12 +0000 Received: from AM4PEPF00027A68.eurprd04.prod.outlook.com (2603:10a6:20b:5da:cafe::25) by AS4P192CA0002.outlook.office365.com (2603:10a6:20b:5da::9) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00027A68.mail.protection.outlook.com (10.167.16.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 Received: from seroius18814.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18814.sero.gic.ericsson.se (Postfix) with ESMTP id 00B344020B71; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id DC943700DB94; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 3/9] upgrade-helper.py: allow to set the git directory from layer-dir Date: Fri, 24 Apr 2026 13:45:57 +0200 Message-ID: <20260424114603.2444938-4-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00027A68:EE_|PR3PR07MB8209:EE_ X-MS-Office365-Filtering-Correlation-Id: 69c8fac0-36d9-4ad7-9955-08dea1f71551 X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700016|1800799024|82310400026|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: zSjDG1JZYo3qiEoW8bSJAOpGNVtwbYsk10Ky2oxk0hiVqqyapCmaWeHorc9+FBzBlITyqKVH+z5fqoYOGsiBEsK9dqYz5waOn7JrawxLlulMW8Kf4qajlwvxGDa376RpAmI/JHPmPIKfg69HRm9A+3go8l3cEtgzdomV9pbp83/F/BqSA38Pvp++PswvuoPjlicW9/sLnIEW2hHXCVnvMPGKcDcbjivzk1s4WPtWnCqm4jTWY8qeM8bthaliiKlbJh+jFcDjmwM7j0jv5jf1yk35OYATSB3K6u431JsfbjsTDjkvHmDdvimtOhXTphHNp/aprsXaXqE5zHsmLMNuF1eWLxIgvwzxeui5n0QdpyDplqHlUJoUTZHvV/mYMSe8OTO32HeI0GJO9dEFCWSlv0QSOZezRV9s4KbOyqajOmnma5Pchqp9Je6MvoKgtIRNW92RSfTb5yjqL7lLNlfbV4y8hElJn3/3hlfgzmiDpzllbyCM+a9zbsjGBemikmmISdpWkvehnwIUPnmrc8ayM8XDKpC7lvxIEAQ7+b0TwlF6ytnpMrq8LsJZn1b0P2/xZ3OKsfl6BCcwVy07fpyXAkOzEVF+VPmifX65iAGaLuZ0euHS9koF77nqIABkUIcqIJg5HrYB7Z+fLpHZ4CXGUqW2NcLeQOIFG9lgXxkTtN17m81kPxrZW26Mkyzs1r7Yk1ZnZv3x4xrniXTGmGS901oGrBARozDLOHstYoAutlhDooiztAeYO2I4Yb49sNzWe58v9PoiMpdbB7+DNVvR6Q== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(376014)(36860700016)(1800799024)(82310400026)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ss8PIvImIIWNXdE+RuE1rIG89Mo6hWdKMl2vwa91gVcMplYbPNABhBW0OABXz0Z+FvvqFbXTI4AiIvG1ATUsTii+115POYGDkpmqMlJ3MSyw/nfj86XI6HAQuDXFh6mNhBalEe1tmZhryhzdFkekLXAF60nFXyMAPn3dPXaGht1IdmPyvNRIhlzV4ZTTP/qsdV1YBVC71A0uLb6NYxIrbbSktYnezz9R+zUC96InbjN/wzOlKKYqRYfcF0bQJb3IYxJ36F6XhS8+UG/MnI7+F3OntScOvVZOQUkE0otu5RnuA3o2xo2rXLu/O5FMHbEnGDXwTHRH5J75NRJK+FpK36k1769iRQgVrKxSTKPKE2HBTmYfrFTHH3OZaHIdOzQRwGOhTfWFxAXPeJdzPsaxv7Pbyy2xK6N7zi37741xEsnkb+q/O+PO4gO+JZhqeswA X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:11.9219 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69c8fac0-36d9-4ad7-9955-08dea1f71551 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A68.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR07MB8209 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:24 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3791 From: Daniel Turull Replace the assumption that the poky directory is the first PATH entry and reuse --layer-dir to setup directory for git repo. This is useful when not using poky and each meta-layer is its own git. Without this fix, commit_changes fails in non-poky layouts: git add .../openembedded-core/meta/recipes-support/gnupg fatal: '.../openembedded-core/meta/recipes-support/gnupg' is outside repository at '.../yocto-scarthgap' Assisted-by: Claude, Anthropic Signed-off-by: Daniel Turull --- upgrade-helper.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/upgrade-helper.py b/upgrade-helper.py index 98391f3..df927d1 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -184,6 +184,9 @@ class Updater(object): _layer_settings_error(s) self.git = Git(self.opts['layer_dir']) + elif settings.get('layer_dir'): + # allow to set the git repo directory without enabling layer_mode + self.git = Git(settings.get('layer_dir')) else: # use scripts_path to get oe-core directory self.git = Git(os.path.dirname(scripts_path)) From patchwork Fri Apr 24 11:45:58 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7530FB44D8 for ; Fri, 24 Apr 2026 11:46:26 +0000 (UTC) Received: from PA4PR04CU001.outbound.protection.outlook.com (PA4PR04CU001.outbound.protection.outlook.com [40.107.162.63]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.19331.1777031176998279875 for ; Fri, 24 Apr 2026 04:46:17 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=ghx+rQkV; spf=pass (domain: ericsson.com, ip: 40.107.162.63, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b8ZR+L0eOngi7AlWVwcZM7LL2Tt7nFjNxQYG+gUNhQkBPI6CRzZXXIqJrLYwoZuxmSsI+ps/0Gz9WkO2AatZFkHMrVUwC8toABwavJ47H4ddeLhrDrzN065kzxc/UfCBbbHGsc1Q+kyHqfHLpRGddebi5wyhMPABHPT9swtlTDQj6bHEldmY/IBvPBbhBq2YPaNgWpP5xnhsWrIQc+SQRYDG9zpn7Qx+1WJSqePUYCepS4367LEihkXxKrUo50/TDmdVLogonEACymfu1vlBev0ZCSkqmqAzXpw461IY2dWU15hedY/MV1lmACUkZNph9EkfMQdvuoF1aifgHJavjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PPFCL+AY5LMQmFN8t3HXt/qkKIXv5L8+AXkNwrmO5l0=; b=QKoANRUv31SDKFlvZuJ0HLWaEuZmg1AXbRoB+gyE3wbNzjWWLO0MJm0z+J88PgF67FM9Ho1g6mXRP4zNg7KOEHo9gRchTaF11IAdms9psauOM4E1IETaIgcq/6SZJFq5gdNr4s9ECivvvR7LNqVqORTyMomcMEw7UiN5Quh51XADG/45xkY7SE2EEDD7K1eXTXLQKZ1ViKMMV6Si5gW4dFzO2/col8K26K5lOCUOmfI9/pt+0ff21/mEPnIym4dHZQy6Uhn6aKlmfrE2k7ZnF91e304X3fxWVlqJu3QJ+IuunqgLmPq73EPpuQf+2l75P7/SEFE6+x65LeVWHHOr9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PPFCL+AY5LMQmFN8t3HXt/qkKIXv5L8+AXkNwrmO5l0=; b=ghx+rQkVITZeAHHX6yRSOwJCMUhLO3lX4j6zzWFNSOlp4/2aOSU1kxFlLRfrKxSPEO6hDXd0nuZrD4nNJT2a+TE1vn4jAHj65nrTcMFWbVf6NfxZ19kq8sc6nQhtBg0NUVJWOLRTtd3vHmwhTAAsZEmntqtCE36VaFvm00C4KnEzupNRWLe7ElaFTivGpxsfsDznNhUQJw5KY3uAYwLXZa15FszzRUyu9SUMapBoJ1MDrkNTjBaRRqWMhxlYQm4HfFxTIgLorIzEnwD5EvFFy0ACRveepqS8kFW7oQkuD8h5ya8TTKCSYpwwrT2cfC+od1XyqMBdAPy0P/Kn/jFPqw== Received: from DUZPR01CA0162.eurprd01.prod.exchangelabs.com (2603:10a6:10:4bd::10) by DBBPR07MB7547.eurprd07.prod.outlook.com (2603:10a6:10:1e9::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18; Fri, 24 Apr 2026 11:46:11 +0000 Received: from DB1PEPF000509EC.eurprd03.prod.outlook.com (2603:10a6:10:4bd:cafe::43) by DUZPR01CA0162.outlook.office365.com (2603:10a6:10:4bd::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509EC.mail.protection.outlook.com (10.167.242.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9791.48 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 Received: from seroius18815.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18815.sero.gic.ericsson.se (Postfix) with ESMTP id 069524020840; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id E0E8A700DB95; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 4/9] upgrade-helper.py: add changelog flag Date: Fri, 24 Apr 2026 13:45:58 +0200 Message-ID: <20260424114603.2444938-5-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509EC:EE_|DBBPR07MB7547:EE_ X-MS-Office365-Filtering-Correlation-Id: 76388c9a-7fae-4455-03a5-08dea1f714db X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|82310400026|36860700016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: HFM/lQ3xNFRwFTsY4lkXshl+Acq4rG24/Myt8tvnKz8xfTK32FkOqrmo/tMfWWOs8X592cDcvqWwbUdoBU9DLsA2IINvRR4lA9rmfi+eBZ8+6mQA3XWozCidHC4habwziw0M1xt+NE184LwEzwlZjBAaby30TDBgmyjBiUjBMwMV4528X6juoHRfFzQHOYjt3Kor7YDDbYlrSyMy1diyzKgNX1glvZPWniv1HrL6gq5Zx1iFsdWkMgizCmUngDm7JIVrvKwKGdU89TQYLToKBMQmOzERNOUz3inqdtZKIHDExxSDsI9lO7Xj4GfYTR9L9lycf6BiRZ7yxB7nfo65xMevEyP/wwGJqsMwCTs1SLDfXBjdAVl6bYpm1OKz8Y3S1jj/+ihINCtuRv2fdgVBmaTWwkF5LDFBjTwXMOv0YUwYq7yUj4C/nbgkUEbzWVD6lI+CkCcrbiPmtlXvkafJCqMk4U42f7qDtAAaXUtMJ8etpW8+mHJ7l3pgqbfpkkAk3/r3ygqVEKuRiZYw4hctGfvW9OdaO/CbI2uc3Z+5MASTdAS6FNU5u+9R7UTvoWnwDi/elEGbkqLx9EyyF4GtGT6pxnJa+63Ga3yIPRR+IYWLc7Iac3jJowWVrf5w1kVIvhFocSiJaDMHuL7wOW5N/wNYXyQgIxuAttdCKSRAmoMNb9X6+f2ersu3bsJ1erXmusoLQYbt4/fWrhsZZlFJctBOZiGTBvm53DbnNWT/DC0i9MIQQlq9sP8+ZHPPSIDF5qon1kymLgCsXyEUZacRTw== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(376014)(1800799024)(82310400026)(36860700016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KhuC5VbSbgOh276Tye9e2O3HLkdROomfoP08nFD+s3Q1ZM0w2SG4Gn+BzcdsUyBQE2TY1jfdRJ+esOMiLc7NLg7YKpaNi4HtSrsjHS/WYLYypXNzQi7y2sWjSnmk4oJIrGpDKmGc15ljTkbQmVrUFzmE4tqclX4N7oGcv4JZ1Q5mLyzBWKcnjf6HTQhmdt19zFS9piwe7mgRUH1pUlqhVDxuIy5vpXr2HDkoxkIYajJXgWXclfhDl/r5MEQLUc9IZbybeGrvY/X4XXozPl5wu8H4ejGdizlSoiJ/yYtmtxorPJqsh/TnF2m8UsDY75j4NzSZfcaBDcxpVbHsCdbuJymZtDe30WwUOCuIG5k2UGMZreLDfrhT3h+f9TdUrJ962fME1XVl9WtleHIkNOFwk8Hf/HXOicQ64vHyj6OpxeMbgU/NMR8wcM2MgU96J0DA X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:11.1274 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 76388c9a-7fae-4455-03a5-08dea1f714db X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509EC.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR07MB7547 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:26 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3797 From: Daniel Turull Add --changelog option to extract git log between old and new versions, highlighting CVE references. The changelog summary is appended to the commit message and included in the email notification. Signed-off-by: Daniel Turull Assisted-by: Claude, Anthropic --- modules/changelog.py | 256 +++++++++++++++++++++++++++++++++++++++++++ modules/steps.py | 20 ++++ upgrade-helper.py | 16 +++ 3 files changed, 292 insertions(+) create mode 100644 modules/changelog.py diff --git a/modules/changelog.py b/modules/changelog.py new file mode 100644 index 0000000..c1cc14b --- /dev/null +++ b/modules/changelog.py @@ -0,0 +1,256 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +import os +import re +import glob +import functools +import subprocess + +from logging import info as I +from logging import debug as D + +import bb.utils + +CHANGELOG_FILENAMES = [ + 'ChangeLog', 'CHANGELOG', 'CHANGELOG.md', 'CHANGELOG.txt', + 'Changes', 'CHANGES', 'NEWS', 'NEWS.md', 'NEWS.txt', + 'RELEASE_NOTES', 'RELEASE_NOTES.md', 'RELEASE-NOTES', + 'HISTORY', 'HISTORY.md', + 'debian/changelog', +] + +CVE_PATTERN = re.compile(r'(CVE-\d{4}-\d{4,})', re.IGNORECASE) + + +def _find_changelog_files(srcdir): + found = [] + for name in CHANGELOG_FILENAMES: + for f in glob.glob(os.path.join(srcdir, '**', name), recursive=True): + if os.path.isfile(f) and f not in found: + found.append(f) + return found + + +RST_INCLUDE_RE = re.compile(r'^\.\.\s+include::\s+(.+)$') +RST_COMMENT_RE = re.compile(r'^\.\.\s*$|^\.\.\s') + + +def _strip_rst_comments(text): + """Remove RST comment blocks (license headers etc.).""" + lines = text.split('\n') + result = [] + in_comment = False + for line in lines: + if RST_COMMENT_RE.match(line): + in_comment = True + continue + if in_comment: + if line.startswith(' ') or line.strip() == '': + continue + in_comment = False + result.append(line) + return '\n'.join(result) + + +def _resolve_rst_includes(content, base_dir, srcdir): + """Inline RST .. include:: directives.""" + lines = content.split('\n') + result = [] + for line in lines: + m = RST_INCLUDE_RE.match(line.strip()) + if m: + inc_rel = m.group(1).strip() + inc_path = os.path.normpath(os.path.join(base_dir, inc_rel)) + if not os.path.isfile(inc_path): + # Search for the filename within the source tree + fname = os.path.basename(inc_rel) + for f in glob.glob(os.path.join(srcdir, '**', fname), + recursive=True): + if os.path.isfile(f): + inc_path = f + break + if os.path.isfile(inc_path): + try: + with open(inc_path, 'r', encoding='utf-8', + errors='replace') as f: + result.append(f.read()) + continue + except OSError: + pass + result.append(line) + return '\n'.join(result) + + +GIT_LOG_RE = re.compile(r'^commit [0-9a-f]{7,}', re.MULTILINE) + + +def _condense_git_log(text): + """Condense git-log-style content to subject lines only.""" + if not GIT_LOG_RE.search(text): + return text + lines = text.split('\n') + subjects = [] + i = 0 + while i < len(lines): + if GIT_LOG_RE.match(lines[i]): + short = lines[i].split()[1][:12] + # Skip Author/Date, blank line, then grab subject + i += 1 + while i < len(lines) and (lines[i].startswith('Author:') or + lines[i].startswith('Date:') or not lines[i].strip()): + i += 1 + if i < len(lines): + subjects.append('%s %s' % (short, lines[i].strip())) + continue + i += 1 + return '\n'.join(subjects) if subjects else text + + +def _extract_entries_between_versions(content, old_ver, new_ver): + lines = content.split('\n') + new_pattern = re.compile(re.escape(new_ver)) + old_pattern = re.compile(re.escape(old_ver)) + + start_idx = None + end_idx = None + + for i, line in enumerate(lines): + if start_idx is None and new_pattern.search(line): + start_idx = i + elif start_idx is not None and old_pattern.search(line): + end_idx = i + break + + if start_idx is not None: + return '\n'.join(lines[start_idx:end_idx]) + return None + + +def _find_per_version_files(srcdir, old_ver, new_ver): + """Find individual per-version changelog files (e.g. changelog-1.2.3.rst).""" + ver_file_re = re.compile( + r'(?:changelog|changes|news|release|relnotes)[-_./\\]?' + r'v?(?P(\d+[\.\-_])*\d+)\.\w+$', + re.IGNORECASE) + + candidates = {} + for f in glob.glob(os.path.join(srcdir, '**', '*'), recursive=True): + if not os.path.isfile(f): + continue + # Match against last two path components (e.g. RelNotes/v1.47.4.txt) + tail = os.sep.join(f.rsplit(os.sep, 2)[-2:]) + m = ver_file_re.search(tail) + if m: + ver = m.group('pver').replace('_', '.').replace('-', '.') + candidates[ver] = f + + # Select versions > old_ver and <= new_ver + selected = [] + for ver, path in candidates.items(): + if bb.utils.vercmp_string(ver, old_ver) > 0 and \ + bb.utils.vercmp_string(ver, new_ver) <= 0: + selected.append((ver, path)) + + selected.sort(key=lambda x: functools.cmp_to_key( + bb.utils.vercmp_string)(x[0])) + return [path for _, path in selected] + + +def extract_changelog(srcdir, pn, old_ver, new_ver, workdir): + if not srcdir or not os.path.isdir(srcdir): + D(" %s: source directory %s not available" % (pn, srcdir)) + return None + + entries = [] + cves = [] + + # Strategy 1: extract sections between version markers in changelog files + changelog_files = _find_changelog_files(srcdir) + for fpath in changelog_files: + try: + with open(fpath, 'r', encoding='utf-8', errors='replace') as f: + content = f.read() + except OSError: + continue + content = _resolve_rst_includes(content, os.path.dirname(fpath), srcdir) + section = _extract_entries_between_versions(content, old_ver, new_ver) + if section: + section = _condense_git_log(section) + entries.append(section) + cves.extend(CVE_PATTERN.findall(section)) + break + + # Strategy 2: concatenate per-version files (e.g. changelog-9.18.42.rst) + if not entries: + ver_files = _find_per_version_files(srcdir, old_ver, new_ver) + for fpath in ver_files: + try: + with open(fpath, 'r', encoding='utf-8', errors='replace') as f: + content = f.read() + except OSError: + continue + entries.append(content) + cves.extend(CVE_PATTERN.findall(content)) + + # Strategy 3: git log between version tags + if not entries and os.path.isdir(os.path.join(srcdir, '.git')): + tag_prefixes = ['v', '', pn + '-'] + old_tag = new_tag = None + for prefix in tag_prefixes: + try: + subprocess.check_output( + ['git', 'rev-parse', prefix + old_ver], + cwd=srcdir, stderr=subprocess.DEVNULL) + subprocess.check_output( + ['git', 'rev-parse', prefix + new_ver], + cwd=srcdir, stderr=subprocess.DEVNULL) + old_tag, new_tag = prefix + old_ver, prefix + new_ver + break + except (subprocess.CalledProcessError, OSError): + continue + if old_tag and new_tag: + try: + out = subprocess.check_output( + ['git', 'log', '--oneline', '%s..%s' % (old_tag, new_tag)], + cwd=srcdir, stderr=subprocess.DEVNULL).decode('utf-8', errors='replace') + if out.strip(): + entries.append(out.strip()) + cves.extend(CVE_PATTERN.findall(out)) + except (subprocess.CalledProcessError, OSError): + pass + + if not entries: + D(" %s: no changelog entries found" % pn) + return None + + I(" %s: found %d changelog entries" % (pn, len(entries))) + + cves = sorted(set(cves)) + text = "Changelog for %s: %s -> %s\n" % (pn, old_ver, new_ver) + text += "=" * 60 + "\n\n" + if cves: + text += "SECURITY FIXES / CVEs FOUND:\n" + for cve in cves: + text += " - %s\n" % cve + text += "\n" + "-" * 60 + "\n\n" + text += CVE_PATTERN.sub(r'*** \1 ***', _strip_rst_comments('\n\n'.join(entries))) + + # Collapse multiple blank lines into one + text = re.sub(r'\n{3,}', '\n\n', text) + + changelog_path = os.path.join(workdir, "changelog-%s.txt" % pn) + with open(changelog_path, 'w', encoding='utf-8') as f: + f.write(text) + + I(" %s: changelog saved to %s" % (pn, os.path.basename(changelog_path))) + if cves: + I(" %s: CVEs found: %s" % (pn, ', '.join(cves))) + + commit_text = text + if len(commit_text) > 3000: + commit_text = commit_text[:3000] + "\n\n[... changelog truncated ...]\n" + # Sanitize for shell-safe git commit -m "..." + for ch in '"', '`', '$', '\\': + commit_text = commit_text.replace(ch, '') + + return {'text': text, 'commit_text': commit_text, 'cves': cves, 'file': changelog_path} diff --git a/modules/steps.py b/modules/steps.py index b3ec61c..78fcbbe 100644 --- a/modules/steps.py +++ b/modules/steps.py @@ -29,6 +29,7 @@ from logging import warning as W from errors import Error, DevtoolError, CompilationError from buildhistory import BuildHistory +from changelog import extract_changelog def load_env(devtool, bb, git, opts, group): group['workdir'] = os.path.join(group['base_dir'], group['name']) @@ -150,10 +151,29 @@ def devtool_finish(devtool, bb, git, opts, group): pass raise e1 +def changelog_extract(devtool, bb, git, opts, group): + if not opts.get('changelog'): + return + for p in group['pkgs']: + # After devtool_upgrade, source is in workspace/sources// + srcdir = os.path.join(os.environ.get('BUILDDIR', ''), + 'workspace', 'sources', p['PN']) + if not os.path.isdir(srcdir): + # Fallback: derive from env S, replacing old version + srcdir = p['env'].get('S', '') + if p['PV'] in srcdir: + srcdir = srcdir.replace(p['PV'], p['NPV']) + result = extract_changelog(srcdir, p['PN'], p['PV'], + p['NPV'], group['workdir']) + if result: + p['changelog'] = result + group['commit_msg'] += "\n\n" + result['commit_text'] + upgrade_steps = [ (load_env, "Loading environment ..."), (buildhistory_init, None), (devtool_upgrade, "Running 'devtool upgrade' ..."), + (changelog_extract, "Extracting changelog ..."), (devtool_finish, "Running 'devtool finish' ..."), (compile, None), ] diff --git a/upgrade-helper.py b/upgrade-helper.py index df927d1..327bb6d 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -95,6 +95,8 @@ def parse_cmdline(): parser.add_argument("-t", "--to_version", help="version to upgrade the recipe to") + parser.add_argument("--changelog", action="store_true", default=False, + help="extract changelog between old and new versions, highlighting CVEs") parser.add_argument("-d", "--debug-level", type=int, default=4, choices=range(1, 6), help="set the debug level: CRITICAL=1, ERROR=2, WARNING=3, INFO=4, DEBUG=5") @@ -198,6 +200,7 @@ class Updater(object): self.opts['skip_compilation'] = self.args.skip_compilation self.opts['buildhistory'] = self._buildhistory_is_enabled() self.opts['testimage'] = self._testimage_is_enabled() + self.opts['changelog'] = self.args.changelog def _make_dirs(self, build_dir): self.uh_dir = os.path.join(build_dir, "upgrade-helper") @@ -358,6 +361,19 @@ class Updater(object): if 'patch_file' in g and g['patch_file'] is not None: msg_body += next_steps_info % (os.path.basename(g['patch_file'])) + # Add changelog summary if available + for pkg_ctx in g['pkgs']: + if 'changelog' in pkg_ctx: + cl = pkg_ctx['changelog'] + msg_body += ("\n--- Changelog Summary for %s ---\n" + % pkg_ctx['PN']) + if cl['cves']: + msg_body += "\nSECURITY FIXES / CVEs:\n" + for cve in cl['cves']: + msg_body += " - %s\n" % cve + msg_body += "\n" + msg_body += cl['text'] + "\n" + msg_body += mail_footer # Add possible attachments to email From patchwork Fri Apr 24 11:45:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0871FE5214 for ; Fri, 24 Apr 2026 11:46:24 +0000 (UTC) Received: from GVXPR05CU001.outbound.protection.outlook.com (GVXPR05CU001.outbound.protection.outlook.com [52.101.83.68]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.19334.1777031177712176426 for ; Fri, 24 Apr 2026 04:46:18 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=nobyEmAV; spf=pass (domain: ericsson.com, ip: 52.101.83.68, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Lyz6cdcHcteD1bL8HCvXFgiqZJpNUxhA2FGywePHUWwSILAW9g26hm/C1enIEAyfLw2foTFiNuATgM6Kh858e276syRMUvBneBXEKHr/ja0/K0hP1tA6AGdkOJ3IebgxvgjnLWMbjQik32c5SX28vK8mdMToydb7moiRqv7H+dVDWdM/emzhzCgNWTAuE+Zi1u05mreGe/x0c+cV1+jMXMTgAQ4mFcxUCgLcG2MFsKSIScBa+UWij74iJH5hWlk8mVeFj/AMduzhpY3QBqCkp99nVRjWstpLff8FbOhEO7XNx8cHWp0NsJZAJ+f6VUypW2SiiHcbQvJNXOaRxXhcxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EOelNwgqd2IQGlaTJJ62kmFACCk5CW5XwaCagwSPHiY=; b=LC0NJUIT8qzNrE1PHg+SFiWvPBjxdZQHkKihyYbiZvVOzWVxHDmjv8kv5BCcMZsUvIByALJflOVp58YznaBk0H1tdBVXWbAJNessRwTKv0Tm+TRBee9FXaYM5QiKTLbtste632nScSGKu/ckpM/SMCSkiwzEfk2BYFnlWnPN3bS8dpZEe1a58p5+/450GrkRMx96s3OZs7Zqs6zwiK3TlVBnijTLHYf9vAyFlnnoPxuQTowgVfBoU25zEuiBloQQpNYDZYv7JSTJE0xx2sSNAg2j9JTpiAJfHTOxw6Fj2TcA3X+I+kMlTkiqMZG3iFrj+mhmUeyXZmHjygZNJIV5Kw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EOelNwgqd2IQGlaTJJ62kmFACCk5CW5XwaCagwSPHiY=; b=nobyEmAVZ0HHbIrlnYby7Neq4qNA5ZfUk3B5a5jzcMS7z1vbHAtxwq+b8m0BGhZeCEjKJ27GZKOixq/SqvA8UYhv4UsNi2IEOVopTYqfi50ivbHYcjunb+eSP0OrXgyT9KKteXa6ZndKkFDE+V5Kgcb5//ashFIykO86M9YiX3KF48Epd+qyWW8galjHShYan5/ZHn4T9y9XIQ/QBxr3JZWETVRVIaMVJPlniAeaQDmMQbUTu+0P/fGhH9Oi9UX/fg11nR9Y6b6m0suXGi1HF7TKhp4GFzOVkGnmks1Ydy6RURO+hBz3gWYBMAX7d+EQua2/4QEcJLZm6sPPAwrHtQ== Received: from CWLP265CA0440.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1b7::21) by DBBPR07MB7612.eurprd07.prod.outlook.com (2603:10a6:10:1e3::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.21; Fri, 24 Apr 2026 11:46:11 +0000 Received: from AMS0EPF000001A3.eurprd05.prod.outlook.com (2603:10a6:400:1b7:cafe::96) by CWLP265CA0440.outlook.office365.com (2603:10a6:400:1b7::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AMS0EPF000001A3.mail.protection.outlook.com (10.167.16.228) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 Received: from seroius18815.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.65) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18815.sero.gic.ericsson.se (Postfix) with ESMTP id 3BC544020410; Fri, 24 Apr 2026 13:46:10 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id E5CE5700DBAF; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 5/9] upgrade-helper.py: Add stable option for patch-only upgrades Date: Fri, 24 Apr 2026 13:45:59 +0200 Message-ID: <20260424114603.2444938-6-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF000001A3:EE_|DBBPR07MB7612:EE_ X-MS-Office365-Filtering-Correlation-Id: dc8a4609-5b5c-448b-9283-08dea1f7152b X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700016|1800799024|82310400026|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: THjcDqXGEKdZUIMeDliFzX8nRdyMPKfl4Du1CjcaLbtR8GGy4a8Xbjc8sWO4xnoDNtWybP1crgEJXP9VfmAxLTH9hgVQk5plv5mM2K7vew118/h3uJPxE4ZTfGnIjJyD/uHC3T3/yeaTyk2ZMukGXSJnpnwuLgKVp7X1kiqzYFIvCMep6Er5cN7vHaR78+ohQpEpO1md2gVywRcfQQzKGCIed+oXoEQIFpelttvy9qVAPOPoWX2I9A5hHZpZmDNDpA88eekIMyo1CWejHJTgTl89azpx1nYj10pUp3usUFi1Cnza8tHl0lpVWKZZ1ezBov8i0iXGp2Wh6W99FPeYf5u3dwabfYRDNULBAplATw5JtqBf7ZGk+qZHC5dOaV91eO7ym/do1kY823IZxwopRlUGPie/6AZC8ZbOdPSbMvLtWMebAM9c0WbVqwR7bYVA3n64IGx+a7lwtpJaiNggamJk1qBAvHcz6PaWMMNri1l+yAGPluYx+/4R1bWlbRbNUNFja33546nDpsTzILUGdExGn9h7qvRe+Y8qdLfaznIFbGL763FL9kXPxqc/r7J1jasN/3dryV2wXsMPZ7tSLs4e3yZgyGfgAsN6rAkQX1HtrbysZURhmO4u2zNdmvhKPnAB08J6JfQfOcwgdfRiYULrK9mh/8lUIn3ekFR62uqZKQMBv14yVLIfyGgINCpRtkWkxIsD1RmQxJ0Kfu8AT7sVlxuYKTRKxOPanz9+YFHiu+XfZ5sYQ9nFSDu/vLfMRjemOQwmDRavt7BzmzZjRw== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(376014)(36860700016)(1800799024)(82310400026)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ZSXkJUTq5NObvv8wldvY3Vw3/vhhVap9GoSwvCzMMVvjMTRk7JdnZfp6gJC0sIOFica+s/YJ9b+g6CdXXuD3iDywOX/1YKrIKJsIMDlLDDpFsOvCWMtQeHmfcERlQ0yrkuuNRkQkvMAxABYIKVI4vDX8SVtnESIwxz7i7I7+e9bwyztLdqJxM4r6/uG6vZBI2WM47+8f2RzfEmMsv/Fp+zRMP3o5sDqK8g7VUlVpUx+XMtBoVuWDNGEnPlJP3A0dwEezn9J8Mp3Ju6fjqWfhPyIWSTux+uGUJMZfd/ypfx/yrS50+Ia5C04DRk+9KwxITrKEWncR6v4M0ezFfTAxcs7TGBAXeU88ZonjyeMZzGviEqRiIZRx5IhnsqC7+aAGPEzzUwE6NvaR322ZxLJlRsYmi/dI5gbtGm5MIVIhKuW1UTgSztFI6sMDK4GxSyDF X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:11.6713 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc8a4609-5b5c-448b-9283-08dea1f7152b X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A3.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR07MB7612 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:24 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3799 From: Daniel Turull Add --stable flag to restrict upgrades to the next patch version within the current stable branch (e.g. 1.2.3 -> 1.2.4). When the latest upstream version is a major/minor bump, AUH queries all available versions and picks the best patch-level update. Signed-off-by: Daniel Turull Assisted-by: Claude, Anthropic --- modules/utils/version.py | 160 +++++++++++++++++++++++++++++++++++++++ upgrade-helper.py | 51 +++++++++++++ 2 files changed, 211 insertions(+) create mode 100644 modules/utils/version.py diff --git a/modules/utils/version.py b/modules/utils/version.py new file mode 100644 index 0000000..b40eb1c --- /dev/null +++ b/modules/utils/version.py @@ -0,0 +1,160 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Version utilities for --stable patch-only upgrades. +# +# get_all_upstream_versions() collects *all* available upstream versions so +# that we can pick the highest patch-level release within the current stable +# branch. The existing bitbake/oe-core APIs (latest_versionstring, +# get_recipe_upstream_version) only return the single highest version, so we +# must re-implement the inner loops here. +# +# The HTTP path mirrors bb.fetch2.wget.Wget._check_latest_version() and the +# git path mirrors bb.fetch2.git.Git.latest_versionstring(), both from +# bitbake scarthgap. We cannot modify oe-core on a stable release, hence +# the duplication. + +import functools +import re +from logging import warning as W + +import bb.utils +import bb.fetch2 +from bs4 import BeautifulSoup, SoupStrainer + + +def _split_version(ver): + # Split on '.', '-', '_' and letter-digit boundaries (e.g. 10.0p2 -> [10, 0, p, 2]) + parts = re.split(r'[\.\-_]', ver) + result = [] + for p in parts: + result.extend(re.split(r'(?<=[a-zA-Z])(?=\d)|(?<=\d)(?=[a-zA-Z])', p)) + return result + + +def is_patch_update(current_ver, candidate_ver): + cur = _split_version(current_ver) + cand = _split_version(candidate_ver) + if len(cur) != len(cand) or len(cur) < 3: + return False + if cur[:-1] != cand[:-1]: + return False + try: + return int(cand[-1]) > int(cur[-1]) + except ValueError: + return bb.utils.vercmp_string(candidate_ver, current_ver) > 0 + + +def _find_best_version(current_ver, all_versions, filter_fn): + candidates = [v for v in all_versions if filter_fn(current_ver, v)] + if not candidates: + return None + candidates.sort( + key=functools.cmp_to_key(bb.utils.vercmp_string), reverse=True + ) + return candidates[0] + + +def find_patch_version(current_ver, all_versions): + return _find_best_version(current_ver, all_versions, is_patch_update) + + +def get_all_upstream_versions(rd): + """Get all upstream versions using the fetcher infrastructure. + + Unlike ud.method.latest_versionstring() which returns only the highest + version, this collects every version so the caller can filter for + patch-only updates. + """ + src_uris = rd.getVar('SRC_URI') + if not src_uris: + return [] + + src_uri = src_uris.split()[0] + ud = bb.fetch2.FetchData(src_uri, rd) + + if ud.type == 'git': + return _get_git_versions(ud, rd) + return _get_http_versions(ud, rd) + + +def _get_http_versions(ud, rd): + """Collect all upstream versions from an HTTP index page. + + Adapted from bb.fetch2.wget.Wget._check_latest_version() and + ._init_regexes(). The upstream code only keeps the highest version; + we collect them all. Uses BeautifulSoup to parse tags, matching + the upstream behaviour. + """ + try: + package = ud.path.split("/")[-1] + + regex_uri = rd.getVar('UPSTREAM_CHECK_URI') + if not regex_uri: + path = ud.path.split(package)[0] + regex_uri = bb.fetch.encodeurl([ud.type, ud.host, path, + ud.user, ud.pswd, {}]) + + page = ud.method._fetch_index(regex_uri, ud, rd) + if not page: + return [] + + regex = rd.getVar('UPSTREAM_CHECK_REGEX') + if regex: + regex = re.compile(regex) + else: + regex = ud.method._init_regexes(package, ud, rd) + if not regex: + return [] + + # Parse HTML links, same as Wget._check_latest_version() + soup = BeautifulSoup(page, "html.parser", + parse_only=SoupStrainer("a")) + if not soup: + return [] + + versions = set() + for link in soup.find_all('a', href=True): + for text in (link['href'], str(link)): + m = regex.search(text) + if m and 'pver' in m.groupdict() and m.group('pver'): + versions.add(re.sub('_', '.', m.group('pver'))) + break + return list(versions) + except Exception as e: + W(" Failed to get HTTP versions: %s" % str(e)) + return [] + + +def _get_git_versions(ud, rd): + """Collect all tagged versions from a git remote. + + Adapted from bb.fetch2.git.Git.latest_versionstring(). The upstream + code only keeps the highest version; we collect them all. + """ + try: + output = ud.method._lsremote(ud, rd, "refs/tags/*") + except (bb.fetch2.FetchError, bb.fetch2.NetworkAccess, OSError) as e: + W(" Failed to list remote tags: %s" % str(e)) + return [] + + rev_tag_re = re.compile(r"([0-9a-f]{40})\s+refs/tags/(.*)") + pver_re = re.compile( + rd.getVar('UPSTREAM_CHECK_GITTAGREGEX') + or r"(?P([0-9][\.|_]?)+)" + ) + nonrel_re = re.compile(r"(alpha|beta|rc|final)+") + + versions = set() + for line in output.split("\n"): + if not line: + break + m = rev_tag_re.match(line) + if not m: + continue + tag = m.group(2) + if nonrel_re.search(tag): + continue + m = pver_re.search(tag) + if m: + versions.add(m.group('pver').replace("_", ".")) + return list(versions) diff --git a/upgrade-helper.py b/upgrade-helper.py index 327bb6d..b7b8ddf 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -59,6 +59,7 @@ from utils.emailhandler import Email from statistics import Statistics from steps import upgrade_steps from testimage import TestImage +from utils.version import is_patch_update, find_patch_version, get_all_upstream_versions if not os.getenv('BUILDDIR', False): E(" You must source oe-init-build-env before running this script!\n") @@ -74,6 +75,7 @@ scriptpath.add_bitbake_lib_path() scriptpath.add_oe_lib_path() import oe.recipeutils +import bb.tinfoil help_text = """Usage examples: * To upgrade xmodmap recipe to the latest available version: @@ -97,6 +99,8 @@ def parse_cmdline(): help="version to upgrade the recipe to") parser.add_argument("--changelog", action="store_true", default=False, help="extract changelog between old and new versions, highlighting CVEs") + parser.add_argument("--stable", action="store_true", default=False, + help="only upgrade to the next patch version within the stable branch (e.g. 1.2.3 -> 1.2.4)") parser.add_argument("-d", "--debug-level", type=int, default=4, choices=range(1, 6), help="set the debug level: CRITICAL=1, ERROR=2, WARNING=3, INFO=4, DEBUG=5") @@ -698,6 +702,30 @@ class UniverseUpdater(Updater): def _get_packagegroups_to_upgrade(self, packages=None): + def _resolve_stable_version(pn, cur_ver, next_ver, tinfoil): + """Find the latest patch version within the current stable branch.""" + if is_patch_update(cur_ver, next_ver): + return next_ver, None + I(" %s: latest version %s is not a patch update from %s," + " searching for versions..." % + (pn, next_ver, cur_ver)) + try: + rd = tinfoil.parse_recipe(pn) + if not rd: + I(" %s: could not parse recipe, skipping" % pn) + return None, None + all_versions = get_all_upstream_versions(rd) + ver = find_patch_version(cur_ver, all_versions) + if ver: + I(" %s: found version %s" % (pn, ver)) + return ver, "N/A" + else: + I(" %s: no suitable version available, skipping" % pn) + return None, None + except Exception as e: + I(" %s: failed to search for versions: %s" % (pn, e)) + return None, None + # Prepare a single pkg dict data (or None is not upgradable) from recipeutils.get_recipe_upgrade_status data. def _get_pkg_to_upgrade(self, layer_name, pn, status, cur_ver, next_ver, maintainer, revision, no_upgrade_reason): pkg_to_upgrade = None @@ -763,6 +791,29 @@ class UniverseUpdater(Updater): upgrade_group.append(pkg_to_upgrade) if upgrade_group: upgrade_pkggroups.append(upgrade_group) + + if self.args.stable and upgrade_pkggroups: + stable_tinfoil = bb.tinfoil.Tinfoil() + stable_tinfoil.prepare(config_only=False) + try: + filtered = [] + for group in upgrade_pkggroups: + filtered_group = [] + for pkg in group: + stable_ver, stable_rev = _resolve_stable_version( + pkg['pn'], pkg['cur_ver'], pkg['next_ver'], + stable_tinfoil) + if stable_ver is not None: + pkg['next_ver'] = stable_ver + if stable_rev is not None: + pkg['revision'] = stable_rev + filtered_group.append(pkg) + if filtered_group: + filtered.append(filtered_group) + upgrade_pkggroups = filtered + finally: + stable_tinfoil.shutdown() + return upgrade_pkggroups def pkg_upgrade_handler(self, pkg_ctx): From patchwork Fri Apr 24 11:46:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83704FE521D for ; Fri, 24 Apr 2026 11:46:26 +0000 (UTC) Received: from DU2PR03CU002.outbound.protection.outlook.com (DU2PR03CU002.outbound.protection.outlook.com [52.101.65.33]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.19329.1777031176537108924 for ; Fri, 24 Apr 2026 04:46:16 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=RQevbE1u; spf=pass (domain: ericsson.com, ip: 52.101.65.33, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FO2BRpAbKdA+kvnKpkvGENXDFDaR/OfP0CkYZCb3daw46tEm7muanxm6SuFDluPhfEE5Wo3jHHypWzyD9p8VNC4cG7g5Qd528Uo8l0fbNuVnEES4KPjatHVGdTlYpMZ8mrPjtKyuE6jAX0ESHOR/o7l6yqVP4W0l4vfIqtUcop91MN1BFUGkGwZTIfYkwqpVDYni0LSh7TVy0UuoKvX7pMR40wlotpJTEaTOqBi7T1mVk0CcXwdEMs/H/tlBEWivaSdRhoXiAOeVHTqmn3eUQGpQ+qskQoR2dfwRS/KFCHiaNgGDBSzhkq3EXppp1DIFQs3qbQcJxG6uk8XeDiyv5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zOg1y81jVZuRC+Q/KVbDWc0oE7XxVFESSn1XIN4cm/k=; b=XX64Y60jJpTdGcir6lDuHOaYJeY5bloqjnKVoeA5TBle4jOTzrrzV8GdKhP2sFg+C8ubNbW9pOyqQm9WSSbPVLDaeC4uIpSID+rlQpL/ocf7LzTuH8dIMxjh4I/T6A7ZXT0oksL1l1eMXD8imX9sxt+kWK/r5s+EMJTqpeyriF7Ew34/RfkMbpkj41qNT7PkBa3nsn9oL2m67GKdY3LSiym2ZxsNXufuv4DBDl4e0WL0/BAbnEU3XGiP69T7rZXzoifs26o16nHQerU/+397wqSGdFZnkx6gIdpb1q1P19B2/0w6ebuMCHj74U4q0e/dlWvFnDsEphVbtFrR7qKfSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zOg1y81jVZuRC+Q/KVbDWc0oE7XxVFESSn1XIN4cm/k=; b=RQevbE1umdTON+G/wG8+zXzOI3jZ2FXFcf1vwet8jdp1w+T/e1d1qGhPkWv5zuRAkkxGpD/TTbNkmu8qV7sKp2llUVSHhU12EBaYGUmbGfPXAhLO73k0UGzFBWgr7uKhkR1lXpG64zVMk1pOTa/lWkHWU6WCWpyryK4iHrSdAL8xRyGpXW6mQ0gpqzVxVp6UaWr2cqGJUPWLfA84aZdiGuQKtGZxkUMY542q6iUbJTq/Exontre3USJGAWysdYgrajQ6f6AU+LcrY5xvB9Sl4Z52HrTAWEmJSVACdHcszYWqnm9ov63c+8MWPSeEsvHapBdqVPt4msJbGGe1WII08A== Received: from CWLP265CA0540.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:18d::15) by AMBPR07MB11464.eurprd07.prod.outlook.com (2603:10a6:20b:734::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Fri, 24 Apr 2026 11:46:11 +0000 Received: from AM3PEPF0000A793.eurprd04.prod.outlook.com (2603:10a6:400:18d:cafe::83) by CWLP265CA0540.outlook.office365.com (2603:10a6:400:18d::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM3PEPF0000A793.mail.protection.outlook.com (10.167.16.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 Received: from seroius18814.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18814.sero.gic.ericsson.se (Postfix) with ESMTP id 40B7F4020846; Fri, 24 Apr 2026 13:46:10 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id EA063700DBB4; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 6/9] upgrade-helper.py: Add minor option for minor version upgrades Date: Fri, 24 Apr 2026 13:46:00 +0200 Message-ID: <20260424114603.2444938-7-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF0000A793:EE_|AMBPR07MB11464:EE_ X-MS-Office365-Filtering-Correlation-Id: 292970f6-30ce-431c-f60a-08dea1f714ee X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700016|376014|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: MVrcBSaMiyMKkuX87aMIj/d16B9xxJTnKar0S4wuzHM3tcthKxuyQmvEIYJLbatasouUm1p740P+UqHZj806vzh410O2YRVt6J0Tuku3K7udSTo4VSM4hp21LRJwMzpcK8ChZBefkVr+sBsaJXGK7vwM9+1S+41QepPyvM4ykrJP/NMtizjfGsuTX1f+WFoq2OEgyXiz+GLN+7hGLHq6X3Kaa1EfNOm0jIIPVL2aeUOOFg6eu09Hg350Bl+AieHKOy8OsBi/tEuyYfTZ7NhZ8U1h/IHPdZZuYmVKc5unGdG352T2Mx8KUVC+R0ZnxpXRUV+qoJaxjh+lKkV3As62JxAIECihS7JAcck0CZaAyD6l8+uq7Ap5mwQjKLY0FrdyJnClPgyH888YnGKA8pkzhKTJyhLrX/d7Vs3P++PWkt4FHM2XxLbPW6YIhHxb/EYYZEawG/lKtEXzJ88cpwcNYxvg9qBsCXzX0tBE8SvULanJGEh6XI9A8EN+3Z0DS9D7cVYhyXCBuq2Dl55GhlDariqIwzB0obq3tkkTc3YRT3wmBx7ETLIwgjlNoF/xVD8LKDrUMOlfzKEJbwpzgvOosJpAXVABbQgEAh1IVX/NUObKF9K3u8xGzBG/SE6a+3RV04WCOgaHNlVB2wHiSzrzlnPUD3oiwcdnwq7B1voXY3KCIMo2iNR+T1/M/t6YplkgtKTBEazCIayMtJoxnUNrxBaiAulMeu+yHV4CahugbN3andpnB9ez7WdIJ/kC+VuyWtdeTVnqih/HmSmoB9vsDQ== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700016)(376014)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bSe+JV9pqf5R67QIAIeBqcOkZv56aQfhHPPxY2wAVtUYjnQG2VoLYzx4XCXkNGMKhXi4wGLaO1pWy26YhQqDBrEU8elFjk3wS+dXPoJ81Rk5NPUZ8yFrvmmEB+hMclzNA2NR5wQgqwFvzmsirGKZxuZx9dcZNYXENrpepTqfIbpToXCB+AKN5hVx/gcPO0hRyS4Q6UOLk0PCtRxQVa4CmAQORnihbpV5gGaKWt2Q2jntwvl80zl+glvACM87tvBN1slf4m7GJ77uptWcullI37dVdXj1vpOmTKXmL/PCHq+ES4vKctuUBFaF3zzN0f7T99qcjkvJufn6dmy6U/oDhJRGSzV7z6BzvAcflR/VoGcIjIaWTUV5oCdLOYUIGzuHc/POZiAHr00toDovXvmvKmiw+aPyyW6YjP3oSKGWtc11Pax4SFtEvI4VfonGt3Eu X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:11.2809 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 292970f6-30ce-431c-f60a-08dea1f714ee X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF0000A793.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AMBPR07MB11464 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:26 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3793 From: Daniel Turull Add --minor option that allows upgrades within the same major version (e.g. 3.5.x -> 3.6.x), complementing --stable which only allows patch-level changes. Extends _resolve_stable_version with a minor flag and adds is_minor_update / find_minor_version to utils/version.py. Assisted-by: Claude, Anthropic Signed-off-by: Daniel Turull --- modules/utils/version.py | 15 +++++++++++++++ upgrade-helper.py | 23 ++++++++++++++--------- 2 files changed, 29 insertions(+), 9 deletions(-) diff --git a/modules/utils/version.py b/modules/utils/version.py index b40eb1c..71e0a1e 100644 --- a/modules/utils/version.py +++ b/modules/utils/version.py @@ -44,6 +44,17 @@ def is_patch_update(current_ver, candidate_ver): return bb.utils.vercmp_string(candidate_ver, current_ver) > 0 +def is_minor_update(current_ver, candidate_ver): + """Check if candidate is a newer release within the same major version.""" + cur = _split_version(current_ver) + cand = _split_version(candidate_ver) + if len(cur) < 2 or len(cand) < 2: + return False + if cur[0] != cand[0]: + return False + return bb.utils.vercmp_string(candidate_ver, current_ver) > 0 + + def _find_best_version(current_ver, all_versions, filter_fn): candidates = [v for v in all_versions if filter_fn(current_ver, v)] if not candidates: @@ -58,6 +69,10 @@ def find_patch_version(current_ver, all_versions): return _find_best_version(current_ver, all_versions, is_patch_update) +def find_minor_version(current_ver, all_versions): + return _find_best_version(current_ver, all_versions, is_minor_update) + + def get_all_upstream_versions(rd): """Get all upstream versions using the fetcher infrastructure. diff --git a/upgrade-helper.py b/upgrade-helper.py index b7b8ddf..ffa7449 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -59,7 +59,8 @@ from utils.emailhandler import Email from statistics import Statistics from steps import upgrade_steps from testimage import TestImage -from utils.version import is_patch_update, find_patch_version, get_all_upstream_versions +from utils.version import is_patch_update, find_patch_version, get_all_upstream_versions, \ + is_minor_update, find_minor_version if not os.getenv('BUILDDIR', False): E(" You must source oe-init-build-env before running this script!\n") @@ -101,6 +102,8 @@ def parse_cmdline(): help="extract changelog between old and new versions, highlighting CVEs") parser.add_argument("--stable", action="store_true", default=False, help="only upgrade to the next patch version within the stable branch (e.g. 1.2.3 -> 1.2.4)") + parser.add_argument("--minor", action="store_true", default=False, + help="allow upgrades within the same major version (e.g. 3.5.2 -> 3.6.0, 3.99.0)") parser.add_argument("-d", "--debug-level", type=int, default=4, choices=range(1, 6), help="set the debug level: CRITICAL=1, ERROR=2, WARNING=3, INFO=4, DEBUG=5") @@ -702,20 +705,22 @@ class UniverseUpdater(Updater): def _get_packagegroups_to_upgrade(self, packages=None): - def _resolve_stable_version(pn, cur_ver, next_ver, tinfoil): - """Find the latest patch version within the current stable branch.""" - if is_patch_update(cur_ver, next_ver): + def _resolve_stable_version(pn, cur_ver, next_ver, tinfoil, minor=False): + """Find the latest patch (or minor) version within the allowed range.""" + check_fn = is_minor_update if minor else is_patch_update + find_fn = find_minor_version if minor else find_patch_version + if check_fn(cur_ver, next_ver): return next_ver, None - I(" %s: latest version %s is not a patch update from %s," + I(" %s: latest version %s is not a %s update from %s," " searching for versions..." % - (pn, next_ver, cur_ver)) + (pn, next_ver, "minor" if minor else "patch", cur_ver)) try: rd = tinfoil.parse_recipe(pn) if not rd: I(" %s: could not parse recipe, skipping" % pn) return None, None all_versions = get_all_upstream_versions(rd) - ver = find_patch_version(cur_ver, all_versions) + ver = find_fn(cur_ver, all_versions) if ver: I(" %s: found version %s" % (pn, ver)) return ver, "N/A" @@ -792,7 +797,7 @@ class UniverseUpdater(Updater): if upgrade_group: upgrade_pkggroups.append(upgrade_group) - if self.args.stable and upgrade_pkggroups: + if (self.args.stable or self.args.minor) and upgrade_pkggroups: stable_tinfoil = bb.tinfoil.Tinfoil() stable_tinfoil.prepare(config_only=False) try: @@ -802,7 +807,7 @@ class UniverseUpdater(Updater): for pkg in group: stable_ver, stable_rev = _resolve_stable_version( pkg['pn'], pkg['cur_ver'], pkg['next_ver'], - stable_tinfoil) + stable_tinfoil, minor=self.args.minor) if stable_ver is not None: pkg['next_ver'] = stable_ver if stable_rev is not None: From patchwork Fri Apr 24 11:46:01 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 938A9FE5210 for ; Fri, 24 Apr 2026 11:46:24 +0000 (UTC) Received: from DB3PR0202CU003.outbound.protection.outlook.com (DB3PR0202CU003.outbound.protection.outlook.com [52.101.84.57]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.19327.1777031175730191117 for ; Fri, 24 Apr 2026 04:46:16 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=l3dMQAis; spf=pass (domain: ericsson.com, ip: 52.101.84.57, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=S4HDIHAZ6EWo/KDOMxlxxYweIivbXcpFBiNsnwDMV7RwrdGGZ8ZKOOO664Cd/edO5lM5VKLVx+K8BJkTK9pF9VMSnyT9YR6RA9mC9B6cVVUW4m068Hk8QksfPUP9Jnd9zpIgAs30xb4mofOPyotrWeBwFnCaeSVquZDYy+TkuLx8Cgga0nJFbp9bdMH4+YQD4v7hfJlElhY3dw8EoD9B7T03dOCiNoqn00QH5SBYl0H10vrldl2JNDar5TqrR9d3VT9jdIh38vYu8+N00MviI8lmgIaOc2FmKSAM+UWhe+pp1x7BIaSV3b+mPLkAstRpCB/zmisA2hBcOiElFEpkcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ho30g4I1ujBGrv3V0wW9BB8Rupz2/VYv+LsZhi/P5c8=; b=E7XGUVFAbXvJpfbaFpkqjUDNhNGoe27n05/IDr265JJFYSNxenANix5t0b7jVkXi6Jhgg+4Ib/zdUb/E9DjexLbnB2OGYW6KNZKPE7NUOeX/lY4WaKymBGe34cQ/PF7Faid3HSPiPDXFFqO0Dm02LcOO5GqFsnbnkM2uCR3bdTh9lFEItASWN6cUJ966YKIcat99taHolwCHqJOP6esjEGWEzh/zghmi/A2lBNmoQDkeDjvntcskPf0eyKfAapU+yKt8FFAMFR6Vur/zFT/6St7eQLfPppeGEK7sg+CNq7FgyWVmqLnub9q8EF1wuNzpXlQvqF9N+4GZw0KvgsYdDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ho30g4I1ujBGrv3V0wW9BB8Rupz2/VYv+LsZhi/P5c8=; b=l3dMQAisuVW0L9oRQf5/jSERKcvY7E7pQT3/v3h8dj+gybnSXO9JA8/2yuY0XSOHsmM5niKm2OdchCq5jj8qFYKdWWhvelShP7XLRKuEZmSvSWnOudtwo5LqSZLxPWU1D37ZKFQkitLTeBDMBHMwZTaU6rORmwlmk7ggRz+/NeBDHG7+tvveZibsjrkXgZeUhGO6Z0MlSlhI3DlWNTv+zlH9GYOJRPWZIrCJXjIVeVa2uw7tHZh/rFbFms5exJN3DJSmpgkzmFlqTLp5V96sM+z51vH7Fpht+VsYoo8Hk02Y/7Uc47TZmZcNjVF9FFedbuAwexWQ6XGShj94jOVTQQ== Received: from DUZPR01CA0156.eurprd01.prod.exchangelabs.com (2603:10a6:10:4bd::19) by AM7PR07MB6376.eurprd07.prod.outlook.com (2603:10a6:20b:131::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Fri, 24 Apr 2026 11:46:11 +0000 Received: from DB1PEPF000509EC.eurprd03.prod.outlook.com (2603:10a6:10:4bd:cafe::17) by DUZPR01CA0156.outlook.office365.com (2603:10a6:10:4bd::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509EC.mail.protection.outlook.com (10.167.242.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9791.48 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 Received: from seroius18815.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18815.sero.gic.ericsson.se (Postfix) with ESMTP id 3DCF640215A8; Fri, 24 Apr 2026 13:46:10 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id EE31E700DBB5; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 7/9] upgrade-helper.py: Add prebuild option Date: Fri, 24 Apr 2026 13:46:01 +0200 Message-ID: <20260424114603.2444938-8-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509EC:EE_|AM7PR07MB6376:EE_ X-MS-Office365-Filtering-Correlation-Id: 195e733e-47a0-41cd-f9ef-08dea1f71503 X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700016|1800799024|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: E/6ECrHXY/oE+igJFVyrvK/Lp4unsagoQcfnmJthNEpFr4IacNF9ogaNDP9oZVUKdGir0Xv0WZvnf1eORhId+WAd+dfcPrx512aD2CZzvQG+TL+hLlAlggfkWSrDUT/5UTRKKnRHs3POHM6SfGByN9+abYrKEZGz/XWFXGeE8tlYXjkio/fW55v2fx/RTWaga31S0EYFClm2J4skW4Sa16FI2BH184CLJgctA7lMCJQk+StuGkDx/hkZGJgpx0rMJJc+0YgAvbZNYKHON1rDPvJNIJ0Sep5/un++XOItSfiuta0hAS0AOwQgut6U8EZA3GBcX01pfb5qnxh2C0oRD8lMTl1GGYAVQvwL/T29a9xbhpJp7aF6+FTLtOT7ohlgghJAnu32sEzr4HReEyYDD11qHq59UPsTRvEwYDSqUj8HN6UK79G8QCDC8TAbkJjbLIrzQs091EIuzKHRYzka/ICkjvZx0Y1ZDoAfdFWqQ2qoBA0Yf1WBmGlJISZ1gXQdDMPAMIFrg18sGgLlvDXF+cTvLSZLMlJrtyOHMBpBQouJRYZ7RuGeljUpj1MewaKjhJUbejp0P/iphQbmZMXaKXc9OSODh5qPVBvlqNSgQ9lMEsnwBjJzCHqypEq0ed6VBh2PnSOWDiG7zfgL/xW+cpwxetAVODlPZkwQ4vs7qsAqlyEI+grwdjfB+5A0JLv319RQ3mnLNGmaeqObL5moBmC32ehgquIrOEj4qs+FTSRG+21Hs9JiJtHplMJ0rf0nfKgYgyismynZEt4hCO7Kpg== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700016)(1800799024)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Kb3ruZyQ+W08P3V4Nw7arOTvIxuNXPdNl8ALeRrpouVZ5aYRBJKeTw5LJu6/6Mdl4nvJeMSiy4pnzKWsq+tOS7Xvak7KLc9OyeFwdmgwai9xYcgx3h+uiwapY+j9QjL5UQ65mhz8eanHdHeC9aiysHMrvCdenqP81IcuvxxtdvqyzI33jOY/faOtG0LDiUrJk9FuQ2NhwXAGfSdRjiJhhob9A3AKPz40bgQuuBKZLKE2ha+hc4dx/g+wNOLxKPnr/Nr//ta5Yoeies3CxGzrli08JwHGGRya+DDuEDWiM1tPeBT3+RxmrflYIktxd/FgD/sezL/AOhP+AhT4JdPNPzbfETLAeSlt2w/o+t7YUAal9Cz1lfBGT2AePM5My29FxdMmpqm7izWuqnffbgwyy6MuRGJOjj1/9btbtrz0rez0doutQ8CwWKVWeml+BNBA X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:11.3960 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 195e733e-47a0-41cd-f9ef-08dea1f71503 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509EC.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6376 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:24 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3790 From: Daniel Turull Add --prebuild option to build each recipe at its current version before attempting the upgrade. This allows verifying that recipes build before applying the upgrade and enables comparing binaries produced before and after, for example for build size or ABI compatibility checks. Recipes that fail the pre-build are skipped and recorded as failures. Assisted-by: Claude, Anthropic Signed-off-by: Daniel Turull --- upgrade-helper.py | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/upgrade-helper.py b/upgrade-helper.py index ffa7449..11a97d2 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -105,6 +105,9 @@ def parse_cmdline(): parser.add_argument("--minor", action="store_true", default=False, help="allow upgrades within the same major version (e.g. 3.5.2 -> 3.6.0, 3.99.0)") + parser.add_argument("--prebuild", action="store_true", default=False, + help="build each recipe at its current version before attempting upgrade") + parser.add_argument("-d", "--debug-level", type=int, default=4, choices=range(1, 6), help="set the debug level: CRITICAL=1, ERROR=2, WARNING=3, INFO=4, DEBUG=5") parser.add_argument("-e", "--send-emails", action="store_true", default=False, @@ -516,6 +519,21 @@ class Updater(object): attempted_pkggroups += 1 pkggroup_name = g["name"] I(" ATTEMPT PACKAGE GROUP %d/%d" % (attempted_pkggroups, total_pkggroups)) + + # Build current version first to verify it builds and seed ABI reference + if self.args.prebuild and not self.args.skip_compilation: + I(" %s: Pre-building current version ..." % pkggroup_name) + try: + for machine in self.opts['machines']: + self.bb.complete(" ".join([p['PN'] for p in g['pkgs']]), machine) + except Exception as e: + E(" %s: Pre-build FAILED, skipping upgrade" % pkggroup_name) + if isinstance(e, Error): + E(e.stdout + e.stderr) + g['error'] = e if isinstance(e, Error) else Error(message=str(e)) + failed_pkggroups_ctx.append(g) + continue + try: for pkg_ctx in g['pkgs']: I(" %s: Upgrading to %s" % (pkg_ctx['PN'], pkg_ctx['NPV'])) @@ -571,6 +589,8 @@ class Updater(object): tim.run() for g in pkggroups_ctx: + if 'workdir' not in g: + continue if g in succeeded_pkggroups_ctx: os.symlink(g['workdir'], os.path.join( \ From patchwork Fri Apr 24 11:46:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86834 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 453DDFE521B for ; Fri, 24 Apr 2026 11:46:26 +0000 (UTC) Received: from DB3PR0202CU003.outbound.protection.outlook.com (DB3PR0202CU003.outbound.protection.outlook.com [52.101.84.62]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.19022.1777031176968726017 for ; Fri, 24 Apr 2026 04:46:17 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=yZYtPihR; spf=pass (domain: ericsson.com, ip: 52.101.84.62, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ABM9QYYoD7KERyM1LrlpQtAixMlXj+LE4Heq/ZdVB22ZjDAvTiHxRcDeXAfbYf9CiGe8xteqNByyKCeafOJorO/MomhZ4vMQEHwO2DriE8HmO0SURdt8OAqQJX3mHETZUb491iaqIbWw2K7d6UNcLIdM/xHpA+/k+JRbcCFrCgWmzXVFQhG7Xgx87AlB964lvANRaRfQvmaKEKePfClzEdzrsj1hjiHKmvfelYpkyHOVeM2B/HJvw6oJktf6NOE8bLehkIPM9/MWfR6abLgSJ22+BaCbCokFnWAERTYqcnMTEpWNeb5yIDmjO5TPiLpq1xilm6bAsJ8DXgGhf2jAZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YSRS1yMVickLJ4NkgWpNLELVJpF+msKyA+/kY7Z2h7w=; b=PGUmzEyB1UwI0tLm3xN7J8l7hvbm4axD1Pj1VvKCKAdvFTD2Mw6pDye1t62NkUbei4KFQCDrssE2O8FiEvAFpo5sjzdqFjwetXG4NNqPvnuAD4QcqpEzPlXPnc/LnIcTQBH+GAAo4U3qdB/B4+Cxfc5IcU3zkoY8XDXBIIiBMFPwhXqs305TLP/XCPU1M45Ugm6RxE4wFF9lZUgwKU3YBgma9CPxUToEY1Ky6GLhmsCFmECIcJhsViHqnAz5mUKTR/ZzyS1KxsPafeqqSnP15uV96grpFtQwyebJvBF23Y5d8NkuZw17BwE64chpmd8GfUUH7KGr2itghtk5pc7zZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YSRS1yMVickLJ4NkgWpNLELVJpF+msKyA+/kY7Z2h7w=; b=yZYtPihRj8pGmYn+tl3zNxVnAPF2KHwyotJuJk2n1LsemmqPY50b8MB+P30i3WrThNj3QHKFiaFQKO0A4qXoJT8cWYqm92p5QZj/mvPI2PxdD7IaWrB2btqUgFrjpTjN7uWOenP4+D+biut9awqRA4DJrWjsKIc+f4SDjMa8QHl34bt6y4YZa5mno3Q4y9peYlmERPx0Q8Zq9eKEUNctRHD7G+rgvwnLQUhqbB9w4qvvo9wfGTlrLEDbBaXP5z1pb0+hf7n7qlt0FWy1mqCymkmVIDNxTsb5momT2x6+wW5WrNbLFZ1fwD/elVdiALqeRc44S9Qo0Og5hekFSowmAA== Received: from AS4P192CA0012.EURP192.PROD.OUTLOOK.COM (2603:10a6:20b:5da::20) by DUYPR07MB12299.eurprd07.prod.outlook.com (2603:10a6:10:65f::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Fri, 24 Apr 2026 11:46:12 +0000 Received: from AM4PEPF00027A68.eurprd04.prod.outlook.com (2603:10a6:20b:5da:cafe::8f) by AS4P192CA0012.outlook.office365.com (2603:10a6:20b:5da::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.22 via Frontend Transport; Fri, 24 Apr 2026 11:46:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00027A68.mail.protection.outlook.com (10.167.16.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:12 +0000 Received: from seroius18813.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18813.sero.gic.ericsson.se (Postfix) with ESMTP id 417CE9569E; Fri, 24 Apr 2026 13:46:10 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id F2A8D700DBBF; Fri, 24 Apr 2026 13:46:09 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 8/9] upgrade-helper.py: Add resume option to skip already attempted recipes Date: Fri, 24 Apr 2026 13:46:02 +0200 Message-ID: <20260424114603.2444938-9-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00027A68:EE_|DUYPR07MB12299:EE_ X-MS-Office365-Filtering-Correlation-Id: 24b33ff1-7e10-4932-12a6-08dea1f71598 X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|376014|82310400026|1800799024|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: k/p7Q5zPXoLEM5kxFLlJQRgPig0lhU17R9qrul7LWExjzh7OI8CiIfrcTNA9beBNruxGkvvbgN9DdLrZLqnNjjmOj+D73sWE0+u2xIeq2siDzIJl3B/XtSNU7wYnFlXSybwgXYIhCPiNjqgk8Ag/7ftWGz+dE9/BGNzT9KC0tqEFgqkDgbcgLUO3qdT1Ur263Xh5IbQ0/+EHO96ORw6jtDArpUUvRV0cSgjRFYIy4Bmzhf5jRH5tr24lJKbCk58Ruh4tb7ovaPV/Qlq3mwCDcuZiD1oivElgpGIwy+jmPO32h3Ymhim+qLD3FXkOMMsu0QBOk7Ip3/3Y+y3ZqyiffAzV8jVCLNu8EVjHVQ8+HEPoAxLyTKY0FUw3cfSYTmmHhciqsDyIiATyRCRS4RELCwJ19eDsJqjXMiliCqzLouhnF97vrDMA8F2Kx/zqdZwN1ki//Cf7Upw1JJleCNSYcepBrZlNggRBmhzdgZAGvRhQr5NR3aXxxptD/UNNqFj4CD4Ca/ir+2ACy8QoXw+n6ylVHbi1qpbKU5+JMb0SlcZFymLDvR7LodJ2GufwD5jJMmZOO/OH67I4lzUv2HpMn0nLFm18g85UAzka8m4UWpKoDrMpdqVi58AptvAPEZur2fmSXH/j+FHX4c9/xKK0GFeHMNAf2jboHORIU+oGz+r30HroqH5z2Gfam5LHyGcFmwrS3biyzJ/0nUOF6xF8H/i57V/y+E51rNeD9RmbdLcKLXjULe2NhYDuUPySgUsnuvMT9bfMiOOjHi/oJr5KcA== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(36860700016)(376014)(82310400026)(1800799024)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zbR1rwOfwnMjc74a59BX6aLBCdKjEJGiaVHgpowZHfNDICwbfQ6f6MrxTlAHGAozKrVYazwbVMBi2WEJQbBEpt2biOgpWtINxyiXdh08O/YFJ/ODJD9w6Xnbw7N68jpe1p1gQW/zyGvNWFP8KywrnIncf850GuaTiVrCoTBfiq6XmoMorSqIYidjA5VesnMdopF44cYq15u4NYesgCN7VJL96nzrIbfZobyJlrJpit9kJUzLdyXWhq8JW4PxclEUzXEWRBqKTzGGibDRTUrHvCGdorOiWZ9Nu72SEOaw0Xnv3Y10Vjb2ZvQryGjlelWD144JWRpejHJA9fOAjeMHN8ftM/BZMOWcekHcT+4wLfMuqeuFCh3BVqOzG/pPyTEjq21XKaliDfikK6wT3ymX5J/6cWAqc7R6ghyatQYexPyrDpQBttcBdDCLK6WaKARS X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:12.3816 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 24b33ff1-7e10-4932-12a6-08dea1f71598 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A68.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DUYPR07MB12299 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:26 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3795 From: Daniel Turull Add --resume option that reads the most recent run's log and skips recipes that were already attempted (either upgraded or failed pre-build). This allows restarting an interrupted run without re-processing recipes that have already been handled, saving significant time on large runs (e.g. 12h for 135 recipes). The skip set is built by parsing 'Upgrading to' and 'Pre-build FAILED' lines from the previous run's upgrade-helper.log. Assisted-by: Claude, Anthropic Signed-off-by: Daniel Turull --- upgrade-helper.py | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/upgrade-helper.py b/upgrade-helper.py index 11a97d2..58fd660 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -108,6 +108,9 @@ def parse_cmdline(): parser.add_argument("--prebuild", action="store_true", default=False, help="build each recipe at its current version before attempting upgrade") + parser.add_argument("--resume", action="store_true", default=False, + help="skip recipes already attempted in the last run") + parser.add_argument("-d", "--debug-level", type=int, default=4, choices=range(1, 6), help="set the debug level: CRITICAL=1, ERROR=2, WARNING=3, INFO=4, DEBUG=5") parser.add_argument("-e", "--send-emails", action="store_true", default=False, @@ -515,9 +518,33 @@ class Updater(object): succeeded_pkggroups_ctx = [] failed_pkggroups_ctx = [] attempted_pkggroups = 0 + + # --resume: skip recipes already attempted in the most recent run + resume_skip = set() + if self.args.resume: + import glob as _glob + prev_runs = sorted(_glob.glob(os.path.join(self.uh_dir, "20*"))) + # skip the current run dir (last entry) if it exists + prev_runs = [r for r in prev_runs if r != self.uh_work_dir] + if prev_runs: + prev_log = os.path.join(prev_runs[-1], "upgrade-helper.log") + if os.path.isfile(prev_log): + with open(prev_log) as f: + for line in f: + m = re.match(r'\s+(\S+): (?:Upgrading to|Pre-build FAILED)', line) + if m: + resume_skip.add(m.group(1)) + I(" --resume: skipping %d recipes from %s" % + (len(resume_skip), prev_runs[-1])) for g in pkggroups_ctx: attempted_pkggroups += 1 pkggroup_name = g["name"] + + if any(p['PN'] in resume_skip for p in g['pkgs']): + I(" SKIP PACKAGE GROUP %d/%d (already attempted): %s" % + (attempted_pkggroups, total_pkggroups, pkggroup_name)) + continue + I(" ATTEMPT PACKAGE GROUP %d/%d" % (attempted_pkggroups, total_pkggroups)) # Build current version first to verify it builds and seed ABI reference From patchwork Fri Apr 24 11:46:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 86835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C02CFFE521F for ; Fri, 24 Apr 2026 11:46:26 +0000 (UTC) Received: from GVXPR05CU001.outbound.protection.outlook.com (GVXPR05CU001.outbound.protection.outlook.com [52.101.83.12]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.19330.1777031176623999700 for ; Fri, 24 Apr 2026 04:46:17 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=XKzqyakv; spf=pass (domain: ericsson.com, ip: 52.101.83.12, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kEyCz85+RbX2be1/ljn+m+Er4jYDxMY0PO77hYCht3PAbyS6FqLNTAQsEzB+jqhoZFZ5N0XKpc+bUgXSoFnFaykv99pnkcgnoxVP/VqLO/iF7XaH5qx1ybLYB8yUOXTYNhUULXOaZndJgFLbjO6qVFx6WjfEiQp/0h7KJwEcPfIwo0DcSYEu6vyp/NGpT1wfLQ6xSDxOxnPSUTA6hpeQMmJQJCw1Z45g568/MbHPZpORmWjyO/PmoVrWATjK/ZJZEmBATCWMfRFnVF9KWaASAZ0ziilvUM2o6cctkDIOkgmfDiwAZbjMKF4ik+7JovP+5Xw6RBcFSZaMwlUVusfU4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MIUd0tLuuUs1eiRDNKt4w7Um0JpMG7TboLW+XBO5kcY=; b=UtyS3Pz/updWbjXbAJ4Ib6bg/zznn9BZODPY1PpFLuctvpcsYGTK7gz7MiAWj9rsBjJHM885Zcxx7PH366iiTwjUfQMxpPFgXxE9tmISBpJWIq4oN1GajIuNP+usxVhiXEJVRsntiMeA1+EIk5+7jaPQsYvMcKhd/CBX7/wW1El5CFN9ogbRyw1rRvWNZM50ASZElQBvdTc9REaE1+roDW6hl8iCtQi33iEYZsUjbNoqPK3cuG5qcaN4B7Q266I+VajleeOIR31IfWeicf79PlLHAeDB+1+qfvh2glfirSCsmP4MfrwVA7V8jvqt8JeLruI3f5y21BFXBMSyUQLtaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MIUd0tLuuUs1eiRDNKt4w7Um0JpMG7TboLW+XBO5kcY=; b=XKzqyakvFc7kxJajQOOp8XepDf7hoXqlBLzGi92ge79JuFcECmceAQlVnM/iQ/RfBuNXHXZJVN1zdketI5Nx43rMcpzQN/Ku/ZZM6DVHg8/btw1l4/A7Z/v2johZxFk4T809Z8yeSCg1biS6mb+60QJSGXpnuV513qrZQDpv061l138s+JbcwZEmiYAcOHtYzpvb6VZb8pVn0HDGSblLfwsaS8ZWMjnriaBjxMptFQWr3GnL66fqtzWXCr+gvnbU3IQIk39bSGUEls3r5gWyUi3kWDoGXNQDbFgE/c/Qump0edTq47d+au3gHsc6qZpE9zHfAT2AO8L4Mfn0wvLTQw== Received: from AM0PR10CA0050.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:150::30) by AM7PR07MB6737.eurprd07.prod.outlook.com (2603:10a6:20b:1b7::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Fri, 24 Apr 2026 11:46:11 +0000 Received: from AM4PEPF00027A6B.eurprd04.prod.outlook.com (2603:10a6:20b:150:cafe::3) by AM0PR10CA0050.outlook.office365.com (2603:10a6:20b:150::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.21 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00027A6B.mail.protection.outlook.com (10.167.16.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Fri, 24 Apr 2026 11:46:11 +0000 Received: from seroius18815.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Fri, 24 Apr 2026 13:46:10 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18815.sero.gic.ericsson.se (Postfix) with ESMTP id 4FEF840215B5; Fri, 24 Apr 2026 13:46:10 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id 02863700DF4A; Fri, 24 Apr 2026 13:46:10 +0200 (CEST) From: To: CC: , , , , Daniel Turull Subject: [AUH][PATCH v2 9/9] upgrade-helper.py: Limit number of requests to remote servers Date: Fri, 24 Apr 2026 13:46:03 +0200 Message-ID: <20260424114603.2444938-10-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424114603.2444938-1-daniel.turull@ericsson.com> References: <20260424114603.2444938-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00027A6B:EE_|AM7PR07MB6737:EE_ X-MS-Office365-Filtering-Correlation-Id: 58d14696-fa05-4132-496c-08dea1f714d9 X-SMTP-Server: smtp-central.internal.ericsson.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|36860700016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: mGU25RgbS0JQIJ3UvKuWmNh2t5OXRe1pZaHrpYujtfuBOfu9fXDUrdkeb6MW+XUzaaTCiWMEDZ632QGZwI4m756E3ngXi4sg2Ck12oNfLYDo1ie5InW13wBTS5a3ulpMeA83hN2HGDIPPqnPGRM3Z+0czIX1bkFnXjJP13jsyPaLdcKjg2iEWu4/e238hItFKaIExosMGpmuYTszpqgbFDne+/8uZUKJRgm8rAw8DARIJA6IqZma90EtjzA5cRbPnbUpXdKAHDPhcuq9owG6Qz5IBRR9bTjwTM/AjQezVd919yEdgVvnmpL5vr3b5Ztq/jEaOPPv/m1MVl152Fijmdpx+NHvOgnXTtOfxf4eONbX6n3gESCvRnAvysBg6eyqBMY5tliSK9OdYF6f224DXUAsGSoETTHItn0S5xV9YTwDJN4Suzx/mLLS+O2sQFE7K2vNA7fe3W0Ieja6mWLOqn7RllYQwRziOQ6sQXW7u8fiOjYcgAcy21AfJLxSZLzM75AZQpOMvGvXOUeO7XmYJ6qlFL/kwEZe8la4r636kepoAM/YLUFs8HIMf0UJIXtz4gXwPlf0DXdvOKuZ02ghZyrBG+c4eqTrIgyO7Lagoq3NQ9n9aVZnaZ8MaI5CJifFVyFdiiCXoxqGWEq5Vo8ytIAEQo2CKeqhrwelQAXCM7RVnCWH25qyb0HNYvI3DIwaHip920uLiz5/Z7Fl1o4hherE1d3g9VmHRZddUmLQmsBgLzNuM2lRaN3Vp390JVCt44zMA51hLdCskbhcH+tCUA== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(36860700016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8FUikh5AGvFc4obU8mDFTy3c7a6r9oelVNFu4U2RQKHBMOVaZrSXhpphZuizGiyTaSkWbLjHefolVeks67UANSH3baJQNuJhQ7NIH8tSbpl6C94vORam2U9xLy73lx7itSxUkNUaLAp0EI5yibdTDdi+jNmMcn3GfMB2cmYX726aUSI7m9wsbuXVZblxWAK37I/FKahEpUVMs99BckmBkzZk2KzdA98V1QL61EgOh4yu06rbQQX3+Vq7BbyciybzrIvTBayFDSRZdaFNEm5CVyji6nnVBpyzxl25dDjwNK8ThTTAG9j1a/TbICIqItk277OQ4DzoGLU+fKeQVYKhwT3zmJ5vdODhM8/GI+30cqETmQc49jh1sfUrsGH1vAf8eQVLA/4oW/1zW1smlD0gJbObTxMLbQsgjlLrp31i9gquhCkPEmLO0wjXRGUpWZxW X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 11:46:11.1323 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 58d14696-fa05-4132-496c-08dea1f714d9 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A6B.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6737 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 24 Apr 2026 11:46:26 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto-patches/message/3794 From: Daniel Turull In systems with lots of CPUs, we overflood some remote servers and then we get timeouts because of rate control. Be more conservative. Defaults to 8 parallel workers instead of 64. Configurable via scan_workers=N in upgrade-helper.conf For example, ftp.gnu.org was timing out Assisted-by: Claude, Anthropic Signed-off-by: Daniel Turull --- upgrade-helper.py | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/upgrade-helper.py b/upgrade-helper.py index 58fd660..11b8d58 100755 --- a/upgrade-helper.py +++ b/upgrade-helper.py @@ -808,9 +808,17 @@ class UniverseUpdater(Updater): return pkg_to_upgrade upgrade_pkggroups = [] + scan_workers = int(settings.get("scan_workers", "8")) for layer_name, layer_recipes in self.recipes: - pkggroups_iter = oe.recipeutils.get_recipe_upgrade_status(layer_recipes) + # Limit parallel version checks to avoid overwhelming upstream + # servers (oe-core defaults to cpu_count which can be 64+). + _orig_cpu_count = oe.utils.cpu_count + oe.utils.cpu_count = lambda at_least=1, at_most=64: min(max(at_least, scan_workers), at_most) + try: + pkggroups_iter = oe.recipeutils.get_recipe_upgrade_status(layer_recipes) + finally: + oe.utils.cpu_count = _orig_cpu_count # Collect results, logging any fetch error that terminates # the iterator early (oe-core raises inside executor.map,