From patchwork Wed May 21 13:43:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 63457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDF75C54ED0 for ; Wed, 21 May 2025 13:44:38 +0000 (UTC) Received: from OSPPR02CU001.outbound.protection.outlook.com (OSPPR02CU001.outbound.protection.outlook.com [40.107.159.7]) by mx.groups.io with SMTP id smtpd.web11.11557.1747835070785205816 for ; Wed, 21 May 2025 06:44:31 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=YSK5hkiz; spf=pass (domain: ericsson.com, ip: 40.107.159.7, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MIyGI7Uf8NSFQxppGT2VaaptSMFIfocSK6w7WWe0Diq4bqWYDDZlnlIvXD38cKHvZJLc1eRelnxHELGcyL/fisx03Z3jgWUtQ9ruQVYOXgwDcjWWahC3PKkjZWCZ3gzsOS9fJWaqqtRA9HB9ZXeX0nAvbt4fWN7RkdOXitzgl6nVSwqKrxqPceKt+BFsMDU5VF+8xSCq9VCerrElbAoGB7oJ6vNsksrrKuaGKfMElx/dJ4sYwF9Qp9jtNZlUJGgbDLhh2kk4+W+cn1UriTuMHmkIMMBZsDj12uaYbzB4Sn9kQbIgb2pnctjIPcIid6OfpeMTDx8gOtc8YRut428E9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=12kb6Kn5H79VrvTi2boSFD4Jz0AIztSFHuCK9aRb6f4=; b=BDSpPwsYq+AxBbrXMpOYbRmHLDnUg76K8RjVfan2ZDKGlAF12RcL7eoTWAGjJuKnRGzytdtrloEG0CJ1C0XGRnfnaVOEYriLuGZhgOYdnrsAz2CsZEDHPnpW+OspdC5GmO12RstOQ/iy/WZsKVH0yP7JLXBpEWXT869usuzyPDrZAa76xgHewPNL8ljMzZ5FEO626pC4gimHMksOAF6hDIXeBQ6LDz8Rbo5mlDIgjWzyxxwOqpABKZQJaAgIu6nfs+ScExmBkPkBNdBgCqE+4u9lY8PHXMULgToC7FIblwTCYnm1CgduhlMswT35c0eoHyopw+yfmDbN0f7YhF60GQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=lists.openembedded.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=12kb6Kn5H79VrvTi2boSFD4Jz0AIztSFHuCK9aRb6f4=; b=YSK5hkizxSTTHlBUvSNb25OvocFbaPVBmMUXZakFqgtducAzPjkLAMG2xX9t3kOAgr7GX6DQOmnB9yvylwS29xf6MQjIjv+NX9j+HVFeJGY14JnebL/as2A17aFdEyw62ymPmG9ulV4ZuTG8DZh5ImiMcdvTnwxXK5Un3Wm6i2QX05zz1vfy4pERnK0r7mZcXv3ty86IRblXlWJFTe2HevlOowroyJH+01ZbZNM+ggWWPAfbKe+U5Vzw0Fa7Fx4GwRwENiDbudG1m86m7ag/XzqjIbXaVqzyH6N3xq7Gc0uJukhGbSY0IC6/HH95Y5NMl4P3/3bkuzbdtrgp78b27A== Received: from DB6PR0301CA0070.eurprd03.prod.outlook.com (2603:10a6:6:30::17) by AS8PR07MB7814.eurprd07.prod.outlook.com (2603:10a6:20b:353::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8699.26; Wed, 21 May 2025 13:44:25 +0000 Received: from DB1PEPF000509F9.eurprd02.prod.outlook.com (2603:10a6:6:30:cafe::83) by DB6PR0301CA0070.outlook.office365.com (2603:10a6:6:30::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.19 via Frontend Transport; Wed, 21 May 2025 13:44:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509F9.mail.protection.outlook.com (10.167.242.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Wed, 21 May 2025 13:44:25 +0000 Received: from seroius18813.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Wed, 21 May 2025 15:44:24 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18813.sero.gic.ericsson.se (Postfix) with ESMTP id 31DA29582F; Wed, 21 May 2025 15:44:24 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id 1E83E70B5B03; Wed, 21 May 2025 15:44:24 +0200 (CEST) From: To: CC: Daniel Turull Subject: [PATCH v5 1/3] package: change location of debugsources to PKGDESTWORK Date: Wed, 21 May 2025 15:43:58 +0200 Message-ID: <20250521134400.1733473-2-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250521134400.1733473-1-daniel.turull@ericsson.com> References: <20250521134400.1733473-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509F9:EE_|AS8PR07MB7814:EE_ X-MS-Office365-Filtering-Correlation-Id: 92f46631-ab19-433d-1c2d-08dd986d9993 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: zDOm3XET0nJgcFfBGIWxaIR2h2sdj0Ij2OrfKZowKsIQGFIBRf74AKlXQRy9zH5wWu+xDeM1tNNSbAwYvn6zDbB2oA7A89aLtqzA5wiQyUdjeYSv1SO5XaOEPjzgzERGZS9aNgRcvrsj9Qkg6LE3kBd/FWAAjk7QRXmvK1qPIC6o7SkCV78U+t+CzI8ktVcH6FHrmhjT3O+rSN+EUBbUM6aI0K5gC5W8AcHh29gQyz/wdiTg847sYvbVfffSRbLuVHcQzjVwWW94XetTLHE0zZ/AuledZBnNSc8Pen9LnPe3aQCBb1M3YyjKJS4nuMYLzfil/0PjqlRlOANscc25zHbeIKlKy5+cR4BTfdekRJYeDAyaNxhvM3zz0i3BqYoL4dsqjjJUbvW2LKdA7a0qB0NCNWrDeOSy02cTtfJeIdvfyTAcQJkuXzvC40/8xXLStJPHwWxKhK0OTGiBvuCd/nbnSmK467stlIjsTfxfx0q6WK2OVTwuPaWOYs9zFcu8LF1x/lRQzuEySIHBMMQA25GCrFCkQoDT4WIsfmxj6XN/af/is8hRojAvjnMK7HBraBDBbOL2DgoORclDCxwuSvscJCaR20ltgTqc1mTvmjtQBHlNB2RoSXv4HS4DMS7eZDiWCTcsDbleiBm0yT12cfWaGjba9G0o1t1QPFP8BrA3hcx30t16CJ+nGZHOnePYxXxMYI7GeZ56rb9qxc492mYghy4gW18FKiZtQLpBz779O0TAury1m8tkAfS0iZdoom0lYIizsOmo5b9mxdYEUoT0/5ZNVxSs4fJQP2fha5g0XvTuFqYqX11D0GsvDf9ElcIGywKguc9nvpCpMuPB5c6I+LfaFKfu/CINMnTO3DVaGNqjjMFoqolLEbYpWJdS5oOR8qPT+sDs3n41ZgvCbQbNBLoYHKjLdKl501MI+3qPrFTSrC7NZfRbokU2hFspq0B7CS+Yzau7aGO3jreiGabucrFx9OSRsRhJpi5EM5D9oeg9TUtxTJ/8XWcKNADwkiowVTkMEoN+3kDrt0GDknp+0ALRjaPwxQyYylNFx0LNWeq3qjHPIjsUo6vPhuzGenvSufWE778ygoEnz7sWRXESZ76DFwU8XyyKIGeILCVRclWrdieX+zLUmMk95sDZCOMI3a04DxLq67PpiZyLfsdfSQuAnBc0c/0XgovVLdJztK50TQymIBk5t4fI/nnmQnFGoGYIPaS9MgvgYXYpBv6oXyPJcJVfCG2pvaBCNfeH4Oh4izp1Sv8BHcc5atxd/GXB5oAyc8Rb8XE8xj3VmaKnNQ+bsOMshcea9eiPSBbY09bNzyi9YehEjjfZUVBC15A40go7OcK5JUDAGOCQpOXAWhsIqqhx06F6Fngme90znCHq1denSJKY4C9JWWGLa3CvGYCARDg2ANTQFxtgr63xG4kIk/E6X//U3g/pxM27eL1qaEa2rW1ossemoy3d407NP5IbBOzgr0sZyG5IPnnnfbDmQPoRZSVHHemacob1say9Qb9wuKs0Mfg1bueY X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2025 13:44:25.0854 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92f46631-ab19-433d-1c2d-08dd986d9993 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509F9.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7814 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 May 2025 13:44:38 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/217025 From: Daniel Turull Storing file generated doing packaging in WORKDIR, doesn't get cached in sstate and task depending on them fail to find them. Store it instead in pkgdata/debugsources. Signed-off-by: Daniel Turull --- meta/lib/oe/package.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 0bcc04ea54..6fcd001db1 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -972,7 +972,10 @@ def copydebugsources(debugsrcdir, sources, d): cpath = oe.cachedpath.CachedPath() if debugsrcdir and sources: - sourcefile = d.expand("${WORKDIR}/debugsources.list") + sourcefile = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.list") + debugdir = os.path.dirname(sourcefile) + if not os.path.isdir(debugdir): + bb.utils.mkdirhier(debugdir) bb.utils.remove(sourcefile) # filenames are null-separated - this is an artefact of the previous use From patchwork Wed May 21 13:43:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 63458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEDEFC54E71 for ; Wed, 21 May 2025 13:44:38 +0000 (UTC) Received: from EUR02-DB5-obe.outbound.protection.outlook.com (EUR02-DB5-obe.outbound.protection.outlook.com [40.107.249.64]) by mx.groups.io with SMTP id smtpd.web11.11558.1747835070934477864 for ; Wed, 21 May 2025 06:44:31 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=W7JbrFzo; spf=pass (domain: ericsson.com, ip: 40.107.249.64, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=i9v0iz7CdyKWw6NKYR//YgZ2Euk1e3Bj1c9g2KbYmcoxsqH7VHVPFffQTED8N28gV541m/DYAIE09mEAim0zviZDHkzPolLy5yCDkh2xYz9edWkVLJgGgU1P7gG/Sfzi31Fn8bktQefE+i1NjgoQ+AXd5E8zORFPALqvhafzjopDPzWGkOkRIMo8EDS4EJiP+f9fAuyHb6cRGVzciHRPFStlC8mfU6ZTIrdziqCOl4XCqywcgacy387Xxc3kzcXUP808n4VyaAbxvcXeFHkYjhVTOxPq6FvgrK4Q4+XzWdhDplyyXpjX0VfCaYYvRypRi6Rv44OxhB7mQDjmMtTAVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2CRHsESif/f0/kgSztC4khaC5Vizk6B0ZcwVLWkgPDM=; b=IM30UEdf1OSis9mmQ+R35ESTbHKivn9+rqWCj4C3lot00Lk3ZwyZ+c6EQc1T2Fsj7hyuQqSLd4yUpQxTov1O0Qim7TvXzboNKg8XZBFIvnQA0W5MBCQo7ubqut9fnEuboG1Fd/J76jiT1Yzyab0s4oay7jWLw6LwkfgSjcrDG4vrIHOVIkyxNh9wgIzLfISkFTLn2kyKsupyieQfRCDO7TBFwdXS6XREOFCZ4Pi+S/sCDkbeKDTEeRy3EOSknn6zwu4PTp/WZqgZRE7WBtCJ/5DU0uYqmB2+K6xGR5zh+2xKBZrHP8p0lWLX72Oi2rQa5j+mbcVNHpJCNtYbDU8dzQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=cherry.de smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2CRHsESif/f0/kgSztC4khaC5Vizk6B0ZcwVLWkgPDM=; b=W7JbrFzoWp47OiZDzpHfog/LWcgnfjXs0BJ4B8gkzbHBiw0vDULGEGoDOcDC3AOy1G/i3WdnXB1xAq8ujCBSf9r0fMAy0E6n3m1azefkE+wTUqUPVLY8bT4WU+pU9mOa3CEpZI8QwULwVYGyMd897kFITN9mpf2M1NEeZoU92I1BpgKOk0e3IPhlkz6Yt/fwjXf+Z9SaMOeo6JMVduTlx0K22bUO3BOdkHHC6thjjfVN12aPE5gUGAe73G/2KWtj/MMxqtSFelLyc30NVxaNfQJlV+xqVLR9ORUpcdwk4LBEHdnDIp4w5tbbDI5prxfJLFkdG2HybtKOqFBCvfEk9w== Received: from DU7P194CA0019.EURP194.PROD.OUTLOOK.COM (2603:10a6:10:553::34) by AM7PR07MB6818.eurprd07.prod.outlook.com (2603:10a6:20b:1c1::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8746.31; Wed, 21 May 2025 13:44:27 +0000 Received: from DU2PEPF00028CFF.eurprd03.prod.outlook.com (2603:10a6:10:553:cafe::9e) by DU7P194CA0019.outlook.office365.com (2603:10a6:10:553::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.20 via Frontend Transport; Wed, 21 May 2025 13:44:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF00028CFF.mail.protection.outlook.com (10.167.242.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Wed, 21 May 2025 13:44:27 +0000 Received: from seroius18814.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Wed, 21 May 2025 15:44:26 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18814.sero.gic.ericsson.se (Postfix) with ESMTP id 84ECC4020B71; Wed, 21 May 2025 15:44:26 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id 72EC470B5B03; Wed, 21 May 2025 15:44:26 +0200 (CEST) From: To: CC: Daniel Turull , Quentin Schulz , Joshua Watt , Peter Marko Subject: [PATCH v5 2/3] spdx: add option to include only compiled sources Date: Wed, 21 May 2025 15:43:59 +0200 Message-ID: <20250521134400.1733473-3-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250521134400.1733473-1-daniel.turull@ericsson.com> References: <20250521134400.1733473-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028CFF:EE_|AM7PR07MB6818:EE_ X-MS-Office365-Filtering-Correlation-Id: 0dab7722-34b8-4d23-5a0d-08dd986d9ae7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|1800799024|36860700013|7053199007; X-Microsoft-Antispam-Message-Info: VIDDLPp/13j9vSvLXTiIXYECRHE0LB3gHJTpBM+T+ohOBgoOfT9CyU2Vbj93Q+DwOcUbumVN7z6i91ZJpFI8UL/QnJaikuxHA5E88C18wet2/ao1ieB3c+UpnzgtfLzOuziHlPwjaBj+s6bSalsVYZhI6L1quq498VTxlmeY7qo46960QpAKct6GEobFpDKK/jEUfWhPIw013R+yL0NGJ8jFOJ5siXrxKBWc1AxypGX8pm2Um1UIO7gUESzM81DIG7CLaNi8Es5YLf/enzmei7icEsGD2m/JAZs+5KIvs2NQJbwjcQbyrR8fbI6DREr9skr+XB0zK+wbiYm3wNutOHOh86TLAv7ceeEaJXOc3ex2R3ftcCPtJgitvWjOhvqhSh+eeEgWQCqlQiZwfIKWbu6rr1zQ5IYn6WuEZp3859h7ML0I88Ca9fJtQ+3bVH4Bp8Uj4AGl3EifmVKMjLQP0TgmcbGXRde+Mkl31RLDtrUA455ui+1iSlGilwHH/uM4QulfNzdao+l4tGtaLgI1aKSCOD1DFKcgkv4YvrjlZrr5qACfzQ457CHW8c/7Tul++irOabUzsqX9a5vpxFKCbqYk/cI6DIxtP4q8Zm2H+v86Wb1lj9qTNMe1buAQmrpF5H5aK4hOjYJsqpoVBMLAsp2Khk9WPExC5/3Cs41zqVsfDGug8jgqqr7sHrQB6lF3IHTDguUAnTZ+hWOiT3pUrTqh4J0f6f4YlLoJkeKzFZ+xCu+ZHz3zJnm7TNtbjcm65nB3FINR2zhmVmR1Qw/J+FlZxzEif2L1E2syf7QZmre5AQo0oqpzCBL3nR1ONJjl2ABDZgn1X9834Q5Al2Vzn7U04LN7NdmSf/jFNxWqgesdrW6Yz+gNIO3oZzsIuv5k2j3kRnPvKf/7bofEoNzlt99B0rvcLsyIvvM6VIiwGekUHn/3pio4YGEuZhPHztyoTFa1x9R0/ROxV2sdCaryUVMj9wprMSwEm5Dki2HtewzKU0fDxJeEFh0kdoKmdqKUSCepF9fkl3o9ghyHdNhjjwpeI6BjpA9ZoR/h64lkp7l14D20WIO+H5uCEtQH6bmtxM4+MDkeQfJ6Zkv6Us7o+9YzG8cUu7brgpaGMkeB7CgEYkYVkHXTL3WRVNyfsG09c93qh74UD3ez1Ts2eCMhOM4nQMWnUfDzgTeiZqhdX4YuaFGS3B7oTi5LzDnj5/z7lFaX1plvSX/Jdj5XwjKfsMlvye9kvfjo5eTWfUv/AI/JwmoQW/ewe9QBmBofuvq/H+Ppwqmk7bXYwVYGxuJToE9OaF7X8dxrDELdYI9s3/dTN/ONkhTsy7ogx6xbDYYm/AYtGuA1E6TflWXf5nmcocvitBzx+rKLmIAqx/K8ik5IhDgnfQEX5+BmHaJ9hiMQZolhE3TjLT+aENLhijSXkko1KU4nbBq3/V1CBhIMMrIsRtR5XzpDJwXBwpZQAodFzoJqBS9hbUw4YgwOxyYuxA== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013)(7053199007);DIR:OUT;SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2025 13:44:27.3322 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0dab7722-34b8-4d23-5a0d-08dd986d9ae7 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028CFF.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6818 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 May 2025 13:44:38 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/217026 From: Daniel Turull When SPDX_INCLUDE_COMPILED_SOURCES is enabled, only include the source code files that are used during compilation. It uses debugsource information generated during do_package. This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled. As example, when used with the default config with linux-yocto, the spdx size is reduced from 156MB to 61MB. CC: Quentin Schulz CC: Joshua Watt CC: Peter Marko Signed-off-by: Daniel Turull --- meta/classes/create-spdx-2.2.bbclass | 9 +++++ meta/classes/spdx-common.bbclass | 3 ++ meta/lib/oe/spdx30_tasks.py | 10 ++++++ meta/lib/oe/spdx_common.py | 49 ++++++++++++++++++++++++++++ 4 files changed, 71 insertions(+) diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 7e8f8b9ff5..6fc60a1d97 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -137,6 +137,11 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv spdx_files = [] file_counter = 1 + + check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1" + if check_compiled_sources: + compiled_sources, types = oe.spdx_common.get_compiled_sources(d) + bb.debug(1, f"Total compiled files: {len(compiled_sources)}") for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir): @@ -147,6 +152,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # Check if file is compiled + if check_compiled_sources: + if not oe.spdx_common.is_compiled_source(filename, compiled_sources, types): + continue spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath): diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index 713a7fc651..ca0416d1c7 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -26,6 +26,7 @@ SPDX_TOOL_VERSION ??= "1.0" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" SPDX_INCLUDE_SOURCES ??= "0" +SPDX_INCLUDE_COMPILED_SOURCES ??= "0" SPDX_UUID_NAMESPACE ??= "sbom.openembedded.org" SPDX_NAMESPACE_PREFIX ??= "http://spdx.org/spdxdocs" @@ -40,6 +41,8 @@ SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" python () { from oe.cve_check import extend_cve_status extend_cve_status(d) + if d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1": + d.setVar("SPDX_INCLUDE_SOURCES", "1") } def create_spdx_source_deps(d): diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 61d7ba45e3..beeafc2bb7 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -156,6 +156,11 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1" + if check_compiled_sources: + compiled_sources, types = oe.spdx_common.get_compiled_sources(d) + bb.debug(1, f"Total compiled files: {len(compiled_sources)}") + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir): @@ -171,6 +176,11 @@ def add_package_files( filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) + # Check if file is compiled + if check_compiled_sources: + if not oe.spdx_common.is_compiled_source(filename, compiled_sources, types): + continue + spdx_file = objset.new_file( get_spdxid(file_counter), filename, diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 4caefc7673..e4959fb755 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -242,3 +242,52 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_compiled_source (filename, compiled_sources, types): + """ + Check if the file, is a compiled file + """ + import os + # If we don't have compiled source, we assume all are compiled. + if len(compiled_sources) == 0: + return True + # We remove the top directory, to match the format in compiled sources + relative = filename[filename.find("/")+1:] + basename = os.path.basename(filename) + # We return always true if the file type is not in the list of compiled files + if basename[basename.find("."):] not in types: + return True + # Check that the file is in the list + return relative in compiled_sources + +def get_compiled_sources(d): + """ + Get list of compiled sources from debug information and normalize the paths + """ + sourcefile = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.list") + pn = d.getVar('PN') + pv = d.getVar('PV') + + if not os.path.isfile(sourcefile): + bb.debug(1, "Do not have debugsources.list. Skipping") + return [], [] + with open(sourcefile, 'r') as sf: + # We need to normalize the path to match the one in the package + # kernel is special case that doesn't match pn + # filenames are null-separated - this is an artefact of the previous use + # of rpm's debugedit + sources = sf.readline()\ + .replace(f"/usr/src/debug/{pn}/","")\ + .replace(f"/usr/src/kernel/","")\ + .replace(f"/usr/src/{pn}/","")\ + .replace(f"{pv}/","")\ + .split('\0') + # Check extensions of files + types = [] + for src in sources: + basename = os.path.basename(src) + ext = basename[basename.find("."):] + if ext not in types and len(ext)>0: + types.append(ext) + bb.debug(1, f"Num of sources: {len(sources)} and types: {len(types)} {str(types)}") + return sources, types From patchwork Wed May 21 13:44:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Turull X-Patchwork-Id: 63459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6EEDC54ED1 for ; Wed, 21 May 2025 13:44:38 +0000 (UTC) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (EUR05-DB8-obe.outbound.protection.outlook.com [40.107.20.86]) by mx.groups.io with SMTP id smtpd.web10.11346.1747835073841297015 for ; Wed, 21 May 2025 06:44:34 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ericsson.com header.s=selector1 header.b=IkVEJm8H; spf=pass (domain: ericsson.com, ip: 40.107.20.86, mailfrom: edaturu@ericsson.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QO5AadS2zWy4gNX+PAw/Ah58r0TSacUATzFOCMvtjWVFRs/U8S/sHmACd5AyQq2pWLIXttiF724VS03fr+mg58b3cbMx2uhjCbRdOl3Tq/Z0ABsEglqIh5uOVuuw6vcoams5j8OWAfgKXChroC0JkXCWpN94eDJtaoQzeFDS357//SdDXkDn+RL2J3AJyKu6QTOY4ajV7vvBGm1jnlh+OVeN+RdmTbKU/z0n7wp4fdZv40pfnUjyBFtAULwFzyWf5ZHDbFbRzd+NRwS7XvWGFqvw6qO3Rzt3t+GQNBz84bxqo1qxxUXgc1g5sjUzZSLSDF1TdDEYoePCjuDieWyeDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=G5X9QZc11nQWAqvLmsG0r1jxwdzW7MbuUQhXgBH1/S4=; b=HG0so6FB4+Zx+uJxE7Ct34N3wUq2+WE0IgUW4wEXrmRY0sB0Tm3oze30w5tWHXYO7O7rWwFtE3nMLjeL+FsUT3SSV6DysJAGIaxrWTfrom7Azwmb1XB2fvxzcX1ifDW/8DCMYaWC078xCKL6mBwX801xKechisvUyON2N3P0m3f7GFnXdWwNMhgjkWHY4H6SCfmtNEMe96RdeGOW3THHyexlBU9EDb2YgZOR8vITMtOy/aGtHHXynORwY/xiDrL6H6KR1yHOGO51jIRnRvU0BcjABEZ8EvCjiByFMlN00uMq4MFZH6mbN1reCbzH0WSImxzylpkWWr9lIjGOWvwwoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=gmail.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G5X9QZc11nQWAqvLmsG0r1jxwdzW7MbuUQhXgBH1/S4=; b=IkVEJm8HUefcbViUaIVhhSzbvJ1GdSlaleaCMZZCtIESUmp7NzyXkRJpxy1N+z4QvDP/kqBB46pFNE9eFlaGL3Wle+SJtKE33xZ0c6i/fwwXF4gJrow/q+igkJL1Qh/+t1yFN9+3jhuypWBuCdzOrQdbMqFA+ssX7ve6ouJlrKlv3/KMyk95h7VHwEH1Ry6kwOOLdSbWGxYV+1Bg7YX9PLBDDovqzHi1NNxJlSTTQgsSR8qobQzk4dqhlSwxPVmXTizhc1/r0NqaWVti7T2uFh/fXs1/4nqDzvOmyZi7NnEEiF7mQv3vgg+2R0U6JBvUD0M5sR9A/vcrluGfEok5zg== Received: from DU7PR01CA0015.eurprd01.prod.exchangelabs.com (2603:10a6:10:50f::13) by VI1PR0701MB6782.eurprd07.prod.outlook.com (2603:10a6:800:17e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8746.31; Wed, 21 May 2025 13:44:29 +0000 Received: from DU2PEPF00028CFD.eurprd03.prod.outlook.com (2603:10a6:10:50f:cafe::62) by DU7PR01CA0015.outlook.office365.com (2603:10a6:10:50f::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Wed, 21 May 2025 13:44:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF00028CFD.mail.protection.outlook.com (10.167.242.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.18 via Frontend Transport; Wed, 21 May 2025 13:44:29 +0000 Received: from seroius18814.sero.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Wed, 21 May 2025 15:44:28 +0200 Received: from seroius08462.sero.gic.ericsson.se (seroius08462.sero.gic.ericsson.se [10.63.237.245]) by seroius18814.sero.gic.ericsson.se (Postfix) with ESMTP id A1D454020B71; Wed, 21 May 2025 15:44:28 +0200 (CEST) Received: by seroius08462.sero.gic.ericsson.se (Postfix, from userid 160155) id 8B48370B5B03; Wed, 21 May 2025 15:44:28 +0200 (CEST) From: To: CC: Daniel Turull , Peter Marko , Marta Rybczynska Subject: [PATCH v5 3/3] improve_kernel_cve_report: add script for postprocesing of kernel CVE data Date: Wed, 21 May 2025 15:44:00 +0200 Message-ID: <20250521134400.1733473-4-daniel.turull@ericsson.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250521134400.1733473-1-daniel.turull@ericsson.com> References: <20250521134400.1733473-1-daniel.turull@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028CFD:EE_|VI1PR0701MB6782:EE_ X-MS-Office365-Filtering-Correlation-Id: 39e73b08-d883-4c8b-c57a-08dd986d9c38 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|1800799024|36860700013|13003099007; X-Microsoft-Antispam-Message-Info: W1ypftYqxG5zs4/myaGx3o00lx9MAF34l8HQJO2bJUIV861+ul5b5+Ey7ZWN5z6sXWrujqJKkPd1ij+oUacs4lAbb17KDN3P2NW7StcKYU9f/KBxF/egzqtn7ob/5f8CDOOqhdWVbBPnP4NPkmVgAUzi0v2MFo6DOtD/3IBKAee6gvsFH8V4OjOA0zDcegLEGozkfAkOfdBWG1OxJSrg6A7hfOyB3loahucB/RO6DJf3Sti5eGfrga63pgnaUllq/Q0BrNJ8a8fVbtP2NwCzfD3qmT4bjIEiXhknqBlMz50wCjpbzYRmdk+u0fqwSptxMz+8Lk56lS3sXDWmtyPbTq/SDW413cHI1ChAOQzqvLTC/jfCPRV0oDQLjFEJR5kKdTXjNSb0jkeXaAk6DWucDNx69N4xvFXb1d9wUs0OxClRbxNg2VyKV4uqRVSf/dXyJLT311Gv9estVmNj12qOuJPU22o9uuz4S8o7jjUy5udroaPb1/QEPFs2GqbxO85uherWvRihdxL/DnfuVKOv7prMIUWMespLWlWVOyxoCt02nzPlAuS0kH29a3F56sRMw8HlkKueWdw3Pu6bKYxAjRplZP/AZ52jwvIjaVZoMqmKuF2CszguVPVx32c0NPAntPGqQD8Nrsf+rqgyUzTCKk/ufVZlnA4a/kvMkaWDWwOOttXDQfO6vIXok6iv0PZ5eFCEZ85tOOtTVZB3zqazQ7KjkMElKblSy60Uh4rBqxVKIZnf4fsNk1LpFgmY2KILeLixm+bovT+IrTEIkZJZm3FW6h/16Y60xVsRFXBk6hzMOeHy0pMUzL5co+gYe0t7VmA98rU/S2M0oKRQ2sUx9g+fV/ryXFlwW6mQIDRDKy6In24ejm3CGVbM+WhPHOyiL3taPG8v9d9zA3eXfgBBO1KhCqzJHKGcwFX48T69d79VKgC0TGpiL4ydbqZgMXZoY9EerNykM6n+3HTziRT40BC1FoabfvCoYV+/WU1ntJeJDJwHrZYm7pO2MmuafR6X5Tl5fSuTw/xziZqlg3zM6nLeG+SiBtGgiD3e2Hd8mnZ3cYtVsBbVj3XnpH6uCnGNNzINrSNw7pf7/f666+F3k+DxADmi1uz/eKRbvBN019KImBYDyeYTsI3BXqq3etMz8qO0jLm+hXF54jHsOZd5KGRAOBLD/qMqdUJRaYAHWK5GKDGBFr/W3NzvalDZJEvQ1CQqJ4iFCKINSwGSACiGdtHY98QCp7FEVzzPoVw+WCHxQwZNmdEtnfh6JiZIGAjQ97O3AXgFnugV75zQL1x+RUiLwSZpEEletnVhmp7qKNfBl6ZK5t78668EvvYah2Mt2YMe6MpXkQAsNefOcUZWxCVnP79WAkV7RTdPPe5YpCSRUwGvHG9Zv2QW40V0JS6HJtytOAK7ceYP+6gL7WhPeAsWLoA+2rwm3LyhAZJi8KgOMCoW1OSbYOGwvOtsyve8s5UZYm9oBRWxahZx8ZjcXQ== X-Forefront-Antispam-Report: CIP:192.176.1.74;CTRY:SE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:oa.msg.ericsson.com;PTR:office365.se.ericsson.net;CAT:NONE;SFS:(13230040)(82310400026)(376014)(1800799024)(36860700013)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2025 13:44:29.5131 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 39e73b08-d883-4c8b-c57a-08dd986d9c38 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f;Ip=[192.176.1.74];Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028CFD.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0701MB6782 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 21 May 2025 13:44:38 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/217027 From: Daniel Turull Adding postprocessing script to process data from linux CNA that includes more accurate metadata and it is updated directly by the source. Example of enhanced CVE from a report from cve-check: { "id": "CVE-2024-26710", "status": "Ignored", "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710", "summary": "In the Linux kernel, the following vulnerability [...]", "scorev2": "0.0", "scorev3": "5.5", "scorev4": "0.0", "modified": "2025-03-17T15:36:11.620", "vector": "LOCAL", "vectorString": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H", "detail": "not-applicable-config", "description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']" }, And same from a report generated with vex: { "id": "CVE-2024-26710", "status": "Ignored", "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710", "detail": "not-applicable-config", "description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']" }, For unpatched CVEs, provide more context in the description: Tested with 6.12.22 kernel { "id": "CVE-2025-39728", "status": "Unpatched", "link": "https://nvd.nist.gov/vuln/detail/CVE-2025-39728", "summary": "In the Linux kernel, the following vulnerability has been [...], "scorev2": "0.0", "scorev3": "0.0", "scorev4": "0.0", "modified": "2025-04-21T14:23:45.950", "vector": "UNKNOWN", "vectorString": "UNKNOWN", "detail": "version-in-range", "description": "Needs backporting (fixed from 6.12.23)" }, CC: Peter Marko CC: Marta Rybczynska Signed-off-by: Daniel Turull --- scripts/contrib/improve_kernel_cve_report.py | 467 +++++++++++++++++++ 1 file changed, 467 insertions(+) create mode 100755 scripts/contrib/improve_kernel_cve_report.py diff --git a/scripts/contrib/improve_kernel_cve_report.py b/scripts/contrib/improve_kernel_cve_report.py new file mode 100755 index 0000000000..d729f8ed64 --- /dev/null +++ b/scripts/contrib/improve_kernel_cve_report.py @@ -0,0 +1,467 @@ +#! /usr/bin/env python3 +# +# Copyright OpenEmbedded Contributors +# +# The script uses another source of CVE information from linux-vulns +# to enrich the cve-summary from cve-check or vex. +# It can also use the list of compiled files from the kernel spdx to ignore CVEs +# that are not affected since the files are not compiled. +# +# It creates a new json file with updated CVE information +# +# Compiled files can be extracted adding the following in local.conf +# SPDX_INCLUDE_COMPILED_SOURCES:pn-linux-yocto = "1" +# +# Tested with the following CVE sources: +# - https://git.kernel.org/pub/scm/linux/security/vulns.git +# - https://github.com/CVEProject/cvelistV5 +# +# Example: +# python3 ./openembedded-core/scripts/contrib/improve_kernel_cve_report.py --spdx tmp/deploy/spdx/3.0.1/qemux86_64/recipes/recipe-linux-yocto.spdx.json --kernel-version 6.12.27 --datadir ./vulns +# python3 ./openembedded-core/scripts/contrib/improve_kernel_cve_report.py --spdx tmp/deploy/spdx/3.0.1/qemux86_64/recipes/recipe-linux-yocto.spdx.json --datadir ./vulns --old-cve-report build/tmp/log/cve/cve-summary.json +# +# SPDX-License-Identifier: GPLv2 + +import argparse +import json +import sys +import logging +import glob +import os +import pathlib +from packaging.version import Version + +def is_linux_cve(cve_info): + '''Return true is the CVE belongs to Linux''' + if not "affected" in cve_info["containers"]["cna"]: + return False + for affected in cve_info["containers"]["cna"]["affected"]: + if not "product" in affected: + return False + if affected["product"] == "Linux" and affected["vendor"] == "Linux": + return True + return False + +def get_kernel_cves(datadir, compiled_files, version): + """ + Get CVEs for the kernel + """ + cves = {} + + check_config = len(compiled_files) > 0 + + base_version = Version(f"{version.major}.{version.minor}") + + # Check all CVES from kernel vulns + pattern = os.path.join(datadir, '**', "CVE-*.json") + cve_files = glob.glob(pattern, recursive=True) + not_applicable_config = 0 + fixed_as_later_backport = 0 + vulnerable = 0 + not_vulnerable = 0 + for cve_file in sorted(cve_files): + cve_info = {} + with open(cve_file, "r", encoding='ISO-8859-1') as f: + cve_info = json.load(f) + + if len(cve_info) == 0: + logging.error("Not valid data in %s. Aborting", cve_file) + break + + if not is_linux_cve(cve_info): + continue + cve_id = os.path.basename(cve_file)[:-5] + description = cve_info["containers"]["cna"]["descriptions"][0]["value"] + if cve_file.find("rejected") >= 0: + logging.debug("%s is rejected by the CNA", cve_id) + cves[cve_id] = { + "id": cve_id, + "status": "Ignored", + "detail": "rejected", + "summary": description, + "description": f"Rejected by CNA" + } + continue + if any(elem in cve_file for elem in ["review", "reverved", "testing"]): + continue + + is_vulnerable, first_affected, last_affected, better_match_first, better_match_last, affected_versions = get_cpe_applicability(cve_info, version) + + logging.debug("%s: %s (%s - %s) (%s - %s)", cve_id, is_vulnerable, better_match_first, better_match_last, first_affected, last_affected) + + if is_vulnerable is None: + logging.warning("%s doesn't have good metadata", cve_id) + if is_vulnerable: + is_affected = True + affected_files = [] + if check_config: + is_affected, affected_files = check_kernel_compiled_files(compiled_files, cve_info) + + if not is_affected and len(affected_files) > 0: + logging.debug( + "%s - not applicable configuration since affected files not compiled: %s", + cve_id, affected_files) + cves[cve_id] = { + "id": cve_id, + "status": "Ignored", + "detail": "not-applicable-config", + "summary": description, + "description": f"Source code not compiled by config. {affected_files}" + } + not_applicable_config +=1 + # Check if we have backport + else: + if not better_match_last: + fixed_in = last_affected + else: + fixed_in = better_match_last + logging.debug("%s needs backporting (fixed from %s)", cve_id, fixed_in) + cves[cve_id] = { + "id": cve_id, + "status": "Unpatched", + "detail": "version-in-range", + "summary": description, + "description": f"Needs backporting (fixed from {fixed_in})" + } + vulnerable += 1 + if (better_match_last and + Version(f"{better_match_last.major}.{better_match_last.minor}") == base_version): + fixed_as_later_backport += 1 + # Not vulnerable + else: + if not first_affected: + logging.debug("%s - not known affected %s", + cve_id, + better_match_last) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "version-not-in-range", + "summary": description, + "description": "No CPE match" + } + not_vulnerable += 1 + continue + backport_base = Version(f"{better_match_last.major}.{better_match_last.minor}") + if version < first_affected: + logging.debug('%s - fixed-version: only affects %s onwards', + cve_id, + first_affected) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "fixed-version", + "summary": description, + "description": f"only affects {first_affected} onwards" + } + not_vulnerable += 1 + elif last_affected <= version: + logging.debug("%s - fixed-version: Fixed from version %s", + cve_id, + last_affected) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "fixed-version", + "summary": description, + "description": f"fixed-version: Fixed from version {last_affected}" + } + not_vulnerable += 1 + elif backport_base == base_version: + logging.debug("%s - cpe-stable-backport: Backported in %s", + cve_id, + better_match_last) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "cpe-stable-backport", + "summary": description, + "description": f"Backported in {better_match_last}" + } + not_vulnerable += 1 + else: + logging.debug("%s - version not affected %s", cve_id, str(affected_versions)) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "version-not-in-range", + "summary": description, + "description": f"Range {affected_versions}" + } + not_vulnerable += 1 + + logging.info("Total CVEs ignored due to not applicable config: %d", not_applicable_config) + logging.info("Total CVEs not vulnerable due version-not-in-range: %d", not_vulnerable) + logging.info("Total vulnerable CVEs: %d", vulnerable) + + logging.info("Total CVEs already backported in %s: %s", base_version, + fixed_as_later_backport) + return cves + +def read_spdx(spdx_file): + '''Open SPDX file and extract compiled files''' + with open(spdx_file, 'r', encoding='ISO-8859-1') as f: + spdx = json.load(f) + if "spdxVersion" in spdx: + if spdx["spdxVersion"] == "SPDX-2.2": + return read_spdx2(spdx) + if "@graph" in spdx: + return read_spdx3(spdx) + return [] + +def read_spdx2(spdx): + ''' + Read spdx2 compiled files from spdx + ''' + cfiles = [] + if 'files' not in spdx: + return cfiles + for item in spdx['files']: + for ftype in item['fileTypes']: + if ftype == "SOURCE": + filename = item["fileName"][item["fileName"].find("/")+1:] + cfiles.append(filename) + return cfiles + +def read_spdx3(spdx): + ''' + Read spdx3 compiled files from spdx + ''' + cfiles = [] + for item in spdx["@graph"]: + if "software_primaryPurpose" not in item: + continue + if item["software_primaryPurpose"] == "source": + filename = item['name'][item['name'].find("/")+1:] + cfiles.append(filename) + return cfiles + +def check_kernel_compiled_files(compiled_files, cve_info): + """ + Return if a CVE affected us depending on compiled files + """ + files_affected = [] + is_affected = False + + for item in cve_info['containers']['cna']['affected']: + if "programFiles" in item: + for f in item['programFiles']: + if f not in files_affected: + files_affected.append(f) + + if len(files_affected) > 0: + for f in files_affected: + if f in compiled_files: + logging.debug("File match: %s", f) + is_affected = True + return is_affected, files_affected + +def get_cpe_applicability(cve_info, v): + ''' + Check if version is affected and return affected versions + ''' + base_branch = Version(f"{v.major}.{v.minor}") + affected = [] + if not 'cpeApplicability' in cve_info["containers"]["cna"]: + return None, None, None, None, None, None + + for nodes in cve_info["containers"]["cna"]["cpeApplicability"]: + for node in nodes.values(): + vulnerable = False + matched_branch = False + first_affected = Version("5000") + last_affected = Version("0") + better_match_first = Version("0") + better_match_last = Version("5000") + + if len(node[0]['cpeMatch']) == 0: + first_affected = None + last_affected = None + better_match_first = None + better_match_last = None + + for cpe_match in node[0]['cpeMatch']: + version_start_including = Version("0") + version_end_excluding = Version("0") + if 'versionStartIncluding' in cpe_match: + version_start_including = Version(cpe_match['versionStartIncluding']) + else: + version_start_including = Version("0") + # if versionEndExcluding is missing we are in a branch, which is not fixed. + if "versionEndExcluding" in cpe_match: + version_end_excluding = Version(cpe_match["versionEndExcluding"]) + else: + # if versionEndExcluding is missing we are in a branch, which is not fixed. + version_end_excluding = Version( + f"{version_start_including.major}.{version_start_including.minor}.5000" + ) + affected.append(f" {version_start_including}-{version_end_excluding}") + # Detect if versionEnd is in fixed in base branch. It has precedence over the rest + branch_end = Version(f"{version_end_excluding.major}.{version_end_excluding.minor}") + if branch_end == base_branch: + if version_start_including <= v < version_end_excluding: + vulnerable = cpe_match['vulnerable'] + # If we don't match in our branch, we are not vulnerable, + # since we have a backport + matched_branch = True + better_match_first = version_start_including + better_match_last = version_end_excluding + if version_start_including <= v < version_end_excluding and not matched_branch: + if version_end_excluding < better_match_last: + better_match_first = max(version_start_including, better_match_first) + better_match_last = min(better_match_last, version_end_excluding) + vulnerable = cpe_match['vulnerable'] + matched_branch = True + + first_affected = min(version_start_including, first_affected) + last_affected = max(version_end_excluding, last_affected) + # Not a better match, we use the first and last affected instead of the fake .5000 + if vulnerable and better_match_last == Version(f"{base_branch}.5000"): + better_match_last = last_affected + better_match_first = first_affected + return vulnerable, first_affected, last_affected, better_match_first, better_match_last, affected + +def copy_data(old, new): + '''Update dictionary with new entries, while keeping the old ones''' + for k in new.keys(): + old[k] = new[k] + return old + +# Function taken from cve_check.bbclass. Adapted to cve fields +def cve_update(cve_data, cve, entry): + # If no entry, just add it + if cve not in cve_data: + cve_data[cve] = entry + return + # If we are updating, there might be change in the status + if cve_data[cve]['status'] == "Unknown": + cve_data[cve] = copy_data(cve_data[cve], entry) + return + if cve_data[cve]['status'] == entry['status']: + return + if entry['status'] == "Unpatched" and cve_data[cve]['status'] == "Patched": + logging.warning("CVE entry %s update from Patched to Unpatched from the scan result", cve) + cve_data[cve] = copy_data(cve_data[cve], entry) + return + if entry['status'] == "Patched" and cve_data[cve]['status'] == "Unpatched": + logging.warning("CVE entry %s update from Unpatched to Patched from the scan result", cve) + cve_data[cve] = copy_data(cve_data[cve], entry) + return + # If we have an "Ignored", it has a priority + if cve_data[cve]['status'] == "Ignored": + logging.debug("CVE %s not updating because Ignored", cve) + return + # If we have an "Ignored", it has a priority + if entry['status'] == "Ignored": + cve_data[cve] = copy_data(cve_data[cve], entry) + logging.debug("CVE entry %s updated from Unpatched to Ignored", cve) + return + logging.warning("Unhandled CVE entry update for %s %s from %s %s to %s", + cve, cve_data[cve]['status'], cve_data[cve]['detail'], entry['status'], entry['detail']) + +def main(): + parser = argparse.ArgumentParser( + description="Update cve-summary with kernel compiled files and kernel CVE information" + ) + parser.add_argument( + "-s", + "--spdx", + help="SPDX2/3 for the kernel. Needs to include compiled sources", + ) + parser.add_argument( + "--datadir", + type=pathlib.Path, + help="Directory where CVE data is", + required=True + ) + parser.add_argument( + "--old-cve-report", + help="CVE report to update. (Optional)", + ) + parser.add_argument( + "--kernel-version", + help="Kernel version. Needed if old cve_report is not provided (Optional)", + type=Version + ) + parser.add_argument( + "--new-cve-report", + help="Output file", + default="cve-summary-enhance.json" + ) + parser.add_argument( + "-D", + "--debug", + help='Enable debug ', + action="store_true") + + args = parser.parse_args() + + if args.debug: + log_level=logging.DEBUG + else: + log_level=logging.INFO + logging.basicConfig(format='[%(filename)s:%(lineno)d] %(message)s', level=log_level) + + if not args.kernel_version and not args.old_cve_report: + parser.error("either --kernel-version or --old-cve-report are needed") + return -1 + + # by default we don't check the compiled files, unless provided + compiled_files = [] + if args.spdx: + compiled_files = read_spdx(args.spdx) + logging.info("Total compiled files %d", len(compiled_files)) + + if args.old_cve_report: + with open(args.old_cve_report, encoding='ISO-8859-1') as f: + cve_report = json.load(f) + else: + #If summary not provided, we create one + cve_report = { + "version": "1", + "package": [ + { + "name": "linux-yocto", + "version": str(args.kernel_version), + "products": [ + { + "product": "linux_kernel", + "cvesInRecord": "Yes" + } + ], + "issue": [] + } + ] + } + + for pkg in cve_report['package']: + is_kernel = False + for product in pkg['products']: + if product['product'] == "linux_kernel": + is_kernel=True + if not is_kernel: + continue + + kernel_cves = get_kernel_cves(args.datadir, + compiled_files, + Version(pkg["version"])) + logging.info("Total kernel cves from kernel CNA: %s", len(kernel_cves)) + cves = {issue["id"]: issue for issue in pkg["issue"]} + logging.info("Total kernel before processing cves: %s", len(cves)) + + for cve in kernel_cves: + cve_update(cves, cve, kernel_cves[cve]) + + pkg["issue"] = [] + for cve in sorted(cves): + pkg["issue"].extend([cves[cve]]) + logging.info("Total kernel cves after processing: %s", len(pkg['issue'])) + + with open(args.new_cve_report, "w", encoding='ISO-8859-1') as f: + json.dump(cve_report, f, indent=2) + + return 0 + +if __name__ == "__main__": + sys.exit(main()) +