From patchwork Tue Jul 21 04:31:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Deepesh Varatharajan X-Patchwork-Id: 93010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 912F9C44529 for ; Tue, 21 Jul 2026 04:31:48 +0000 (UTC) Received: from mx0a-0064b401.pphosted.com (mx0a-0064b401.pphosted.com [205.220.166.238]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.15637.1784608306694021345 for ; Mon, 20 Jul 2026 21:31:46 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@windriver.com header.s=PPS06212021 header.b=IIie4EKt; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.166.238, mailfrom: prvs=1662381085=deepesh.varatharajan@windriver.com) Received: from pps.filterd (m0250809.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 66L46B1r1635816 for ; Mon, 20 Jul 2026 21:31:46 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=cc:content-transfer-encoding:content-type:date:from :in-reply-to:message-id:mime-version:references:subject:to; s= PPS06212021; bh=Q1PQchsry7TqxfKAw092qGwn6B/m4k513lmPNJJrN8Q=; b= IIie4EKt3bvO3l6kFKTGVRvdNtTmnIeBory/p4eKtAvG7WzfKkGqzNf9R+LGUPxQ 0CTwMrTLK1DpmhdNTnupF3Xpkf+gkCC1onOvvGjvInSsbcKxyMUasMXYbQxYPPWI qgILCiNjAB1swocpOSthZvy3rm5+PugSw2ve2OSFlrx8/2WucCUfr5xtMYyyg3mi dHKckDSUC3DtLIz09TDAwjAftZOA0sAwYpVQJVJKXhY83Y+mvbIUzU8Qz6jIDvYr jGpTGr1Imwy/mI4JL0ABT9WdaU6CS3ViXWBkbaSJAF8DEZVO/oItzSCTI+VlEnJG Fv90N4n6BtVwIVLfBJDvgQ== Received: from ch5pr02cu005.outbound.protection.outlook.com (mail-northcentralusazon11012015.outbound.protection.outlook.com [40.107.200.15]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 4fg90cutfp-1 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Mon, 20 Jul 2026 21:31:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IgRed0xSTfaKzoWu+g4uhNOMnGvSH2q7aPagzeLUKicZCNALSQIJTD7ib8eyPLW45jB4b2/UqF0xrqPB/2zdaFbnxmi8L4YdLBX6jYJ3nbIGox7FPcLJG0WYi5+EgF2MfQJ0XKH8jzNkiazCXd6Z55XhkDDc5AcwEtO8UWkyrkBuqTEVk4QBuGUJXNmTZJcNdAO7yD77bt0MacBjTc7D3aUbaii0i4p97oV9916ZdbqQjB/FfznNBCbRoMPDQ9A9mN+H+g1sLNQvgV57S27LEQkRLYJlZ8xU7zmpEN5K1q8ZoKthTPBReQEgYy7NWlZcsQcL/i8Ruho3+O2lrV2trQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q1PQchsry7TqxfKAw092qGwn6B/m4k513lmPNJJrN8Q=; b=L2Gk8phchp7qzdpXdqI1oQk3h/W8OJVHjotwboZutdTHNjWwlMUJsksaFkAfZpqXIRKwe1iluPfKs1TkqlfzamDw34gzL1T+fxMxUZp1pixIBO9LbJQvaFK4yOo5ABmFntjCBNdT4aSKPg8tRyQ3kBV2mvEAnJ0Vw44c23IRemfXrMKk+AXae8xsOEYexVMpmnXb1rQQHazTbQiNiM0kpCm4UUEJvIjOAabU9sBvQqIwQFufAdozqKeesLUMWdTFi8WZz9BlJTAj48DL8xc4VPruLSMAy+iCOn8i8t46HbBBwK7Yl+mCYuWQbRiQD0/ZA8G/ggIVgoMzLDCjm4eyIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Received: from SN7PR11MB6677.namprd11.prod.outlook.com (2603:10b6:806:26b::6) by DSVPR11MB445312.namprd11.prod.outlook.com (2603:10b6:8:37a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.223.18; Tue, 21 Jul 2026 04:31:44 +0000 Received: from SN7PR11MB6677.namprd11.prod.outlook.com ([fe80::490a:519b:d657:99ab]) by SN7PR11MB6677.namprd11.prod.outlook.com ([fe80::490a:519b:d657:99ab%4]) with mapi id 15.21.0223.017; Tue, 21 Jul 2026 04:31:44 +0000 From: Deepesh.Varatharajan@windriver.com To: openembedded-core@lists.openembedded.org Cc: randy.macleod@windriver.com, Sundeep.Kokkonda@windriver.com, Deepesh.Varatharajan@windriver.com Subject: [PATCH 2/3] rust: refactor source handling into a shared work-shared recipe Date: Mon, 20 Jul 2026 21:31:11 -0700 Message-ID: <20260721043124.3538696-2-Deepesh.Varatharajan@windriver.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20260721043124.3538696-1-Deepesh.Varatharajan@windriver.com> References: <20260721043124.3538696-1-Deepesh.Varatharajan@windriver.com> X-ClientProxiedBy: SJ0PR13CA0099.namprd13.prod.outlook.com (2603:10b6:a03:2c5::14) To SN7PR11MB6677.namprd11.prod.outlook.com (2603:10b6:806:26b::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN7PR11MB6677:EE_|DSVPR11MB445312:EE_ X-MS-Office365-Filtering-Correlation-Id: de5bad6c-3a53-4ccc-cdcf-08dee6e0f80b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|52116014|376014|23010399003|1800799024|366016|13003099007|38350700014|29003799003|25016099003|6133799003|12006099003|56012099006|11063799006|5023799004|10067099003|22082099003|18002099003|3023799007; X-Microsoft-Antispam-Message-Info: 2qGmiQ/BitD54Gw3tuULfnorRdHkbVyr7arAsbNspzFYzOy7S8vAdGrIh83hKsZPGC/HfowyUfjQxbYmdVerfgGOnlktw98RjM3yeuL+jVYxMG+zt+MZNq0qWQeFv3gqsZrxYtOxgmEkFItsgTd+umkzrllVE0bF4l/GtmPay3kc0J3ZZnimUNkoeJ8GJfaqHb8aUPenLLUZHsj+2CFjaVx9so38mMUnyq4FSPzx0g8tSji37d8fy473dP7ajNvubAMiMQRH2PWbqTqMoOJ+gfZU2QlNklbbT5Z7DuGo8vRXQUGcrWyz317JDpSQTcAAO7ebZRyWkNEZqL4D7NMOWN0F9v8GW68sR623/hj90U6Z/wxr5XwwguWtxomzR5BG5SaPL+QlGVmpRWAuHHaG/WLUT1VHHTiTOgpRmmoB5ipjDWQpsnJICd60PzoLGX0/h7pib164kg26efHfJdMu68yzRJ/KmvGlqw5QvqQz09K4eY4EpFvI00eNg1+wCZKbxrGD3F7ydoVDfJrfSzwBE9MZfydZ9bG2wmT5LscEkI+s0Gy5OKRG1chtVsCDnB/J6TBcJIDk6yUwTUYYj9ZyhVnow9QMgV6y9jnrYJtCv98lmwnAaRSR234KK23++5wdr4gyE9LOnAqZZLhMt6+uKpFcbDlYX3spdQn118MAhco= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN7PR11MB6677.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(52116014)(376014)(23010399003)(1800799024)(366016)(13003099007)(38350700014)(29003799003)(25016099003)(6133799003)(12006099003)(56012099006)(11063799006)(5023799004)(10067099003)(22082099003)(18002099003)(3023799007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?d2TRn8368mSOVAuUZgi32Jv+tsQT?= =?utf-8?q?rxryAkUaAOu3lIir2M9QMJhKvUP1398F7p5efnWbi3Tqdcaq4PlX9Jd5tLuDi3Avk?= =?utf-8?q?5pLEK5yIEVJR/E0OwABdyK8yoivaS4lCHWupEUq2RqiWfECl0bMPsBN9gDbJEh/Cn?= =?utf-8?q?bCdhz5sLlH2yDQFQ6J/80hb48oQqfHoEfA+4NzNKNvTc1BoZlz9RQxi1q8P5QzaWT?= =?utf-8?q?mQUooKp9/esx2IujHzMACIoeLPr6T2pNBKmrBtdjMd2HRkt2N77fdcJ8ER63oHhk5?= =?utf-8?q?WM82PQGfZkoHnPphjYxV0gDLW3n4fy1450t+21khj/7c4r1aevMWZ1yj2VdWiadxZ?= =?utf-8?q?lDc7zZnwJqVm8K9PffzIeJu17vTv2Yg493DdQyq9z2DvGBW2qao7RJPBFhCxJjuos?= =?utf-8?q?K0/t5P6c/1e/LuJR9tairhTMg9+YiXifdqSYXqKnvznen5h983eL52x/WEdjatZ4X?= =?utf-8?q?UaT2CrZZmDHgRxrmmBHXJUvQ0KxPdy1/exVE/STXL/2q754uNIwKZ/O83aJy0pVop?= =?utf-8?q?QqrghIeIM2FGvdL/sKSp0UrtKUy3tHFWoHnXTUOmNqmG4NrXfBaRwHSP3WUgeXLHI?= =?utf-8?q?oGnKLBVxahWLFDFl6xwzmTl/ijTxBlGSImAn0qQenWD0YvV4xyPGMh9PK3EAUUck3?= =?utf-8?q?Ky09oauXxFXoWlDBRLEcz2SeyrQBfwMF1qMq/OVdxNAlWRUyr0mNOT5EOCWtvKTjZ?= =?utf-8?q?w5urN8i+YQD7porPEvTb2PsNDTpFFbt9d8o7QRJNgG0MgEzgDIGFiFB9xRY0p0juT?= =?utf-8?q?CPuQFc4K5NH/4OwiaaGIdoLXLoESA0oyooCyLPTPYAX4l9deyvVR0w5+bCKM5iLiu?= =?utf-8?q?ItW9GqXwonZ3TCUSjw2kEX+M7OkPpuTyiKntpEfeRoUX47l9Zf+sAXB74z81OgpkW?= =?utf-8?q?l6jRUrlwRSlYfGnAV8JZj9kcywx4Ed9tcozZL80Eyykf1YYklAIs5FVi3HmCYBiL2?= =?utf-8?q?Lv4xqKp33DlGD9vXqsmAG1pMdGIJ+UE5BsjMZInsfccKH6FsbIUvgOua7ncv6dp6R?= =?utf-8?q?hSJmY6yJhpbLh8PwjsOeWbY2C3pPhEmz8sDkLDcVOgEInPGrWg4nlLeWjYsl9qzkQ?= =?utf-8?q?gd2/J6FTS93jPgj4DJROgzP82iozy32XR5UL3cIwar81vaOBE2uIlatorJb6sHV3S?= =?utf-8?q?7SOhm+5BFXDBikA878EaTrEFL64cN8dpAS3OkO0VwPgGC9pnfqVXz+GaHWHwHbKJy?= =?utf-8?q?G4Vfyg/BAqFGabvY8untVj1L+IbwkmrH8uFPMje0ZAXe917X++4pE9BnEOlFIL/hp?= =?utf-8?q?kpXmgsDrXCG+YCTkB/nyyqQs0M0Kjv+Sl7/CTV/CqOk/cmsz5h+eI7mx4CZO7xaqW?= =?utf-8?q?Ykhgnilwm/AR48ZxfMkeB5eG2Q9YKInKLbQf3esuIeaP7qtQqDlHzeNQaZ0MgYGM3?= =?utf-8?q?fmU2UhkQxzkc5lnCec7MG0axpqRLMIigv4tnChQXa4RFV9AVSX8lqjX0C4dFmD3or?= =?utf-8?q?yKmfm6XfJVlb/BJxjx2Uenla30Tyzahy87fAht9eYd2sX47HeuzwHrbBoiuppTJ4J?= =?utf-8?q?AReErKmtaSzhkpJSHyxj0hibOEbeaAeayAedomR0VFpab2lOGverxtfNLYpidv7WZ?= =?utf-8?q?a/YYFpf4IR+mseFPjt8+/EJ3SVmz59IyMJ96f3tly7wCvk+3QrTgo4bpEUa2X6iLN?= =?utf-8?q?KVR/aeD75qy12vhHom/uNffqzxP1DI5BUata/GVa0cy6PAd2qCYcEgp4wvuDorNjl?= =?utf-8?q?46NAiI6l3?= X-Exchange-RoutingPolicyChecked: pa7cDk7ME8CELp15kD8kz7xHvuE0uWb9j+wE6YMGiHgo17iee+if7O4qGi7kCmcIpi3SGYq79odmPCWK+R9qx3mrlnD7EhhWqU1m4N7Op8YxVBZgwTduCeIRtSgf2OqRzrbss76yxbcLwQ8kF1OSBNgsiIHCGcRv5XtUTpZ0H4VANYLLZIRlmaNahoFrkgTqDZBd6TV6x7XtAqbcDGU3EldQZc+1xx7Klqig57YqHXWYmahXWXXxVEjzzegW2qUT7py3wMOxbzSGt4KKUlw9NKnTfO/N4f479xVbJKV6+UOwTQBM/kTX1Xj+PQWKNZJzGva52JeHKybF6Pv6scuVkQ== X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: de5bad6c-3a53-4ccc-cdcf-08dee6e0f80b X-MS-Exchange-CrossTenant-AuthSource: SN7PR11MB6677.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2026 04:31:44.3373 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jXvOZ6/sl4BPXLNcIC8DY1BQ/JdFUvgYBuedPDGrPcsouqrwbPjahF0XWQy+psYzYPotqGSxltg9iYuk/vC3CS20hzz01R7suaXKBbZujrhy4KaX8kkDZYBHQQuzJZhC X-MS-Exchange-Transport-CrossTenantHeadersStamped: DSVPR11MB445312 X-Proofpoint-ORIG-GUID: E6YdjWugzT77yL4fxdB8qKmmWbWuKnyh X-Proofpoint-GUID: E6YdjWugzT77yL4fxdB8qKmmWbWuKnyh X-Authority-Analysis: v=2.4 cv=AOkSgtoa c=1 sm=1 tr=0 ts=6a5ef632 cx=c_pps a=CD2IJdD5qp8L8UJuiUkm2w==:117 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=RAioF0-LDSMA:10 a=VkNPw1HP01LnGYTKEx00:22 a=bi6dqmuHe4P4UrxVR6um:22 a=iKiJcTA2PjBS6x5JeXcw:22 a=ei4SEBeUAAAA:8 a=t7CeM3EgAAAA:8 a=7CQSdrXTAAAA:8 a=iGHA9ds3AAAA:8 a=8wEic9LhAAAA:8 a=ag1SF4gXAAAA:8 a=Tr0CIQbrg0mKLZOxC6IA:9 a=5O9rwaaqg39D_ep3:21 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=8zIOOLb7Ym0NljyPXbuS:22 a=FdTzh2GWekK77mhwV6Dw:22 a=a-qgeE7W1pNrGK8U0ZQC:22 a=nM-MV4yxpKKO9kiQg6Ot:22 a=M9VCcFR7d36vLOFuqMn8:22 a=Yupwre4RP9_Eg_Bd0iYG:22 X-Proofpoint-Spam-Info: AW1haW4tMjYwNzIxMDA0MyBTYWx0ZWRfXyCYK//DzmmRW yIUT+C4poJjdjVvXzg8qwjjj+Ou44C+nHz8Ptq1OpFWaEUsQNFIRGLtRctvD22FGZEOQQruFC7n 5DncIkeE7x1MaqfECdSRwKE1fAoMGlhUBiXKqL2Y/dvk8mhPdwCR X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzIxMDA0MyBTYWx0ZWRfXz8ZV33W9JCgG eJl5m97a/0/VLsJiE1g1GzK+iXxtWGtQCp7lznIDukIwCphUcNyNeBkGcazEJV9DKLrAX5v55EZ zst7GwPdqpG/KGDBF4T4uAM0nZrnYOk+oPVitKQJ1ulU2V4zsUXjxrUfLb9/7c6Yf783NHWLpYs vlHef2DpaxmRdCQFpmed/bQFww0Ga7xOGpmn7E5hLGdXh6RwddQtQm68R8BNZ5jd7hEp300RiaO MLHJeW80XPxXXsBPNeejlQUSqVZYq4PDMD3qKa/h5fAD0rzNzSQxvu0wiwPHJKRNjGUTy0OsBoN YYExNAgjyUOEC+eG4rt6dtu6rWDTZTok9A8fyki3uzDfNiPTdbtOzYs0Vbw2b9z40bjOgL+aTeQ 4S3TKt2CXffdluHxXl2QX4+vnG/goK53Cq4xPQ2ltQbN/E5yxmdI1MyWJ1LtjsS9LsXo5x3co0S RUOc7YSS79GucQjkPNA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-07-20_06,2026-07-20_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 clxscore=1015 adultscore=0 priorityscore=1501 impostorscore=0 phishscore=0 bulkscore=0 lowpriorityscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2607210043 X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-0064b401.pphosted.com id 66L46B1r1635816 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 21 Jul 2026 04:31:48 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/241497 From: Deepesh Varatharajan [YOCTO #15808] Consolidate Rust source management across all Rust recipes (rust, cargo, libstd-rs and their native/nativesdk variants) into a single shared source tree using the work-shared pattern. Disk savings: a world build with multilibs enabled, together with nativesdk-rust and nativesdk-cargo, now shares a single ~4.2 GB rustc-src tree instead of maintaining up to 11 independent copies (~46.2 GB total). Performance (clean core-image-sato build, qemux86-64): Before: 68m53.507s After: 65m46.159s Improvement: 3m07.348s (~4.5%) New files: - rust-source.bb: A shared recipe that fetches, unpacks, and patches the rustc source tree into ${TMPDIR}/work-shared/rust-source-${PV}-${PR}. All build/install/packaging tasks are disabled; this recipe exists solely to provide a single copy of the source. Inherits allarch/nopackages, excludes itself from rm_work, and is hidden from world builds. Also defines RUST_BUILD_ARCH locally since this recipe does not inherit rust-common.bbclass but still needs the mapping for snapshot fetching via rust-snapshot.inc. - common-source.inc: Included by consumer recipes (rust, cargo, libstd-rs) to disable their local do_fetch/do_unpack/do_patch tasks, clear SRC_URI, and add task dependencies on rust-source-${PV} for configure, licensing, and SPDX generation. Also provides is_work_shared_spdx() override for correct SPDX shared-workdir detection. Modified files: - rust-source.inc: - Replace RUST_VERSION with explicit MAJOR_VER/MINOR_VER/PATCH_VER and PV. - Set UNPACKDIR to the work-shared location so all consumers share one source tree via RUSTSRC and S. - rust_git.bb (renamed from rust_1.96.1.bb): - Add 'require common-source.inc' to use the shared source tree. - Set B = ${WORKDIR}/build to separate build artifacts from shared source. - Set CARGO_HOME = ${WORKDIR}/cargo_home to isolate cargo caching. - Set build-dir = "rust-build" in config.toml to distinguish Yocto's build directory (B), which holds config.toml, from the bootstrap's own build output directory. Without this, bootstrap defaults to "build/" inside B, creating an ambiguous build/build/ nesting. - Use absolute ${S}/src/bootstrap/bootstrap.py path since cwd is now B. - Update install functions to reference rust-build/ instead of build/. - Update do_test_compile to use absolute ${S}/src/tools/ path. - Change do_rust_setup_snapshot from 'after do_unpack' to depend on rust-source-${PV}:do_unpack (since local do_unpack is removed). - cargo_git.bb (renamed from cargo_1.96.1.bb): - Add 'require common-source.inc' to use the shared source tree. - Change do_cargo_setup_snapshot from 'after do_unpack' to depend on rust-source-${PV}:do_unpack. - libstd-rs_git.bb (renamed from libstd-rs_1.96.1.bb): - Add 'require common-source.inc' to use the shared source tree. - rust-cross-canadian_git.bb (renamed from rust-cross-canadian_1.96.1.bb): - Rename only (version now comes from rust-source.inc via PV). - rust-common.bbclass: - Add a second --remap-path-prefix for ${TMPDIR}/work-shared so debug info from the shared source tree is correctly remapped. - Retain RUST_BUILD_ARCH definition so that consumer recipes (rust, cargo, libstd-rs) which inherit this class can resolve snapshot directory names used by do_rust_setup_snapshot and do_cargo_setup_snapshot. - lib/oeqa/selftest/cases/rust.py: - Adapt the selftest to the build-dir/source-dir split: use B for build artifacts and RUSTSRC for source paths. - Update remote-test-server copy path from build/ to rust-build/. - Pass --build-dir and --config explicitly to bootstrap.py. - conf/distro/include/maintainers.inc: - Add maintainer entry for the new rust-source recipe. Signed-off-by: Deepesh Varatharajan --- meta/classes-recipe/rust-common.bbclass | 4 +- meta/conf/distro/include/maintainers.inc | 1 + meta/lib/oeqa/selftest/cases/rust.py | 9 ++- .../rust/{cargo_1.96.1.bb => cargo_git.bb} | 8 ++- meta/recipes-devtools/rust/common-source.inc | 28 ++++++++++ .../{libstd-rs_1.96.1.bb => libstd-rs_git.bb} | 4 ++ ...n_1.96.1.bb => rust-cross-canadian_git.bb} | 0 meta/recipes-devtools/rust/rust-source.bb | 56 +++++++++++++++++++ meta/recipes-devtools/rust/rust-source.inc | 16 +++++- .../rust/{rust_1.96.1.bb => rust_git.bb} | 31 +++++++--- 10 files changed, 140 insertions(+), 17 deletions(-) rename meta/recipes-devtools/rust/{cargo_1.96.1.bb => cargo_git.bb} (91%) create mode 100644 meta/recipes-devtools/rust/common-source.inc rename meta/recipes-devtools/rust/{libstd-rs_1.96.1.bb => libstd-rs_git.bb} (92%) rename meta/recipes-devtools/rust/{rust-cross-canadian_1.96.1.bb => rust-cross-canadian_git.bb} (100%) create mode 100644 meta/recipes-devtools/rust/rust-source.bb rename meta/recipes-devtools/rust/{rust_1.96.1.bb => rust_git.bb} (93%) diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass index 057aeda67a..6bc42016d1 100644 --- a/meta/classes-recipe/rust-common.bbclass +++ b/meta/classes-recipe/rust-common.bbclass @@ -14,7 +14,9 @@ FILES:${PN}-dev += "${rustlibdir}/*.rlib ${rustlibdir}/*.rmeta" FILES:${PN}-dbg += "${rustlibdir}/.debug" RUSTLIB ?= "-L ${STAGING_DIR_HOST}${rustlibdir}" -RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR}" +RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR} \ + --remap-path-prefix=${TMPDIR}/work-shared=${TARGET_DBGSRC_DIR} \ +" RUSTFLAGS += "${RUSTLIB} ${RUST_DEBUG_REMAP}" RUSTLIB_DEP ??= "libstd-rs" RUST_PANIC_STRATEGY ??= "unwind" diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc index 4c6307086c..bfff5082c2 100644 --- a/meta/conf/distro/include/maintainers.inc +++ b/meta/conf/distro/include/maintainers.inc @@ -784,6 +784,7 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton " RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton " RECIPE_MAINTAINER:pn-rust = "Randy MacLeod " RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod " +RECIPE_MAINTAINER:pn-rust-source-1.96.1 = "Deepesh Varatharajan " RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned " RECIPE_MAINTAINER:pn-sassc = "Simone Weiß " RECIPE_MAINTAINER:pn-sato-icon-theme = "Richard Purdie " diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py index c961a8f652..4111d72044 100644 --- a/meta/lib/oeqa/selftest/cases/rust.py +++ b/meta/lib/oeqa/selftest/cases/rust.py @@ -46,7 +46,8 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): recipe = "rust" start_time = time.time() bitbake("{} -c test_compile".format(recipe)) - builddir = get_bb_var("RUSTSRC", "rust") + builddir = get_bb_var("B", "rust") + sourcedir = get_bb_var("RUSTSRC", "rust") # build core-image-minimal with required packages default_installed_packages = ["libgcc", "libstdc++", "libatomic", "libgomp", "libzstd", "llvm", "openssl"] features = [] @@ -111,7 +112,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): # Copy remote-test-server to image through scp host_sys = get_bb_var("RUST_BUILD_SYS", "rust") ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user="root") - ssh.copy_to(builddir + "/build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/") + ssh.copy_to(builddir + "/rust-build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/") # Execute remote-test-server on image through background ssh command = '~/remote-test-server --bind 0.0.0.0:12345 -v' sshrun=subprocess.Popen(("ssh", '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-f', "root@%s" % qemu.ip, command), shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE) @@ -131,8 +132,10 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): # PowerPC mac99 QEMU has 768MB RAM limit, so we need to minimize test binary sizes cmd = cmd + " export RUSTFLAGS='-C strip=debuginfo -Clink-arg=-lz -Clink-arg=-lzstd';" # Trigger testing. + # Run bootstrap from sourcedir with explicit --build-dir and --config + # pointing to the separate build directory. cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s" % (builddir, testargs, targetsys) + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py --build-dir %s/rust-build --config %s/config.toml test %s --target %s" % (sourcedir, builddir, builddir, testargs, targetsys) retval = runCmd(cmd) end_time = time.time() diff --git a/meta/recipes-devtools/rust/cargo_1.96.1.bb b/meta/recipes-devtools/rust/cargo_git.bb similarity index 91% rename from meta/recipes-devtools/rust/cargo_1.96.1.bb rename to meta/recipes-devtools/rust/cargo_git.bb index f16688fc76..8047a2b255 100644 --- a/meta/recipes-devtools/rust/cargo_1.96.1.bb +++ b/meta/recipes-devtools/rust/cargo_git.bb @@ -13,6 +13,8 @@ LIC_FILES_CHKSUM = " \ require rust-source.inc require rust-snapshot.inc +# Use shared source tree from rust-source.bb; disables local fetch/unpack/patch +require common-source.inc S = "${RUSTSRC}/src/tools/cargo" CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor" @@ -21,6 +23,9 @@ CVE_PRODUCT = "rust-lang:cargo" inherit cargo pkgconfig +# Explicit CARGO_HOME avoids cargo caching in the shared source tree +CARGO_HOME = "${WORKDIR}/cargo_home" + DEBUG_PREFIX_MAP += "-ffile-prefix-map=${RUSTSRC}/vendor=${TARGET_DBGSRC_DIR}" do_cargo_setup_snapshot () { @@ -32,9 +37,10 @@ do_cargo_setup_snapshot () { fi } -addtask cargo_setup_snapshot after do_unpack before do_configure +addtask cargo_setup_snapshot before do_configure do_cargo_setup_snapshot[dirs] += "${WORKDIR}/${CARGO_SNAPSHOT}" do_cargo_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER" +do_cargo_setup_snapshot[depends] += "rust-source-${PV}:do_unpack" do_compile:prepend () { export RUSTC_BOOTSTRAP="1" diff --git a/meta/recipes-devtools/rust/common-source.inc b/meta/recipes-devtools/rust/common-source.inc new file mode 100644 index 0000000000..a0d4dc542c --- /dev/null +++ b/meta/recipes-devtools/rust/common-source.inc @@ -0,0 +1,28 @@ +# common-source.inc - Disable source tasks for recipes using shared rust-source +# +# Recipes that include this file rely on rust-source-${PV} for fetching, +# unpacking, and patching the Rust source tree. This avoids duplicating the +# ~4.2 GB rustc-src tree per recipe variant (rust, cargo, libstd-rs and their +# native/nativesdk counterparts). + +# Disable local fetch/unpack/patch since rust-source-${PV} handles them +do_fetch() { + : +} +do_fetch[noexec] = "1" +deltask do_unpack +deltask do_patch + +# Clear SRC_URI to prevent fetcher warnings from stale URIs inherited via .inc +SRC_URI = "" + +# Ensure the shared source is fully ready before dependent tasks run +do_configure[depends] += "rust-source-${PV}:do_patch" +do_populate_lic[depends] += "rust-source-${PV}:do_unpack" +do_create_spdx[depends] += "rust-source-${PV}:do_patch" + +# The SPDX class uses is_work_shared_spdx() to detect shared workdirs. +# Since we redirect S rather than WORKDIR, the default detection fails. +# Override to always return True so SPDX processing works correctly. +def is_work_shared_spdx(d): + return True diff --git a/meta/recipes-devtools/rust/libstd-rs_1.96.1.bb b/meta/recipes-devtools/rust/libstd-rs_git.bb similarity index 92% rename from meta/recipes-devtools/rust/libstd-rs_1.96.1.bb rename to meta/recipes-devtools/rust/libstd-rs_git.bb index 2a5e977a5e..31d42a8525 100644 --- a/meta/recipes-devtools/rust/libstd-rs_1.96.1.bb +++ b/meta/recipes-devtools/rust/libstd-rs_git.bb @@ -5,6 +5,7 @@ LICENSE = "(MIT | Apache-2.0) & Unicode-3.0" LIC_FILES_CHKSUM = "file://../../COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc" require rust-source.inc +require common-source.inc # The dummy crate named `sysroot` represents the standard library target. # @@ -30,6 +31,9 @@ CARGO_FEATURES ?= "panic-unwind backtrace" CARGO_BUILD_FLAGS += "--features '${CARGO_FEATURES}'" CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor" +# Explicit CARGO_HOME avoids cargo caching in the shared source tree +CARGO_HOME = "${WORKDIR}/cargo_home" + do_compile:prepend () { # For Rust 1.13.0 and newer export RUSTC_BOOTSTRAP="1" diff --git a/meta/recipes-devtools/rust/rust-cross-canadian_1.96.1.bb b/meta/recipes-devtools/rust/rust-cross-canadian_git.bb similarity index 100% rename from meta/recipes-devtools/rust/rust-cross-canadian_1.96.1.bb rename to meta/recipes-devtools/rust/rust-cross-canadian_git.bb diff --git a/meta/recipes-devtools/rust/rust-source.bb b/meta/recipes-devtools/rust/rust-source.bb new file mode 100644 index 0000000000..ba696984b3 --- /dev/null +++ b/meta/recipes-devtools/rust/rust-source.bb @@ -0,0 +1,56 @@ +# rust-source.bb - Shared source recipe for the Rust toolchain +# +# This recipe fetches, unpacks, and patches the rustc source tree into a +# work-shared location so that rust, cargo, libstd-rs, and their native/nativesdk +# variants all share a single copy of the source. This eliminates duplication +# (up to 11 copies of the ~4.2 GB source tree in multilib + nativesdk builds) +# and significantly reduces disk usage (~46.2 GB -> ~4.2 GB for rustc-src). +# +# Only do_fetch, do_unpack, and do_patch are meaningful here. All other +# tasks are disabled since this recipe exists solely to provide sources. + +SUMMARY = "Rust compiler and runtime libaries" +HOMEPAGE = "http://www.rust-lang.org" +SECTION = "devel" +LICENSE = "(MIT | Apache-2.0) & Unicode-3.0" +LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc" + +# This recipe only provides source; disable build/install/packaging tasks +deltask do_configure +deltask do_compile +deltask do_install +deltask do_populate_sysroot +deltask do_populate_lic + +# Prevent rm_work from removing the shared source tree +RM_WORK_EXCLUDE += "${PN}" + +require rust-source.inc +require rust-snapshot.inc + +inherit allarch nopackages + +PN = "rust-source-${PV}" +BPN = "rust-source" + +# Place WORKDIR under work-shared so all consumers reference the same location +WORKDIR = "${TMPDIR}/work-shared/rust-source-${PV}-${PR}" +SSTATE_SWSPEC = "sstate:rust-source::${PV}:${PR}::${SSTATE_VERSION}:" + +# Use work-shared stamp directory so sstate is shared across all architectures +STAMP = "${STAMPS_DIR}/work-shared/rust-source-${PV}-${PR}" +STAMPCLEAN = "${STAMPS_DIR}/work-shared/rust-source-${PV}-*" + +DEPENDS = "" +PACKAGES = "" +baselib = "lib" +PACKAGE_ARCH = "all" + +B = "${WORKDIR}/build" + +# rust-source.bb does not inherit rust-common.bbclass (it is allarch/nopackages +# and has no toolchain dependencies), so we define RUST_BUILD_ARCH locally for +# snapshot URL resolution in rust-snapshot.inc. +RUST_BUILD_ARCH = "${@{'ppc': 'powerpc', 'ppc64': 'powerpc64', 'ppc64le': 'powerpc64le', 'riscv64': 'riscv64gc'}.get(d.getVar('BUILD_ARCH'), d.getVar('BUILD_ARCH'))}" + +EXCLUDE_FROM_WORLD = "1" diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc index 5678e00671..49207a3947 100644 --- a/meta/recipes-devtools/rust/rust-source.inc +++ b/meta/recipes-devtools/rust/rust-source.inc @@ -1,6 +1,12 @@ -RUST_VERSION ?= "${@d.getVar('PV').split('-')[0]}" +# Version is split into MAJOR_VER and MINOR_VER for easy bumping and +# to allow consistent use of PV across all consuming recipes. +MAJOR_VER = "1" +MINOR_VER = "96" +PATCH_VER = "1" -SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;name=rust \ +PV = "${MAJOR_VER}.${MINOR_VER}.${PATCH_VER}" + +SRC_URI += "https://static.rust-lang.org/dist/rustc-${PV}-src.tar.xz;name=rust \ file://0001-Define-more-ioctl-codes-on-riscv32gc-unknown-linux-g.patch;patchdir=${RUSTSRC} \ file://rust-oe-selftest.patch;patchdir=${RUSTSRC} \ file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} \ @@ -13,7 +19,11 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n " SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98" -RUSTSRC = "${UNPACKDIR}/rustc-${RUST_VERSION}-src" +# Point UNPACKDIR into the work-shared location so rust-source.bb and all +# consumers (via S/RUSTSRC) reference a single shared source tree. +UNPACKDIR = "${TMPDIR}/work-shared/rust-source-${PV}-${PR}/sources" +RUSTSRC = "${UNPACKDIR}/rustc-${PV}-src" +S = "${RUSTSRC}" UPSTREAM_CHECK_URI = "https://forge.rust-lang.org/infra/other-installation-methods.html" UPSTREAM_CHECK_REGEX = "rustc-(?P\d+(\.\d+)+)-src" diff --git a/meta/recipes-devtools/rust/rust_1.96.1.bb b/meta/recipes-devtools/rust/rust_git.bb similarity index 93% rename from meta/recipes-devtools/rust/rust_1.96.1.bb rename to meta/recipes-devtools/rust/rust_git.bb index 8b0f1d1459..da042ffaae 100644 --- a/meta/recipes-devtools/rust/rust_1.96.1.bb +++ b/meta/recipes-devtools/rust/rust_git.bb @@ -4,6 +4,10 @@ SECTION = "devel" LICENSE = "(MIT | Apache-2.0) & Unicode-3.0" LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc" +require rust-source.inc +require rust-snapshot.inc +require common-source.inc + inherit rust inherit cargo_common @@ -24,6 +28,9 @@ INHIBIT_DEFAULT_RUST_DEPS:class-native = "1" PROVIDES:class-native = "virtual/${TARGET_PREFIX}rust" S = "${RUSTSRC}" +# Separate build directory from the shared source tree so that multiple +# variants (target, native, nativesdk) don't conflict in the source dir. +B = "${WORKDIR}/build" # Use at your own risk, accepted values are stable, beta and nightly RUST_CHANNEL ?= "stable" @@ -38,6 +45,9 @@ RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config" # own vendoring. CARGO_DISABLE_BITBAKE_VENDORING = "1" +# Explicit CARGO_HOME avoids cargo caching in the shared source tree +CARGO_HOME = "${WORKDIR}/cargo_home" + setup_cargo_environment () { # The first step is to build bootstrap and some early stage tools, # these are build for the same target as the snapshot, e.g. @@ -66,11 +76,11 @@ do_rust_setup_snapshot () { done fi } -addtask rust_setup_snapshot after do_unpack before do_configure +addtask rust_setup_snapshot before do_configure addtask do_test_compile after do_configure do_rust_gen_targets do_rust_setup_snapshot[dirs] += "${WORKDIR}/rust-snapshot" do_rust_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER" -do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot" +do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot rust-source-${PV}:do_unpack" RUSTC_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/rustc" CARGO_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/cargo" @@ -173,6 +183,12 @@ python do_configure() { # nothing about when trying to build some stage0 tools (like fabricate) config.set("build", "build", e(d.getVar("RUST_BUILD_SYS"))) + # Yocto's B (${WORKDIR}/build) is where config.toml lives. Rust's + # bootstrap creates its own build output directory inside B; name it + # "rust-build" to avoid ambiguity with Yocto's build directory and + # prevent a confusing build/build/ nesting. + config.set("build", "build-dir", e("rust-build")) + # [install] config.add_section("install") # ./x.py install doesn't have any notion of "destdir" @@ -223,13 +239,10 @@ rust_runx () { oe_cargo_fix_env - python3 src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose + python3 ${S}/src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose } rust_runx[vardepsexclude] += "PARALLEL_MAKE" -require rust-source.inc -require rust-snapshot.inc - INSANE_SKIP:${PN}:class-native = "already-stripped" FILES:${PN} += "${libdir}/rustlib" FILES:${PN} += "${libdir}/*.so" @@ -241,7 +254,7 @@ do_compile () { do_test_compile[dirs] = "${B}" do_test_compile () { replace_llvm_config_path - rust_runx build src/tools/remote-test-server --target "${RUST_TARGET_SYS}" + rust_runx build ${S}/src/tools/remote-test-server --target "${RUST_TARGET_SYS}" } ALLOW_EMPTY:${PN} = "1" @@ -282,7 +295,7 @@ rust_do_install:class-nativesdk() { install -d ${D}${bindir} for i in cargo-clippy clippy-driver rustfmt; do - cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} + cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i done @@ -322,7 +335,7 @@ rust_do_install:class-target() { install -d ${D}${bindir} for i in ${EXTRA_TOOLS}; do - cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} + cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i done