From patchwork Fri Apr 11 12:48:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikko Rapeli X-Patchwork-Id: 61173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC225C36010 for ; Fri, 11 Apr 2025 12:49:00 +0000 (UTC) Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by mx.groups.io with SMTP id smtpd.web11.25093.1744375733226670839 for ; Fri, 11 Apr 2025 05:48:53 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linaro.org header.s=google header.b=uRGrGBsO; spf=pass (domain: linaro.org, ip: 209.85.167.52, mailfrom: mikko.rapeli@linaro.org) Received: by mail-lf1-f52.google.com with SMTP id 2adb3069b0e04-54954fa61c9so2503010e87.1 for ; Fri, 11 Apr 2025 05:48:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1744375731; x=1744980531; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=+TbvSrs3dUDHRsGAnWj9LV1nR+tqAZxnMtuyRTrLy6I=; b=uRGrGBsOwad8XlZ/t3ipaKvmhYjdAUQ0y2P/6CcE606EtOXDJZnnWPVxRY/VO8e2dv U3KgTu8vRrOTiEHL4Ny09G19zOAaM8ElKDhtEE2ac+EPShNv8bt4g6+ke/vb9n6mj+K6 awU3G+BIMwMimvNvYvzvevc09ccGqurnQBj8WK/QDahapUVN4xvuLZPWQRc7IfDiFTvZ TBNQH8hOz+kqIZk3PCLBJsMNbBTTEfw5F6eaNOhD+GT6Jt8zj8QXsdaV8ZgAVHp4dd46 JyEVC6ELwyfjba88rv9fRk6J0iR9S+aLEXYGoidb/YyukV3VRy9RL2bmCMj1cVWWAnaL AOrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744375731; x=1744980531; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+TbvSrs3dUDHRsGAnWj9LV1nR+tqAZxnMtuyRTrLy6I=; b=djWL9cMwnbZJ8z+PnWsDQZk9i/ZN0yRjvogpODnu2JO8LvutkCH3/iHmmXfF+h8dv1 IAuUZQjwEQFjvh0V9fNtKZglopZTMamqCETE0R19u++dQEP/3AWYf7OEo8E4fkCvn0Wi VHLlzCtL2dZXE4mdwUHvPisWiVx0gEUvqqTTYSwLCkeSXiBG2fgAAlgh9dbdXO3RgAWG i1lWAWpQyXtobgxStAqQ0DAkKYKbgyT/uzjgn9DaTgu9yOFGqQRE2QhM0XxKJ0vY01Sf r8mrjzD08TcMmXnM1DPLegz0aQIGw+C/YzBxG7h8b5Asz3vJffQDCyaewNB4xDIwuy21 nrQg== X-Gm-Message-State: AOJu0YzMO2QvoDa7ZSH6gVQiTVwp0wtq5+JQZWRRR1tack9q+7lYL/DF MA7Z5DPr2LZB9rAeZED348x7JglwTdy/JYwBKsM6gU6pFi+OkFNXBkUw+5AQs0uxXoQMCrwGCn3 BafE= X-Gm-Gg: ASbGncvA4Dga8wSKFxr8CDHz2BTwM9Fp82Hd1wIedmTpKE69en1J0FE8673ehdxxKpB sZPlUpSaemEnnMzMCDxkCoTxLM21sxP11e7HSFPuAw0ZwutSedZU9iCtUrMXaeXDWn3B/psSpCb Y9WQlKdUPDwL3NVDZ4P4NEGTj1aOD43D3xhGfgGQdGag/h1V9FEk3bx1Xj7pLkPvp3PaFkZQqgv BIuCrRHys+0EXaXx3PHxslhAMPeKfJx51gcPVwA3//GB61bTtzOslAJOkQjv83LZSebR7eFbZD6 tJZ0om+XEBO5xvA5akJhoubrKc60cAHNMGqYWdeVt8tXxEqLXmza/yMqs1rmZ4PGDPyWpibAouS 891YcKjOocZnBzSc0QQ== X-Google-Smtp-Source: AGHT+IGe7rMbruuXpb8dlaMX876a7XSJXNoP7E4uBKIwxA0t5tsg5L7UbFpkMYDn7FUBSpEXF2h4QA== X-Received: by 2002:a05:6512:3181:b0:54a:fa58:a6f7 with SMTP id 2adb3069b0e04-54d45294fe0mr792735e87.24.1744375730953; Fri, 11 Apr 2025 05:48:50 -0700 (PDT) Received: from localhost.localdomain (87-100-218-141.bb.dnainternet.fi. [87.100.218.141]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-54d3d234204sm421936e87.54.2025.04.11.05.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Apr 2025 05:48:50 -0700 (PDT) From: Mikko Rapeli To: openembedded-core@lists.openembedded.org Cc: Mikko Rapeli Subject: [PATCH v4] kernel.bbclass: add kernel-initrd-modules meta package Date: Fri, 11 Apr 2025 15:48:13 +0300 Message-ID: <20250411124814.436689-1-mikko.rapeli@linaro.org> X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 11 Apr 2025 12:49:00 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/214737 At the moment linux-yocto kernels for various architectures are not very modular and a lot of drivers are built into the kernel even when they are not needed at runtime. These make the main kernel binary big and slow to boot. This also impacts udev in userspace which takes a long time processing events from all these built in drivers, for example when udev runs in initrd. Then constructing the initrd is very device and kernel configuration specific. initrd image needs explicitly define which binary packages to install to avoid pulling in complex dependencies. A full set of kernel modules via kernel-modules meta package is too big for initrd and most of the drivers are not needed for use cases like "just load modules to mount main rootfs". Then the initrd configuration breaks if kernel driver is built into the kernel since the binary package doesn't exist. Introduce kernel-initrd-modules meta package to solve these problems. The meta package adds dependencies to real kernel modules based on the kernel module file paths so that it will include several kernel subsystems and their drivers which are often needed to find main rootfs from some block device. This works when drivers are built as modules but does not break if drivers are built into the kernel. The resulting initrd is also smaller since only a subset of drivers are needed for "mount the rootfs" usecase. Tested on genericarm64 kernel and qemu and AMD KV260 HW. Signed-off-by: Mikko Rapeli --- .../kernel-module-split.bbclass | 48 +++++++++++++++++++ meta/classes-recipe/kernel.bbclass | 5 +- meta/classes-recipe/module.bbclass | 3 ++ 3 files changed, 55 insertions(+), 1 deletion(-) v4: remove duplication of KERNEL_INITRD_MODULES_REGEX and KERNEL_INITRD_MODULES_META_PACKAGE from modules.bbclass since both modules.bbclass and kernel.bbclass include kernel-module-split.bbclass. Sending v4 patch separately from full series which is unchanged from v3 to avoid sending too many messages to oe-core list. v3: full series https://lists.openembedded.org/g/openembedded-core/message/214351 v2: full series https://lists.openembedded.org/g/openembedded-core/message/213458 v1: full series https://lists.openembedded.org/g/openembedded-core/message/212832 diff --git a/meta/classes-recipe/kernel-module-split.bbclass b/meta/classes-recipe/kernel-module-split.bbclass index 9487365eb7..101c5cd959 100644 --- a/meta/classes-recipe/kernel-module-split.bbclass +++ b/meta/classes-recipe/kernel-module-split.bbclass @@ -42,6 +42,40 @@ KERNEL_MODULE_PACKAGE_PREFIX ?= "" KERNEL_MODULE_PACKAGE_SUFFIX ?= "-${KERNEL_VERSION}" KERNEL_MODULE_PROVIDE_VIRTUAL ?= "1" +# subset of kernel modules needed in initrd, to e.g. mount rootfs from block device +KERNEL_INITRD_MODULES_META_PACKAGE ?= "${@ d.getVar("KERNEL_PACKAGE_NAME") or "kernel" }-initrd-modules" + +# match regex to path or file name. E.g. include all drivers with files in path /drivers/ata/ +KERNEL_INITRD_MODULES_REGEX ?= "(.*)(\ +/drivers/acpi/|\ +/drivers/ata/|\ +/drivers/block/|\ +/drivers/cdrom/|\ +/drivers/char/hw_random/|\ +/drivers/char/tpm/|\ +/drivers/char/|\ +/drivers/crypto/|\ +/drivers/dax/|\ +/drivers/firmware/arm_scmi/|\ +/drivers/gpu/drm/|\ +/drivers/md/|\ +/drivers/mmc/|\ +/drivers/mtd/|\ +/drivers/nvdimm/|\ +/drivers/nvme/|\ +/drivers/pci/|\ +/drivers/scsi/|\ +/drivers/tee/|\ +/drivers/tty/serial/|\ +/drivers/virtio/|\ +/drivers/watchdog/|\ +/kernel/arch/|\ +/kernel/block/|\ +/kernel/crypto/|\ +/kernel/fs/|\ +/kernel/lib/\ +)(.*)" + python split_kernel_module_packages () { import re @@ -183,6 +217,20 @@ python split_kernel_module_packages () { modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', file_regex=module_regex, output_pattern=module_pattern, description='%s kernel module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='%s-%s' % (kernel_package_name, kernel_version)) if modules: d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules)) + + initrd_metapkg = d.getVar('KERNEL_INITRD_MODULES_META_PACKAGE') or "" + initrd_module_regex = d.getVar('KERNEL_INITRD_MODULES_REGEX') or "" + if (initrd_metapkg != "") and (initrd_module_regex != ""): + initrd_module_regex = re.compile(initrd_module_regex) + initrd_modules = [] + for module in modules: + files = d.getVar('FILES:' + module) + m = re.match(initrd_module_regex, files) + if m: + initrd_modules.append(module) + + if initrd_modules: + d.appendVar('RDEPENDS:' + initrd_metapkg, ' '+' '.join(initrd_modules)) } do_package[vardeps] += '${@" ".join(map(lambda s: "module_conf_" + s, (d.getVar("KERNEL_MODULE_PROBECONF") or "").split()))}' diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index 36ce659762..3dcaebcaed 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -695,13 +695,14 @@ EXPORT_FUNCTIONS do_compile do_transform_kernel do_transform_bundled_initramfs d # kernel-base becomes kernel-${KERNEL_VERSION} # kernel-image becomes kernel-image-${KERNEL_VERSION} -PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules ${KERNEL_PACKAGE_NAME}-dbg" +PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules ${KERNEL_PACKAGE_NAME}-initrd-modules ${KERNEL_PACKAGE_NAME}-dbg" FILES:${PN} = "" FILES:${KERNEL_PACKAGE_NAME}-base = "${nonarch_base_libdir}/modules/${KERNEL_VERSION}/modules.order ${nonarch_base_libdir}/modules/${KERNEL_VERSION}/modules.builtin ${nonarch_base_libdir}/modules/${KERNEL_VERSION}/modules.builtin.modinfo" FILES:${KERNEL_PACKAGE_NAME}-image = "" FILES:${KERNEL_PACKAGE_NAME}-dev = "/${KERNEL_IMAGEDEST}/System.map* /${KERNEL_IMAGEDEST}/Module.symvers* /${KERNEL_IMAGEDEST}/config* ${KERNEL_SRC_PATH} ${nonarch_base_libdir}/modules/${KERNEL_VERSION}/build" FILES:${KERNEL_PACKAGE_NAME}-vmlinux = "/${KERNEL_IMAGEDEST}/vmlinux-${KERNEL_VERSION_NAME}" FILES:${KERNEL_PACKAGE_NAME}-modules = "" +FILES:${KERNEL_PACKAGE_NAME}-initrd-modules = "" FILES:${KERNEL_PACKAGE_NAME}-dbg = "/usr/lib/debug /usr/src/debug" RDEPENDS:${KERNEL_PACKAGE_NAME} = "${KERNEL_PACKAGE_NAME}-base (= ${EXTENDPKGV})" # Allow machines to override this dependency if kernel image files are @@ -716,7 +717,9 @@ ALLOW_EMPTY:${KERNEL_PACKAGE_NAME} = "1" ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-base = "1" ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-image = "1" ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-modules = "1" +ALLOW_EMPTY:${KERNEL_PACKAGE_NAME}-initrd-modules = "1" DESCRIPTION:${KERNEL_PACKAGE_NAME}-modules = "Kernel modules meta package" +DESCRIPTION:${KERNEL_PACKAGE_NAME}-initrd-modules = "Kernel initrd modules meta package" pkg_postinst:${KERNEL_PACKAGE_NAME}-base () { if [ ! -e "$D/lib/modules/${KERNEL_VERSION}" ]; then diff --git a/meta/classes-recipe/module.bbclass b/meta/classes-recipe/module.bbclass index f2f0b25a2d..4661501ded 100644 --- a/meta/classes-recipe/module.bbclass +++ b/meta/classes-recipe/module.bbclass @@ -86,3 +86,6 @@ EXPORT_FUNCTIONS do_compile do_install KERNEL_MODULES_META_PACKAGE = "${PN}" FILES:${PN} = "" ALLOW_EMPTY:${PN} = "1" + +FILES:${PN}-initrd = "" +ALLOW_EMPTY:${PN}-initrd = "1"