| Message ID | 20250314111942.304800-1-mikko.rapeli@linaro.org |
|---|---|
| Headers | show |
| Series | systemd based initrd and modular kernel support | expand |
Note that the feature freeze is now. This has to wait. Alex On Fri, 14 Mar 2025 at 12:20, Mikko Rapeli via lists.openembedded.org <mikko.rapeli=linaro.org@lists.openembedded.org> wrote: > > systemd based initrd supports more security, encryption etc features > than custom shell script ones but oe-core lacks support for it. > Convert core-image-initramfs-boot to create systemd based initrd > if "systemd-initramfs" is set in DISTRO_FEATURES. Includes a test > for this initrd to mount rootfs via Unified Kernel Image uki.py. > > Sadly the boot time is really slow, e.g. over 30 seconds with > systemd based initrd. One of the major reasons is the large > amount of kernel modules built into the linux-yocto kernel and > processing of their udev events in userspace. Thus add > kernel-initrd-modules meta package to easily install > subset of all kernel modules to initrd. The subset > supports "mount rootfs from any block device" but > doesn't include graphics, UBS etc support. Without this > generated meta package, initrd recipe needs to manually define > which exact kernel modules to include which breaks if > those modules are built into the kernel. Thus the initrd > becomes machine and kernel config specific and hard to maintain. > With this meta package a generic initrd can be created. > > Separate changes to linux-yocto compile a lot more drivers > as modules but they depend on these initrd changes. > > Mikko Rapeli (6): > systemd: use serial-getty-generator on genericarm64 > systemd: enable efi support if in MACHINE_FEATURES too > uki.bbclass: drop serial console from kernel command line > kernel.bbclass: add kernel-initrd-modules meta package > core-image-initramfs-boot: add option to build systemd based initrd > oeqa selftest uki.py: add aarch64/arm test with systemd based initrd > > .../recipes-core/systemd/systemd_%.bbappend | 3 + > .../kernel-module-split.bbclass | 46 +++++++++++ > meta/classes-recipe/kernel.bbclass | 5 +- > meta/classes-recipe/module.bbclass | 37 +++++++++ > meta/classes-recipe/uki.bbclass | 2 +- > meta/lib/oeqa/selftest/cases/uki.py | 76 +++++++++++++++++++ > .../images/core-image-initramfs-boot.bb | 28 ++++++- > meta/recipes-core/systemd/systemd_257.3.bb | 1 + > 8 files changed, 193 insertions(+), 5 deletions(-) > create mode 100644 meta-yocto-bsp/recipes-core/systemd/systemd_%.bbappend > > -- > 2.43.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#212832): https://lists.openembedded.org/g/openembedded-core/message/212832 > Mute This Topic: https://lists.openembedded.org/mt/111696849/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hi, On Mon, Mar 17, 2025 at 11:37:29AM +0100, Alexander Kanavin wrote: > Note that the feature freeze is now. This has to wait. Yes, I'm aware. Just posted this out as it is 'done' on my side and need feedback. The kernel config change RFC is also out but that in current form breaks things so need feedback how this could be fixed, for example in specific machines configure features to compile as modules, and/or which usecases must work with the single kernel binary without initrd. Cheers, -Mikko > Alex > > On Fri, 14 Mar 2025 at 12:20, Mikko Rapeli via lists.openembedded.org > <mikko.rapeli=linaro.org@lists.openembedded.org> wrote: > > > > systemd based initrd supports more security, encryption etc features > > than custom shell script ones but oe-core lacks support for it. > > Convert core-image-initramfs-boot to create systemd based initrd > > if "systemd-initramfs" is set in DISTRO_FEATURES. Includes a test > > for this initrd to mount rootfs via Unified Kernel Image uki.py. > > > > Sadly the boot time is really slow, e.g. over 30 seconds with > > systemd based initrd. One of the major reasons is the large > > amount of kernel modules built into the linux-yocto kernel and > > processing of their udev events in userspace. Thus add > > kernel-initrd-modules meta package to easily install > > subset of all kernel modules to initrd. The subset > > supports "mount rootfs from any block device" but > > doesn't include graphics, UBS etc support. Without this > > generated meta package, initrd recipe needs to manually define > > which exact kernel modules to include which breaks if > > those modules are built into the kernel. Thus the initrd > > becomes machine and kernel config specific and hard to maintain. > > With this meta package a generic initrd can be created. > > > > Separate changes to linux-yocto compile a lot more drivers > > as modules but they depend on these initrd changes. > > > > Mikko Rapeli (6): > > systemd: use serial-getty-generator on genericarm64 > > systemd: enable efi support if in MACHINE_FEATURES too > > uki.bbclass: drop serial console from kernel command line > > kernel.bbclass: add kernel-initrd-modules meta package > > core-image-initramfs-boot: add option to build systemd based initrd > > oeqa selftest uki.py: add aarch64/arm test with systemd based initrd > > > > .../recipes-core/systemd/systemd_%.bbappend | 3 + > > .../kernel-module-split.bbclass | 46 +++++++++++ > > meta/classes-recipe/kernel.bbclass | 5 +- > > meta/classes-recipe/module.bbclass | 37 +++++++++ > > meta/classes-recipe/uki.bbclass | 2 +- > > meta/lib/oeqa/selftest/cases/uki.py | 76 +++++++++++++++++++ > > .../images/core-image-initramfs-boot.bb | 28 ++++++- > > meta/recipes-core/systemd/systemd_257.3.bb | 1 + > > 8 files changed, 193 insertions(+), 5 deletions(-) > > create mode 100644 meta-yocto-bsp/recipes-core/systemd/systemd_%.bbappend > > > > -- > > 2.43.0 > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#212832): https://lists.openembedded.org/g/openembedded-core/message/212832 > > Mute This Topic: https://lists.openembedded.org/mt/111696849/1686489 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > >
On Fri, 2025-03-14 at 13:19 +0200, Mikko Rapeli via lists.openembedded.org wrote: > systemd based initrd supports more security, encryption etc features > than custom shell script ones but oe-core lacks support for it. > Convert core-image-initramfs-boot to create systemd based initrd > if "systemd-initramfs" is set in DISTRO_FEATURES. Includes a test > for this initrd to mount rootfs via Unified Kernel Image uki.py. > > Sadly the boot time is really slow, e.g. over 30 seconds with > systemd based initrd. One of the major reasons is the large > amount of kernel modules built into the linux-yocto kernel and > processing of their udev events in userspace. Thus add > kernel-initrd-modules meta package to easily install > subset of all kernel modules to initrd. The subset > supports "mount rootfs from any block device" but > doesn't include graphics, UBS etc support. Without this > generated meta package, initrd recipe needs to manually define > which exact kernel modules to include which breaks if > those modules are built into the kernel. Thus the initrd > becomes machine and kernel config specific and hard to maintain. > With this meta package a generic initrd can be created. > > Separate changes to linux-yocto compile a lot more drivers > as modules but they depend on these initrd changes. > > Mikko Rapeli (6): > systemd: use serial-getty-generator on genericarm64 > systemd: enable efi support if in MACHINE_FEATURES too > uki.bbclass: drop serial console from kernel command line > kernel.bbclass: add kernel-initrd-modules meta package > core-image-initramfs-boot: add option to build systemd based initrd > oeqa selftest uki.py: add aarch64/arm test with systemd based initrd > > .../recipes-core/systemd/systemd_%.bbappend | 3 + > .../kernel-module-split.bbclass | 46 +++++++++++ > meta/classes-recipe/kernel.bbclass | 5 +- > meta/classes-recipe/module.bbclass | 37 +++++++++ > meta/classes-recipe/uki.bbclass | 2 +- > meta/lib/oeqa/selftest/cases/uki.py | 76 +++++++++++++++++++ > .../images/core-image-initramfs-boot.bb | 28 ++++++- > meta/recipes-core/systemd/systemd_257.3.bb | 1 + > 8 files changed, 193 insertions(+), 5 deletions(-) > create mode 100644 meta-yocto-bsp/recipes-core/systemd/systemd_%.bbappend Unfortunately this series seemed to result in oe-selftest failures on both aarch64 and x86-64: https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1242 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1110/steps/14/logs/stdio Cheers, Richard
Hi, On Mon, Mar 17, 2025 at 11:50:58AM +0000, Richard Purdie wrote: > On Fri, 2025-03-14 at 13:19 +0200, Mikko Rapeli via lists.openembedded.org wrote: > > systemd based initrd supports more security, encryption etc features > > than custom shell script ones but oe-core lacks support for it. > > Convert core-image-initramfs-boot to create systemd based initrd > > if "systemd-initramfs" is set in DISTRO_FEATURES. Includes a test > > for this initrd to mount rootfs via Unified Kernel Image uki.py. > > > > Sadly the boot time is really slow, e.g. over 30 seconds with > > systemd based initrd. One of the major reasons is the large > > amount of kernel modules built into the linux-yocto kernel and > > processing of their udev events in userspace. Thus add > > kernel-initrd-modules meta package to easily install > > subset of all kernel modules to initrd. The subset > > supports "mount rootfs from any block device" but > > doesn't include graphics, UBS etc support. Without this > > generated meta package, initrd recipe needs to manually define > > which exact kernel modules to include which breaks if > > those modules are built into the kernel. Thus the initrd > > becomes machine and kernel config specific and hard to maintain. > > With this meta package a generic initrd can be created. > > > > Separate changes to linux-yocto compile a lot more drivers > > as modules but they depend on these initrd changes. > > > > Mikko Rapeli (6): > > � systemd: use serial-getty-generator on genericarm64 > > � systemd: enable efi support if in MACHINE_FEATURES too > > � uki.bbclass: drop serial console from kernel command line > > � kernel.bbclass: add kernel-initrd-modules meta package > > � core-image-initramfs-boot: add option to build systemd based initrd > > � oeqa selftest uki.py: add aarch64/arm test with systemd based initrd > > > > �.../recipes-core/systemd/systemd_%.bbappend�� |� 3 + > > �.../kernel-module-split.bbclass�������������� | 46 +++++++++++ > > �meta/classes-recipe/kernel.bbclass����������� |� 5 +- > > �meta/classes-recipe/module.bbclass����������� | 37 +++++++++ > > �meta/classes-recipe/uki.bbclass�������������� |� 2 +- > > �meta/lib/oeqa/selftest/cases/uki.py���������� | 76 +++++++++++++++++++ > > �.../images/core-image-initramfs-boot.bb������ | 28 ++++++- > > �meta/recipes-core/systemd/systemd_257.3.bb��� |� 1 + > > �8 files changed, 193 insertions(+), 5 deletions(-) > > �create mode 100644 meta-yocto-bsp/recipes-core/systemd/systemd_%.bbappend > > Unfortunately this series seemed to result in oe-selftest failures on > both aarch64 and x86-64: > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1242 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1110/steps/14/logs/stdio Thanks, I will investigate these. I ran into various issues when executing selftests on my aarch64 build machine. Is the basehash error related to this series or something else? https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1782873/raw_inline stdio: ERROR: When reparsing /tmp/selftest-fetchoxfe4t0p/test.bb:do_checkuri, the basehash value changed from 84b9e23a83a764e7a5d8a96ee523199e2b1c604f4280a45030e248de0ff52962 to 4123da244f465feffecb2b4fd2b132b64851981ec2fb70bf9445df8b0ea6a69e. The metadata is not deterministic and this needs to be fixed. stdio: ERROR: The following commands may help: stdio: ERROR: $ bitbake test -cdo_checkuri -Snone stdio: ERROR: Then: stdio: ERROR: $ bitbake test -cdo_checkuri -Sprintdiff stdio: ERROR: When reparsing /tmp/selftest-fetchoxfe4t0p/test.bb:do_fetch, the basehash value changed from 6d55d6743729b615749dc016857d7e5f9c884a8b92f0d57f68e743f4910333d3 to fa7d6ea0563b60362418d2a6c5e41a6684183d01ec578b65abd0c64a9cfba7cf. The metadata is not deterministic and this needs to be fixed. ... There are a lot of these. If there are any boot failures in qemu then I will have a look. The changes were not supposed to change existing rootfs or initrd images. The new test in uki.py may fail on some configurations which I did no manage to test. Maybe "uki.bbclass: drop serial console from kernel command line" is breaking serial console prompt on some machines. I'll try to reproduce. Did the test run contain the linux-yocto changes which build a lot of kernel drivers as modules? I don't think that is ready for AB testing yet, hence RFC in https://lists.yoctoproject.org/g/linux-yocto/message/14866 The changes were sent out in-reply-to this thread so it's possible some tool picked them up. That could explain shell script based initrd boot error message: ERROR: There's no '/dev' on rootfs. Cheers, -Mikko
On Mon, 2025-03-17 at 14:20 +0200, Mikko Rapeli wrote: > Hi, > > On Mon, Mar 17, 2025 at 11:50:58AM +0000, Richard Purdie wrote: > > On Fri, 2025-03-14 at 13:19 +0200, Mikko Rapeli via lists.openembedded.org wrote: > > > systemd based initrd supports more security, encryption etc features > > > than custom shell script ones but oe-core lacks support for it. > > > Convert core-image-initramfs-boot to create systemd based initrd > > > if "systemd-initramfs" is set in DISTRO_FEATURES. Includes a test > > > for this initrd to mount rootfs via Unified Kernel Image uki.py. > > > > > > Sadly the boot time is really slow, e.g. over 30 seconds with > > > systemd based initrd. One of the major reasons is the large > > > amount of kernel modules built into the linux-yocto kernel and > > > processing of their udev events in userspace. Thus add > > > kernel-initrd-modules meta package to easily install > > > subset of all kernel modules to initrd. The subset > > > supports "mount rootfs from any block device" but > > > doesn't include graphics, UBS etc support. Without this > > > generated meta package, initrd recipe needs to manually define > > > which exact kernel modules to include which breaks if > > > those modules are built into the kernel. Thus the initrd > > > becomes machine and kernel config specific and hard to maintain. > > > With this meta package a generic initrd can be created. > > > > > > Separate changes to linux-yocto compile a lot more drivers > > > as modules but they depend on these initrd changes. > > > > > > Mikko Rapeli (6): > > > systemd: use serial-getty-generator on genericarm64 > > > systemd: enable efi support if in MACHINE_FEATURES too > > > uki.bbclass: drop serial console from kernel command line > > > kernel.bbclass: add kernel-initrd-modules meta package > > > core-image-initramfs-boot: add option to build systemd based initrd > > > oeqa selftest uki.py: add aarch64/arm test with systemd based initrd > > > > > > .../recipes-core/systemd/systemd_%.bbappend | 3 + > > > .../kernel-module-split.bbclass | 46 +++++++++++ > > > meta/classes-recipe/kernel.bbclass | 5 +- > > > meta/classes-recipe/module.bbclass | 37 +++++++++ > > > meta/classes-recipe/uki.bbclass | 2 +- > > > meta/lib/oeqa/selftest/cases/uki.py | 76 +++++++++++++++++++ > > > .../images/core-image-initramfs-boot.bb | 28 ++++++- > > > meta/recipes-core/systemd/systemd_257.3.bb | 1 + > > > 8 files changed, 193 insertions(+), 5 deletions(-) > > > create mode 100644 meta-yocto-bsp/recipes-core/systemd/systemd_%.bbappend > > > > Unfortunately this series seemed to result in oe-selftest failures on > > both aarch64 and x86-64: > > > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1242 > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1110/steps/14/logs/stdio > > Thanks, I will investigate these. I ran into various issues when executing > selftests on my aarch64 build machine. > > Is the basehash error related to this series or something else? > > https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1782873/raw_inline > > stdio: ERROR: When reparsing /tmp/selftest-fetchoxfe4t0p/test.bb:do_checkuri, the basehash value changed from 84b9e23a83a764e7a5d8a96ee523199e2b1c604f4280a45030e248de0ff52962 to 4123da244f465feffecb2b4fd2b132b64851981ec2fb70bf9445df8b0ea6a69e. The metadata is not deterministic and this needs to be fixed. > stdio: ERROR: The following commands may help: > stdio: ERROR: $ bitbake test -cdo_checkuri -Snone > stdio: ERROR: Then: > stdio: ERROR: $ bitbake test -cdo_checkuri -Sprintdiff > stdio: ERROR: When reparsing /tmp/selftest-fetchoxfe4t0p/test.bb:do_fetch, the basehash value changed from 6d55d6743729b615749dc016857d7e5f9c884a8b92f0d57f68e743f4910333d3 to fa7d6ea0563b60362418d2a6c5e41a6684183d01ec578b65abd0c64a9cfba7cf. The metadata is not deterministic and this needs to be fixed. > ... > > There are a lot of these. I don't remember anything in the branch that would do that. It is possible these are normal test output and only show up due to the other failures. The test output shows everything from that execution thread of selftest (due to the way python unittest parallelism was enabled). > If there are any boot failures in qemu then I will have a look. The changes were not > supposed to change existing rootfs or initrd images. The new test in uki.py > may fail on some configurations which I did no manage to test. > Maybe "uki.bbclass: drop serial console from kernel command line" is breaking > serial console prompt on some machines. I'll try to reproduce. > > Did the test run contain the linux-yocto changes which build a lot of kernel > drivers as modules? I don't think that is ready for AB testing yet, hence RFC in > https://lists.yoctoproject.org/g/linux-yocto/message/14866 > The changes were sent out in-reply-to this thread so it's possible some tool > picked them up. That could explain shell script based initrd boot error message: > > ERROR: There's no '/dev' on rootfs. Those were not included. Cheers, Richard