| Message ID | 20250603082419.409564-1-adrian.freihofer@siemens.com |
|---|---|
| Headers | show |
| Series | FIT image improvements | expand |
On Tue, 2025-06-03 at 10:23 +0200, Adrian Freihofer via lists.openembedded.org wrote: > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > This patch series re-writes the FIT image related code. The goal is > to > fix [YOCTO #12912] which is a long standing issue. > > Changes in comparison to v6 > --------------------------- > > * Move the code for exporting the FIT image specific kernel artifacts > in a new, > separate kernel-fit-extra-artifacts.bbclass rather than adding it > to the > existing kernel-uimage.bbclass. > * Improve the devicetree.bbclass to skip error messages of shell > globs > * Improve the error message in the kernel.bbclass to: "fitImage is no > longer > supported as a KERNEL_IMAGETYPE(S). FIT images are built by the > linux-yocto-fitimage recipe." > > Revert a non essential refactoring in the kernel.bbclass v7 seems to have some new failures: https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/497 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/1810 both in musl. Cheers, Richard
On Tue, 2025-06-03 at 11:48 +0100, Richard Purdie wrote: > On Tue, 2025-06-03 at 10:23 +0200, Adrian Freihofer via > lists.openembedded.org wrote: > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > This patch series re-writes the FIT image related code. The goal is > > to > > fix [YOCTO #12912] which is a long standing issue. > > > > Changes in comparison to v6 > > --------------------------- > > > > * Move the code for exporting the FIT image specific kernel > > artifacts > > in a new, > > separate kernel-fit-extra-artifacts.bbclass rather than adding it > > to the > > existing kernel-uimage.bbclass. > > * Improve the devicetree.bbclass to skip error messages of shell > > globs > > * Improve the error message in the kernel.bbclass to: "fitImage is > > no > > longer > > supported as a KERNEL_IMAGETYPE(S). FIT images are built by the > > linux-yocto-fitimage recipe." > > > > Revert a non essential refactoring in the kernel.bbclass > > v7 seems to have some new failures: > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/497 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/1810 > > both in musl. Hi Richard My conclusion is that the change requested by this discussion https://lists.openembedded.org/g/openembedded-core/message/217730 makes the linux-yocto-fitimage.bb incompatible with bitbake world. I guess that bitbake world would now fail for all build configurations, not only for musl. The background is that the kernel artifact (linux.bin) which gets included into the Fit image is no longer built by default. It requires KERNEL_CLASSES += "kernel-fit-extra-artifacts". I guess that this !fixup patch is the solution: diff --git a/meta/classes-recipe/kernel-fit-image.bbclass b/meta/classes-recipe/kernel-fit-image.bbclass index 6d80cd4bb47..d2eebb88bc2 100644 --- a/meta/classes-recipe/kernel-fit-image.bbclass +++ b/meta/classes-recipe/kernel-fit-image.bbclass @@ -7,6 +7,9 @@ UNPACKDIR = "${S}" PACKAGE_ARCH = "${MACHINE_ARCH}" +# This bbclass requires KERNEL_CLASSES += "kernel-fit-extra-artifacts" +EXCLUDE_FROM_WORLD = "1" + DEPENDS += "\ u-boot-tools-native dtc-native \ But I need to test it before sending a new version. And today I have no access to my build machine. Thank you for testing Adrian > > Cheers, > > Richard
On Tue, 2025-06-03 at 11:48 +0100, Richard Purdie via lists.openembedded.org wrote: > On Tue, 2025-06-03 at 10:23 +0200, Adrian Freihofer via > lists.openembedded.org wrote: > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > This patch series re-writes the FIT image related code. The goal is > > to > > fix [YOCTO #12912] which is a long standing issue. > > > > Changes in comparison to v6 > > --------------------------- > > > > * Move the code for exporting the FIT image specific kernel > > artifacts > > in a new, > > separate kernel-fit-extra-artifacts.bbclass rather than adding it > > to the > > existing kernel-uimage.bbclass. > > * Improve the devicetree.bbclass to skip error messages of shell > > globs > > * Improve the error message in the kernel.bbclass to: "fitImage is > > no > > longer > > supported as a KERNEL_IMAGETYPE(S). FIT images are built by the > > linux-yocto-fitimage recipe." > > > > Revert a non essential refactoring in the kernel.bbclass > > v7 seems to have some new failures: > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/497 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/1810 > > both in musl. Build is continuing and this showed up in the world build: https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/1748 Cheers, Richard
On Tue, 2025-06-03 at 13:35 +0000, Freihofer, Adrian wrote: > On Tue, 2025-06-03 at 11:48 +0100, Richard Purdie wrote: > > On Tue, 2025-06-03 at 10:23 +0200, Adrian Freihofer via > > lists.openembedded.org wrote: > > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > > > This patch series re-writes the FIT image related code. The goal is > > > to > > > fix [YOCTO #12912] which is a long standing issue. > > > > > > Changes in comparison to v6 > > > --------------------------- > > > > > > * Move the code for exporting the FIT image specific kernel > > > artifacts > > > in a new, > > > separate kernel-fit-extra-artifacts.bbclass rather than adding it > > > to the > > > existing kernel-uimage.bbclass. > > > * Improve the devicetree.bbclass to skip error messages of shell > > > globs > > > * Improve the error message in the kernel.bbclass to: "fitImage is > > > no > > > longer > > > supported as a KERNEL_IMAGETYPE(S). FIT images are built by the > > > linux-yocto-fitimage recipe." > > > > > > Revert a non essential refactoring in the kernel.bbclass > > > > v7 seems to have some new failures: > > > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/497 > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/1810 > > > > both in musl. > > Hi Richard > > My conclusion is that the change requested by this discussion > https://lists.openembedded.org/g/openembedded-core/message/217730 > makes the linux-yocto-fitimage.bb incompatible with bitbake world. > I guess that bitbake world would now fail for all build configurations, > not only for musl. > The background is that the kernel artifact (linux.bin) which gets > included into the Fit image is no longer built by default. It requires > KERNEL_CLASSES += "kernel-fit-extra-artifacts". > > I guess that this !fixup patch is the solution: > > diff --git a/meta/classes-recipe/kernel-fit-image.bbclass > b/meta/classes-recipe/kernel-fit-image.bbclass > index 6d80cd4bb47..d2eebb88bc2 100644 > --- a/meta/classes-recipe/kernel-fit-image.bbclass > +++ b/meta/classes-recipe/kernel-fit-image.bbclass > @@ -7,6 +7,9 @@ UNPACKDIR = "${S}" > > PACKAGE_ARCH = "${MACHINE_ARCH}" > > +# This bbclass requires KERNEL_CLASSES += "kernel-fit-extra-artifacts" > +EXCLUDE_FROM_WORLD = "1" > + > DEPENDS += "\ > u-boot-tools-native dtc-native \ > > But I need to test it before sending a new version. And today I have no > access to my build machine. I just saw this and it matches my other email. I also just realised there are mutliple other world build failures. To expedite things, I'll include the above change in a test build. Cheers, Richard
On Tue, 2025-06-03 at 15:07 +0100, Richard Purdie via lists.openembedded.org wrote: > On Tue, 2025-06-03 at 13:35 +0000, Freihofer, Adrian wrote: > > My conclusion is that the change requested by this discussion > > https://lists.openembedded.org/g/openembedded-core/message/217730 > > makes the linux-yocto-fitimage.bb incompatible with bitbake world. > > I guess that bitbake world would now fail for all build configurations, > > not only for musl. > > The background is that the kernel artifact (linux.bin) which gets > > included into the Fit image is no longer built by default. It requires > > KERNEL_CLASSES += "kernel-fit-extra-artifacts". > > > > I guess that this !fixup patch is the solution: > > > > diff --git a/meta/classes-recipe/kernel-fit-image.bbclass > > b/meta/classes-recipe/kernel-fit-image.bbclass > > index 6d80cd4bb47..d2eebb88bc2 100644 > > --- a/meta/classes-recipe/kernel-fit-image.bbclass > > +++ b/meta/classes-recipe/kernel-fit-image.bbclass > > @@ -7,6 +7,9 @@ UNPACKDIR = "${S}" > > > > PACKAGE_ARCH = "${MACHINE_ARCH}" > > > > +# This bbclass requires KERNEL_CLASSES += "kernel-fit-extra-artifacts" > > +EXCLUDE_FROM_WORLD = "1" > > + > > DEPENDS += "\ > > u-boot-tools-native dtc-native \ > > > > But I need to test it before sending a new version. And today I have no > > access to my build machine. > > I just saw this and it matches my other email. I also just realised > there are mutliple other world build failures. To expedite things, I'll > include the above change in a test build. I an confirm this did build successfully. I'll just clean up my patch in master-next with the above in it? Cheers, Richard
Hi Richard The status on master-next looks exactly like what I wanted to send as a next version. Thank you very much! Adrian
On Tue, Jun 3, 2025 at 4:24 AM Adrian Freihofer via lists.openembedded.org <adrian.freihofer=siemens.com@lists.openembedded.org> wrote: > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > This patch series re-writes the FIT image related code. The goal is to > fix [YOCTO #12912] which is a long standing issue. > > Changes in comparison to v6 > --------------------------- > > * Move the code for exporting the FIT image specific kernel artifacts in a > new, > separate kernel-fit-extra-artifacts.bbclass rather than adding it to the > existing kernel-uimage.bbclass. > * Improve the devicetree.bbclass to skip error messages of shell globs > * Improve the error message in the kernel.bbclass to: "fitImage is no > longer > supported as a KERNEL_IMAGETYPE(S). FIT images are built by the > linux-yocto-fitimage recipe." > > Thanks Adrian, Now that you've sorted out that one build issue, this series looks good to me. Bruce > Revert a non essential refactoring in the kernel.bbclass > > Changes in comparison to v5 > --------------------------- > > * Improve the linux-yocto-fitimage recipe: > Replace LINUX_VERSION ?= "6.12.23" by > PKGV = "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}" > * Extend recipe for testing the external devicetree: bbb-dtbs-as-ext.bb > * am335x-bonegreen-ext.dts > Add a dtb file with a compatible string (before there was only a dtbo) > * am335x-bonegreen-ext-alias.dtb > Add a sym-link which should end up as an additional configuration node > in > the FIT image > * Fix handling of external device trees > * Create an extra configuration node for sym-links > * If an external devicetree has a compatible property add the same > property > to the corresponding FIT image configuration node. > * Sort the output of os.listdir > * Fix handling of FIT_CONF_DEFAULT_DTB > * Improve tests: > * FIT_CONF_PREFIX > * FIT_CONF_DEFAULT_DTB > * Add a test for an external devicetree (not only a dtbo) > * Add a test for an external devicetree sym-link > * Test the new get_compatible_from_dtb function > * Improve robustness of testiing FIT_GENERATE_KEYS = "1" > * Improve some minor finding in devicetree.bbclass > > Changes in comparison to v4 > --------------------------- > > * maintainers: add myself for kernel-signing-keys-native > oe-selftest -r distrodata.Distrodata.test_maintainers > > The intention is to provide exactly the state of today's master-next. > > Changes in comparison to v3 > --------------------------- > > * Fix ERROR: linux-yocto-6.12.27+git-r0 do_uboot_mkimage: Execution... > * Pass the kernel artifacts via the deploy directory. This makes the task > dependencies slightly more complicated. But passing the artifacts via > sysroot requires having some artifacts twice in the sstate-cache (once > in sysroot and once in deploy-dir). > * Support the bundled kernel+initramfs in FIT image again. After some > discussions, it turned out that some BSPs make use of this unusual > combination and dropping support for it would probably lead to issues. > Re-introducing this was also part of the motivation for passing the > artifacts via deploy rather than via sysroot. > * Drop the patch which merged kernel-uboot.bbclass into > kernel-uimage.bbclass. This can be done later if it seems useful. > * Drop doc patches again > > Changes in comparison to v2 > --------------------------- > > * Fix the multilib exclusion check linux-fit-image -> kernel-fit-image > This should solve the issue with bitbake world and multilib builds > > Changes in comparison to v1 > --------------------------- > > * Exclude recipes which inherit the kernel-fit-image.bbclass from > multilib builds. This fixes an issue discovered by the AB when > running bitbake world with a multilib configuraton enabled. > * Remove some new tests which expected openssl from the host. > * Rebase the patches on the new bug-fix commit > "u-boot: ensure keys are generated before assembling U-Boot FIT image" > from Rogerio Guerra Borin <rogerio.borin@toradex.com> > This fix required splitting the key generation into a separate recipe. > Note: These patches depend on this commit, which is included in this > series. As of today, the patch is already present on master-next. > * Add a cover letter with correct subject for the patch series > > What gets fixed > --------------- > > * sstate does not work well if a FIT image contains an initramfs. > The kernel gets re-built from scratch if the build runs from an > empty TMPDIR: > https://lists.openembedded.org/g/openembedded-core/message/203510. > This is also problematic for SDK use cases since working with the > SDK is annoying if the kernel gets re-built from scratch for no > good reason. > * A FIT image kernel is not available as a package, but all other > kernel image types are. > * The its-file is generated by complicated shell code with lots of > code duplications. Switching to Python simplifies the maintenance > (also because of the indentation with spaces and tabs in the shell > code). > * Separating the FIT image related complexity from the kernel build > complexity simplifies the maintenance of the kernel but also of > the FIT image related code. > * There is already a new (but unfortunately completely unaligned) > implementation for creating FIT images in meta-openembedded: > https://github.com/openembedded/meta-openembedded/blob/master/meta-oe/ > classes/fitimage.bbclass. Let's hope this patch series will lay a > solid foundation for a future merge of the two implementations. > * The new implementation in Python is also a preparation for additional > features, such as adding different types of artifacts or generating > the configuration nodes with greater flexibility. Currently, exactly > one configuration per device node is supported. > > Architectural change > -------------------- > > The existing kernel-fitimage.bbclass is designed to be added to > KERNEL_CLASSES. It appends code to the kernel's tasks and injects > additional tasks between the existing ones. Some functions rely on > running within the kernel's build folder structure. > > The new implementation introduces the kernel-fit-image.bbclass, > which is intended to be inherited by an independent recipe. This > recipe takes the kernel artifacts from the sstate-cache and assembles > the FIT image entirely independently of the kernel's build tasks and > directory structure. > > An example of using the new kernel-fit-image.bbclass is the > linux-yocto-fitimage.bb recipe, which builds the FIT image for the > linux-yocto kernel. The recipe looks like this: > > SUMMARY = "The Linux kernel as a FIT image (optionally with initramfs)" > SECTION = "kernel" > LICENSE = "GPL-2.0-with-Linux-syscall-note" > LIC_FILES_CHKSUM = "\ > > file://${COREBASE}/meta/files/common-licenses/GPL-2.0-with-Linux-syscall-note; > md5=0bad96c422c41c3a94009dcfe1bff992" > > inherit kernel-fit-image > > The configuration variables defined in the conf/image-fitimage.conf > file are handled by the kernel-fit-image.bbclass in the same way as > they are by the kernel-fitimage.bbclass. The new implementation is > 99% backward compatible with the existing kernel-fitimage.bbclass. > The existing test-suite runs with minimal changes. > > With the kernel-fitimage.bbclass, the FIT image was built as part of > the kernel itself. To ensure the new recipe is built automatically, > the following variables can be set, for example, in the machine > configuration file: > > # Do not install the kernel image package > RRECOMMENDS:${KERNEL_PACKAGE_NAME}-base = "" > > # Install the FIT image package into the rootfs (there is now a package > :-) > MACHINE_EXTRA_RDEPENDS += "${PREFERRED_PROVIDER_virtual/kernel}-fitimage" > > # Configure the image.bbclass to depend on the fitImage instead of only > # the kernel to ensure the FIT image is built with the image > KERNEL_DEPLOY_DEPEND = > "${PREFERRED_PROVIDER_virtual/kernel}-fitimage:do_deploy" > > Breaking changes > ---------------- > > * Building a kernel FIT image changes. > Before this patch series: > > A configuration like: > KERNEL_IMAGETYPE = "fitImage" > KERNEL_CLASSES = "kernel-fitimage" > and building a kernel like: > bitbake linux-yocto > generated a FIT image including the kernel and maybe additional > artifacts. > > With this patch series merged, the same can be achieved like this: > bitbake linux-yocto-fitimage > > While this simple example is even simpler than before, there are > also other examples which might look more complicated with the > new implementation than with the old implementation. > > Proposal for merging these patch series > --------------------------------------- > > The patches are split into small steps which allow a step-by-step > merging: > > 1. New tests without impact on code: > 2. Add the new implementation without changing the existing implementation > 3. Refactor the old kernel-fitimage.bbclass to Python to share code > 4. Remove the old kernel-fitimage.bbclass and clean up the code from > > Adrian Freihofer (20): > devicetree: minor improvements > oe-selftest: add new ext dtb recipe > oe-selftest: fitimage: test external dtb > oe-selftest: fitimage: test FIT_CONF_PREFIX > oe-selftest: fitimage: test FIT_CONF_DEFAULT_DTB > kernel-signing-keys-native: refactor key generation into a new recipe > maintainers: add myself for kernel-signing-keys-native > oe-selftest: fitimage: cleanup FIT_GENERATE_KEYS > kernel-fitimage: refactor order in its > kernel-uboot.bbclass: do not require the kernel build folder > kernel-fit-image.bbclass: add a new FIT image implementation > maintainers: add myself for linux-yocto-fitimage > oe-selftest: fitimage: add tests for fitimage.py > oe-selftest: fitimage: support new FIT recipe as well > oe-selftest: fitimage: run all tests for both FIT implementations > oe-selftest: fitimage refactor classes > kernel-fitimage: re-write its code in Python > oe-selftest: fitimage: remove kernel-fitimage tests > kernel.bbclass: remove support for type fitImage > kernel-fitimage.bbclass: remove it > > .../recipes-test/ext-dtb/bbb-dtbs-as-ext.bb | 29 + > .../ext-dtb/files/BBORG_RELAY-00A2.dts | 49 + > .../ext-dtb/files/am335x-bonegreen-ext.dts | 14 + > meta/classes-recipe/devicetree.bbclass | 22 +- > .../kernel-fit-extra-artifacts.bbclass | 19 + > meta/classes-recipe/kernel-fit-image.bbclass | 187 ++++ > meta/classes-recipe/kernel-fitimage.bbclass | 839 ------------------ > meta/classes-recipe/kernel-uboot.bbclass | 38 +- > meta/classes-recipe/kernel-uimage.bbclass | 3 +- > meta/classes-recipe/kernel.bbclass | 18 +- > meta/classes-recipe/uboot-sign.bbclass | 5 +- > meta/classes/multilib.bbclass | 1 + > meta/conf/distro/include/maintainers.inc | 2 + > meta/lib/oe/fitimage.py | 547 ++++++++++++ > meta/lib/oeqa/selftest/cases/fitimage.py | 365 ++++++-- > .../kernel-signing-keys-native.bb | 75 ++ > .../linux/linux-yocto-fitimage.bb | 13 + > 17 files changed, 1286 insertions(+), 940 deletions(-) > create mode 100644 meta-selftest/recipes-test/ext-dtb/bbb-dtbs-as-ext.bb > create mode 100644 > meta-selftest/recipes-test/ext-dtb/files/BBORG_RELAY-00A2.dts > create mode 100644 > meta-selftest/recipes-test/ext-dtb/files/am335x-bonegreen-ext.dts > create mode 100644 meta/classes-recipe/kernel-fit-extra-artifacts.bbclass > create mode 100644 meta/classes-recipe/kernel-fit-image.bbclass > delete mode 100644 meta/classes-recipe/kernel-fitimage.bbclass > create mode 100644 meta/lib/oe/fitimage.py > create mode 100644 meta/recipes-kernel/kernel-signing-keys/ > kernel-signing-keys-native.bb > create mode 100644 meta/recipes-kernel/linux/linux-yocto-fitimage.bb > > -- > 2.49.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#217767): > https://lists.openembedded.org/g/openembedded-core/message/217767 > Mute This Topic: https://lists.openembedded.org/mt/113443923/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Hello Adrian, On Tue, Jun 03, 2025 at 10:23:11AM +0200, AdrianF wrote: > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > This patch series re-writes the FIT image related code. The goal is to > fix [YOCTO #12912] which is a long standing issue. I did not have time to look into the details yet, but it seems that this breaks build with meta-ti. | 06-05:18:06:33 ERROR: ParseError at /workdir/oe/build/../layers/openembedded-core/meta/classes-recipe/kernel.bbclass:183: Could not inherit file classes/kernel-fitimage.bbclass | 06-05:18:06:33 ERROR: Parsing halted due to errors, see error messages above https://gist.github.com/dolcini/8240a229516e7a93356ebba56631d3c0 Francesco
Hi Francesco Congratulate, your CI is faster than I am with updating the documentation :-). But in the mean time published the corresponding documentation update too: https://lists.yoctoproject.org/g/docs/message/7017 Yes, unfortunately there is a breaking change. The modification has undergone extensive review through several months of technical meetings and patch review sessions. A very brief migration guide is here. Coming from a configuration like: KERNEL_IMAGETYPE = "fitImage" KERNEL_CLASSES = "kernel-fitimage" and building a kernel like: bitbake linux-yocto needs to be changed to: KERNEL_CLASSES = "kernel-fit-extra-artifacts" and building a kernel like: bitbake linux-yocto-fitimage I hope that helps. Otherwise, please let me know. Adrian On Fri, 2025-06-06 at 12:29 +0200, Francesco Dolcini wrote: > Hello Adrian, > > On Tue, Jun 03, 2025 at 10:23:11AM +0200, AdrianF wrote: > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > This patch series re-writes the FIT image related code. The goal is > > to > > fix [YOCTO #12912] which is a long standing issue. > > I did not have time to look into the details yet, but it seems that > this > breaks build with meta-ti. > > > 06-05:18:06:33 ERROR: ParseError at > > /workdir/oe/build/../layers/openembedded-core/meta/classes- > > recipe/kernel.bbclass:183: Could not inherit file classes/kernel- > > fitimage.bbclass > > 06-05:18:06:33 ERROR: Parsing halted due to errors, see error > > messages above > > https://gist.github.com/dolcini/8240a229516e7a93356ebba56631d3c0 > > Francesco >
On Fri, Jun 06, 2025 at 02:11:57PM +0000, Adrian Freihofer via lists.yoctoproject.org wrote: > Hi Francesco > > Congratulate, your CI is faster than I am with updating the > documentation :-). But in the mean time published the corresponding > documentation update too: > https://lists.yoctoproject.org/g/docs/message/7017 > > Yes, unfortunately there is a breaking change. The modification has > undergone extensive review through several months of technical meetings > and patch review sessions. A very brief migration guide is here. > > Coming from a configuration like: > KERNEL_IMAGETYPE = "fitImage" > KERNEL_CLASSES = "kernel-fitimage" > and building a kernel like: > bitbake linux-yocto > > needs to be changed to: > KERNEL_CLASSES = "kernel-fit-extra-artifacts" > and building a kernel like: > bitbake linux-yocto-fitimage Quick question here - can you still produce multple kernel types in a single go from a single kernel recipe? E.g. old KERNEL_IMAGETYPES = "Image fitImage" Or are those now mutually exclusive? > I hope that helps. Otherwise, please let me know. > Adrian > > > > > On Fri, 2025-06-06 at 12:29 +0200, Francesco Dolcini wrote: > > Hello Adrian, > > > > On Tue, Jun 03, 2025 at 10:23:11AM +0200, AdrianF wrote: > > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > > > This patch series re-writes the FIT image related code. The goal is > > > to > > > fix [YOCTO #12912] which is a long standing issue. > > > > I did not have time to look into the details yet, but it seems that > > this > > breaks build with meta-ti. > > > > > 06-05:18:06:33 ERROR: ParseError at > > > /workdir/oe/build/../layers/openembedded-core/meta/classes- > > > recipe/kernel.bbclass:183: Could not inherit file classes/kernel- > > > fitimage.bbclass > > > 06-05:18:06:33 ERROR: Parsing halted due to errors, see error > > > messages above > > > > https://gist.github.com/dolcini/8240a229516e7a93356ebba56631d3c0 > > > > Francesco > >
On Fri, 2025-06-06 at 11:52 -0400, Denys Dmytriyenko wrote: > On Fri, Jun 06, 2025 at 02:11:57PM +0000, Adrian Freihofer via > lists.yoctoproject.org wrote: > > Hi Francesco > > > > Congratulate, your CI is faster than I am with updating the > > documentation :-). But in the mean time published the corresponding > > documentation update too: > > https://lists.yoctoproject.org/g/docs/message/7017 > > > > Yes, unfortunately there is a breaking change. The modification has > > undergone extensive review through several months of technical > > meetings > > and patch review sessions. A very brief migration guide is here. > > > > Coming from a configuration like: > > KERNEL_IMAGETYPE = "fitImage" > > KERNEL_CLASSES = "kernel-fitimage" > > and building a kernel like: > > bitbake linux-yocto > > > > needs to be changed to: > > KERNEL_CLASSES = "kernel-fit-extra-artifacts" > > and building a kernel like: > > bitbake linux-yocto-fitimage > > Quick question here - can you still produce multple kernel types in a > single > go from a single kernel recipe? E.g. old KERNEL_IMAGETYPES = "Image > fitImage" > > Or are those now mutually exclusive? Everything should be possible. The only little detail which changes is that the kernel.bbclass does no longer handle the KERNEL_IMAGETYPE(S) fitImage. For example if you do: KERNEL_IMAGETYPES = "Image zImage bzImage uImage" and call bitbake linux-yocto You get all these images as packages as well as deployed to DEPLOY_DIR_IMAGE. Nothing changes. if you call bitbake linux-yocto-fitimage the same happens. Additionally you also get the fitImage as a package (which is new) and the fitImage in the DEPLOY_DIR_IMAGE. Adrian > > > > I hope that helps. Otherwise, please let me know. > > Adrian > > > > > > > > > > On Fri, 2025-06-06 at 12:29 +0200, Francesco Dolcini wrote: > > > Hello Adrian, > > > > > > On Tue, Jun 03, 2025 at 10:23:11AM +0200, AdrianF wrote: > > > > From: Adrian Freihofer <adrian.freihofer@siemens.com> > > > > > > > > This patch series re-writes the FIT image related code. The > > > > goal is > > > > to > > > > fix [YOCTO #12912] which is a long standing issue. > > > > > > I did not have time to look into the details yet, but it seems > > > that > > > this > > > breaks build with meta-ti. > > > > > > > 06-05:18:06:33 ERROR: ParseError at > > > > /workdir/oe/build/../layers/openembedded-core/meta/classes- > > > > recipe/kernel.bbclass:183: Could not inherit file > > > > classes/kernel- > > > > fitimage.bbclass > > > > 06-05:18:06:33 ERROR: Parsing halted due to errors, see error > > > > messages above > > > > > > https://gist.github.com/dolcini/8240a229516e7a93356ebba56631d3c0 > > > > > > Francesco > > >
From: Adrian Freihofer <adrian.freihofer@siemens.com> This patch series re-writes the FIT image related code. The goal is to fix [YOCTO #12912] which is a long standing issue. Changes in comparison to v6 --------------------------- * Move the code for exporting the FIT image specific kernel artifacts in a new, separate kernel-fit-extra-artifacts.bbclass rather than adding it to the existing kernel-uimage.bbclass. * Improve the devicetree.bbclass to skip error messages of shell globs * Improve the error message in the kernel.bbclass to: "fitImage is no longer supported as a KERNEL_IMAGETYPE(S). FIT images are built by the linux-yocto-fitimage recipe." Revert a non essential refactoring in the kernel.bbclass Changes in comparison to v5 --------------------------- * Improve the linux-yocto-fitimage recipe: Replace LINUX_VERSION ?= "6.12.23" by PKGV = "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}" * Extend recipe for testing the external devicetree: bbb-dtbs-as-ext.bb * am335x-bonegreen-ext.dts Add a dtb file with a compatible string (before there was only a dtbo) * am335x-bonegreen-ext-alias.dtb Add a sym-link which should end up as an additional configuration node in the FIT image * Fix handling of external device trees * Create an extra configuration node for sym-links * If an external devicetree has a compatible property add the same property to the corresponding FIT image configuration node. * Sort the output of os.listdir * Fix handling of FIT_CONF_DEFAULT_DTB * Improve tests: * FIT_CONF_PREFIX * FIT_CONF_DEFAULT_DTB * Add a test for an external devicetree (not only a dtbo) * Add a test for an external devicetree sym-link * Test the new get_compatible_from_dtb function * Improve robustness of testiing FIT_GENERATE_KEYS = "1" * Improve some minor finding in devicetree.bbclass Changes in comparison to v4 --------------------------- * maintainers: add myself for kernel-signing-keys-native oe-selftest -r distrodata.Distrodata.test_maintainers The intention is to provide exactly the state of today's master-next. Changes in comparison to v3 --------------------------- * Fix ERROR: linux-yocto-6.12.27+git-r0 do_uboot_mkimage: Execution... * Pass the kernel artifacts via the deploy directory. This makes the task dependencies slightly more complicated. But passing the artifacts via sysroot requires having some artifacts twice in the sstate-cache (once in sysroot and once in deploy-dir). * Support the bundled kernel+initramfs in FIT image again. After some discussions, it turned out that some BSPs make use of this unusual combination and dropping support for it would probably lead to issues. Re-introducing this was also part of the motivation for passing the artifacts via deploy rather than via sysroot. * Drop the patch which merged kernel-uboot.bbclass into kernel-uimage.bbclass. This can be done later if it seems useful. * Drop doc patches again Changes in comparison to v2 --------------------------- * Fix the multilib exclusion check linux-fit-image -> kernel-fit-image This should solve the issue with bitbake world and multilib builds Changes in comparison to v1 --------------------------- * Exclude recipes which inherit the kernel-fit-image.bbclass from multilib builds. This fixes an issue discovered by the AB when running bitbake world with a multilib configuraton enabled. * Remove some new tests which expected openssl from the host. * Rebase the patches on the new bug-fix commit "u-boot: ensure keys are generated before assembling U-Boot FIT image" from Rogerio Guerra Borin <rogerio.borin@toradex.com> This fix required splitting the key generation into a separate recipe. Note: These patches depend on this commit, which is included in this series. As of today, the patch is already present on master-next. * Add a cover letter with correct subject for the patch series What gets fixed --------------- * sstate does not work well if a FIT image contains an initramfs. The kernel gets re-built from scratch if the build runs from an empty TMPDIR: https://lists.openembedded.org/g/openembedded-core/message/203510. This is also problematic for SDK use cases since working with the SDK is annoying if the kernel gets re-built from scratch for no good reason. * A FIT image kernel is not available as a package, but all other kernel image types are. * The its-file is generated by complicated shell code with lots of code duplications. Switching to Python simplifies the maintenance (also because of the indentation with spaces and tabs in the shell code). * Separating the FIT image related complexity from the kernel build complexity simplifies the maintenance of the kernel but also of the FIT image related code. * There is already a new (but unfortunately completely unaligned) implementation for creating FIT images in meta-openembedded: https://github.com/openembedded/meta-openembedded/blob/master/meta-oe/ classes/fitimage.bbclass. Let's hope this patch series will lay a solid foundation for a future merge of the two implementations. * The new implementation in Python is also a preparation for additional features, such as adding different types of artifacts or generating the configuration nodes with greater flexibility. Currently, exactly one configuration per device node is supported. Architectural change -------------------- The existing kernel-fitimage.bbclass is designed to be added to KERNEL_CLASSES. It appends code to the kernel's tasks and injects additional tasks between the existing ones. Some functions rely on running within the kernel's build folder structure. The new implementation introduces the kernel-fit-image.bbclass, which is intended to be inherited by an independent recipe. This recipe takes the kernel artifacts from the sstate-cache and assembles the FIT image entirely independently of the kernel's build tasks and directory structure. An example of using the new kernel-fit-image.bbclass is the linux-yocto-fitimage.bb recipe, which builds the FIT image for the linux-yocto kernel. The recipe looks like this: SUMMARY = "The Linux kernel as a FIT image (optionally with initramfs)" SECTION = "kernel" LICENSE = "GPL-2.0-with-Linux-syscall-note" LIC_FILES_CHKSUM = "\ file://${COREBASE}/meta/files/common-licenses/GPL-2.0-with-Linux-syscall-note; md5=0bad96c422c41c3a94009dcfe1bff992" inherit kernel-fit-image The configuration variables defined in the conf/image-fitimage.conf file are handled by the kernel-fit-image.bbclass in the same way as they are by the kernel-fitimage.bbclass. The new implementation is 99% backward compatible with the existing kernel-fitimage.bbclass. The existing test-suite runs with minimal changes. With the kernel-fitimage.bbclass, the FIT image was built as part of the kernel itself. To ensure the new recipe is built automatically, the following variables can be set, for example, in the machine configuration file: # Do not install the kernel image package RRECOMMENDS:${KERNEL_PACKAGE_NAME}-base = "" # Install the FIT image package into the rootfs (there is now a package :-) MACHINE_EXTRA_RDEPENDS += "${PREFERRED_PROVIDER_virtual/kernel}-fitimage" # Configure the image.bbclass to depend on the fitImage instead of only # the kernel to ensure the FIT image is built with the image KERNEL_DEPLOY_DEPEND = "${PREFERRED_PROVIDER_virtual/kernel}-fitimage:do_deploy" Breaking changes ---------------- * Building a kernel FIT image changes. Before this patch series: A configuration like: KERNEL_IMAGETYPE = "fitImage" KERNEL_CLASSES = "kernel-fitimage" and building a kernel like: bitbake linux-yocto generated a FIT image including the kernel and maybe additional artifacts. With this patch series merged, the same can be achieved like this: bitbake linux-yocto-fitimage While this simple example is even simpler than before, there are also other examples which might look more complicated with the new implementation than with the old implementation. Proposal for merging these patch series --------------------------------------- The patches are split into small steps which allow a step-by-step merging: 1. New tests without impact on code: 2. Add the new implementation without changing the existing implementation 3. Refactor the old kernel-fitimage.bbclass to Python to share code 4. Remove the old kernel-fitimage.bbclass and clean up the code from Adrian Freihofer (20): devicetree: minor improvements oe-selftest: add new ext dtb recipe oe-selftest: fitimage: test external dtb oe-selftest: fitimage: test FIT_CONF_PREFIX oe-selftest: fitimage: test FIT_CONF_DEFAULT_DTB kernel-signing-keys-native: refactor key generation into a new recipe maintainers: add myself for kernel-signing-keys-native oe-selftest: fitimage: cleanup FIT_GENERATE_KEYS kernel-fitimage: refactor order in its kernel-uboot.bbclass: do not require the kernel build folder kernel-fit-image.bbclass: add a new FIT image implementation maintainers: add myself for linux-yocto-fitimage oe-selftest: fitimage: add tests for fitimage.py oe-selftest: fitimage: support new FIT recipe as well oe-selftest: fitimage: run all tests for both FIT implementations oe-selftest: fitimage refactor classes kernel-fitimage: re-write its code in Python oe-selftest: fitimage: remove kernel-fitimage tests kernel.bbclass: remove support for type fitImage kernel-fitimage.bbclass: remove it .../recipes-test/ext-dtb/bbb-dtbs-as-ext.bb | 29 + .../ext-dtb/files/BBORG_RELAY-00A2.dts | 49 + .../ext-dtb/files/am335x-bonegreen-ext.dts | 14 + meta/classes-recipe/devicetree.bbclass | 22 +- .../kernel-fit-extra-artifacts.bbclass | 19 + meta/classes-recipe/kernel-fit-image.bbclass | 187 ++++ meta/classes-recipe/kernel-fitimage.bbclass | 839 ------------------ meta/classes-recipe/kernel-uboot.bbclass | 38 +- meta/classes-recipe/kernel-uimage.bbclass | 3 +- meta/classes-recipe/kernel.bbclass | 18 +- meta/classes-recipe/uboot-sign.bbclass | 5 +- meta/classes/multilib.bbclass | 1 + meta/conf/distro/include/maintainers.inc | 2 + meta/lib/oe/fitimage.py | 547 ++++++++++++ meta/lib/oeqa/selftest/cases/fitimage.py | 365 ++++++-- .../kernel-signing-keys-native.bb | 75 ++ .../linux/linux-yocto-fitimage.bb | 13 + 17 files changed, 1286 insertions(+), 940 deletions(-) create mode 100644 meta-selftest/recipes-test/ext-dtb/bbb-dtbs-as-ext.bb create mode 100644 meta-selftest/recipes-test/ext-dtb/files/BBORG_RELAY-00A2.dts create mode 100644 meta-selftest/recipes-test/ext-dtb/files/am335x-bonegreen-ext.dts create mode 100644 meta/classes-recipe/kernel-fit-extra-artifacts.bbclass create mode 100644 meta/classes-recipe/kernel-fit-image.bbclass delete mode 100644 meta/classes-recipe/kernel-fitimage.bbclass create mode 100644 meta/lib/oe/fitimage.py create mode 100644 meta/recipes-kernel/kernel-signing-keys/kernel-signing-keys-native.bb create mode 100644 meta/recipes-kernel/linux/linux-yocto-fitimage.bb