| Message ID | 20250213-multiconfig-doc-v2-6-f2bd35115cb7@bootlin.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Improve multiconfig documentation | expand |
Hi Antonin, On 2/13/25 9:23 AM, Antonin Godard via lists.yoctoproject.org wrote: > After the suggestions from Mark Hatle on the list > (https://lists.yoctoproject.org/g/docs/topic/110487932), add two > sections to the multiconfig doc: > > - Suggested best practices: suggestion for better design of multiconfig > builds. > > - Common use case: baremetal build. > > This section applies the guidelines from the first sections and apply > it to a real-life example of how to use multiconfig. This one to build > some baremetal firmware alongside a regular Linux build. > > Suggested-by: Mark Hatle <mark.hatle@kernel.crashing.org> > Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> > --- > documentation/dev-manual/multiconfig.rst | 136 +++++++++++++++++++++++++++++++ > 1 file changed, 136 insertions(+) > > diff --git a/documentation/dev-manual/multiconfig.rst b/documentation/dev-manual/multiconfig.rst > index 49e32b04a..cdf06d2ca 100644 > --- a/documentation/dev-manual/multiconfig.rst > +++ b/documentation/dev-manual/multiconfig.rst > @@ -171,3 +171,139 @@ and have separate configuration files, BitBake places the artifacts for > each build in the respective temporary build directories (i.e. > :term:`TMPDIR`). > > +Suggested best practices > +======================== > + > +- :term:`TMPDIR` (other than the default set in bitbake.conf) is only set in > + ``local.conf`` by the user. This means that we should **not** manipulate > + :term:`TMPDIR` in any way within the Machine or Distro :term:`configuration > + file`. > + > +- A multiconfig should specify a :term:`TMPDIR`, and should specify it by > + appending the multiconfig name via :term:`BB_CURRENT_MC`. s/via/with/ > + > +- Recipes that are used to transfer the output from a multiconfig build to > + another should use ``do_task[mcdepends]`` to trigger the build of the > + component, and then transfer the item to the current configuration in > + :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy`, assuming the value of > + the deployed item based on :term:`TMPDIR`. > + > + The :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy` tasks should look > + like this:: > + > + do_install() { > + install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${D}/some/path > + } > + > + do_deploy() { > + install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${DEPLOYDIR}/somefile > + } > + > + In the examples above: > + s/examples/example/ > + - ``<multiconfig>`` is the multiconfig name as set by the multiconfig > + :term:`configuration file` (see the :ref:`dev-manual/multiconfig:Setting > + Up and Running a Multiple Configuration Build` section above). > + > + - ``<machine>`` must be the :term:`MACHINE` for which ``somefile`` was built > + and deployed. This value depends on whether the multiconfig > + :term:`configuration file` is overriding the :term:`MACHINE` value or not. > + I can suggest: """ This value may differ from the current :term:`MACHINE` if the multiconfig :term:`configuration file` overrides it. """ > +- Firmware recipes can set the :term:`INHIBIT_DEFAULT_DEPS` variable to ``1`` > + if they don't rely on default dependencies such as the standard C library. > + > +Common use case: building baremetal firmware alongside a Linux build > +==================================================================== > + > +A common use for multiconfig is to use the default configuration as the regular s/use/use case/ ? > +Linux build, while one or more multiconfigs can be used to build special > +components, such as baremetal firmware. It would also apply to a scenario where > +a microcontroller, for example, is companion to a main processor where Linux is > +running. This section details how one can achieve these kinds of scenarios with > +a multiconfig build. > + > +Adding a multiconfig configuration file and recipe for a baremetal firmware > +--------------------------------------------------------------------------- > + > +As described in :ref:`dev-manual/multiconfig:Setting Up and Running a Multiple > +Configuration Build`, each multiconfig will require a separate > +:term:`Configuration File`. In our case, we will make a separate temporary > +directory for our baremetal firmware build configuration. > + Mmmm not sure what "separate temporary directory" should mean here? Are you talking about the conf/multiconfig/ directory or TMPDIR? > +For example, we will define a new ``conf/multiconfig/baremetal-firmware.conf`` > +as follows:: > + > + TMPDIR .= "-${BB_CURRENT_MC}" > + TCLIBC = "newlib" > + > +The ``baremetal-firmware.conf`` configure a separate :term:`TMPDIR` for holding s/configure/configures/ > +binaries compiled with the `newlib <https://sourceware.org/newlib/>`__ toolchain > +(see :term:`TCLIBC`). > + > +.. note:: > + > + Here, the default :term:`MACHINE` is not overriden by the multiconfig s/overriden/overridden/ > + configuration file. As a consequence, the architecture of the built baremetal > + binaries will be the same. In other cases, where the firmware runs on a s/the same/the same as the one for the Linux build/ > + completely different architecture, the :term:`MACHINE` may be overriden. s/may/must/ s/overriden/overridden/ > + > +We then create a recipe ``my-firmware.bb`` that defines how the baremetal > +firmware is built. The recipe should contain enough information for the > +:term:`OpenEmbedded build system` to properly compile the firmware with our > +toolchain. The building tasks may vary depending on the nature of the firmware. > +However, the recipe should define a :ref:`ref-classes-deploy` task that deploys > +the output into the :term:`DEPLOYDIR` directory. We will consider in the > +following that the file is named ``my-firmware.elf``. > + > +Building the firmware > +--------------------- > + > +The firmware can be built with BitBake with the following command:: > + > + $ bitbake mc:baremetal-firmware:my-firmware > + > +However, we would prefer for ``my-firmware`` to be automatically built when > +triggering a normal Linux build. > + > +Using an ``mcdepend``, a recipe belonging to the Linux build can trigger the I read mcdepend as multiconfigdepend or macdepend, and not em-see-depend. So I would use "a" and not "an" but no strong opinion. > +build of ``my-firmware``. For example, let's consider that our Linux build needs > +to assemble a "special" firmware that uses the output of our ``my-firmware`` > +recipe - let's call it ``my-parent-firmware.bb``. Then, we should specify this > +dependency in ``my-parent-firmware.bb`` with:: > + > + do_compile[mcdepends] = "mc::baremetal-firmware:my-firmware:do_deploy" > + > +The above will ensure that when the :ref:`ref-tasks-compile` task of > +``my-parent-firmware`` is triggered, the :ref:`ref-tasks-deploy` task of > +``my-firmware`` will already have run successfully. > + > +Using the output of ``my-firmware`` > +----------------------------------- > + > +After we have deployed ``my-firmware`` by using ``mcdepends``, we need to use I can suggest: """ After ``my-firmware`` recipe has deployed `my-firmware.elf`, we need to use """ otherwise it may imply that mcdepends it what deployed my-firmware which isn't the case, it just made sure my-firmware:do_deploy is finished before my-parent-firmware:do_compile starts. > +the output in some way. We can make a series of assumptions, based on the > +default Yocto Project variables in order to get the binary for packaging. > + > +First, we can set the following in ``my-parent-firmware.bb``:: > + > + FIRMWARE_FILE ??= "${TMPDIR}-baremetal-firmware/deploy/images/<machine>/my-firmware.elf" > + FIRMWARE_FILE[vardepsexclude] += "TMPDIR" > + > +The first assignment stores the value of the path to the firmware built and > +deployed by the ``my-firmware.bb`` recipe. The second assignment excludes the > +:term:`TMPDIR` variable from being part of ``FIRMWARE_FILE``'s dependencies - Michael really wanted to use em-dashes, so maybe use it here? i.e. --- c.f. c29a1bf4dd52 ("profile-manual: usage.rst: formatting fixes") Cheers, Quentin
Hi Quentin, On Mon Feb 17, 2025 at 1:42 PM CET, Quentin Schulz wrote: > Hi Antonin, > > On 2/13/25 9:23 AM, Antonin Godard via lists.yoctoproject.org wrote: >> After the suggestions from Mark Hatle on the list >> (https://lists.yoctoproject.org/g/docs/topic/110487932), add two >> sections to the multiconfig doc: >> >> - Suggested best practices: suggestion for better design of multiconfig >> builds. >> >> - Common use case: baremetal build. >> >> This section applies the guidelines from the first sections and apply >> it to a real-life example of how to use multiconfig. This one to build >> some baremetal firmware alongside a regular Linux build. >> >> Suggested-by: Mark Hatle <mark.hatle@kernel.crashing.org> >> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> >> --- >> documentation/dev-manual/multiconfig.rst | 136 +++++++++++++++++++++++++++++++ >> 1 file changed, 136 insertions(+) >> >> diff --git a/documentation/dev-manual/multiconfig.rst b/documentation/dev-manual/multiconfig.rst >> index 49e32b04a..cdf06d2ca 100644 >> --- a/documentation/dev-manual/multiconfig.rst >> +++ b/documentation/dev-manual/multiconfig.rst >> @@ -171,3 +171,139 @@ and have separate configuration files, BitBake places the artifacts for >> each build in the respective temporary build directories (i.e. >> :term:`TMPDIR`). >> >> +Suggested best practices >> +======================== >> + >> +- :term:`TMPDIR` (other than the default set in bitbake.conf) is only set in >> + ``local.conf`` by the user. This means that we should **not** manipulate >> + :term:`TMPDIR` in any way within the Machine or Distro :term:`configuration >> + file`. >> + >> +- A multiconfig should specify a :term:`TMPDIR`, and should specify it by >> + appending the multiconfig name via :term:`BB_CURRENT_MC`. > > s/via/with/ > >> + >> +- Recipes that are used to transfer the output from a multiconfig build to >> + another should use ``do_task[mcdepends]`` to trigger the build of the >> + component, and then transfer the item to the current configuration in >> + :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy`, assuming the value of >> + the deployed item based on :term:`TMPDIR`. >> + >> + The :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy` tasks should look >> + like this:: >> + >> + do_install() { >> + install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${D}/some/path >> + } >> + >> + do_deploy() { >> + install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${DEPLOYDIR}/somefile >> + } >> + >> + In the examples above: >> + > > s/examples/example/ > >> + - ``<multiconfig>`` is the multiconfig name as set by the multiconfig >> + :term:`configuration file` (see the :ref:`dev-manual/multiconfig:Setting >> + Up and Running a Multiple Configuration Build` section above). >> + >> + - ``<machine>`` must be the :term:`MACHINE` for which ``somefile`` was built >> + and deployed. This value depends on whether the multiconfig >> + :term:`configuration file` is overriding the :term:`MACHINE` value or not. >> + > > I can suggest: > > """ > This value may differ from the current :term:`MACHINE` if the > multiconfig :term:`configuration file` overrides it. > """ > >> +- Firmware recipes can set the :term:`INHIBIT_DEFAULT_DEPS` variable to ``1`` >> + if they don't rely on default dependencies such as the standard C library. >> + >> +Common use case: building baremetal firmware alongside a Linux build >> +==================================================================== >> + >> +A common use for multiconfig is to use the default configuration as the regular > > s/use/use case/ ? > >> +Linux build, while one or more multiconfigs can be used to build special >> +components, such as baremetal firmware. It would also apply to a scenario where >> +a microcontroller, for example, is companion to a main processor where Linux is >> +running. This section details how one can achieve these kinds of scenarios with >> +a multiconfig build. >> + >> +Adding a multiconfig configuration file and recipe for a baremetal firmware >> +--------------------------------------------------------------------------- >> + >> +As described in :ref:`dev-manual/multiconfig:Setting Up and Running a Multiple >> +Configuration Build`, each multiconfig will require a separate >> +:term:`Configuration File`. In our case, we will make a separate temporary >> +directory for our baremetal firmware build configuration. >> + > > Mmmm not sure what "separate temporary directory" should mean here? Are > you talking about the conf/multiconfig/ directory or TMPDIR? Yes, that wasn't clear. I was talking about the TMPDIR. Will rephrase to: """ In addition, we will define a separate :term:`TMPDIR` for our baremetal firmware build configuration. """ >> +For example, we will define a new ``conf/multiconfig/baremetal-firmware.conf`` >> +as follows:: >> + >> + TMPDIR .= "-${BB_CURRENT_MC}" >> + TCLIBC = "newlib" >> + >> +The ``baremetal-firmware.conf`` configure a separate :term:`TMPDIR` for holding > > s/configure/configures/ > >> +binaries compiled with the `newlib <https://sourceware.org/newlib/>`__ toolchain >> +(see :term:`TCLIBC`). >> + >> +.. note:: >> + >> + Here, the default :term:`MACHINE` is not overriden by the multiconfig > > s/overriden/overridden/ > >> + configuration file. As a consequence, the architecture of the built baremetal >> + binaries will be the same. In other cases, where the firmware runs on a > > s/the same/the same as the one for the Linux build/ > >> + completely different architecture, the :term:`MACHINE` may be overriden. > > s/may/must/ > > s/overriden/overridden/ > >> + >> +We then create a recipe ``my-firmware.bb`` that defines how the baremetal >> +firmware is built. The recipe should contain enough information for the >> +:term:`OpenEmbedded build system` to properly compile the firmware with our >> +toolchain. The building tasks may vary depending on the nature of the firmware. >> +However, the recipe should define a :ref:`ref-classes-deploy` task that deploys >> +the output into the :term:`DEPLOYDIR` directory. We will consider in the >> +following that the file is named ``my-firmware.elf``. >> + >> +Building the firmware >> +--------------------- >> + >> +The firmware can be built with BitBake with the following command:: >> + >> + $ bitbake mc:baremetal-firmware:my-firmware >> + >> +However, we would prefer for ``my-firmware`` to be automatically built when >> +triggering a normal Linux build. >> + >> +Using an ``mcdepend``, a recipe belonging to the Linux build can trigger the > > I read mcdepend as multiconfigdepend or macdepend, and not > em-see-depend. So I would use "a" and not "an" but no strong opinion. Ha, that's right, I read it as em-see-depend. It's probably multiconfigdepends tbh, so I'll swap to "a". >> +build of ``my-firmware``. For example, let's consider that our Linux build needs >> +to assemble a "special" firmware that uses the output of our ``my-firmware`` >> +recipe - let's call it ``my-parent-firmware.bb``. Then, we should specify this >> +dependency in ``my-parent-firmware.bb`` with:: >> + >> + do_compile[mcdepends] = "mc::baremetal-firmware:my-firmware:do_deploy" >> + >> +The above will ensure that when the :ref:`ref-tasks-compile` task of >> +``my-parent-firmware`` is triggered, the :ref:`ref-tasks-deploy` task of >> +``my-firmware`` will already have run successfully. >> + >> +Using the output of ``my-firmware`` >> +----------------------------------- >> + >> +After we have deployed ``my-firmware`` by using ``mcdepends``, we need to use > > I can suggest: > > """ > After ``my-firmware`` recipe has deployed `my-firmware.elf`, we need to use > """ > > otherwise it may imply that mcdepends it what deployed my-firmware which > isn't the case, it just made sure my-firmware:do_deploy is finished > before my-parent-firmware:do_compile starts. +1, I agree >> +the output in some way. We can make a series of assumptions, based on the >> +default Yocto Project variables in order to get the binary for packaging. >> + >> +First, we can set the following in ``my-parent-firmware.bb``:: >> + >> + FIRMWARE_FILE ??= "${TMPDIR}-baremetal-firmware/deploy/images/<machine>/my-firmware.elf" >> + FIRMWARE_FILE[vardepsexclude] += "TMPDIR" >> + >> +The first assignment stores the value of the path to the firmware built and >> +deployed by the ``my-firmware.bb`` recipe. The second assignment excludes the >> +:term:`TMPDIR` variable from being part of ``FIRMWARE_FILE``'s dependencies - > > Michael really wanted to use em-dashes, so maybe use it here? > > i.e. > > --- > > c.f. c29a1bf4dd52 ("profile-manual: usage.rst: formatting fixes") Will do Thanks a lot! Antonin
diff --git a/documentation/dev-manual/multiconfig.rst b/documentation/dev-manual/multiconfig.rst index 49e32b04a..cdf06d2ca 100644 --- a/documentation/dev-manual/multiconfig.rst +++ b/documentation/dev-manual/multiconfig.rst @@ -171,3 +171,139 @@ and have separate configuration files, BitBake places the artifacts for each build in the respective temporary build directories (i.e. :term:`TMPDIR`). +Suggested best practices +======================== + +- :term:`TMPDIR` (other than the default set in bitbake.conf) is only set in + ``local.conf`` by the user. This means that we should **not** manipulate + :term:`TMPDIR` in any way within the Machine or Distro :term:`configuration + file`. + +- A multiconfig should specify a :term:`TMPDIR`, and should specify it by + appending the multiconfig name via :term:`BB_CURRENT_MC`. + +- Recipes that are used to transfer the output from a multiconfig build to + another should use ``do_task[mcdepends]`` to trigger the build of the + component, and then transfer the item to the current configuration in + :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy`, assuming the value of + the deployed item based on :term:`TMPDIR`. + + The :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy` tasks should look + like this:: + + do_install() { + install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${D}/some/path + } + + do_deploy() { + install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${DEPLOYDIR}/somefile + } + + In the examples above: + + - ``<multiconfig>`` is the multiconfig name as set by the multiconfig + :term:`configuration file` (see the :ref:`dev-manual/multiconfig:Setting + Up and Running a Multiple Configuration Build` section above). + + - ``<machine>`` must be the :term:`MACHINE` for which ``somefile`` was built + and deployed. This value depends on whether the multiconfig + :term:`configuration file` is overriding the :term:`MACHINE` value or not. + +- Firmware recipes can set the :term:`INHIBIT_DEFAULT_DEPS` variable to ``1`` + if they don't rely on default dependencies such as the standard C library. + +Common use case: building baremetal firmware alongside a Linux build +==================================================================== + +A common use for multiconfig is to use the default configuration as the regular +Linux build, while one or more multiconfigs can be used to build special +components, such as baremetal firmware. It would also apply to a scenario where +a microcontroller, for example, is companion to a main processor where Linux is +running. This section details how one can achieve these kinds of scenarios with +a multiconfig build. + +Adding a multiconfig configuration file and recipe for a baremetal firmware +--------------------------------------------------------------------------- + +As described in :ref:`dev-manual/multiconfig:Setting Up and Running a Multiple +Configuration Build`, each multiconfig will require a separate +:term:`Configuration File`. In our case, we will make a separate temporary +directory for our baremetal firmware build configuration. + +For example, we will define a new ``conf/multiconfig/baremetal-firmware.conf`` +as follows:: + + TMPDIR .= "-${BB_CURRENT_MC}" + TCLIBC = "newlib" + +The ``baremetal-firmware.conf`` configure a separate :term:`TMPDIR` for holding +binaries compiled with the `newlib <https://sourceware.org/newlib/>`__ toolchain +(see :term:`TCLIBC`). + +.. note:: + + Here, the default :term:`MACHINE` is not overriden by the multiconfig + configuration file. As a consequence, the architecture of the built baremetal + binaries will be the same. In other cases, where the firmware runs on a + completely different architecture, the :term:`MACHINE` may be overriden. + +We then create a recipe ``my-firmware.bb`` that defines how the baremetal +firmware is built. The recipe should contain enough information for the +:term:`OpenEmbedded build system` to properly compile the firmware with our +toolchain. The building tasks may vary depending on the nature of the firmware. +However, the recipe should define a :ref:`ref-classes-deploy` task that deploys +the output into the :term:`DEPLOYDIR` directory. We will consider in the +following that the file is named ``my-firmware.elf``. + +Building the firmware +--------------------- + +The firmware can be built with BitBake with the following command:: + + $ bitbake mc:baremetal-firmware:my-firmware + +However, we would prefer for ``my-firmware`` to be automatically built when +triggering a normal Linux build. + +Using an ``mcdepend``, a recipe belonging to the Linux build can trigger the +build of ``my-firmware``. For example, let's consider that our Linux build needs +to assemble a "special" firmware that uses the output of our ``my-firmware`` +recipe - let's call it ``my-parent-firmware.bb``. Then, we should specify this +dependency in ``my-parent-firmware.bb`` with:: + + do_compile[mcdepends] = "mc::baremetal-firmware:my-firmware:do_deploy" + +The above will ensure that when the :ref:`ref-tasks-compile` task of +``my-parent-firmware`` is triggered, the :ref:`ref-tasks-deploy` task of +``my-firmware`` will already have run successfully. + +Using the output of ``my-firmware`` +----------------------------------- + +After we have deployed ``my-firmware`` by using ``mcdepends``, we need to use +the output in some way. We can make a series of assumptions, based on the +default Yocto Project variables in order to get the binary for packaging. + +First, we can set the following in ``my-parent-firmware.bb``:: + + FIRMWARE_FILE ??= "${TMPDIR}-baremetal-firmware/deploy/images/<machine>/my-firmware.elf" + FIRMWARE_FILE[vardepsexclude] += "TMPDIR" + +The first assignment stores the value of the path to the firmware built and +deployed by the ``my-firmware.bb`` recipe. The second assignment excludes the +:term:`TMPDIR` variable from being part of ``FIRMWARE_FILE``'s dependencies - +meaning that changing the value of :term:`TMPDIR` (for example, changing the +host on which the firmware is built) will not invalidate the :ref:`shared state +cache <overview-manual/concepts:shared state cache>`. + +Additionally, ``<machine>`` should be replaced by the :term:`MACHINE` for which +we are building in the baremetal-firmware context. + +We can then add a :ref:`ref-tasks-install` task to ``my-parent-firmware``:: + + do_install() { + install -Dm 0644 ${FIRMWARE_FILE} ${D}/lib/firmware/my-firmware.elf + } + +Doing the above will allow the firmware binary to be transferred and packaged +into the Linux context and rootfs.
After the suggestions from Mark Hatle on the list (https://lists.yoctoproject.org/g/docs/topic/110487932), add two sections to the multiconfig doc: - Suggested best practices: suggestion for better design of multiconfig builds. - Common use case: baremetal build. This section applies the guidelines from the first sections and apply it to a real-life example of how to use multiconfig. This one to build some baremetal firmware alongside a regular Linux build. Suggested-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> --- documentation/dev-manual/multiconfig.rst | 136 +++++++++++++++++++++++++++++++ 1 file changed, 136 insertions(+)