[v2,1/1] wic: extra partition plugin

From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>

The extra_partition plugin allows populating an extra partition with
files listed in the new IMAGE_EXTRA_FILES variable. The implementation
is similar to the bootimg_partition plugin.

This plugin provides an easy way to install files that are not part of
the rootfs.

Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
---
changes in v2:
- missing signed-off-by
- typo
- assertion message
---
 meta/classes-recipe/image_types_wic.bbclass   |   1 +
 meta/lib/oeqa/selftest/cases/wic.py           |  41 ++++++
 .../lib/wic/plugins/source/extra_partition.py | 134 ++++++++++++++++++
 3 files changed, 176 insertions(+)
 create mode 100644 scripts/lib/wic/plugins/source/extra_partition.py

Le jeu. 25 sept. 2025 à 09:23, <pierre-loup.gosse@smile.fr> a écrit :

> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> The extra_partition plugin allows populating an extra partition with
> files listed in the new IMAGE_EXTRA_FILES variable. The implementation
> is similar to the bootimg_partition plugin.
>
> This plugin provides an easy way to install files that are not part of
> the rootfs.
>
> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
>

To confirm, I indeed did review it internally.


> ---
> changes in v2:
> - missing signed-off-by
> - typo
> - assertion message
> ---
>  meta/classes-recipe/image_types_wic.bbclass   |   1 +
>  meta/lib/oeqa/selftest/cases/wic.py           |  41 ++++++
>  .../lib/wic/plugins/source/extra_partition.py | 134 ++++++++++++++++++
>  3 files changed, 176 insertions(+)
>  create mode 100644 scripts/lib/wic/plugins/source/extra_partition.py
>
> diff --git a/meta/classes-recipe/image_types_wic.bbclass
> b/meta/classes-recipe/image_types_wic.bbclass
> index 6180874a4c..549a69db60 100644
> --- a/meta/classes-recipe/image_types_wic.bbclass
> +++ b/meta/classes-recipe/image_types_wic.bbclass
> @@ -17,6 +17,7 @@ WICVARS ?= "\
>         IMAGE_BOOT_FILES \
>         IMAGE_CLASSES \
>         IMAGE_EFI_BOOT_FILES \
> +       IMAGE_EXTRA_FILES \
>         IMAGE_LINK_NAME \
>         IMAGE_ROOTFS \
>         IMGDEPLOYDIR \
> diff --git a/meta/lib/oeqa/selftest/cases/wic.py
> b/meta/lib/oeqa/selftest/cases/wic.py
> index b1c318bd4e..acbece4279 100644
> --- a/meta/lib/oeqa/selftest/cases/wic.py
> +++ b/meta/lib/oeqa/selftest/cases/wic.py
> @@ -18,6 +18,7 @@ from glob import glob
>  from shutil import rmtree, copy
>  from tempfile import NamedTemporaryFile
>  from tempfile import TemporaryDirectory
> +from textwrap import dedent
>
>  from oeqa.selftest.case import OESelftestTestCase
>  from oeqa.core.decorator import OETestTag
> @@ -1647,6 +1648,46 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot"
>              status, output = qemu.run_serial(cmd)
>              self.assertEqual(1, status, 'Failed to run command "%s": %s'
> % (cmd, output))
>
> +    def test_extra_partition_plugin(self):
> +        """Test extra partition plugin"""
> +        config = dedent("""\
> +        IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
> +        IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d =
> "bar.conf;foobar.conf"
> +        IMAGE_EXTRA_FILES = "foo/*"
> +        WICVARS:append = "\
> +            IMAGE_EXTRA_FILES_label-foo \
> +            IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
> +        "
> +        """)
> +        self.append_config(config)
> +
> +        deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE')
> +
> +        testfile = open(os.path.join(deploy_dir, "bar.conf"), "w")
> +        testfile.write("test")
> +        testfile.close()
> +
> +        os.mkdir(os.path.join(deploy_dir, "foo"))
> +        testfile = open(os.path.join(deploy_dir, "foo", "bar.conf"), "w")
> +        testfile.write("test")
> +        testfile.close()
> +
> +        with NamedTemporaryFile("w", suffix=".wks") as wks:
> +            wks.writelines(['part / --source extra_partition --ondisk sda
> --fstype=ext4 --label foo --align 4 --size 5M\n',
> +                            'part / --source extra_partition --ondisk sda
> --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size
> 5M\n',
> +                            'part / --source extra_partition --ondisk sda
> --fstype=ext4 --label bar --align 4 --size 5M\n'])
> +            wks.flush()
> +            _, wicimg = self._get_wic(wks.name)
> +
> +            result = runCmd("wic ls %s | wc -l" % wicimg)
> +            self.assertEqual('4', result.output, msg="Expect 3
> partitions, not %s" % result.output)
> +
> +            for part, file in enumerate(["foo.conf", "foobar.conf",
> "bar.conf"]):
> +                result = runCmd("wic ls %s:%d | grep -q \"%s\"" %
> (wicimg, part + 1, file))
> +                self.assertEqual(0, result.status, msg="File '%s' not
> found in the partition #%d" % (file, part))
> +
> +        self.remove_config(config)
> +
>      def test_fs_types(self):
>          """Test filesystem types for empty and not empty partitions"""
>          img = 'core-image-minimal'
> diff --git a/scripts/lib/wic/plugins/source/extra_partition.py
> b/scripts/lib/wic/plugins/source/extra_partition.py
> new file mode 100644
> index 0000000000..499bede280
> --- /dev/null
> +++ b/scripts/lib/wic/plugins/source/extra_partition.py
> @@ -0,0 +1,134 @@
> +import logging
> +import os
> +import re
> +
> +from glob import glob
> +
> +from wic import WicError
> +from wic.pluginbase import SourcePlugin
> +from wic.misc import exec_cmd, get_bitbake_var
> +
> +logger = logging.getLogger('wic')
> +
> +class ExtraPartitionPlugin(SourcePlugin):
> +    """
> +    Populates an extra partition with files listed in the
> IMAGE_EXTRA_FILES
> +    BitBake variable. Files should be deployed to the DEPLOY_DIR_IMAGE
> directory.
> +
> +    The plugin supports:
> +    - Glob pattern matching for file selection.
> +    - File renaming.
> +    - Suffixes to specify the target partition (by label, UUID, or
> partname),
> +      enabling multiple extra partitions to coexist.
> +
> +    For example:
> +
> +        IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
> +        IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d =
> "bar.conf;foobar.conf"
> +        IMAGE_EXTRA_FILES = "foo/*"
> +        WICVARS:append = "\
> +            IMAGE_EXTRA_FILES_label-foo \
> +            IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
> +        "
> +
> +    """
> +
> +    name = 'extra_partition'
> +    image_extra_files_var_name = 'IMAGE_EXTRA_FILES'
> +
> +    @classmethod
> +    def do_configure_partition(cls, part, source_params, cr, cr_workdir,
> +                             oe_builddir, bootimg_dir, kernel_dir,
> +                             native_sysroot):
> +        """
> +        Called before do_prepare_partition(), list the files to copy
> +        """
> +        extradir = "%s/extra.%d" % (cr_workdir, part.lineno)
> +        install_cmd = "install -d %s" % extradir
> +        exec_cmd(install_cmd)
> +
> +        if not kernel_dir:
> +            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> +            if not kernel_dir:
> +                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
> +
> +        extra_files = None
> +        for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s",
> part.label), ("_part-name-%s", part.part_name), (None, None)):
> +            if fmt:
> +                var = fmt % id
> +            else:
> +                var = ""
> +            extra_files = get_bitbake_var(cls.image_extra_files_var_name
> + var)
> +            if extra_files is not None:
> +                break
> +
> +        if extra_files is None:
> +            raise WicError('No extra files defined, %s unset for entry
> #%d' % (cls.image_extra_files_var_name, part.lineno))
> +
> +        logger.info('Extra files: %s', extra_files)
> +
> +        # list of tuples (src_name, dst_name)
> +        deploy_files = []
> +        for src_entry in re.findall(r'[\w;\-\./\*]+', extra_files):
> +            if ';' in src_entry:
> +                dst_entry = tuple(src_entry.split(';'))
> +                if not dst_entry[0] or not dst_entry[1]:
> +                    raise WicError('Malformed extra file entry: %s' %
> src_entry)
> +            else:
> +                dst_entry = (src_entry, src_entry)
> +
> +            logger.debug('Destination entry: %r', dst_entry)
> +            deploy_files.append(dst_entry)
> +
> +        cls.install_task = [];
> +        for deploy_entry in deploy_files:
> +            src, dst = deploy_entry
> +            if '*' in src:
> +                # by default install files under their basename
> +                entry_name_fn = os.path.basename
> +                if dst != src:
> +                    # unless a target name was given, then treat name
> +                    # as a directory and append a basename
> +                    entry_name_fn = lambda name: \
> +                                    os.path.join(dst,
> +                                                 os.path.basename(name))
> +
> +                srcs = glob(os.path.join(kernel_dir, src))
> +
> +                logger.debug('Globbed sources: %s', ', '.join(srcs))
> +                for entry in srcs:
> +                    src = os.path.relpath(entry, kernel_dir)
> +                    entry_dst_name = entry_name_fn(entry)
> +                    cls.install_task.append((src, entry_dst_name))
> +            else:
> +                cls.install_task.append((src, dst))
> +
> +
> +    @classmethod
> +    def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
> +                             oe_builddir, bootimg_dir, kernel_dir,
> +                             rootfs_dir, native_sysroot):
> +        """
> +        Called to do the actual content population for a partition i.e. it
> +        'prepares' the partition to be incorporated into the image.
> +        In this case, we copies all files listed in IMAGE_EXTRA_FILES
> variable.
> +        """
> +        extradir = "%s/extra.%d" % (cr_workdir, part.lineno)
> +
> +        if not kernel_dir:
> +            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> +            if not kernel_dir:
> +                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
> +
> +        for task in cls.install_task:
> +            src_path, dst_path = task
> +            logger.debug('Install %s as %s', src_path, dst_path)
> +            install_cmd = "install -m 0644 -D %s %s" \
> +                          % (os.path.join(kernel_dir, src_path),
> +                             os.path.join(extradir, dst_path))
> +            exec_cmd(install_cmd)
> +
> +        logger.debug('Prepare extra partition using rootfs in %s',
> extradir)
> +        part.prepare_rootfs(cr_workdir, oe_builddir, extradir,
> +                            native_sysroot, False)
> +
> --
> 2.34.1
>
>

On Thu, 25 Sept 2025 at 09:56, Yoann Congal via lists.openembedded.org
<yoann.congal=smile.fr@lists.openembedded.org> wrote:

>> The extra_partition plugin allows populating an extra partition with
>> files listed in the new IMAGE_EXTRA_FILES variable. The implementation
>> is similar to the bootimg_partition plugin.
>>
>> This plugin provides an easy way to install files that are not part of
>> the rootfs.
>>
>> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>> Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
>
>
> To confirm, I indeed did review it internally.

I don't object to the plugin, but this looks like a very common use
case. How have people been handling it up to now? I'm sure I saw
builds and products with such 'extra' partitions, but I can't for the
life of me remember how those were configured. Did they all write
custom wic plugins for themselves?

Alex

Maybe these 'extra' partitions are populated at runtime (first boot) from
the rootfs?
I wrote this plugin for a project because I couldn’t find an easy way like
bootimg_partition to install 'extra' files at build time.
If a similar solution already exists in Yocto, I’d be interested to know.

Pierre-Loup,

On Thu, Sep 25, 2025 at 10:23 AM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> On Thu, 25 Sept 2025 at 09:56, Yoann Congal via lists.openembedded.org
> <yoann.congal=smile.fr@lists.openembedded.org> wrote:
>
> >> The extra_partition plugin allows populating an extra partition with
> >> files listed in the new IMAGE_EXTRA_FILES variable. The implementation
> >> is similar to the bootimg_partition plugin.
> >>
> >> This plugin provides an easy way to install files that are not part of
> >> the rootfs.
> >>
> >> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> >> Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
> >
> >
> > To confirm, I indeed did review it internally.
>
> I don't object to the plugin, but this looks like a very common use
> case. How have people been handling it up to now? I'm sure I saw
> builds and products with such 'extra' partitions, but I can't for the
> life of me remember how those were configured. Did they all write
> custom wic plugins for themselves?
>
> Alex
>

Hi,

On 9/25/25 11:13, Pierre-loup GOSSE via lists.openembedded.org wrote:
> Maybe these 'extra' partitions are populated at runtime (first boot) 
> from the rootfs?
> I wrote this plugin for a project because I couldn’t find an easy way 
> like |bootimg_partition to install 'extra' files at build time.|
> If a similar solution already exists in Yocto, I’d be interested to know.
>
in the past I have used to rootfs plugin with '--exclude-path' [1] and 
'--change-directory' [2] assuming that I have isolated all files going 
in to the extra partition in to one directory of rootfs.

   part / --source rootfs --exclude-path=config/ ...
   part /config --source rootfs --change-directory=config

[1] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n987

[2] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n1007

Best regards,
/Peter

>
> in the past I have used to rootfs plugin with '--exclude-path' [1] and
> '--change-directory' [2] assuming that I have isolated all files going in
> to the extra partition in to one directory of rootfs.

This is a solution when files are installed in the rootfs but need to be
split across multiple partitions.

However, some files aren't or shouldn’t be in the rootfs. For instance,
I’ve worked on boards requiring a dedicated VFAT partition just for the
U-Boot script. Other use cases include keeping business software separate
from the rootfs, especially when they are installed and updated
independently (with a SWU package).
Pierre-Loup,

On Thu, Sep 25, 2025 at 12:58 PM Peter Bergin <peter@berginkonsult.se>
wrote:

> Hi,
> On 9/25/25 11:13, Pierre-loup GOSSE via lists.openembedded.org wrote:
>
> Maybe these 'extra' partitions are populated at runtime (first boot) from
> the rootfs?
> I wrote this plugin for a project because I couldn’t find an easy way like
> bootimg_partition to install 'extra' files at build time.
> If a similar solution already exists in Yocto, I’d be interested to know.
>
> in the past I have used to rootfs plugin with '--exclude-path' [1] and
> '--change-directory' [2] assuming that I have isolated all files going in
> to the extra partition in to one directory of rootfs.
>
>   part / --source rootfs --exclude-path=config/ ...
>   part /config --source rootfs --change-directory=config
>
> [1] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n987
>
> [2] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n1007
> Best regards,
> /Peter
>

On Thu Sep 25, 2025 at 2:43 PM CEST, Pierre-loup GOSSE via lists.openembedded.org wrote:
>>
>> in the past I have used to rootfs plugin with '--exclude-path' [1] and
>> '--change-directory' [2] assuming that I have isolated all files going in
>> to the extra partition in to one directory of rootfs.
>
> This is a solution when files are installed in the rootfs but need to be
> split across multiple partitions.
>
> However, some files aren't or shouldn’t be in the rootfs. For instance,
> I’ve worked on boards requiring a dedicated VFAT partition just for the
> U-Boot script. Other use cases include keeping business software separate
> from the rootfs, especially when they are installed and updated
> independently (with a SWU package).
> Pierre-Loup,

Why not install all the files in IMAGE_ROOTFS in a single image, then do the
filtering in the WKS?

What I do is:

  part rootfs --source rootfs --fstype=ext4 --exclude-path=data/ ...
  part data --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/data ...

In this case the rootfs partition will not include the files in /data, and the
data partition will include them.

Antonin

>
> Why not install all the files in IMAGE_ROOTFS in a single image, then do
> the
> filtering in the WKS?


This isn’t a suitable solution if you don’t want the files included in the
rootfs image initially. Perhaps my earlier description wasn’t clear, let me
clarify with a practical example.
I use SWUpdate to update the rootfs and business software separately. The
business software shouldn't be part of the rootfs, as it would otherwise be
bundled into the rootfs SWU package. However, I still need to install it in
the output WIC image, specifically in an 'app' partition, and update it
later via the app SWU package.

Based on the bootimg_partition plugin, I wrote this plugin to handle these
extra files, that aren't part of the rootfs but are fetched and deployed in
the DEPLOY_DIR_IMAGE.

Pierre-Loup,



On Thu, Sep 25, 2025 at 5:20 PM Antonin Godard <antonin.godard@bootlin.com>
wrote:

> On Thu Sep 25, 2025 at 2:43 PM CEST, Pierre-loup GOSSE via
> lists.openembedded.org wrote:
> >>
> >> in the past I have used to rootfs plugin with '--exclude-path' [1] and
> >> '--change-directory' [2] assuming that I have isolated all files going
> in
> >> to the extra partition in to one directory of rootfs.
> >
> > This is a solution when files are installed in the rootfs but need to be
> > split across multiple partitions.
> >
> > However, some files aren't or shouldn’t be in the rootfs. For instance,
> > I’ve worked on boards requiring a dedicated VFAT partition just for the
> > U-Boot script. Other use cases include keeping business software separate
> > from the rootfs, especially when they are installed and updated
> > independently (with a SWU package).
> > Pierre-Loup,
>
> Why not install all the files in IMAGE_ROOTFS in a single image, then do
> the
> filtering in the WKS?
>
> What I do is:
>
>   part rootfs --source rootfs --fstype=ext4 --exclude-path=data/ ...
>   part data --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/data ...
>
> In this case the rootfs partition will not include the files in /data, and
> the
> data partition will include them.
>
> Antonin
>
> --
> Antonin Godard, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>