From patchwork Mon Feb 23 21:49:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 397CDEEC292 for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6356.1771883425978131063 for ; Mon, 23 Feb 2026 13:50:26 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=EBesDL35; spf=pass (domain: gmail.com, ip: 209.85.219.41, mailfrom: twoerner@gmail.com) Received: by mail-qv1-f41.google.com with SMTP id 6a1803df08f44-8954c9daaeaso62088856d6.1 for ; Mon, 23 Feb 2026 13:50:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883424; x=1772488224; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=dO9bYfpuijjJJmm/ZR9NTFCM2gLfHvllqzkFSC8Bt18=; b=EBesDL35KZn1VNjWyG1lRlaWYBrTEtbmYMzYt+EOWHuaPcYeYrV1wNoWSGEvmx5fNw 9yLAaed8hgxWR7P96Mo3kfdLlq22K+7OM3pcRaJnB60trjnL2dKXdBUJbVmxjS9wGTQU B5normq37TvBZXwcHklfQhM0t3w2WQRKg+TNVPlLnSY573jnoTLfF1LP4rTAM1fSWSf4 SVxGRzFiFQ0Q824epS3GddnIwkwSniVxJsB2u6LfiErRv8bWwkDC4ErP5yrUfp2ktIRj +pnP5Fs58n6XMemWGPeppc273m0rYgjgvZa2kJhZQunRpKMEm1lYmRXlvKzQ8X6wbTq8 Akcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883424; x=1772488224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=dO9bYfpuijjJJmm/ZR9NTFCM2gLfHvllqzkFSC8Bt18=; b=uOCHrwdJBAH0wntCPajZprdL74DwHyVL2mov/aDIpj1epaEb7xx7tJvAtp3S7UD8TK axV/FYAHvWhTFiFG26Czv31TojyAu7vTpkmIoPFn/RM06u8w5kaLdzl3MsW9ghspY97O +UEECI5isSnH3SovRZ8/1BISgjvzz6wCrrg70rGGncidzhd3Bmbtwee8GnrrGq/GUnMb 4gONdIkMhIe9xjNBUa7Z01MXLQ/M9aa1wTiQMUB0LpJ5CzQplAtEL48pjoNvKKnBCjFw mg04TuQLDQrqF6f7THQs4JYNJaJIUc4EWofLlyWwXFo3QPFQvj9kNondOPLxdLpKdYAH PtwA== X-Gm-Message-State: AOJu0YxyiXANAaZqQV0MiUfZY49meI22dIp9izhfQHwTQ7iB1/OFXK+s 1BATXtlXNqWWgDEMe0+3fbYwJJf8CQFUW2fBoKoG1UkVZYzltmDu7ePMtiEgUqkL X-Gm-Gg: ATEYQzxOyZYppnR7eaiV9osHBgP88ewEVfxiyhwl9v8o5vSA4y8L8qr7BluA4sS+GAf OUYZCEqTyflDsewGViRFmbjQb9qJzOVI9YlFW2x7zV7gCF4S3aZMKGsSqpzK6qCXyIlpPZJmM7P dzcMtkdlaPfsNnQg2Iw9YbH/T1J4EKlI6DefFWx0aCyIp7CFkGl6S+/b2uHyoZB7Mu88G2il+CK ne3ZWflJoTCocgvBDjCdHojZgITWWYv2eYi7nCkasuy4G4hArMfSaGCsDSr34dj/WbYLnsq6C/F 3+fxf97tXhSa81lbvMfoEJBQgoPS3egSlpvVnD7dDvGWy3akKI0vDpkOQCII7SBrhBS3Bgs6HZQ L3Ddhbod8DHNdXJNudMUJlv1fCgqqkeHqU/RuVOnIy52WUWvOWrUKidocm9is7kctE1IdOtIeM3 Mp7ScxGDzsKGjSu79Jnl267ss6Bm0dD/mU1MZkMxNF5liPydfiYxIdhlgQUcZx/ukn5w== X-Received: by 2002:a05:6214:2a49:b0:894:61c8:947f with SMTP id 6a1803df08f44-89979e63a53mr136838306d6.25.1771883424037; Mon, 23 Feb 2026 13:50:24 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:23 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 01/10] wic: re-implement sector-size support Date: Mon, 23 Feb 2026 16:49:59 -0500 Message-ID: <20260223215008.2062721-2-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231702 The previous implementation to add variable sector-size support: - required the variable WIC_SECTOR_SIZE either be defined in a configuration file or be defined in a --vars file - this means that every invocation of "wic ls", "wic cp", or "wic rm" needed this variable defined (config or --vars) - required the user to create separate *wks files for every sector size they wanted to use - required the user to specify the mkfs-extraopts by hand to specify the correct sector size: e.g. bootloader --ptable gpt part --fstype ext4 --source rootfs --label rofs-a --mkfs-extraopts "-b 4096" part --fstype ext4 --source rootfs --use-uuid --mkfs-extraopts "-b 4096" - specifying --mkfs-extraopts replaces the defaults with the user-supplied values - only provided details to support using variable sector-sizes with ext[234] filesystems - it would not be possible to generate images with different sector sizes in the same build since the configuration and *wks files would need to change and the build re-run for each size Update the sector-size handling so that: - the sector-size will now be provided on the cmdline to the "wic ls", "wic cp", "wic rm", and "wic create" commands: default = 512 - this means the configuration and/or --vars file does not need to be changed in order to perform those operations on images with different sector sizes - support is provided implicitly for mkdosfs and ext[234] partitions - the user no longer needs to know and supply the sector-size magic in --mkfs-extraopts (thereby clobbering the other defaults) If the --sector-size command-line argument is not given, allow the sector-size to be provided via the WIC_SECTOR_SIZE bitbake variable. Warn the user whenever the WIC_SECTOR_SIZE bitbake variable is used, mention it is deprecated, and suggest the user use the cmdline arg instead. If both are given, warn the user that the cmdline argument takes precedence. AI-Generated: codex/gpt-5.1-codex-max Signed-off-by: Trevor Woerner --- changes in v5: - fix a bug in the case where WIC_SECTOR_SIZE does not exist at all and is not provided neither in the environment nor a config file changes in v4: - allow the sector-size to be set from WIC_SECTOR_SIZE (environment or vars file) but mark its use deprecated, suggest the user use the cmdline arg instead; if both are given point out that the cmdline value takes precedence changes in v3: - none changes in v2: - none --- meta/lib/oeqa/selftest/cases/wic.py | 39 ++++--------- scripts/lib/wic/engine.py | 45 +++++++-------- scripts/lib/wic/help.py | 23 +++++--- scripts/lib/wic/misc.py | 6 ++ scripts/lib/wic/partition.py | 55 +++++++++++++++++-- scripts/lib/wic/plugins/imager/direct.py | 18 ++---- scripts/lib/wic/plugins/source/bootimg_efi.py | 5 +- .../lib/wic/plugins/source/bootimg_pcbios.py | 11 ++-- .../wic/plugins/source/isoimage_isohybrid.py | 5 +- scripts/wic | 44 +++++++++++++++ 10 files changed, 167 insertions(+), 84 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index ecaee5a29144..7ad61fa700f7 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -935,33 +935,22 @@ bootloader --ptable gpt""") finally: os.remove(wks_file) - def test_wic_sector_size(self): - """Test generation image sector size""" - + def test_wic_sector_size_cli(self): + """Test sector size handling via CLI option.""" + oldpath = os.environ['PATH'] os.environ['PATH'] = get_bb_var("PATH", "wic-tools") try: - # Add WIC_SECTOR_SIZE into config - config = 'WIC_SECTOR_SIZE = "4096"\n'\ - 'WICVARS:append = " WIC_SECTOR_SIZE"\n' - self.append_config(config) bitbake('core-image-minimal') - # Check WIC_SECTOR_SIZE apply to bitbake variable - wic_sector_size_str = get_bb_var('WIC_SECTOR_SIZE', 'core-image-minimal') - wic_sector_size = int(wic_sector_size_str) - self.assertEqual(4096, wic_sector_size) - - self.logger.info("Test wic_sector_size: %d \n" % wic_sector_size) - with NamedTemporaryFile("w", suffix=".wks") as wks: wks.writelines( ['bootloader --ptable gpt\n', - 'part --fstype ext4 --source rootfs --label rofs-a --mkfs-extraopts "-b 4096"\n', - 'part --fstype ext4 --source rootfs --use-uuid --mkfs-extraopts "-b 4096"\n']) + 'part --fstype ext4 --source rootfs --label rofs-a\n', + 'part --fstype ext4 --source rootfs --use-uuid\n']) wks.flush() - cmd = "wic create %s -e core-image-minimal -o %s" % (wks.name, self.resultdir) + cmd = "wic create %s -e core-image-minimal -o %s --sector-size 4096" % (wks.name, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] images = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -969,24 +958,18 @@ bootloader --ptable gpt""") sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') # list partitions - result = runCmd("wic ls %s -n %s" % (images[0], sysroot)) + result = runCmd("wic ls %s -n %s --sector-size 4096" % (images[0], sysroot)) self.assertEqual(3, len(result.output.split('\n'))) - # verify partition size with wic - res = runCmd("export PARTED_SECTOR_SIZE=%d; parted -m %s unit b p" % (wic_sector_size, images[0]), + # verify partition size with parted output + res = runCmd("export PARTED_SECTOR_SIZE=%d; parted -m %s unit b p" % (4096, images[0]), stderr=subprocess.PIPE) - # parse parted output which looks like this: - # BYT;\n - # /var/tmp/wic/build/tmpgjzzefdd-202410281021-sda.direct:78569472B:file:4096:4096:gpt::;\n - # 1:139264B:39284735B:39145472B:ext4:rofs-a:;\n - # 2:39284736B:78430207B:39145472B:ext4:primary:;\n disk_info = res.output.splitlines()[1] - # Check sector sizes sector_size_logical = int(disk_info.split(":")[3]) sector_size_physical = int(disk_info.split(":")[4]) - self.assertEqual(wic_sector_size, sector_size_logical, "Logical sector size is not %d." % wic_sector_size) - self.assertEqual(wic_sector_size, sector_size_physical, "Physical sector size is not %d." % wic_sector_size) + self.assertEqual(4096, sector_size_logical, "Logical sector size is not 4096.") + self.assertEqual(4096, sector_size_physical, "Physical sector size is not 4096.") finally: os.environ['PATH'] = oldpath diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py index 8682ca3176c2..949119edd487 100644 --- a/scripts/lib/wic/engine.py +++ b/scripts/lib/wic/engine.py @@ -252,7 +252,7 @@ def debugfs_version_check(debugfs_path, min_ver=(1, 46, 5)): class Disk: - def __init__(self, imagepath, native_sysroot, fstypes=('fat', 'ext')): + def __init__(self, imagepath, native_sysroot, fstypes=('fat', 'ext'), sector_size=512): self.imagepath = imagepath self.native_sysroot = native_sysroot self.fstypes = fstypes @@ -261,16 +261,7 @@ class Disk: self._lsector_size = None self._psector_size = None self._ptable_format = None - - # define sector size - sector_size_str = get_bitbake_var('WIC_SECTOR_SIZE') - if sector_size_str is not None: - try: - self.sector_size = int(sector_size_str) - except ValueError: - self.sector_size = None - else: - self.sector_size = None + self.sector_size = sector_size # find parted # read paths from $PATH environment variable @@ -299,7 +290,7 @@ class Disk: if self._partitions is None: self._partitions = OrderedDict() - if self.sector_size is not None: + if self.sector_size: out = exec_cmd("export PARTED_SECTOR_SIZE=%d; %s -sm %s unit B print" % \ (self.sector_size, self.parted, self.imagepath), True) else: @@ -340,11 +331,15 @@ class Disk: raise WicError("Partition %s is not in the image" % pnum) part = self.partitions[pnum] # check if fstype is supported + # NOTE: parted is unable to identify dos-type partitions with a 4k sector-size + # if the type is empty and a non-default sector size is used, assume 'fat' + # fdisk identifies them without issue + part_fstype = part.fstype if part.fstype or self.sector_size == 512 else 'fat' for fstype in self.fstypes: - if part.fstype.startswith(fstype): + if part_fstype.startswith(fstype): break else: - raise WicError("Not supported fstype: {}".format(part.fstype)) + raise WicError("Not supported fstype: {}".format(part_fstype)) if pnum not in self._partimages: tmpf = tempfile.NamedTemporaryFile(prefix="wic-part") dst_fname = tmpf.name @@ -615,8 +610,9 @@ class Disk: label = part.get("name") label_str = "-n {}".format(label) if label else '' - cmd = "{} {} -C {} {}".format(self.mkdosfs, label_str, partfname, - part['size']) + sector_str = "-S {}".format(self.sector_size) if self.sector_size else '' + cmd = "{} {} {} -C {} {}".format(self.mkdosfs, label_str, sector_str, partfname, + part['size']) exec_cmd(cmd) # copy content from the temporary directory to the new partition cmd = "{} -snompi {} {}/* ::".format(self.mcopy, partfname, tmpdir) @@ -638,14 +634,19 @@ class Disk: def wic_ls(args, native_sysroot): """List contents of partitioned image or vfat partition.""" - disk = Disk(args.path.image, native_sysroot) + disk = Disk(args.path.image, native_sysroot, sector_size=args.sector_size) if not args.path.part: if disk.partitions: print('Num Start End Size Fstype') for part in disk.partitions.values(): + # size values are in bytes from parted; convert to sectors if a custom sector size was requested + display_size = part.size + if args.sector_size and args.sector_size != disk._lsector_size: + display_size = part.size // args.sector_size print("{:2d} {:12d} {:12d} {:12d} {}".format(\ - part.pnum, part.start, part.end, - part.size, part.fstype)) + part.pnum, part.start // args.sector_size, + part.end // args.sector_size, + display_size, part.fstype)) else: path = args.path.path or '/' print(disk.dir(args.path.part, path)) @@ -656,9 +657,9 @@ def wic_cp(args, native_sysroot): partitioned image. """ if isinstance(args.dest, str): - disk = Disk(args.src.image, native_sysroot) + disk = Disk(args.src.image, native_sysroot, sector_size=args.sector_size) else: - disk = Disk(args.dest.image, native_sysroot) + disk = Disk(args.dest.image, native_sysroot, sector_size=args.sector_size) disk.copy(args.src, args.dest) @@ -667,7 +668,7 @@ def wic_rm(args, native_sysroot): Remove files or directories from the vfat partition of partitioned image. """ - disk = Disk(args.path.image, native_sysroot) + disk = Disk(args.path.image, native_sysroot, sector_size=args.sector_size) disk.remove(args.path.part, args.path.path, args.recursive_delete) def wic_write(args, native_sysroot): diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index 6b49a67de938..5d7c40456a80 100644 --- a/scripts/lib/wic/help.py +++ b/scripts/lib/wic/help.py @@ -118,7 +118,7 @@ wic_create_usage = """ usage: wic create [-o | --outdir ] [-e | --image-name] [-s, --skip-build-check] [-D, --debug] [-r, --rootfs-dir] [-b, --bootimg-dir] - [-k, --kernel-dir] [-n, --native-sysroot] [-f, --build-rootfs] + [-k, --kernel-dir] [-n, --native-sysroot] [--sector-size ] [-f, --build-rootfs] [-c, --compress-with] [-m, --bmap] This command creates an OpenEmbedded image based on the 'OE kickstart @@ -139,13 +139,16 @@ SYNOPSIS wic create [-o | --outdir ] [-e | --image-name] [-s, --skip-build-check] [-D, --debug] [-r, --rootfs-dir] [-b, --bootimg-dir] - [-k, --kernel-dir] [-n, --native-sysroot] [-f, --build-rootfs] + [-k, --kernel-dir] [-n, --native-sysroot] [--sector-size ] [-f, --build-rootfs] [-c, --compress-with] [-m, --bmap] [--no-fstab-update] DESCRIPTION This command creates an OpenEmbedded image based on the 'OE kickstart commands' found in the . + Use the --sector-size option to select the sector size (in bytes) + used for partition layout calculations (default is 512). + In order to do this, wic needs to know the locations of the various build artifacts required to build the image. @@ -278,7 +281,7 @@ wic_ls_usage = """ List content of a partitioned image - usage: wic ls [:[]] [--native-sysroot ] + usage: wic ls [:[]] [--native-sysroot ] [--sector-size ] This command outputs either list of image partitions or directory contents of vfat and ext* partitions. @@ -296,7 +299,7 @@ SYNOPSIS wic ls wic ls : wic ls : - wic ls : --native-sysroot + wic ls : --native-sysroot [--sector-size ] DESCRIPTION This command lists either partitions of the image or directory contents @@ -336,6 +339,8 @@ DESCRIPTION The -n option is used to specify the path to the native sysroot containing the tools(parted and mtools) to use. + The --sector-size option sets the sector size used for partition math + (default is 512 bytes). """ @@ -343,7 +348,7 @@ wic_cp_usage = """ Copy files and directories to/from the vfat or ext* partition - usage: wic cp [--native-sysroot ] + usage: wic cp [--native-sysroot ] [--sector-size ] source/destination image in format :[] @@ -364,7 +369,7 @@ SYNOPSIS wic cp : wic cp : wic cp : - wic cp : --native-sysroot + wic cp : --native-sysroot [--sector-size ] DESCRIPTION This command copies files or directories either @@ -408,13 +413,15 @@ DESCRIPTION The -n option is used to specify the path to the native sysroot containing the tools(parted and mtools) to use. + The --sector-size option sets the sector size used for partition math + (default is 512 bytes). """ wic_rm_usage = """ Remove files or directories from the vfat or ext* partitions - usage: wic rm : [--native-sysroot ] + usage: wic rm : [--native-sysroot ] [--sector-size ] This command removes files or directories from the vfat or ext* partitions of the partitioned image. @@ -466,6 +473,8 @@ DESCRIPTION The -n option is used to specify the path to the native sysroot containing the tools(parted and mtools) to use. + The --sector-size option sets the sector size used for partition math + (default is 512 bytes). The -r option is used to remove directories and their contents recursively,this only applies to ext* partition. diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 1a7c140fa6c8..310e6367a2ad 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -263,4 +263,10 @@ def get_bitbake_var(var, image=None, cache=True): Provide old get_bitbake_var API by wrapping get_var method of BB_VARS singleton. """ + if var == "WIC_SECTOR_SIZE": + env_val = os.environ.get("WIC_SECTOR_SIZE") + if env_val is not None: + logger.warning("DEPRECATED: Using WIC_SECTOR_SIZE from environment; prefer --sector-size to avoid surprises.") + return env_val + return BB_VARS.get_var(var, image, cache) diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index 8fed686e903e..df6e3271649b 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -65,6 +65,50 @@ class Partition(): self.lineno = lineno self.source_file = "" + self.sector_size = 512 + + def _mkdosfs_extraopts(self): + """ + Build mkdosfs extra options ensuring the CLI sector size is applied. + """ + extraopts = self.mkfs_extraopts or '' + tokens = [] + skip_next = False + for tok in extraopts.split(): + if skip_next: + skip_next = False + continue + if tok == '-S': + skip_next = True + continue + if tok.startswith('-S'): + continue + tokens.append(tok) + tokens.extend(['-S', str(self.sector_size)]) + return ' '.join(tokens).strip() + + def _mkfs_ext_extraopts(self, base_opts): + """ + Build mkfs.ext* extra options ensuring the CLI sector size is applied. + """ + extraopts = self.mkfs_extraopts or base_opts + # Only add an explicit block size when a non-default sector size is requested. + if self.sector_size and self.sector_size != 512: + tokens = [] + skip_next = False + for tok in extraopts.split(): + if skip_next: + skip_next = False + continue + if tok == '-b': + skip_next = True + continue + if tok.startswith('-b'): + continue + tokens.append(tok) + tokens.extend(['-b', str(self.sector_size)]) + return ' '.join(tokens).strip() + return extraopts def get_extra_block_count(self, current_blocks): """ @@ -138,6 +182,8 @@ class Partition(): Prepare content for individual partitions, depending on partition command parameters. """ + # capture the sector size requested on the CLI for mkdosfs invocations + self.sector_size = getattr(creator, 'sector_size', 512) or 512 self.updated_fstab_path = updated_fstab_path if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"): self.update_fstab_in_rootfs = True @@ -293,7 +339,7 @@ class Partition(): with open(rootfs, 'w') as sparse: os.ftruncate(sparse.fileno(), rootfs_size * 1024) - extraopts = self.mkfs_extraopts or "-F -i 8192" + extraopts = self._mkfs_ext_extraopts("-F -i 8192") # use hash_seed to generate reproducible ext4 images (extraopts, pseudo) = self.get_hash_seed_ext4(extraopts, pseudo) @@ -401,7 +447,7 @@ class Partition(): size_str = "" - extraopts = self.mkfs_extraopts or '-S 512' + extraopts = self._mkdosfs_extraopts() dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \ (label_str, self.fsuuid, size_str, extraopts, rootfs, @@ -452,7 +498,7 @@ class Partition(): with open(rootfs, 'w') as sparse: os.ftruncate(sparse.fileno(), size * 1024) - extraopts = self.mkfs_extraopts or "-i 8192" + extraopts = self._mkfs_ext_extraopts("-i 8192") # use hash_seed to generate reproducible ext4 images (extraopts, pseudo) = self.get_hash_seed_ext4(extraopts, None) @@ -498,7 +544,7 @@ class Partition(): size_str = "" - extraopts = self.mkfs_extraopts or '-S 512' + extraopts = self._mkdosfs_extraopts() dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \ (label_str, self.fsuuid, extraopts, size_str, rootfs, @@ -559,4 +605,3 @@ class Partition(): logger.warn("%s Inodes (of size %d) are too small." % (get_err_str(self), size)) break - diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py index ad922cfbf122..3adc6eee6280 100644 --- a/scripts/lib/wic/plugins/imager/direct.py +++ b/scripts/lib/wic/plugins/imager/direct.py @@ -67,6 +67,7 @@ class DirectPlugin(ImagerPlugin): self._image = None self.ptable_format = self.ks.bootloader.ptable self.parts = self.ks.partitions + self.sector_size = options.sector_size or 512 # as a convenience, set source to the boot partition source # instead of forcing it to be set via bootloader --source @@ -78,7 +79,7 @@ class DirectPlugin(ImagerPlugin): image_path = self._full_path(self.workdir, self.parts[0].disk, "direct") self._image = PartitionedImage(image_path, self.ptable_format, self.ks.bootloader.diskid, self.parts, self.native_sysroot, - options.extra_space) + options.extra_space, self.sector_size) def setup_workdir(self, workdir): if workdir: @@ -294,15 +295,13 @@ MBR_OVERHEAD = 1 # Overhead of the GPT partitioning scheme GPT_OVERHEAD = 34 -# Size of a sector in bytes -SECTOR_SIZE = 512 - class PartitionedImage(): """ Partitioned image in a file. """ - def __init__(self, path, ptable_format, disk_id, partitions, native_sysroot=None, extra_space=0): + def __init__(self, path, ptable_format, disk_id, partitions, native_sysroot=None, extra_space=0, + sector_size=512): self.path = path # Path to the image file self.numpart = 0 # Number of allocated partitions self.realpart = 0 # Number of partitions in the partition table @@ -332,14 +331,7 @@ class PartitionedImage(): self.partitions = partitions self.partimages = [] # Size of a sector used in calculations - sector_size_str = get_bitbake_var('WIC_SECTOR_SIZE') - if sector_size_str is not None: - try: - self.sector_size = int(sector_size_str) - except ValueError: - self.sector_size = SECTOR_SIZE - else: - self.sector_size = SECTOR_SIZE + self.sector_size = sector_size or 512 self.native_sysroot = native_sysroot num_real_partitions = len([p for p in self.partitions if not p.no_table]) diff --git a/scripts/lib/wic/plugins/source/bootimg_efi.py b/scripts/lib/wic/plugins/source/bootimg_efi.py index 430b0a4b023a..864d6898fc9d 100644 --- a/scripts/lib/wic/plugins/source/bootimg_efi.py +++ b/scripts/lib/wic/plugins/source/bootimg_efi.py @@ -415,8 +415,9 @@ class BootimgEFIPlugin(SourcePlugin): label = part.label if part.label else "ESP" - dosfs_cmd = "mkdosfs -v -n %s -i %s -C %s %d" % \ - (label, part.fsuuid, bootimg, blocks) + sector_size = getattr(creator, 'sector_size', 512) or 512 + dosfs_cmd = "mkdosfs -v -n %s -i %s -S %d -C %s %d" % \ + (label, part.fsuuid, sector_size, bootimg, blocks) exec_native_cmd(dosfs_cmd, native_sysroot) logger.debug("mkdosfs:\n%s" % (str(out))) diff --git a/scripts/lib/wic/plugins/source/bootimg_pcbios.py b/scripts/lib/wic/plugins/source/bootimg_pcbios.py index a7cc5d12c620..32edac47fa09 100644 --- a/scripts/lib/wic/plugins/source/bootimg_pcbios.py +++ b/scripts/lib/wic/plugins/source/bootimg_pcbios.py @@ -132,7 +132,7 @@ class BootimgPcbiosPlugin(SourcePlugin): cls._do_prepare_grub(part, cr_workdir, oe_builddir, kernel_dir, rootfs_dir, native_sysroot) elif source_params['loader-bios'] == 'syslinux': - cls._do_prepare_syslinux(part, cr_workdir, bootimg_dir, + cls._do_prepare_syslinux(part, creator, cr_workdir, bootimg_dir, kernel_dir, native_sysroot) else: raise WicError("unrecognized bootimg_pcbios loader: %s" % source_params['loader-bios']) @@ -142,7 +142,7 @@ class BootimgPcbiosPlugin(SourcePlugin): except KeyError: # Required by do_install_disk cls.loader = 'syslinux' - cls._do_prepare_syslinux(part, cr_workdir, bootimg_dir, + cls._do_prepare_syslinux(part, creator, cr_workdir, bootimg_dir, kernel_dir, native_sysroot) @classmethod @@ -240,7 +240,7 @@ class BootimgPcbiosPlugin(SourcePlugin): cfg.close() @classmethod - def _do_prepare_syslinux(cls, part, cr_workdir, bootimg_dir, + def _do_prepare_syslinux(cls, part, creator, cr_workdir, bootimg_dir, kernel_dir, native_sysroot): """ Called to do the actual content population for a partition i.e. it @@ -292,8 +292,9 @@ class BootimgPcbiosPlugin(SourcePlugin): label = part.label if part.label else "boot" - dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \ - (label, part.fsuuid, bootimg, blocks) + sector_size = getattr(creator, 'sector_size', 512) or 512 + dosfs_cmd = "mkdosfs -n %s -i %s -S %d -C %s %d" % \ + (label, part.fsuuid, sector_size, bootimg, blocks) exec_native_cmd(dosfs_cmd, native_sysroot) mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) diff --git a/scripts/lib/wic/plugins/source/isoimage_isohybrid.py b/scripts/lib/wic/plugins/source/isoimage_isohybrid.py index fdab188db1f8..9195ef5f3184 100644 --- a/scripts/lib/wic/plugins/source/isoimage_isohybrid.py +++ b/scripts/lib/wic/plugins/source/isoimage_isohybrid.py @@ -367,8 +367,9 @@ class IsoImagePlugin(SourcePlugin): esp_label = source_params.get('esp_label', 'EFIimg') - dosfs_cmd = 'mkfs.vfat -n \'%s\' -S 512 -C %s %d' \ - % (esp_label, bootimg, blocks) + sector_size = getattr(creator, 'sector_size', 512) or 512 + dosfs_cmd = "mkfs.vfat -n '%s' -S %d -C %s %d" % \ + (esp_label, sector_size, bootimg, blocks) exec_native_cmd(dosfs_cmd, native_sysroot) mmd_cmd = "mmd -i %s ::/EFI" % bootimg diff --git a/scripts/wic b/scripts/wic index 9137208f5e8f..7006efe5e76c 100755 --- a/scripts/wic +++ b/scripts/wic @@ -103,6 +103,40 @@ class RootfsArgAction(argparse.Action): namespace.__dict__['rootfs_dir'][key] = rootfs_dir +def _apply_sector_size_default(args): + """ + Populate args.sector_size. + Prefer --sector-size if given, WIC_SECTOR_SIZE if not, otherwise fall back to 512. + """ + if not hasattr(args, "sector_size"): + return + + BB_VARS.vars_dir = args.vars_dir + try: + tmp_val = get_bitbake_var("WIC_SECTOR_SIZE", args.image_name) + if tmp_val: + try: + env_val = int(tmp_val) + except ValueError: + raise WicError("Invalid WIC_SECTOR_SIZE value '%s'; please provide an integer or use --sector-size." % tmp_val) + + if args.sector_size is not None: + logger.warning("WIC_SECTOR_SIZE (%d) and --sector-size (%d) were both provided; --sector-size used.", env_val, args.sector_size) + logger.warning("DEPRECATED: WIC_SECTOR_SIZE is deprecated and will be removed, use the --sector-size command-line argument instead.") + return + else: + logger.warning("WIC_SECTOR_SIZE (%s) provided but --sector-size not provided; use --sector-size to avoid confusion.", env_val) + logger.warning("DEPRECATED: WIC_SECTOR_SIZE is deprecated and will be removed, use the --sector-size command-line argument instead.") + args.sector_size = env_val + return + except: + pass + + if args.sector_size is not None: + return + args.sector_size = 512 + + def wic_create_subcommand(options, usage_str): """ Command-line handling for image creation. The real work is done @@ -376,6 +410,8 @@ def wic_init_parser_create(subparser): default="direct", help="the wic imager plugin") subparser.add_argument("--extra-space", type=int, dest="extra_space", default=0, help="additional free disk space to add to the image") + subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, + help="sector size in bytes (default: 512)") return @@ -413,6 +449,8 @@ def imgtype(arg): def wic_init_parser_ls(subparser): subparser.add_argument("path", type=imgtype, help="image spec: [:[]]") + subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, + help="sector size in bytes (default: 512)") subparser.add_argument("-n", "--native-sysroot", help="path to the native sysroot containing the tools") subparser.add_argument("-e", "--image-name", dest="image_name", @@ -433,6 +471,8 @@ def wic_init_parser_cp(subparser): help="image spec: :[] or ") subparser.add_argument("dest", help="image spec: :[] or ") + subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, + help="sector size in bytes (default: 512)") subparser.add_argument("-n", "--native-sysroot", help="path to the native sysroot containing the tools") subparser.add_argument("-e", "--image-name", dest="image_name", @@ -445,6 +485,8 @@ def wic_init_parser_cp(subparser): def wic_init_parser_rm(subparser): subparser.add_argument("path", type=imgpathtype, help="path: :") + subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, + help="sector size in bytes (default: 512)") subparser.add_argument("-n", "--native-sysroot", help="path to the native sysroot containing the tools") subparser.add_argument("-r", dest="recursive_delete", action="store_true", default=False, @@ -569,6 +611,8 @@ def main(argv): if args.debug: logger.setLevel(logging.DEBUG) + _apply_sector_size_default(args) + if "command" in vars(args): if args.command == "help": if args.help_topic is None: From patchwork Mon Feb 23 21:50:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C0A9EEC29E for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6357.1771883426814384598 for ; Mon, 23 Feb 2026 13:50:26 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Dh9u7CFj; spf=pass (domain: gmail.com, ip: 209.85.222.194, mailfrom: twoerner@gmail.com) Received: by mail-qk1-f194.google.com with SMTP id af79cd13be357-8cb3fd71badso460028585a.0 for ; Mon, 23 Feb 2026 13:50:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883425; x=1772488225; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vT9OCIzqto4kBbwc7ymMMgOCcpLjgW+7Z1dIRA2qxVw=; b=Dh9u7CFjhfaLFyxK1gZ0UnRamDUxpDjNRFh4SlPssav4efmfiuvEkfrxJjoxrBzyGI jzFIwrzNF+YarLZ0kgvBbR3zC3ojkCJiHZsdM6pd/OZsNdJiJFs4uAGWV1DzKqGXdt66 R0nzmcg1w53TwaB3wjnu9Cx12oF/wdKs56WPkc3DYAhxfkd+UeLCNggg3+4Q2FD7qaau 55ORygv7uAZs/gCN7bX1BUwMpZB1RQJq1HJ3OqmN4Y8vg41NLWTzdXVbb2sbS4kcsUJL QRtpNec/VVwZc7P4UriEuhX1v+k4nu/I+ydyqaUwiU18kFnNP3PmTI4JH+SAH/kUCryx amiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883425; x=1772488225; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=vT9OCIzqto4kBbwc7ymMMgOCcpLjgW+7Z1dIRA2qxVw=; b=oJUWl1wE6bmgBZzeyQHK1ZCYfxf3OdNQXeetDFEiym2oM5iDwz5QFcCuA0W9x1HJYG OPyCZ8DfozVMLorynpr7b7TpXSa6qDSlE7MJbsEpd9MOUeBBMnK1RCtnrfWW4qDkYSQe +jo2/qBUjk5pMOsgqdf4SVgFuFeJspEuC63tlxFHHmD+qwzDw2a1RCz49joh4q46H5rS 4atWWwD7Ne2bTjYJhjoLm5aU+cdOb6nL1J1tJ5/aHss95rzVqsmZR4WqBeJGajth+4ZK hh0bUjzRfzKgfQNqyc3Dijnqo8Vxm/Fq/rqs821zNL+lfLmY1VV/Jy7l7QJKagbb0so2 21bQ== X-Gm-Message-State: AOJu0YxFd1DrAeZYFZCXRKivDpm+PVLPqTbBOLyLqmhfpilKJumWEdvK MGBQ3UqEP36LBpLMGcPisR4mIijAosc/ostMEgrXdN9BAm+xMr4HBkLdHK7wG71+qYI= X-Gm-Gg: AZuq6aKN7sd+VzR/yoEEFZzl0mBBzr1l7LkM3Z1T8HAPdlJwvOYgi/VX+28YDEQYzuY PhrIOUINRZyCXJsmNDGCHX2822x88+sn+xHnhTOkinvWwGicWgZcrtj57Kax76fL2gqGW/k6izd nl1qu0p6oVFWrbqkmr7L0yeMUwR4AFWqlQnRsDJxuAz2cd3dcP/F3/CVvtxJyujkbq1PWstm//m 0Gf5JUXdFxehG9aLapZoNio834bnhrq379l2YrpIoSaSw5DKFqMb7ydRs7716vYdfawbYfCOMV6 TkG1H3Em8/38uNXLJmehtngpTAaM/8QDuwJHFQ03Wztw1aFIhj30lKartjsIRk/PX+TQi+Xahz+ psSlcFHYGcJXclgnQOe0E+7WEUkEVDe1jFnLIDAtOx8kLHfAOZRlr1nsmGigIwe7a/OxhWTZOgk lwWDo8Vc18oDeAk9zl0WW2+UWvzX9DHSiinL/U3mTmcn29Nrn6QZ8oldN2DbvEZCqO+hx5vVgYZ Et7 X-Received: by 2002:a05:620a:bc2:b0:8cb:3e86:d98b with SMTP id af79cd13be357-8cb8ca9e400mr1342886485a.70.1771883425010; Mon, 23 Feb 2026 13:50:25 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:24 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Cc: Mark Hatle Subject: [PATCH v5 02/10] ufs image class: add Date: Mon, 23 Feb 2026 16:50:00 -0500 Message-ID: <20260223215008.2062721-3-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231703 Add an image class and wks file that demonstrates generating a wic image with a 4096-byte sector size. Signed-off-by: Mark Hatle Signed-off-by: Trevor Woerner --- changes in v5: - none changes in v4: - update the partition table format from mbr (msdos) to gpt changes in v3: - tested more scenarios and make sure to fix the warning from v1 in every case changes in v2: - add Mark as a co-creator (sorry for missing this the first time!) - provide a fix for the following warning: WARNING: core-image-minimal-1.0-r0 do_image_wic_ufs: Function do_image_wic_ufs doesn't exist --- meta/classes-recipe/image.bbclass | 2 +- meta/classes-recipe/image_types_ufs.bbclass | 221 ++++++++++++++++++++ scripts/lib/wic/canned-wks/mkdisk-ufs.wks | 5 + 3 files changed, 227 insertions(+), 1 deletion(-) create mode 100644 meta/classes-recipe/image_types_ufs.bbclass create mode 100644 scripts/lib/wic/canned-wks/mkdisk-ufs.wks diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass index 53f1a9dc45b0..97465836c14a 100644 --- a/meta/classes-recipe/image.bbclass +++ b/meta/classes-recipe/image.bbclass @@ -18,7 +18,7 @@ inherit populate_sdk_base IMGCLASSES += "${@['', 'populate_sdk_ext']['linux' in d.getVar("SDK_OS")]}" IMGCLASSES += "${@bb.utils.contains_any('IMAGE_FSTYPES', 'live iso hddimg', 'image-live', '', d)}" IMGCLASSES += "${@bb.utils.contains('IMAGE_FSTYPES', 'container', 'image-container', '', d)}" -IMGCLASSES += "image_types_wic" +IMGCLASSES += "image_types_wic image_types_ufs" IMGCLASSES += "rootfs-postcommands" IMGCLASSES += "image-postinst-intercepts" IMGCLASSES += "overlayfs-etc" diff --git a/meta/classes-recipe/image_types_ufs.bbclass b/meta/classes-recipe/image_types_ufs.bbclass new file mode 100644 index 000000000000..9a984f084289 --- /dev/null +++ b/meta/classes-recipe/image_types_ufs.bbclass @@ -0,0 +1,221 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +# The WICUFSVARS variable is used to define the base list of bitbake variables used in wic code +# variables from this list are written to .env file +WICUFSVARS ?= "\ + APPEND \ + ASSUME_PROVIDED \ + BBLAYERS \ + DEPLOY_DIR_IMAGE \ + FAKEROOTCMD \ + HOSTTOOLS_DIR \ + IMAGE_BASENAME \ + IMAGE_BOOT_FILES \ + IMAGE_CLASSES \ + IMAGE_EFI_BOOT_FILES \ + IMAGE_EXTRA_PARTITION_FILES \ + IMAGE_LINK_NAME \ + IMAGE_ROOTFS \ + IMGDEPLOYDIR \ + INITRAMFS_FSTYPES \ + INITRAMFS_IMAGE \ + INITRAMFS_IMAGE_BUNDLE \ + INITRAMFS_LINK_NAME \ + INITRD \ + INITRD_LIVE \ + ISODIR \ + KERNEL_CONSOLE \ + KERNEL_IMAGETYPE \ + MACHINE \ + PSEUDO_INCLUDE_PATHS \ + RECIPE_SYSROOT_NATIVE \ + ROOTFS_SIZE \ + STAGING_DATADIR \ + STAGING_DIR \ + STAGING_DIR_HOST \ + STAGING_LIBDIR \ + TARGET_SYS \ +" + +inherit_defer ${@bb.utils.contains('INITRAMFS_IMAGE_BUNDLE', '1', 'kernel-artifact-names', '', d)} + +WKSUFS_FILE ??= "${WKS_FILE}" +WKSUFS_FILES ?= "${WKSUFS_FILE} ${IMAGE_BASENAME}.wks" +WKSUFS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" +WKSUFS_FULL_PATH = "${@wks_search(d.getVar('WKSUFS_FILES').split(), d.getVar('WKSUFS_SEARCH_PATH')) or ''}" + +def wks_search(files, search_path): + for f in files: + if os.path.isabs(f): + if os.path.exists(f): + return f + else: + searched = bb.utils.which(search_path, f) + if searched: + return searched + +def wks_checksums(files, search_path): + ret = "" + for f in files: + found, hist = bb.utils.which(search_path, f, history=True) + ret = ret + " " + " ".join(h + ":False" for h in hist[:-1]) + if found: + ret = ret + " " + found + ":True" + return ret + + +WICUFS_CREATE_EXTRA_ARGS ?= "${WIC_CREATE_EXTRA_ARGS}" + +IMAGE_CMD:wic.ufs () { + out="${IMGDEPLOYDIR}/${IMAGE_NAME}" + build_wic_ufs="${WORKDIR}/build-wic-ufs" + tmp_wic_ufs="${WORKDIR}/tmp-wic-ufs" + wks="${WKSUFS_FULL_PATH}" + if [ -e "$tmp_wic_ufs" ]; then + # Ensure we don't have any junk leftover from a previously interrupted + # do_image_wic_ufs execution + rm -rf "$tmp_wic_ufs" + fi + if [ -z "$wks" ]; then + bbfatal "No kickstart files from WKSUFS_FILES were found: ${WKSUFS_FILES}. Please set WKSUFS_FILE or WKSUFS_FILES appropriately." + fi + BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create --debug "$wks" --sector-size 4096 --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}-ufs" -o "$build_wic_ufs/" -w "$tmp_wic_ufs" ${WICUFS_CREATE_EXTRA_ARGS} + + # look to see if the user specifies a custom imager + IMAGER=direct + eval set -- "${WICUFS_CREATE_EXTRA_ARGS} --" + while [ 1 ]; do + case "$1" in + --imager|-i) + shift + IMAGER=$1 + ;; + --) + shift + break + ;; + esac + shift + done + mv "$build_wic_ufs/$(basename "${wks%.wks}")"*.${IMAGER} "$out.wic.ufs" +} +IMAGE_CMD:wic.ufs[vardepsexclude] = "WKSUFS_FULL_PATH WKSUFS_FILES TOPDIR" +SPDX_IMAGE_PURPOSE:wic.ufs = "diskImage" +do_image_wic_ufs[cleandirs] = "${WORKDIR}/build-wic-ufs" + +# Rebuild when the wks file or vars in WICUFSVARS change +USING_WIC_UFS = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic.ufs ' + ' '.join('wic.ufs.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}" +WKSUFS_FILE_CHECKSUM = "${@wks_checksums(d.getVar('WKSUFS_FILES').split(), d.getVar('WKSUFS_SEARCH_PATH')) if '${USING_WIC_UFS}' else ''}" +do_image_wic_ufs[file-checksums] += "${WKSUFS_FILE_CHECKSUM}" +do_image_wic_ufs[depends] += "${@' '.join('%s-native:do_populate_sysroot' % r for r in ('parted', 'gptfdisk', 'dosfstools', 'mtools'))}" + +# We ensure all artfacts are deployed (e.g virtual/bootloader) +do_image_wic_ufs[recrdeptask] += "do_deploy" +do_image_wic_ufs[deptask] += "do_image_complete" + +WKSUFS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}' +WKSUFS_FILE_DEPENDS_DEFAULT += "bmaptool-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native" +# Unified kernel images need objcopy +WKSUFS_FILE_DEPENDS_DEFAULT += "virtual/cross-binutils" +WKSUFS_FILE_DEPENDS_BOOTLOADERS = "" +WKSUFS_FILE_DEPENDS_BOOTLOADERS:aarch64 = "grub-efi systemd-boot" +WKSUFS_FILE_DEPENDS_BOOTLOADERS:arm = "systemd-boot" +WKSUFS_FILE_DEPENDS_BOOTLOADERS:x86 = "syslinux grub-efi systemd-boot" +WKSUFS_FILE_DEPENDS_BOOTLOADERS:x86-64 = "syslinux grub-efi systemd-boot" +WKSUFS_FILE_DEPENDS_BOOTLOADERS:x86-x32 = "syslinux grub-efi" + +WKSUFS_FILE_DEPENDS ??= "${WKSUFS_FILE_DEPENDS_DEFAULT} ${WKSUFS_FILE_DEPENDS_BOOTLOADERS}" + +DEPENDS += "${@ '${WKSUFS_FILE_DEPENDS}' if d.getVar('USING_WIC_UFS') else '' }" + +python do_write_wksufs_template () { + """Write out expanded template contents to WKSUFS_FULL_PATH.""" + import re + + template_body = d.getVar('_WKSUFS_TEMPLATE') + + # Remove any remnant variable references left behind by the expansion + # due to undefined variables + expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}") + while True: + new_body = re.sub(expand_var_regexp, '', template_body) + if new_body == template_body: + break + else: + template_body = new_body + + wks_file = d.getVar('WKSUFS_FULL_PATH') + with open(wks_file, 'w') as f: + f.write(template_body) + f.close() + # Copy the finalized wks file to the deploy directory for later use + depdir = d.getVar('IMGDEPLOYDIR') + basename = d.getVar('IMAGE_BASENAME') + '-ufs' + bb.utils.copyfile(wks_file, "%s/%s" % (depdir, basename + '-' + os.path.basename(wks_file))) +} + +do_flush_pseudodb() { + ${FAKEROOTENV} ${FAKEROOTCMD} -S +} + +python () { + if d.getVar('USING_WIC_UFS'): + wksufs_file_u = d.getVar('WKSUFS_FULL_PATH', False) + wksufs_file = d.expand(wksufs_file_u) + base, ext = os.path.splitext(wksufs_file) + if ext == '.in' and os.path.exists(wksufs_file): + wksufs_out_file = os.path.join(d.getVar('WORKDIR'), os.path.basename(base)) + d.setVar('WKSUFS_FULL_PATH', wksufs_out_file) + d.setVar('WKSUFS_TEMPLATE_PATH', wksufs_file_u) + d.setVar('WKSUFS_FILE_CHECKSUM', '${WKSUFS_TEMPLATE_PATH}:True') + + # We need to re-parse each time the file changes, and bitbake + # needs to be told about that explicitly. + bb.parse.mark_dependency(d, wksufs_file) + + try: + with open(wksufs_file, 'r') as f: + body = f.read() + except (IOError, OSError) as exc: + pass + else: + # Previously, I used expandWithRefs to get the dependency list + # and add it to WICUFSVARS, but there's no point re-parsing the + # file in process_wks_template as well, so just put it in + # a variable and let the metadata deal with the deps. + d.setVar('_WKSUFS_TEMPLATE', body) + bb.build.addtask('do_write_wksufs_template', 'do_image_wic_ufs', 'do_image', d) + bb.build.addtask('do_image_wic_ufs', 'do_image_complete', 'do_image_wic', d) +} + +# +# Write environment variables used by wic +# to tmp/sysroots//imgdata/-ufs.env +# +python do_rootfs_wicufsenv () { + wicufsvars = d.getVar('WICUFSVARS') + if not wicufsvars: + return + + stdir = d.getVar('STAGING_DIR') + outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata') + bb.utils.mkdirhier(outdir) + basename = d.getVar('IMAGE_BASENAME') + '-ufs' + with open(os.path.join(outdir, basename) + '.env', 'w') as envf: + for var in wicufsvars.split(): + value = d.getVar(var) + if value: + envf.write('%s="%s"\n' % (var, value.strip())) + envf.close() + # Copy .env file to deploy directory for later use with stand alone wic + depdir = d.getVar('IMGDEPLOYDIR') + bb.utils.copyfile(os.path.join(outdir, basename) + '.env', os.path.join(depdir, basename) + '.env') +} +addtask do_flush_pseudodb after do_rootfs before do_image do_image_qa +addtask do_rootfs_wicufsenv after do_image before do_image_wic_ufs +do_rootfs_wicufsenv[vardeps] += "${WICUFSVARS}" +do_rootfs_wicufsenv[prefuncs] = 'set_image_size' diff --git a/scripts/lib/wic/canned-wks/mkdisk-ufs.wks b/scripts/lib/wic/canned-wks/mkdisk-ufs.wks new file mode 100644 index 000000000000..dfd2f3854890 --- /dev/null +++ b/scripts/lib/wic/canned-wks/mkdisk-ufs.wks @@ -0,0 +1,5 @@ +bootloader --ptable gpt + +part /boot --source rootfs --fstype=vfat --label boot --change-directory boot/ +part / --source rootfs --fstype=ext4 --label root --exclude-path boot/ +part swap --fstype=swap --label swap --size 1M From patchwork Mon Feb 23 21:50:01 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B27DEEC296 for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6461.1771883431767782680 for ; Mon, 23 Feb 2026 13:50:32 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=cA4eu1gr; spf=pass (domain: gmail.com, ip: 209.85.222.172, mailfrom: twoerner@gmail.com) Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-8cb4136d865so590989985a.1 for ; Mon, 23 Feb 2026 13:50:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883430; x=1772488230; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=cBOA7F3+VNuyg5IeeikGIeL2tIoMaFoao5UyCIJ2VRk=; b=cA4eu1gr00/Cnabo7KR6wc4UrhezrA/3YyaEJ/3VVIR+/MhuSr3vAbI9rcVGWZkcVy v06pipD/h78rd0NRZ2XZgO7qSJjL/Nmcz030lqpBEzcn03ynOnMt6jfgAOuFn7bTRF3u d2wQzc68YlqCLwyNbaH3uplNEpwBLJ6RDl7F0RjtiAiW3WEDhuxw9NesmFWnI/JLG1kl yuRwH5OiYPbve+hNQTxX3PfD1+Vk+gst1VjWEe1tgzofMugq1cW6LQyjoq0gt1Husl4a pNjdaxXCu29pkM2gJ59pHrIv0O4YP77qQ97dO+NinjI280DRRlUQc5tEFszJiDX3a3tc xbDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883430; x=1772488230; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=cBOA7F3+VNuyg5IeeikGIeL2tIoMaFoao5UyCIJ2VRk=; b=Jk80oSd9OXJi0x6NYuDCuilwghx1wfMOCEWTF81+pcxojktTdlX5JAgDqzJi8b8kQT b93ZpH9IAg7a37xvnZFNx9sYmt0ssj4MiKMWBlQNUOWs6mx3sBwkD2F+e9U7ZvNcmlvY /sGvyD1xl9MFC9y412nTMJyFrtTg4J4BQWQ+909XNgtvDNKDIaoVPvGe6tzpu9uBhs1E PqWsZiyL0TDaJW14CtfkxMeGIXCUmV5d730ol0DNTVapPCMCdVLEoIHhj1wvcOHq/q9W prhe13ZttS9OwpdH3C57qrUqfCaOb/FllihaC39ib5YiYJNTY9ZtjpghmGBvNRMV7awT aFoA== X-Gm-Message-State: AOJu0YwD+VhxNWhggJYWtURe7F9C+xNdPuYwqcVQCGN4wAQBMwzlQANl +I0iculuU77rqsp6Nde/LzfsOtgtfJSExcMGekoVVzMKn96P4dF4JD12SbcGBP2O X-Gm-Gg: AZuq6aImzYjgaD9/25zyszJluKpSFOsJrVWmZOWgSFIBVQ0dXKZRMisENQKGD3VMJ7+ p5trpCGA/Wg6gdqjQtz2U5ZRs0h0tesknhBG1p40vXANF7kLDI8pDRVTPwjsKsVstTI2M+6OsfH +l7fngy2/21WIlspIRBb5rGN4DttYw2qJ2/3nFupTCYZYyEKivblM8owEpB0CPU3IAKkLQI8yhu XxJJYuDlbBJuJUVQxKnO5pdvyrxWo3xRrpjiWXFxn2hluothN6TxZp+ex7OwW/3GIcI1InK/olS 7vmxhU9GDtMEzPIJnV9Rrxt8LJIAlOq6MLCgjw7YOuX2AX8rFrbyLp38mBuEM5syQnhNezKg/hO 8j1+gyJ6wL8qUvWIC9LYqY+43+/90zj5ey1yWE0iyDy0xXkntV993XAHoYVnEOsyZTsigLl7wcu aP3J3+IAc/glV49uImRKB5FP1BgV5j0oVMvuCEr2Trn+/rQ+YPOtjiFtOo5hDYDqBlbwCK/YmPu 93f X-Received: by 2002:a05:620a:d8d:b0:8c9:f4eb:ac71 with SMTP id af79cd13be357-8cb8c9e00a7mr1172583785a.10.1771883426980; Mon, 23 Feb 2026 13:50:26 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:25 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 03/10] wic: remove Date: Mon, 23 Feb 2026 16:50:01 -0500 Message-ID: <20260223215008.2062721-4-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231707 The wic utility will continue to exist under The Yocto Project umbrella, but will no longer be integrated into the project as part of oe-core. The ultimate goal is to make wic a completely independent tool, hosted on PyPI, with its own tests, development roadmap, and releases. Some benefits include: - relieving the oe-core maintainers from having to review wic patches - allow the tool to be used outside of The Yocto Project - provide more flexibility to explore other features, libraries, mechanisms, etc AI-Generated: codex/gpt-5.1-codex-max Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - split patches out again v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large patch changes in v2: - none --- scripts/lib/wic/__init__.py | 10 - scripts/lib/wic/canned-wks/common.wks.inc | 3 - .../directdisk-bootloader-config.cfg | 27 - .../directdisk-bootloader-config.wks | 8 - scripts/lib/wic/canned-wks/directdisk-gpt.wks | 10 - .../canned-wks/directdisk-multi-rootfs.wks | 23 - scripts/lib/wic/canned-wks/directdisk.wks | 8 - .../lib/wic/canned-wks/efi-bootdisk.wks.in | 3 - .../wic/canned-wks/efi-uki-bootdisk.wks.in | 3 - scripts/lib/wic/canned-wks/mkdisk-ufs.wks | 5 - scripts/lib/wic/canned-wks/mkefidisk.wks | 11 - scripts/lib/wic/canned-wks/mkhybridiso.wks | 7 - scripts/lib/wic/canned-wks/qemuloongarch.wks | 3 - scripts/lib/wic/canned-wks/qemuriscv.wks | 3 - .../lib/wic/canned-wks/qemux86-directdisk.wks | 8 - .../lib/wic/canned-wks/sdimage-bootpart.wks | 6 - .../lib/wic/canned-wks/systemd-bootdisk.wks | 11 - scripts/lib/wic/engine.py | 711 ---------- scripts/lib/wic/filemap.py | 583 -------- scripts/lib/wic/help.py | 1197 ----------------- scripts/lib/wic/ksparser.py | 322 ----- scripts/lib/wic/misc.py | 272 ---- scripts/lib/wic/partition.py | 607 --------- scripts/lib/wic/pluginbase.py | 144 -- scripts/lib/wic/plugins/imager/direct.py | 702 ---------- .../wic/plugins/source/bootimg_biosplusefi.py | 213 --- scripts/lib/wic/plugins/source/bootimg_efi.py | 436 ------ .../wic/plugins/source/bootimg_partition.py | 162 --- .../lib/wic/plugins/source/bootimg_pcbios.py | 484 ------- scripts/lib/wic/plugins/source/empty.py | 89 -- .../lib/wic/plugins/source/extra_partition.py | 146 -- .../wic/plugins/source/isoimage_isohybrid.py | 464 ------- scripts/lib/wic/plugins/source/rawcopy.py | 115 -- scripts/lib/wic/plugins/source/rootfs.py | 236 ---- scripts/wic | 644 --------- 35 files changed, 7676 deletions(-) delete mode 100644 scripts/lib/wic/__init__.py delete mode 100644 scripts/lib/wic/canned-wks/common.wks.inc delete mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg delete mode 100644 scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks delete mode 100644 scripts/lib/wic/canned-wks/directdisk-gpt.wks delete mode 100644 scripts/lib/wic/canned-wks/directdisk-multi-rootfs.wks delete mode 100644 scripts/lib/wic/canned-wks/directdisk.wks delete mode 100644 scripts/lib/wic/canned-wks/efi-bootdisk.wks.in delete mode 100644 scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in delete mode 100644 scripts/lib/wic/canned-wks/mkdisk-ufs.wks delete mode 100644 scripts/lib/wic/canned-wks/mkefidisk.wks delete mode 100644 scripts/lib/wic/canned-wks/mkhybridiso.wks delete mode 100644 scripts/lib/wic/canned-wks/qemuloongarch.wks delete mode 100644 scripts/lib/wic/canned-wks/qemuriscv.wks delete mode 100644 scripts/lib/wic/canned-wks/qemux86-directdisk.wks delete mode 100644 scripts/lib/wic/canned-wks/sdimage-bootpart.wks delete mode 100644 scripts/lib/wic/canned-wks/systemd-bootdisk.wks delete mode 100644 scripts/lib/wic/engine.py delete mode 100644 scripts/lib/wic/filemap.py delete mode 100644 scripts/lib/wic/help.py delete mode 100644 scripts/lib/wic/ksparser.py delete mode 100644 scripts/lib/wic/misc.py delete mode 100644 scripts/lib/wic/partition.py delete mode 100644 scripts/lib/wic/pluginbase.py delete mode 100644 scripts/lib/wic/plugins/imager/direct.py delete mode 100644 scripts/lib/wic/plugins/source/bootimg_biosplusefi.py delete mode 100644 scripts/lib/wic/plugins/source/bootimg_efi.py delete mode 100644 scripts/lib/wic/plugins/source/bootimg_partition.py delete mode 100644 scripts/lib/wic/plugins/source/bootimg_pcbios.py delete mode 100644 scripts/lib/wic/plugins/source/empty.py delete mode 100644 scripts/lib/wic/plugins/source/extra_partition.py delete mode 100644 scripts/lib/wic/plugins/source/isoimage_isohybrid.py delete mode 100644 scripts/lib/wic/plugins/source/rawcopy.py delete mode 100644 scripts/lib/wic/plugins/source/rootfs.py delete mode 100755 scripts/wic diff --git a/scripts/lib/wic/__init__.py b/scripts/lib/wic/__init__.py deleted file mode 100644 index 85567934aea5..000000000000 --- a/scripts/lib/wic/__init__.py +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env python3 -# -# Copyright (c) 2007 Red Hat, Inc. -# Copyright (c) 2011 Intel, Inc. -# -# SPDX-License-Identifier: GPL-2.0-only -# - -class WicError(Exception): - pass diff --git a/scripts/lib/wic/canned-wks/common.wks.inc b/scripts/lib/wic/canned-wks/common.wks.inc deleted file mode 100644 index 4a440ddafe64..000000000000 --- a/scripts/lib/wic/canned-wks/common.wks.inc +++ /dev/null @@ -1,3 +0,0 @@ -# This file is included into 3 canned wks files from this directory -part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --use-uuid --fstype=ext4 --label platform --align 1024 diff --git a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg b/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg deleted file mode 100644 index c58e74a853cb..000000000000 --- a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg +++ /dev/null @@ -1,27 +0,0 @@ -# This is an example configuration file for syslinux. -TIMEOUT 50 -ALLOWOPTIONS 1 -SERIAL 0 115200 -PROMPT 0 - -UI vesamenu.c32 -menu title Select boot options -menu tabmsg Press [Tab] to edit, [Return] to select - -DEFAULT Graphics console boot - -LABEL Graphics console boot -KERNEL /vmlinuz -APPEND label=boot rootwait - -LABEL Serial console boot -KERNEL /vmlinuz -APPEND label=boot rootwait console=ttyS0,115200 - -LABEL Graphics console install -KERNEL /vmlinuz -APPEND label=install rootwait - -LABEL Serial console install -KERNEL /vmlinuz -APPEND label=install rootwait console=ttyS0,115200 diff --git a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks b/scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks deleted file mode 100644 index 3529e05c876e..000000000000 --- a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks +++ /dev/null @@ -1,8 +0,0 @@ -# short-description: Create a 'pcbios' direct disk image with custom bootloader config -# long-description: Creates a partitioned legacy BIOS disk image that the user -# can directly dd to boot media. The bootloader configuration source is a user file. - -include common.wks.inc - -bootloader --configfile="directdisk-bootloader-config.cfg" - diff --git a/scripts/lib/wic/canned-wks/directdisk-gpt.wks b/scripts/lib/wic/canned-wks/directdisk-gpt.wks deleted file mode 100644 index cb640056f199..000000000000 --- a/scripts/lib/wic/canned-wks/directdisk-gpt.wks +++ /dev/null @@ -1,10 +0,0 @@ -# short-description: Create a 'pcbios' direct disk image -# long-description: Creates a partitioned legacy BIOS disk image that the user -# can directly dd to boot media. - - -part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid - -bootloader --ptable gpt --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" - diff --git a/scripts/lib/wic/canned-wks/directdisk-multi-rootfs.wks b/scripts/lib/wic/canned-wks/directdisk-multi-rootfs.wks deleted file mode 100644 index 4fd1999ffb69..000000000000 --- a/scripts/lib/wic/canned-wks/directdisk-multi-rootfs.wks +++ /dev/null @@ -1,23 +0,0 @@ -# short-description: Create multi rootfs image using rootfs plugin -# long-description: Creates a partitioned disk image with two rootfs partitions -# using rootfs plugin. -# -# Partitions can use either -# - indirect rootfs references to image recipe(s): -# wic create directdisk-multi-indirect-recipes -e core-image-minimal \ -# --rootfs-dir rootfs1=core-image-minimal -# --rootfs-dir rootfs2=core-image-minimal-dev -# -# - or paths to rootfs directories: -# wic create directdisk-multi-rootfs \ -# --rootfs-dir rootfs1=tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/ -# --rootfs-dir rootfs2=tmp/work/qemux86_64-poky-linux/core-image-minimal-dev/1.0-r0/rootfs/ -# -# - or any combinations of -r and --rootfs command line options - -part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --rootfs-dir=rootfs1 --ondisk sda --fstype=ext4 --label platform --align 1024 -part /rescue --source rootfs --rootfs-dir=rootfs2 --ondisk sda --fstype=ext4 --label secondary --align 1024 - -bootloader --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" - diff --git a/scripts/lib/wic/canned-wks/directdisk.wks b/scripts/lib/wic/canned-wks/directdisk.wks deleted file mode 100644 index 8c8e06b02c66..000000000000 --- a/scripts/lib/wic/canned-wks/directdisk.wks +++ /dev/null @@ -1,8 +0,0 @@ -# short-description: Create a 'pcbios' direct disk image -# long-description: Creates a partitioned legacy BIOS disk image that the user -# can directly dd to boot media. - -include common.wks.inc - -bootloader --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" - diff --git a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in deleted file mode 100644 index 5211972955a5..000000000000 --- a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in +++ /dev/null @@ -1,3 +0,0 @@ -bootloader --ptable gpt -part /boot --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/boot --fstype=vfat --label boot --active --align 1024 --use-uuid --overhead-factor 1.2 -part / --source rootfs --fstype=ext4 --label root --align 1024 --exclude-path boot/ diff --git a/scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in b/scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in deleted file mode 100644 index cac0fa32cdab..000000000000 --- a/scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in +++ /dev/null @@ -1,3 +0,0 @@ -bootloader --ptable gpt --timeout=5 -part /boot --source bootimg_efi --sourceparams="loader=${EFI_PROVIDER}" --label boot --active --align 1024 --use-uuid --part-name="ESP" --part-type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B --overhead-factor=1 -part / --source rootfs --fstype=ext4 --label root --align 1024 --exclude-path boot/ diff --git a/scripts/lib/wic/canned-wks/mkdisk-ufs.wks b/scripts/lib/wic/canned-wks/mkdisk-ufs.wks deleted file mode 100644 index dfd2f3854890..000000000000 --- a/scripts/lib/wic/canned-wks/mkdisk-ufs.wks +++ /dev/null @@ -1,5 +0,0 @@ -bootloader --ptable gpt - -part /boot --source rootfs --fstype=vfat --label boot --change-directory boot/ -part / --source rootfs --fstype=ext4 --label root --exclude-path boot/ -part swap --fstype=swap --label swap --size 1M diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks deleted file mode 100644 index 16dfe76dfe27..000000000000 --- a/scripts/lib/wic/canned-wks/mkefidisk.wks +++ /dev/null @@ -1,11 +0,0 @@ -# short-description: Create an EFI disk image -# long-description: Creates a partitioned EFI disk image that the user -# can directly dd to boot media. - -part /boot --source bootimg_efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 - -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid - -part swap --ondisk sda --size 44 --label swap1 --fstype=swap - -bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=${KERNEL_CONSOLE} console=tty0" diff --git a/scripts/lib/wic/canned-wks/mkhybridiso.wks b/scripts/lib/wic/canned-wks/mkhybridiso.wks deleted file mode 100644 index c3a030e5b40e..000000000000 --- a/scripts/lib/wic/canned-wks/mkhybridiso.wks +++ /dev/null @@ -1,7 +0,0 @@ -# short-description: Create a hybrid ISO image -# long-description: Creates an EFI and legacy bootable hybrid ISO image -# which can be used on optical media as well as USB media. - -part /boot --source isoimage_isohybrid --sourceparams="loader=grub-efi,image_name=HYBRID_ISO_IMG" --ondisk cd --label HYBRIDISO - -bootloader --timeout=15 --append="" diff --git a/scripts/lib/wic/canned-wks/qemuloongarch.wks b/scripts/lib/wic/canned-wks/qemuloongarch.wks deleted file mode 100644 index 8465c7a8c0a2..000000000000 --- a/scripts/lib/wic/canned-wks/qemuloongarch.wks +++ /dev/null @@ -1,3 +0,0 @@ -# short-description: Create qcow2 image for LoongArch QEMU machines - -part / --source rootfs --fstype=ext4 --label root --align 4096 --size 5G diff --git a/scripts/lib/wic/canned-wks/qemuriscv.wks b/scripts/lib/wic/canned-wks/qemuriscv.wks deleted file mode 100644 index 12c68b706917..000000000000 --- a/scripts/lib/wic/canned-wks/qemuriscv.wks +++ /dev/null @@ -1,3 +0,0 @@ -# short-description: Create qcow2 image for RISC-V QEMU machines - -part / --source rootfs --fstype=ext4 --label root --align 4096 --size 5G diff --git a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks deleted file mode 100644 index 808997611a4a..000000000000 --- a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks +++ /dev/null @@ -1,8 +0,0 @@ -# short-description: Create a qemu machine 'pcbios' direct disk image -# long-description: Creates a partitioned legacy BIOS disk image that the user -# can directly use to boot a qemu machine. - -include common.wks.inc - -bootloader --timeout=0 --append="rw oprofile.timer=1 rootfstype=ext4 console=tty console=ttyS0 " - diff --git a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks b/scripts/lib/wic/canned-wks/sdimage-bootpart.wks deleted file mode 100644 index f9f8044f7dc4..000000000000 --- a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks +++ /dev/null @@ -1,6 +0,0 @@ -# short-description: Create SD card image with a boot partition -# long-description: Creates a partitioned SD card image. Boot files -# are located in the first vfat partition. - -part /boot --source bootimg_partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 -part / --source rootfs --ondisk mmcblk0 --fstype=ext4 --label root --align 4 diff --git a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks b/scripts/lib/wic/canned-wks/systemd-bootdisk.wks deleted file mode 100644 index 3fb2c0e35f3a..000000000000 --- a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks +++ /dev/null @@ -1,11 +0,0 @@ -# short-description: Create an EFI disk image with systemd-boot -# long-description: Creates a partitioned EFI disk image that the user -# can directly dd to boot media. The selected bootloader is systemd-boot. - -part /boot --source bootimg_efi --sourceparams="loader=systemd-boot" --ondisk sda --label msdos --active --align 1024 --use-uuid - -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid - -part swap --ondisk sda --size 44 --label swap1 --fstype=swap --use-uuid - -bootloader --ptable gpt --timeout=5 --append="rootwait rootfstype=ext4 console=ttyS0,115200 console=tty0" diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py deleted file mode 100644 index 949119edd487..000000000000 --- a/scripts/lib/wic/engine.py +++ /dev/null @@ -1,711 +0,0 @@ -# -# Copyright (c) 2013, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION - -# This module implements the image creation engine used by 'wic' to -# create images. The engine parses through the OpenEmbedded kickstart -# (wks) file specified and generates images that can then be directly -# written onto media. -# -# AUTHORS -# Tom Zanussi -# - -import logging -import os -import tempfile -import json -import subprocess -import shutil -import re - -from collections import namedtuple, OrderedDict - -from wic import WicError -from wic.filemap import sparse_copy -from wic.pluginbase import PluginMgr -from wic.misc import get_bitbake_var, exec_cmd - -logger = logging.getLogger('wic') - -def verify_build_env(): - """ - Verify that the build environment is sane. - - Returns True if it is, false otherwise - """ - if not os.environ.get("BUILDDIR"): - raise WicError("BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)") - - return True - - -CANNED_IMAGE_DIR = "lib/wic/canned-wks" # relative to scripts -SCRIPTS_CANNED_IMAGE_DIR = "scripts/" + CANNED_IMAGE_DIR -WIC_DIR = "wic" - -def build_canned_image_list(path): - layers_path = get_bitbake_var("BBLAYERS") - canned_wks_layer_dirs = [] - - if layers_path is not None: - for layer_path in layers_path.split(): - for wks_path in (WIC_DIR, SCRIPTS_CANNED_IMAGE_DIR): - cpath = os.path.join(layer_path, wks_path) - if os.path.isdir(cpath): - canned_wks_layer_dirs.append(cpath) - - cpath = os.path.join(path, CANNED_IMAGE_DIR) - canned_wks_layer_dirs.append(cpath) - - return canned_wks_layer_dirs - -def find_canned_image(scripts_path, wks_file): - """ - Find a .wks file with the given name in the canned files dir. - - Return False if not found - """ - layers_canned_wks_dir = build_canned_image_list(scripts_path) - - for canned_wks_dir in layers_canned_wks_dir: - for root, dirs, files in os.walk(canned_wks_dir): - for fname in files: - if fname.endswith("~") or fname.endswith("#"): - continue - if ((fname.endswith(".wks") and wks_file + ".wks" == fname) or \ - (fname.endswith(".wks.in") and wks_file + ".wks.in" == fname)): - fullpath = os.path.join(canned_wks_dir, fname) - return fullpath - return None - - -def list_canned_images(scripts_path): - """ - List the .wks files in the canned image dir, minus the extension. - """ - layers_canned_wks_dir = build_canned_image_list(scripts_path) - - for canned_wks_dir in layers_canned_wks_dir: - for root, dirs, files in os.walk(canned_wks_dir): - for fname in files: - if fname.endswith("~") or fname.endswith("#"): - continue - if fname.endswith(".wks") or fname.endswith(".wks.in"): - fullpath = os.path.join(canned_wks_dir, fname) - with open(fullpath) as wks: - for line in wks: - desc = "" - idx = line.find("short-description:") - if idx != -1: - desc = line[idx + len("short-description:"):].strip() - break - basename = fname.split('.')[0] - print(" %s\t\t%s" % (basename.ljust(30), desc)) - - -def list_canned_image_help(scripts_path, fullpath): - """ - List the help and params in the specified canned image. - """ - found = False - with open(fullpath) as wks: - for line in wks: - if not found: - idx = line.find("long-description:") - if idx != -1: - print() - print(line[idx + len("long-description:"):].strip()) - found = True - continue - if not line.strip(): - break - idx = line.find("#") - if idx != -1: - print(line[idx + len("#:"):].rstrip()) - else: - break - - -def list_source_plugins(): - """ - List the available source plugins i.e. plugins available for --source. - """ - plugins = PluginMgr.get_plugins('source') - - for plugin in plugins: - print(" %s" % plugin) - - -def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, - native_sysroot, options): - """ - Create image - - wks_file - user-defined OE kickstart file - rootfs_dir - absolute path to the build's /rootfs dir - bootimg_dir - absolute path to the build's boot artifacts directory - kernel_dir - absolute path to the build's kernel directory - native_sysroot - absolute path to the build's native sysroots dir - image_output_dir - dirname to create for image - options - wic command line options (debug, bmap, etc) - - Normally, the values for the build artifacts values are determined - by 'wic -e' from the output of the 'bitbake -e' command given an - image name e.g. 'core-image-minimal' and a given machine set in - local.conf. If that's the case, the variables get the following - values from the output of 'bitbake -e': - - rootfs_dir: IMAGE_ROOTFS - kernel_dir: DEPLOY_DIR_IMAGE - native_sysroot: STAGING_DIR_NATIVE - - In the above case, bootimg_dir remains unset and the - plugin-specific image creation code is responsible for finding the - bootimg artifacts. - - In the case where the values are passed in explicitly i.e 'wic -e' - is not used but rather the individual 'wic' options are used to - explicitly specify these values. - """ - try: - oe_builddir = os.environ["BUILDDIR"] - except KeyError: - raise WicError("BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)") - - if not os.path.exists(options.outdir): - os.makedirs(options.outdir) - - pname = options.imager - # Don't support '-' in plugin names - pname = pname.replace("-", "_") - plugin_class = PluginMgr.get_plugins('imager').get(pname) - if not plugin_class: - raise WicError('Unknown plugin: %s' % pname) - - plugin = plugin_class(wks_file, rootfs_dir, bootimg_dir, kernel_dir, - native_sysroot, oe_builddir, options) - - plugin.do_create() - - logger.info("The image(s) were created using OE kickstart file:\n %s", wks_file) - - -def wic_list(args, scripts_path): - """ - Print the list of images or source plugins. - """ - if args.list_type is None: - return False - - if args.list_type == "images": - - list_canned_images(scripts_path) - return True - elif args.list_type == "source-plugins": - list_source_plugins() - return True - elif len(args.help_for) == 1 and args.help_for[0] == 'help': - wks_file = args.list_type - fullpath = find_canned_image(scripts_path, wks_file) - if not fullpath: - raise WicError("No image named %s found, exiting. " - "(Use 'wic list images' to list available images, " - "or specify a fully-qualified OE kickstart (.wks) " - "filename)" % wks_file) - - list_canned_image_help(scripts_path, fullpath) - return True - - return False - -_DEBUGFS_VERSION = None - -def debugfs_version_check(debugfs_path, min_ver=(1, 46, 5)): - global _DEBUGFS_VERSION - - if _DEBUGFS_VERSION is None: - out = "" - for flag in ("-V", "-v"): - try: - out = exec_cmd(f"{debugfs_path} {flag}") - break - except Exception: - continue - - import re - m = re.search(r"(\d+)\.(\d+)\.(\d+)", out or "") - _DEBUGFS_VERSION = tuple(map(int, m.groups())) if m else None - - ver = _DEBUGFS_VERSION - - if ver is not None and ver < min_ver: - raise WicError( - "Sorry, debugfs 1.46.5 or later is required for this script. " - "Older versions of debugfs can make directory copies into ext* partitions " - "via scripted debugfs (-f) unreliable or broken. Detected version: %s" - % (".".join(map(str, ver)) if ver else "unknown") - ) - - -class Disk: - def __init__(self, imagepath, native_sysroot, fstypes=('fat', 'ext'), sector_size=512): - self.imagepath = imagepath - self.native_sysroot = native_sysroot - self.fstypes = fstypes - self._partitions = None - self._partimages = {} - self._lsector_size = None - self._psector_size = None - self._ptable_format = None - self.sector_size = sector_size - - # find parted - # read paths from $PATH environment variable - # if it fails, use hardcoded paths - pathlist = "/bin:/usr/bin:/usr/sbin:/sbin/" - try: - self.paths = os.environ['PATH'] + ":" + pathlist - except KeyError: - self.paths = pathlist - - if native_sysroot: - for path in pathlist.split(':'): - self.paths = "%s%s:%s" % (native_sysroot, path, self.paths) - - self.parted = shutil.which("parted", path=self.paths) - if not self.parted: - raise WicError("Can't find executable parted") - - self.partitions = self.get_partitions() - - def __del__(self): - for path in self._partimages.values(): - os.unlink(path) - - def get_partitions(self): - if self._partitions is None: - self._partitions = OrderedDict() - - if self.sector_size: - out = exec_cmd("export PARTED_SECTOR_SIZE=%d; %s -sm %s unit B print" % \ - (self.sector_size, self.parted, self.imagepath), True) - else: - out = exec_cmd("%s -sm %s unit B print" % (self.parted, self.imagepath)) - - parttype = namedtuple("Part", "pnum start end size fstype") - splitted = out.splitlines() - # skip over possible errors in exec_cmd output - try: - idx =splitted.index("BYT;") - except ValueError: - raise WicError("Error getting partition information from %s" % (self.parted)) - lsector_size, psector_size, self._ptable_format = splitted[idx + 1].split(":")[3:6] - self._lsector_size = int(lsector_size) - self._psector_size = int(psector_size) - for line in splitted[idx + 2:]: - pnum, start, end, size, fstype = line.split(':')[:5] - partition = parttype(int(pnum), int(start[:-1]), int(end[:-1]), - int(size[:-1]), fstype) - self._partitions[pnum] = partition - - return self._partitions - - def __getattr__(self, name): - """Get path to the executable in a lazy way.""" - if name in ("mdir", "mcopy", "mdel", "mdeltree", "sfdisk", "e2fsck", - "resize2fs", "mkswap", "mkdosfs", "debugfs","blkid"): - aname = "_%s" % name - if aname not in self.__dict__: - setattr(self, aname, shutil.which(name, path=self.paths)) - if aname not in self.__dict__ or self.__dict__[aname] is None: - raise WicError("Can't find executable '{}'".format(name)) - return self.__dict__[aname] - return self.__dict__[name] - - def _get_part_image(self, pnum): - if pnum not in self.partitions: - raise WicError("Partition %s is not in the image" % pnum) - part = self.partitions[pnum] - # check if fstype is supported - # NOTE: parted is unable to identify dos-type partitions with a 4k sector-size - # if the type is empty and a non-default sector size is used, assume 'fat' - # fdisk identifies them without issue - part_fstype = part.fstype if part.fstype or self.sector_size == 512 else 'fat' - for fstype in self.fstypes: - if part_fstype.startswith(fstype): - break - else: - raise WicError("Not supported fstype: {}".format(part_fstype)) - if pnum not in self._partimages: - tmpf = tempfile.NamedTemporaryFile(prefix="wic-part") - dst_fname = tmpf.name - tmpf.close() - sparse_copy(self.imagepath, dst_fname, skip=part.start, length=part.size) - self._partimages[pnum] = dst_fname - - return self._partimages[pnum] - - def _put_part_image(self, pnum): - """Put partition image into partitioned image.""" - sparse_copy(self._partimages[pnum], self.imagepath, - seek=self.partitions[pnum].start) - - def dir(self, pnum, path): - if pnum not in self.partitions: - raise WicError("Partition %s is not in the image" % pnum) - - if self.partitions[pnum].fstype.startswith('ext'): - return exec_cmd("{} {} -R 'ls -l {}'".format(self.debugfs, - self._get_part_image(pnum), - path), as_shell=True) - else: # fat - return exec_cmd("{} -i {} ::{}".format(self.mdir, - self._get_part_image(pnum), - path)) - - def copy(self, src, dest): - """Copy files or directories to/from the vfat or ext* partition.""" - pnum = dest.part if isinstance(src, str) else src.part - partimg = self._get_part_image(pnum) - - if self.partitions[pnum].fstype.startswith('ext'): - if isinstance(src, str): # host to image case - if os.path.isdir(src): - debugfs_version_check(self.debugfs) - base = os.path.abspath(src) - base_parent = os.path.dirname(base) - cmds = [] - made = set() - - for root, dirs, files in os.walk(base): - for fname in files: - host_file = os.path.join(root, fname) - rel = os.path.relpath(host_file, base_parent) - dest_file = os.path.join(dest.path, rel) - dest_dir = os.path.dirname(dest_file) - - # create dir structure (mkdir -p) - parts = dest_dir.strip('/').split('/') - cur = '' - for p in parts: - cur = cur + '/' + p - if cur not in made: - cmds.append(f'mkdir "{cur}"') - made.add(cur) - - cmds.append(f'write "{host_file}" "{dest_file}"') - - # write script to a temp file - with tempfile.NamedTemporaryFile(mode='w', delete=False, - prefix='wic-debugfs-') as tf: - for line in cmds: - tf.write(line + '\n') - scriptname = tf.name - - cmd = f"{self.debugfs} -w -f {scriptname} {partimg}" - - else: # single file - cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\ - format(os.path.dirname(dest.path), src, - os.path.basename(src), self.debugfs, partimg) - - else: # image to host case - cmd = "printf 'cd {}\ndump /{} {}\nrdump /{} {}\n' | {} {}".\ - format(os.path.dirname(src.path), src.path, - dest, src.path, dest, self.debugfs, partimg) - - else: # fat - if isinstance(src, str): - cmd = "{} -i {} -snop {} ::{}".format(self.mcopy, - partimg, - src, dest.path) - else: - cmd = "{} -i {} -snop ::{} {}".format(self.mcopy, - partimg, - src.path, dest) - - exec_cmd(cmd, as_shell=True) - self._put_part_image(pnum) - - def remove_ext(self, pnum, path, recursive): - """ - Remove files/dirs and their contents from the partition. - This only applies to ext* partition. - """ - abs_path = re.sub(r'\/\/+', '/', path) - cmd = "{} {} -wR 'rm \"{}\"'".format(self.debugfs, - self._get_part_image(pnum), - abs_path) - out = exec_cmd(cmd , as_shell=True) - for line in out.splitlines(): - if line.startswith("rm:"): - if "file is a directory" in line: - if recursive: - # loop through content and delete them one by one if - # flaged with -r - subdirs = iter(self.dir(pnum, abs_path).splitlines()) - next(subdirs) - for subdir in subdirs: - dir = subdir.split(':')[1].split(" ", 1)[1] - if not dir == "." and not dir == "..": - self.remove_ext(pnum, "%s/%s" % (abs_path, dir), recursive) - - rmdir_out = exec_cmd("{} {} -wR 'rmdir \"{}\"'".format(self.debugfs, - self._get_part_image(pnum), - abs_path.rstrip('/')) - , as_shell=True) - - for rmdir_line in rmdir_out.splitlines(): - if "directory not empty" in rmdir_line: - raise WicError("Could not complete operation: \n%s \n" - "use -r to remove non-empty directory" % rmdir_line) - if rmdir_line.startswith("rmdir:"): - raise WicError("Could not complete operation: \n%s " - "\n%s" % (str(line), rmdir_line)) - - else: - raise WicError("Could not complete operation: \n%s " - "\nUnable to remove %s" % (str(line), abs_path)) - - def remove(self, pnum, path, recursive): - """Remove files/dirs from the partition.""" - partimg = self._get_part_image(pnum) - if self.partitions[pnum].fstype.startswith('ext'): - self.remove_ext(pnum, path, recursive) - - else: # fat - cmd = "{} -i {} ::{}".format(self.mdel, partimg, path) - try: - exec_cmd(cmd) - except WicError as err: - if "not found" in str(err) or "non empty" in str(err): - # mdel outputs 'File ... not found' or 'directory .. non empty" - # try to use mdeltree as path could be a directory - cmd = "{} -i {} ::{}".format(self.mdeltree, - partimg, path) - exec_cmd(cmd) - else: - raise err - self._put_part_image(pnum) - - def write(self, target, expand): - """Write disk image to the media or file.""" - def write_sfdisk_script(outf, parts): - for key, val in parts['partitiontable'].items(): - if key in ("partitions", "device", "firstlba", "lastlba"): - continue - if key == "id": - key = "label-id" - outf.write("{}: {}\n".format(key, val)) - outf.write("\n") - for part in parts['partitiontable']['partitions']: - line = '' - for name in ('attrs', 'name', 'size', 'type', 'uuid'): - if name == 'size' and part['type'] == 'f': - # don't write size for extended partition - continue - val = part.get(name) - if val: - line += '{}={}, '.format(name, val) - if line: - line = line[:-2] # strip ', ' - if part.get('bootable'): - line += ' ,bootable' - outf.write("{}\n".format(line)) - outf.flush() - - def read_ptable(path): - out = exec_cmd("{} -J {}".format(self.sfdisk, path)) - return json.loads(out) - - def write_ptable(parts, target): - with tempfile.NamedTemporaryFile(prefix="wic-sfdisk-", mode='w') as outf: - write_sfdisk_script(outf, parts) - cmd = "{} --no-reread {} < {} ".format(self.sfdisk, target, outf.name) - exec_cmd(cmd, as_shell=True) - - if expand is None: - sparse_copy(self.imagepath, target) - else: - # copy first sectors that may contain bootloader - sparse_copy(self.imagepath, target, length=2048 * self._lsector_size) - - # copy source partition table to the target - parts = read_ptable(self.imagepath) - write_ptable(parts, target) - - # get size of unpartitioned space - free = None - for line in exec_cmd("{} -F {}".format(self.sfdisk, target)).splitlines(): - if line.startswith("Unpartitioned space ") and line.endswith("sectors"): - free = int(line.split()[-2]) - # Align free space to a 2048 sector boundary. YOCTO #12840. - free = free - (free % 2048) - if free is None: - raise WicError("Can't get size of unpartitioned space") - - # calculate expanded partitions sizes - sizes = {} - num_auto_resize = 0 - for num, part in enumerate(parts['partitiontable']['partitions'], 1): - if num in expand: - if expand[num] != 0: # don't resize partition if size is set to 0 - sectors = expand[num] // self._lsector_size - free -= sectors - part['size'] - part['size'] = sectors - sizes[num] = sectors - elif part['type'] != 'f': - sizes[num] = -1 - num_auto_resize += 1 - - for num, part in enumerate(parts['partitiontable']['partitions'], 1): - if sizes.get(num) == -1: - part['size'] += free // num_auto_resize - - # write resized partition table to the target - write_ptable(parts, target) - - # read resized partition table - parts = read_ptable(target) - - # copy partitions content - for num, part in enumerate(parts['partitiontable']['partitions'], 1): - pnum = str(num) - fstype = self.partitions[pnum].fstype - - # copy unchanged partition - if part['size'] == self.partitions[pnum].size // self._lsector_size: - logger.info("copying unchanged partition {}".format(pnum)) - sparse_copy(self._get_part_image(pnum), target, seek=part['start'] * self._lsector_size) - continue - - # resize or re-create partitions - if fstype.startswith('ext') or fstype.startswith('fat') or \ - fstype.startswith('linux-swap'): - - partfname = None - with tempfile.NamedTemporaryFile(prefix="wic-part{}-".format(pnum)) as partf: - partfname = partf.name - - if fstype.startswith('ext'): - logger.info("resizing ext partition {}".format(pnum)) - partimg = self._get_part_image(pnum) - sparse_copy(partimg, partfname) - exec_cmd("{} -pf {}".format(self.e2fsck, partfname)) - exec_cmd("{} {} {}s".format(\ - self.resize2fs, partfname, part['size'])) - elif fstype.startswith('fat'): - logger.info("copying content of the fat partition {}".format(pnum)) - with tempfile.TemporaryDirectory(prefix='wic-fatdir-') as tmpdir: - # copy content to the temporary directory - cmd = "{} -snompi {} :: {}".format(self.mcopy, - self._get_part_image(pnum), - tmpdir) - exec_cmd(cmd) - # create new msdos partition - label = part.get("name") - label_str = "-n {}".format(label) if label else '' - - sector_str = "-S {}".format(self.sector_size) if self.sector_size else '' - cmd = "{} {} {} -C {} {}".format(self.mkdosfs, label_str, sector_str, partfname, - part['size']) - exec_cmd(cmd) - # copy content from the temporary directory to the new partition - cmd = "{} -snompi {} {}/* ::".format(self.mcopy, partfname, tmpdir) - exec_cmd(cmd, as_shell=True) - elif fstype.startswith('linux-swap'): - logger.info("creating swap partition {}".format(pnum)) - label = part.get("name") - label_str = "-L {}".format(label) if label else '' - out = exec_cmd("{} --probe {}".format(self.blkid, self._get_part_image(pnum))) - uuid = out[out.index("UUID=\"")+6:out.index("UUID=\"")+42] - uuid_str = "-U {}".format(uuid) if uuid else '' - with open(partfname, 'w') as sparse: - os.ftruncate(sparse.fileno(), part['size'] * self._lsector_size) - exec_cmd("{} {} {} {}".format(self.mkswap, label_str, uuid_str, partfname)) - sparse_copy(partfname, target, seek=part['start'] * self._lsector_size) - os.unlink(partfname) - elif part['type'] != 'f': - logger.warning("skipping partition {}: unsupported fstype {}".format(pnum, fstype)) - -def wic_ls(args, native_sysroot): - """List contents of partitioned image or vfat partition.""" - disk = Disk(args.path.image, native_sysroot, sector_size=args.sector_size) - if not args.path.part: - if disk.partitions: - print('Num Start End Size Fstype') - for part in disk.partitions.values(): - # size values are in bytes from parted; convert to sectors if a custom sector size was requested - display_size = part.size - if args.sector_size and args.sector_size != disk._lsector_size: - display_size = part.size // args.sector_size - print("{:2d} {:12d} {:12d} {:12d} {}".format(\ - part.pnum, part.start // args.sector_size, - part.end // args.sector_size, - display_size, part.fstype)) - else: - path = args.path.path or '/' - print(disk.dir(args.path.part, path)) - -def wic_cp(args, native_sysroot): - """ - Copy file or directory to/from the vfat/ext partition of - partitioned image. - """ - if isinstance(args.dest, str): - disk = Disk(args.src.image, native_sysroot, sector_size=args.sector_size) - else: - disk = Disk(args.dest.image, native_sysroot, sector_size=args.sector_size) - disk.copy(args.src, args.dest) - - -def wic_rm(args, native_sysroot): - """ - Remove files or directories from the vfat partition of - partitioned image. - """ - disk = Disk(args.path.image, native_sysroot, sector_size=args.sector_size) - disk.remove(args.path.part, args.path.path, args.recursive_delete) - -def wic_write(args, native_sysroot): - """ - Write image to a target device. - """ - disk = Disk(args.image, native_sysroot, ('fat', 'ext', 'linux-swap')) - disk.write(args.target, args.expand) - -def find_canned(scripts_path, file_name): - """ - Find a file either by its path or by name in the canned files dir. - - Return None if not found - """ - if os.path.exists(file_name): - return file_name - - layers_canned_wks_dir = build_canned_image_list(scripts_path) - for canned_wks_dir in layers_canned_wks_dir: - for root, dirs, files in os.walk(canned_wks_dir): - for fname in files: - if fname == file_name: - fullpath = os.path.join(canned_wks_dir, fname) - return fullpath - -def get_custom_config(boot_file): - """ - Get the custom configuration to be used for the bootloader. - - Return None if the file can't be found. - """ - # Get the scripts path of poky - scripts_path = os.path.abspath("%s/../.." % os.path.dirname(__file__)) - - cfg_file = find_canned(scripts_path, boot_file) - if cfg_file: - with open(cfg_file, "r") as f: - config = f.read() - return config diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py deleted file mode 100644 index 85b39d5d743e..000000000000 --- a/scripts/lib/wic/filemap.py +++ /dev/null @@ -1,583 +0,0 @@ -# -# Copyright (c) 2012 Intel, Inc. -# -# SPDX-License-Identifier: GPL-2.0-only -# - -""" -This module implements python implements a way to get file block. Two methods -are supported - the FIEMAP ioctl and the 'SEEK_HOLE / SEEK_DATA' features of -the file seek syscall. The former is implemented by the 'FilemapFiemap' class, -the latter is implemented by the 'FilemapSeek' class. Both classes provide the -same API. The 'filemap' function automatically selects which class can be used -and returns an instance of the class. -""" - -# Disable the following pylint recommendations: -# * Too many instance attributes (R0902) -# pylint: disable=R0902 - -import errno -import os -import struct -import array -import fcntl -import tempfile -import logging - -def get_block_size(file_obj): - """ - Returns block size for file object 'file_obj'. Errors are indicated by the - 'IOError' exception. - """ - # Get the block size of the host file-system for the image file by calling - # the FIGETBSZ ioctl (number 2). - try: - binary_data = fcntl.ioctl(file_obj, 2, struct.pack('I', 0)) - bsize = struct.unpack('I', binary_data)[0] - except OSError: - bsize = None - - # If ioctl causes OSError or give bsize to zero failback to os.fstat - if not bsize: - import os - stat = os.fstat(file_obj.fileno()) - if hasattr(stat, 'st_blksize'): - bsize = stat.st_blksize - else: - raise IOError("Unable to determine block size") - - # The logic in this script only supports a maximum of a 4KB - # block size - max_block_size = 4 * 1024 - if bsize > max_block_size: - bsize = max_block_size - - return bsize - -class ErrorNotSupp(Exception): - """ - An exception of this type is raised when the 'FIEMAP' or 'SEEK_HOLE' feature - is not supported either by the kernel or the file-system. - """ - pass - -class Error(Exception): - """A class for all the other exceptions raised by this module.""" - pass - - -class _FilemapBase(object): - """ - This is a base class for a couple of other classes in this module. This - class simply performs the common parts of the initialization process: opens - the image file, gets its size, etc. The 'log' parameter is the logger object - to use for printing messages. - """ - - def __init__(self, image, log=None): - """ - Initialize a class instance. The 'image' argument is full path to the - file or file object to operate on. - """ - - self._log = log - if self._log is None: - self._log = logging.getLogger(__name__) - - self._f_image_needs_close = False - - if hasattr(image, "fileno"): - self._f_image = image - self._image_path = image.name - else: - self._image_path = image - self._open_image_file() - - try: - self.image_size = os.fstat(self._f_image.fileno()).st_size - except IOError as err: - raise Error("cannot get information about file '%s': %s" - % (self._f_image.name, err)) - - try: - self.block_size = get_block_size(self._f_image) - except IOError as err: - raise Error("cannot get block size for '%s': %s" - % (self._image_path, err)) - - self.blocks_cnt = self.image_size + self.block_size - 1 - self.blocks_cnt //= self.block_size - - try: - self._f_image.flush() - except IOError as err: - raise Error("cannot flush image file '%s': %s" - % (self._image_path, err)) - - try: - os.fsync(self._f_image.fileno()), - except OSError as err: - raise Error("cannot synchronize image file '%s': %s " - % (self._image_path, err.strerror)) - - self._log.debug("opened image \"%s\"" % self._image_path) - self._log.debug("block size %d, blocks count %d, image size %d" - % (self.block_size, self.blocks_cnt, self.image_size)) - - def __del__(self): - """The class destructor which just closes the image file.""" - if self._f_image_needs_close: - self._f_image.close() - - def _open_image_file(self): - """Open the image file.""" - try: - self._f_image = open(self._image_path, 'rb') - except IOError as err: - raise Error("cannot open image file '%s': %s" - % (self._image_path, err)) - - self._f_image_needs_close = True - - def block_is_mapped(self, block): # pylint: disable=W0613,R0201 - """ - This method has has to be implemented by child classes. It returns - 'True' if block number 'block' of the image file is mapped and 'False' - otherwise. - """ - - raise Error("the method is not implemented") - - def get_mapped_ranges(self, start, count): # pylint: disable=W0613,R0201 - """ - This method has has to be implemented by child classes. This is a - generator which yields ranges of mapped blocks in the file. The ranges - are tuples of 2 elements: [first, last], where 'first' is the first - mapped block and 'last' is the last mapped block. - - The ranges are yielded for the area of the file of size 'count' blocks, - starting from block 'start'. - """ - - raise Error("the method is not implemented") - - -# The 'SEEK_HOLE' and 'SEEK_DATA' options of the file seek system call -_SEEK_DATA = 3 -_SEEK_HOLE = 4 - -def _lseek(file_obj, offset, whence): - """This is a helper function which invokes 'os.lseek' for file object - 'file_obj' and with specified 'offset' and 'whence'. The 'whence' - argument is supposed to be either '_SEEK_DATA' or '_SEEK_HOLE'. When - there is no more data or hole starting from 'offset', this function - returns '-1'. Otherwise the data or hole position is returned.""" - - try: - return os.lseek(file_obj.fileno(), offset, whence) - except OSError as err: - # The 'lseek' system call returns the ENXIO if there is no data or - # hole starting from the specified offset. - if err.errno == errno.ENXIO: - return -1 - elif err.errno == errno.EINVAL: - raise ErrorNotSupp("the kernel or file-system does not support " - "\"SEEK_HOLE\" and \"SEEK_DATA\"") - else: - raise - -class FilemapSeek(_FilemapBase): - """ - This class uses the 'SEEK_HOLE' and 'SEEK_DATA' to find file block mapping. - Unfortunately, the current implementation requires the caller to have write - access to the image file. - """ - - def __init__(self, image, log=None): - """Refer the '_FilemapBase' class for the documentation.""" - - # Call the base class constructor first - _FilemapBase.__init__(self, image, log) - self._log.debug("FilemapSeek: initializing") - - self._probe_seek_hole() - - def _probe_seek_hole(self): - """ - Check whether the system implements 'SEEK_HOLE' and 'SEEK_DATA'. - Unfortunately, there seems to be no clean way for detecting this, - because often the system just fakes them by just assuming that all - files are fully mapped, so 'SEEK_HOLE' always returns EOF and - 'SEEK_DATA' always returns the requested offset. - - I could not invent a better way of detecting the fake 'SEEK_HOLE' - implementation than just to create a temporary file in the same - directory where the image file resides. It would be nice to change this - to something better. - """ - - directory = os.path.dirname(self._image_path) - - try: - tmp_obj = tempfile.TemporaryFile("w+", dir=directory) - except IOError as err: - raise ErrorNotSupp("cannot create a temporary in \"%s\": %s" \ - % (directory, err)) - - try: - os.ftruncate(tmp_obj.fileno(), self.block_size) - except OSError as err: - raise ErrorNotSupp("cannot truncate temporary file in \"%s\": %s" - % (directory, err)) - - offs = _lseek(tmp_obj, 0, _SEEK_HOLE) - if offs != 0: - # We are dealing with the stub 'SEEK_HOLE' implementation which - # always returns EOF. - self._log.debug("lseek(0, SEEK_HOLE) returned %d" % offs) - raise ErrorNotSupp("the file-system does not support " - "\"SEEK_HOLE\" and \"SEEK_DATA\" but only " - "provides a stub implementation") - - tmp_obj.close() - - def block_is_mapped(self, block): - """Refer the '_FilemapBase' class for the documentation.""" - offs = _lseek(self._f_image, block * self.block_size, _SEEK_DATA) - if offs == -1: - result = False - else: - result = (offs // self.block_size == block) - - self._log.debug("FilemapSeek: block_is_mapped(%d) returns %s" - % (block, result)) - return result - - def _get_ranges(self, start, count, whence1, whence2): - """ - This function implements 'get_mapped_ranges()' depending - on what is passed in the 'whence1' and 'whence2' arguments. - """ - - assert whence1 != whence2 - end = start * self.block_size - limit = end + count * self.block_size - - while True: - start = _lseek(self._f_image, end, whence1) - if start == -1 or start >= limit or start == self.image_size: - break - - end = _lseek(self._f_image, start, whence2) - if end == -1 or end == self.image_size: - end = self.blocks_cnt * self.block_size - if end > limit: - end = limit - - start_blk = start // self.block_size - end_blk = end // self.block_size - 1 - self._log.debug("FilemapSeek: yielding range (%d, %d)" - % (start_blk, end_blk)) - yield (start_blk, end_blk) - - def get_mapped_ranges(self, start, count): - """Refer the '_FilemapBase' class for the documentation.""" - self._log.debug("FilemapSeek: get_mapped_ranges(%d, %d(%d))" - % (start, count, start + count - 1)) - return self._get_ranges(start, count, _SEEK_DATA, _SEEK_HOLE) - - -# Below goes the FIEMAP ioctl implementation, which is not very readable -# because it deals with the rather complex FIEMAP ioctl. To understand the -# code, you need to know the FIEMAP interface, which is documented in the -# "Documentation/filesystems/fiemap.txt" file in the Linux kernel sources. - -# Format string for 'struct fiemap' -_FIEMAP_FORMAT = "=QQLLLL" -# sizeof(struct fiemap) -_FIEMAP_SIZE = struct.calcsize(_FIEMAP_FORMAT) -# Format string for 'struct fiemap_extent' -_FIEMAP_EXTENT_FORMAT = "=QQQQQLLLL" -# sizeof(struct fiemap_extent) -_FIEMAP_EXTENT_SIZE = struct.calcsize(_FIEMAP_EXTENT_FORMAT) -# The FIEMAP ioctl number -_FIEMAP_IOCTL = 0xC020660B -# This FIEMAP ioctl flag which instructs the kernel to sync the file before -# reading the block map -_FIEMAP_FLAG_SYNC = 0x00000001 -# Size of the buffer for 'struct fiemap_extent' elements which will be used -# when invoking the FIEMAP ioctl. The larger is the buffer, the less times the -# FIEMAP ioctl will be invoked. -_FIEMAP_BUFFER_SIZE = 256 * 1024 - -class FilemapFiemap(_FilemapBase): - """ - This class provides API to the FIEMAP ioctl. Namely, it allows to iterate - over all mapped blocks and over all holes. - - This class synchronizes the image file every time it invokes the FIEMAP - ioctl in order to work-around early FIEMAP implementation kernel bugs. - """ - - def __init__(self, image, log=None): - """ - Initialize a class instance. The 'image' argument is full the file - object to operate on. - """ - - # Call the base class constructor first - _FilemapBase.__init__(self, image, log) - self._log.debug("FilemapFiemap: initializing") - - self._buf_size = _FIEMAP_BUFFER_SIZE - - # Calculate how many 'struct fiemap_extent' elements fit the buffer - self._buf_size -= _FIEMAP_SIZE - self._fiemap_extent_cnt = self._buf_size // _FIEMAP_EXTENT_SIZE - assert self._fiemap_extent_cnt > 0 - self._buf_size = self._fiemap_extent_cnt * _FIEMAP_EXTENT_SIZE - self._buf_size += _FIEMAP_SIZE - - # Allocate a mutable buffer for the FIEMAP ioctl - self._buf = array.array('B', [0] * self._buf_size) - - # Check if the FIEMAP ioctl is supported - self.block_is_mapped(0) - - def _invoke_fiemap(self, block, count): - """ - Invoke the FIEMAP ioctl for 'count' blocks of the file starting from - block number 'block'. - - The full result of the operation is stored in 'self._buf' on exit. - Returns the unpacked 'struct fiemap' data structure in form of a python - list (just like 'struct.upack()'). - """ - - if self.blocks_cnt != 0 and (block < 0 or block >= self.blocks_cnt): - raise Error("bad block number %d, should be within [0, %d]" - % (block, self.blocks_cnt)) - - # Initialize the 'struct fiemap' part of the buffer. We use the - # '_FIEMAP_FLAG_SYNC' flag in order to make sure the file is - # synchronized. The reason for this is that early FIEMAP - # implementations had many bugs related to cached dirty data, and - # synchronizing the file is a necessary work-around. - struct.pack_into(_FIEMAP_FORMAT, self._buf, 0, block * self.block_size, - count * self.block_size, _FIEMAP_FLAG_SYNC, 0, - self._fiemap_extent_cnt, 0) - - try: - fcntl.ioctl(self._f_image, _FIEMAP_IOCTL, self._buf, 1) - except IOError as err: - # Note, the FIEMAP ioctl is supported by the Linux kernel starting - # from version 2.6.28 (year 2008). - if err.errno == errno.EOPNOTSUPP: - errstr = "FilemapFiemap: the FIEMAP ioctl is not supported " \ - "by the file-system" - self._log.debug(errstr) - raise ErrorNotSupp(errstr) - if err.errno == errno.ENOTTY: - errstr = "FilemapFiemap: the FIEMAP ioctl is not supported " \ - "by the kernel" - self._log.debug(errstr) - raise ErrorNotSupp(errstr) - raise Error("the FIEMAP ioctl failed for '%s': %s" - % (self._image_path, err)) - - return struct.unpack(_FIEMAP_FORMAT, self._buf[:_FIEMAP_SIZE]) - - def block_is_mapped(self, block): - """Refer the '_FilemapBase' class for the documentation.""" - struct_fiemap = self._invoke_fiemap(block, 1) - - # The 3rd element of 'struct_fiemap' is the 'fm_mapped_extents' field. - # If it contains zero, the block is not mapped, otherwise it is - # mapped. - result = bool(struct_fiemap[3]) - self._log.debug("FilemapFiemap: block_is_mapped(%d) returns %s" - % (block, result)) - return result - - def _unpack_fiemap_extent(self, index): - """ - Unpack a 'struct fiemap_extent' structure object number 'index' from - the internal 'self._buf' buffer. - """ - - offset = _FIEMAP_SIZE + _FIEMAP_EXTENT_SIZE * index - return struct.unpack(_FIEMAP_EXTENT_FORMAT, - self._buf[offset : offset + _FIEMAP_EXTENT_SIZE]) - - def _do_get_mapped_ranges(self, start, count): - """ - Implements most the functionality for the 'get_mapped_ranges()' - generator: invokes the FIEMAP ioctl, walks through the mapped extents - and yields mapped block ranges. However, the ranges may be consecutive - (e.g., (1, 100), (100, 200)) and 'get_mapped_ranges()' simply merges - them. - """ - - block = start - while block < start + count: - struct_fiemap = self._invoke_fiemap(block, count) - - mapped_extents = struct_fiemap[3] - if mapped_extents == 0: - # No more mapped blocks - return - - extent = 0 - while extent < mapped_extents: - fiemap_extent = self._unpack_fiemap_extent(extent) - - # Start of the extent - extent_start = fiemap_extent[0] - # Starting block number of the extent - extent_block = extent_start // self.block_size - # Length of the extent - extent_len = fiemap_extent[2] - # Count of blocks in the extent - extent_count = extent_len // self.block_size - - # Extent length and offset have to be block-aligned - assert extent_start % self.block_size == 0 - assert extent_len % self.block_size == 0 - - if extent_block > start + count - 1: - return - - first = max(extent_block, block) - last = min(extent_block + extent_count, start + count) - 1 - yield (first, last) - - extent += 1 - - block = extent_block + extent_count - - def get_mapped_ranges(self, start, count): - """Refer the '_FilemapBase' class for the documentation.""" - self._log.debug("FilemapFiemap: get_mapped_ranges(%d, %d(%d))" - % (start, count, start + count - 1)) - iterator = self._do_get_mapped_ranges(start, count) - first_prev, last_prev = next(iterator) - - for first, last in iterator: - if last_prev == first - 1: - last_prev = last - else: - self._log.debug("FilemapFiemap: yielding range (%d, %d)" - % (first_prev, last_prev)) - yield (first_prev, last_prev) - first_prev, last_prev = first, last - - self._log.debug("FilemapFiemap: yielding range (%d, %d)" - % (first_prev, last_prev)) - yield (first_prev, last_prev) - -class FilemapNobmap(_FilemapBase): - """ - This class is used when both the 'SEEK_DATA/HOLE' and FIEMAP are not - supported by the filesystem or kernel. - """ - - def __init__(self, image, log=None): - """Refer the '_FilemapBase' class for the documentation.""" - - # Call the base class constructor first - _FilemapBase.__init__(self, image, log) - self._log.debug("FilemapNobmap: initializing") - - def block_is_mapped(self, block): - """Refer the '_FilemapBase' class for the documentation.""" - return True - - def get_mapped_ranges(self, start, count): - """Refer the '_FilemapBase' class for the documentation.""" - self._log.debug("FilemapNobmap: get_mapped_ranges(%d, %d(%d))" - % (start, count, start + count - 1)) - yield (start, start + count -1) - -def filemap(image, log=None): - """ - Create and return an instance of a Filemap class - 'FilemapFiemap' or - 'FilemapSeek', depending on what the system we run on supports. If the - FIEMAP ioctl is supported, an instance of the 'FilemapFiemap' class is - returned. Otherwise, if 'SEEK_HOLE' is supported an instance of the - 'FilemapSeek' class is returned. If none of these are supported, the - function generates an 'Error' type exception. - """ - - try: - return FilemapFiemap(image, log) - except ErrorNotSupp: - try: - return FilemapSeek(image, log) - except ErrorNotSupp: - return FilemapNobmap(image, log) - -def sparse_copy(src_fname, dst_fname, skip=0, seek=0, - length=0, api=None): - """ - Efficiently copy sparse file to or into another file. - - src_fname: path to source file - dst_fname: path to destination file - skip: skip N bytes at thestart of src - seek: seek N bytes from the start of dst - length: read N bytes from src and write them to dst - api: FilemapFiemap or FilemapSeek object - """ - if not api: - api = filemap - fmap = api(src_fname) - try: - dst_file = open(dst_fname, 'r+b') - except IOError: - dst_file = open(dst_fname, 'wb') - if length: - dst_size = length + seek - else: - dst_size = os.path.getsize(src_fname) + seek - skip - dst_file.truncate(dst_size) - - written = 0 - for first, last in fmap.get_mapped_ranges(0, fmap.blocks_cnt): - start = first * fmap.block_size - end = (last + 1) * fmap.block_size - - if skip >= end: - continue - - if start < skip < end: - start = skip - - fmap._f_image.seek(start, os.SEEK_SET) - - written += start - skip - written - if length and written >= length: - dst_file.seek(seek + length, os.SEEK_SET) - dst_file.close() - return - - dst_file.seek(seek + start - skip, os.SEEK_SET) - - chunk_size = 1024 * 1024 - to_read = end - start - read = 0 - - while read < to_read: - if read + chunk_size > to_read: - chunk_size = to_read - read - size = chunk_size - if length and written + size > length: - size = length - written - chunk = fmap._f_image.read(size) - dst_file.write(chunk) - read += size - written += size - if written == length: - dst_file.close() - return - dst_file.close() diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py deleted file mode 100644 index 5d7c40456a80..000000000000 --- a/scripts/lib/wic/help.py +++ /dev/null @@ -1,1197 +0,0 @@ -# Copyright (c) 2013, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This module implements some basic help invocation functions along -# with the bulk of the help topic text for the OE Core Image Tools. -# -# AUTHORS -# Tom Zanussi -# - -import subprocess -import logging - -from wic.pluginbase import PluginMgr, PLUGIN_TYPES - -logger = logging.getLogger('wic') - -def subcommand_error(args): - logger.info("invalid subcommand %s", args[0]) - - -def display_help(subcommand, subcommands): - """ - Display help for subcommand. - """ - if subcommand not in subcommands: - return False - - hlp = subcommands.get(subcommand, subcommand_error)[2] - if callable(hlp): - hlp = hlp() - pager = subprocess.Popen('less', stdin=subprocess.PIPE) - pager.communicate(hlp.encode('utf-8')) - - return True - - -def wic_help(args, usage_str, subcommands): - """ - Subcommand help dispatcher. - """ - if args.help_topic == None or not display_help(args.help_topic, subcommands): - print(usage_str) - - -def get_wic_plugins_help(): - """ - Combine wic_plugins_help with the help for every known - source plugin. - """ - result = wic_plugins_help - for plugin_type in PLUGIN_TYPES: - result += '\n\n%s PLUGINS\n\n' % plugin_type.upper() - for name, plugin in PluginMgr.get_plugins(plugin_type).items(): - result += "\n %s plugin:\n" % name - if plugin.__doc__: - result += plugin.__doc__ - else: - result += "\n %s is missing docstring\n" % plugin - return result - - -def invoke_subcommand(args, parser, main_command_usage, subcommands): - """ - Dispatch to subcommand handler borrowed from combo-layer. - Should use argparse, but has to work in 2.6. - """ - if not args.command: - logger.error("No subcommand specified, exiting") - parser.print_help() - return 1 - elif args.command == "help": - wic_help(args, main_command_usage, subcommands) - elif args.command not in subcommands: - logger.error("Unsupported subcommand %s, exiting\n", args.command) - parser.print_help() - return 1 - else: - subcmd = subcommands.get(args.command, subcommand_error) - usage = subcmd[1] - subcmd[0](args, usage) - - -## -# wic help and usage strings -## - -wic_usage = """ - - Create a customized OpenEmbedded image - - usage: wic [--version] | [--help] | [COMMAND [ARGS]] - - Current 'wic' commands are: - help Show help for command or one of the topics (see below) - create Create a new OpenEmbedded image - list List available canned images and source plugins - - Help topics: - overview wic overview - General overview of wic - plugins wic plugins - Overview and API - kickstart wic kickstart - wic kickstart reference -""" - -wic_help_usage = """ - - usage: wic help - - This command displays detailed help for the specified subcommand. -""" - -wic_create_usage = """ - - Create a new OpenEmbedded image - - usage: wic create [-o | --outdir ] - [-e | --image-name] [-s, --skip-build-check] [-D, --debug] - [-r, --rootfs-dir] [-b, --bootimg-dir] - [-k, --kernel-dir] [-n, --native-sysroot] [--sector-size ] [-f, --build-rootfs] - [-c, --compress-with] [-m, --bmap] - - This command creates an OpenEmbedded image based on the 'OE kickstart - commands' found in the . - - The -o option can be used to place the image in a directory with a - different name and location. - - See 'wic help create' for more detailed instructions. -""" - -wic_create_help = """ - -NAME - wic create - Create a new OpenEmbedded image - -SYNOPSIS - wic create [-o | --outdir ] - [-e | --image-name] [-s, --skip-build-check] [-D, --debug] - [-r, --rootfs-dir] [-b, --bootimg-dir] - [-k, --kernel-dir] [-n, --native-sysroot] [--sector-size ] [-f, --build-rootfs] - [-c, --compress-with] [-m, --bmap] [--no-fstab-update] - -DESCRIPTION - This command creates an OpenEmbedded image based on the 'OE - kickstart commands' found in the . - - Use the --sector-size option to select the sector size (in bytes) - used for partition layout calculations (default is 512). - - In order to do this, wic needs to know the locations of the - various build artifacts required to build the image. - - Users can explicitly specify the build artifact locations using - the -r, -b, -k, and -n options. See below for details on where - the corresponding artifacts are typically found in a normal - OpenEmbedded build. - - Alternatively, users can use the -e option to have 'wic' determine - those locations for a given image. If the -e option is used, the - user needs to have set the appropriate MACHINE variable in - local.conf, and have sourced the build environment. - - The -e option is used to specify the name of the image to use the - artifacts from e.g. core-image-sato. - - The -r option is used to specify the path to the /rootfs dir to - use as the .wks rootfs source. - - The -b option is used to specify the path to the dir containing - the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the - .wks bootimg source. - - The -k option is used to specify the path to the dir containing - the kernel to use in the .wks bootimg. - - The -n option is used to specify the path to the native sysroot - containing the tools to use to build the image. - - The -f option is used to build rootfs by running "bitbake " - - The -s option is used to skip the build check. The build check is - a simple sanity check used to determine whether the user has - sourced the build environment so that the -e option can operate - correctly. If the user has specified the build artifact locations - explicitly, 'wic' assumes the user knows what he or she is doing - and skips the build check. - - The -D option is used to display debug information detailing - exactly what happens behind the scenes when a create request is - fulfilled (or not, as the case may be). It enumerates and - displays the command sequence used, and should be included in any - bug report describing unexpected results. - - When 'wic -e' is used, the locations for the build artifacts - values are determined by 'wic -e' from the output of the 'bitbake - -e' command given an image name e.g. 'core-image-minimal' and a - given machine set in local.conf. In that case, the image is - created as if the following 'bitbake -e' variables were used: - - -r: IMAGE_ROOTFS - -k: STAGING_KERNEL_DIR - -n: STAGING_DIR_NATIVE - -b: empty (plugin-specific handlers must determine this) - - If 'wic -e' is not used, the user needs to select the appropriate - value for -b (as well as -r, -k, and -n). - - The -o option can be used to place the image in a directory with a - different name and location. - - The -c option is used to specify compressor utility to compress - an image. gzip, bzip2 and xz compressors are supported. - - The -m option is used to produce .bmap file for the image. This file - can be used to flash image using bmaptool utility. - - The --no-fstab-update option is used to doesn't change fstab file. When - using this option the final fstab file will be same that in rootfs and - wic doesn't update file, e.g adding a new mount point. User can control - the fstab file content in base-files recipe. -""" - -wic_list_usage = """ - - List available OpenEmbedded images and source plugins - - usage: wic list images - wic list help - wic list source-plugins - - This command enumerates the set of available canned images as well as - help for those images. It also can be used to list of available source - plugins. - - The first form enumerates all the available 'canned' images. - - The second form lists the detailed help information for a specific - 'canned' image. - - The third form enumerates all the available --sources (source - plugins). - - See 'wic help list' for more details. -""" - -wic_list_help = """ - -NAME - wic list - List available OpenEmbedded images and source plugins - -SYNOPSIS - wic list images - wic list help - wic list source-plugins - -DESCRIPTION - This command enumerates the set of available canned images as well - as help for those images. It also can be used to list available - source plugins. - - The first form enumerates all the available 'canned' images. - These are actually just the set of .wks files that have been moved - into the /scripts/lib/wic/canned-wks directory). - - The second form lists the detailed help information for a specific - 'canned' image. - - The third form enumerates all the available --sources (source - plugins). The contents of a given partition are driven by code - defined in 'source plugins'. Users specify a specific plugin via - the --source parameter of the partition .wks command. Normally - this is the 'rootfs' plugin but can be any of the more specialized - sources listed by the 'list source-plugins' command. Users can - also add their own source plugins - see 'wic help plugins' for - details. -""" - -wic_ls_usage = """ - - List content of a partitioned image - - usage: wic ls [:[]] [--native-sysroot ] [--sector-size ] - - This command outputs either list of image partitions or directory contents - of vfat and ext* partitions. - - See 'wic help ls' for more detailed instructions. - -""" - -wic_ls_help = """ - -NAME - wic ls - List contents of partitioned image or partition - -SYNOPSIS - wic ls - wic ls : - wic ls : - wic ls : --native-sysroot [--sector-size ] - -DESCRIPTION - This command lists either partitions of the image or directory contents - of vfat or ext* partitions. - - The first form it lists partitions of the image. - For example: - $ wic ls tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic - Num Start End Size Fstype - 1 1048576 24438783 23390208 fat16 - 2 25165824 50315263 25149440 ext4 - - Second and third form list directory content of the partition: - $ wic ls tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 - Volume in drive : is boot - Volume Serial Number is 2DF2-5F02 - Directory for ::/ - - efi 2017-05-11 10:54 - startup nsh 26 2017-05-11 10:54 - vmlinuz 6922288 2017-05-11 10:54 - 3 files 6 922 314 bytes - 15 818 752 bytes free - - - $ wic ls tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/EFI/boot/ - Volume in drive : is boot - Volume Serial Number is 2DF2-5F02 - Directory for ::/EFI/boot - - . 2017-05-11 10:54 - .. 2017-05-11 10:54 - grub cfg 679 2017-05-11 10:54 - bootx64 efi 571392 2017-05-11 10:54 - 4 files 572 071 bytes - 15 818 752 bytes free - - The -n option is used to specify the path to the native sysroot - containing the tools(parted and mtools) to use. - The --sector-size option sets the sector size used for partition math - (default is 512 bytes). - -""" - -wic_cp_usage = """ - - Copy files and directories to/from the vfat or ext* partition - - usage: wic cp [--native-sysroot ] [--sector-size ] - - source/destination image in format :[] - - This command copies files or directories either - - from local to vfat or ext* partitions of partitioned image - - from vfat or ext* partitions of partitioned image to local - - See 'wic help cp' for more detailed instructions. - -""" - -wic_cp_help = """ - -NAME - wic cp - copy files and directories to/from the vfat or ext* partitions - -SYNOPSIS - wic cp : - wic cp : - wic cp : - wic cp : --native-sysroot [--sector-size ] - -DESCRIPTION - This command copies files or directories either - - from local to vfat or ext* partitions of partitioned image - - from vfat or ext* partitions of partitioned image to local - - The first form of it copies file or directory to the root directory of - the partition: - $ wic cp test.wks tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 - $ wic ls tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 - Volume in drive : is boot - Volume Serial Number is DB4C-FD4C - Directory for ::/ - - efi 2017-05-24 18:15 - loader 2017-05-24 18:15 - startup nsh 26 2017-05-24 18:15 - vmlinuz 6926384 2017-05-24 18:15 - test wks 628 2017-05-24 21:22 - 5 files 6 927 038 bytes - 15 677 440 bytes free - - The second form of the command copies file or directory to the specified directory - on the partition: - $ wic cp test tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/ - $ wic ls tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/ - Volume in drive : is boot - Volume Serial Number is DB4C-FD4C - Directory for ::/efi - - . 2017-05-24 18:15 - .. 2017-05-24 18:15 - boot 2017-05-24 18:15 - test 2017-05-24 21:27 - 4 files 0 bytes - 15 675 392 bytes free - - The third form of the command copies file or directory from the specified directory - on the partition to local: - $ wic cp tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/vmlinuz test - - The -n option is used to specify the path to the native sysroot - containing the tools(parted and mtools) to use. - The --sector-size option sets the sector size used for partition math - (default is 512 bytes). -""" - -wic_rm_usage = """ - - Remove files or directories from the vfat or ext* partitions - - usage: wic rm : [--native-sysroot ] [--sector-size ] - - This command removes files or directories from the vfat or ext* partitions of - the partitioned image. - - See 'wic help rm' for more detailed instructions. - -""" - -wic_rm_help = """ - -NAME - wic rm - remove files or directories from the vfat or ext* partitions - -SYNOPSIS - wic rm : - wic rm : --native-sysroot - wic rm -r : - -DESCRIPTION - This command removes files or directories from the vfat or ext* partition of the - partitioned image: - - $ wic ls ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 - Volume in drive : is boot - Volume Serial Number is 11D0-DE21 - Directory for ::/ - - libcom32 c32 186500 2017-06-02 15:15 - libutil c32 24148 2017-06-02 15:15 - syslinux cfg 209 2017-06-02 15:15 - vesamenu c32 27104 2017-06-02 15:15 - vmlinuz 6926384 2017-06-02 15:15 - 5 files 7 164 345 bytes - 16 582 656 bytes free - - $ wic rm ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/libutil.c32 - - $ wic ls ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 - Volume in drive : is boot - Volume Serial Number is 11D0-DE21 - Directory for ::/ - - libcom32 c32 186500 2017-06-02 15:15 - syslinux cfg 209 2017-06-02 15:15 - vesamenu c32 27104 2017-06-02 15:15 - vmlinuz 6926384 2017-06-02 15:15 - 4 files 7 140 197 bytes - 16 607 232 bytes free - - The -n option is used to specify the path to the native sysroot - containing the tools(parted and mtools) to use. - The --sector-size option sets the sector size used for partition math - (default is 512 bytes). - - The -r option is used to remove directories and their contents - recursively,this only applies to ext* partition. -""" - -wic_write_usage = """ - - Write image to a device - - usage: wic write [--expand [rules]] [--native-sysroot ] - - This command writes partitioned image to a target device (USB stick, SD card etc). - - See 'wic help write' for more detailed instructions. - -""" - -wic_write_help = """ - -NAME - wic write - write an image to a device - -SYNOPSIS - wic write - wic write --expand auto - wic write --expand 1:100M,2:300M - wic write --native-sysroot - -DESCRIPTION - This command writes an image to a target device (USB stick, SD card etc) - - $ wic write ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic /dev/sdb - - The --expand option is used to resize image partitions. - --expand auto expands partitions to occupy all free space available on the target device. - It's also possible to specify expansion rules in a format - :[,:...] for one or more partitions. - Specifying size 0 will keep partition unmodified. - Note: Resizing boot partition can result in non-bootable image for non-EFI images. It is - recommended to use size 0 for boot partition to keep image bootable. - - The --native-sysroot option is used to specify the path to the native sysroot - containing the tools(parted, resize2fs) to use. -""" - -wic_plugins_help = """ - -NAME - wic plugins - Overview and API - -DESCRIPTION - plugins allow wic functionality to be extended and specialized by - users. This section documents the plugin interface, which is - currently restricted to 'source' plugins. - - 'Source' plugins provide a mechanism to customize various aspects - of the image generation process in wic, mainly the contents of - partitions. - - Source plugins provide a mechanism for mapping values specified in - .wks files using the --source keyword to a particular plugin - implementation that populates a corresponding partition. - - A source plugin is created as a subclass of SourcePlugin (see - scripts/lib/wic/pluginbase.py) and the plugin file containing it - is added to scripts/lib/wic/plugins/source/ to make the plugin - implementation available to the wic implementation. - - Source plugins can also be implemented and added by external - layers - any plugins found in a scripts/lib/wic/plugins/source/ - or lib/wic/plugins/source/ directory in an external layer will - also be made available. - - When the wic implementation needs to invoke a partition-specific - implementation, it looks for the plugin that has the same name as - the --source param given to that partition. For example, if the - partition is set up like this: - - part /boot --source bootimg_pcbios ... - - then the methods defined as class members of the plugin having the - matching bootimg_pcbios .name class member would be used. - - To be more concrete, here's the plugin definition that would match - a '--source bootimg_pcbios' usage, along with an example method - that would be called by the wic implementation when it needed to - invoke an implementation-specific partition-preparation function: - - class BootimgPcbiosPlugin(SourcePlugin): - name = 'bootimg_pcbios' - - @classmethod - def do_prepare_partition(self, part, ...) - - If the subclass itself doesn't implement a function, a 'default' - version in a superclass will be located and used, which is why all - plugins must be derived from SourcePlugin. - - The SourcePlugin class defines the following methods, which is the - current set of methods that can be implemented/overridden by - --source plugins. Any methods not implemented by a SourcePlugin - subclass inherit the implementations present in the SourcePlugin - class (see the SourcePlugin source for details): - - do_prepare_partition() - Called to do the actual content population for a - partition. In other words, it 'prepares' the final partition - image which will be incorporated into the disk image. - - do_post_partition() - Called after the partition is created. It is useful to add post - operations e.g. signing the partition. - - do_configure_partition() - Called before do_prepare_partition(), typically used to - create custom configuration files for a partition, for - example syslinux or grub config files. - - do_install_disk() - Called after all partitions have been prepared and assembled - into a disk image. This provides a hook to allow - finalization of a disk image, for example to write an MBR to - it. - - do_stage_partition() - Special content-staging hook called before - do_prepare_partition(), normally empty. - - Typically, a partition will just use the passed-in - parameters, for example the unmodified value of bootimg_dir. - In some cases however, things may need to be more tailored. - As an example, certain files may additionally need to be - take from bootimg_dir + /boot. This hook allows those files - to be staged in a customized fashion. Note that - get_bitbake_var() allows you to access non-standard - variables that you might want to use for these types of - situations. - - This scheme is extensible - adding more hooks is a simple matter - of adding more plugin methods to SourcePlugin and derived classes. - Please see the implementation for details. -""" - -wic_overview_help = """ - -NAME - wic overview - General overview of wic - -DESCRIPTION - The 'wic' command generates partitioned images from existing - OpenEmbedded build artifacts. Image generation is driven by - partitioning commands contained in an 'Openembedded kickstart' - (.wks) file (see 'wic help kickstart') specified either directly - on the command-line or as one of a selection of canned .wks files - (see 'wic list images'). When applied to a given set of build - artifacts, the result is an image or set of images that can be - directly written onto media and used on a particular system. - - The 'wic' command and the infrastructure it's based on is by - definition incomplete - its purpose is to allow the generation of - customized images, and as such was designed to be completely - extensible via a plugin interface (see 'wic help plugins'). - - Background and Motivation - - wic is meant to be a completely independent standalone utility - that initially provides easier-to-use and more flexible - replacements for a couple bits of existing functionality in - oe-core: directdisk.bbclass and mkefidisk.sh. The difference - between wic and those examples is that with wic the functionality - of those scripts is implemented by a general-purpose partitioning - 'language' based on Red Hat kickstart syntax). - - The initial motivation and design considerations that lead to the - current tool are described exhaustively in Yocto Bug #3847 - (https://bugzilla.yoctoproject.org/show_bug.cgi?id=3847). - - Implementation and Examples - - wic can be used in two different modes, depending on how much - control the user needs in specifying the Openembedded build - artifacts that will be used in creating the image: 'raw' and - 'cooked'. - - If used in 'raw' mode, artifacts are explicitly specified via - command-line arguments (see example below). - - The more easily usable 'cooked' mode uses the current MACHINE - setting and a specified image name to automatically locate the - artifacts used to create the image. - - OE kickstart files (.wks) can of course be specified directly on - the command-line, but the user can also choose from a set of - 'canned' .wks files available via the 'wic list images' command - (example below). - - In any case, the prerequisite for generating any image is to have - the build artifacts already available. The below examples assume - the user has already build a 'core-image-minimal' for a specific - machine (future versions won't require this redundant step, but - for now that's typically how build artifacts get generated). - - The other prerequisite is to source the build environment: - - $ source oe-init-build-env - - To start out with, we'll generate an image from one of the canned - .wks files. The following generates a list of availailable - images: - - $ wic list images - mkefidisk Create an EFI disk image - directdisk Create a 'pcbios' direct disk image - - You can get more information about any of the available images by - typing 'wic list xxx help', where 'xxx' is one of the image names: - - $ wic list mkefidisk help - - Creates a partitioned EFI disk image that the user can directly dd - to boot media. - - At any time, you can get help on the 'wic' command or any - subcommand (currently 'list' and 'create'). For instance, to get - the description of 'wic create' command and its parameters: - - $ wic create - - Usage: - - Create a new OpenEmbedded image - - usage: wic create [-o | ...] - [-i | --infile ] - [-e | --image-name] [-s, --skip-build-check] [-D, --debug] - [-r, --rootfs-dir] [-b, --bootimg-dir] [-k, --kernel-dir] - [-n, --native-sysroot] [-f, --build-rootfs] - - This command creates an OpenEmbedded image based on the 'OE - kickstart commands' found in the . - - The -o option can be used to place the image in a directory - with a different name and location. - - See 'wic help create' for more detailed instructions. - ... - - As mentioned in the command, you can get even more detailed - information by adding 'help' to the above: - - $ wic help create - - So, the easiest way to create an image is to use the -e option - with a canned .wks file. To use the -e option, you need to - specify the image used to generate the artifacts and you actually - need to have the MACHINE used to build them specified in your - local.conf (these requirements aren't necessary if you aren't - using the -e options.) Below, we generate a directdisk image, - pointing the process at the core-image-minimal artifacts for the - current MACHINE: - - $ wic create directdisk -e core-image-minimal - - Checking basic build environment... - Done. - - Creating image(s)... - - Info: The new image(s) can be found here: - /var/tmp/wic/build/directdisk-201309252350-sda.direct - - The following build artifacts were used to create the image(s): - - ROOTFS_DIR: ... - BOOTIMG_DIR: ... - KERNEL_DIR: ... - NATIVE_SYSROOT: ... - - The image(s) were created using OE kickstart file: - .../scripts/lib/wic/canned-wks/directdisk.wks - - The output shows the name and location of the image created, and - so that you know exactly what was used to generate the image, each - of the artifacts and the kickstart file used. - - Similarly, you can create a 'mkefidisk' image in the same way - (notice that this example uses a different machine - because it's - using the -e option, you need to change the MACHINE in your - local.conf): - - $ wic create mkefidisk -e core-image-minimal - Checking basic build environment... - Done. - - Creating image(s)... - - Info: The new image(s) can be found here: - /var/tmp/wic/build/mkefidisk-201309260027-sda.direct - - ... - - Here's an example that doesn't take the easy way out and manually - specifies each build artifact, along with a non-canned .wks file, - and also uses the -o option to have wic create the output - somewhere other than the default /var/tmp/wic: - - $ wic create ./test.wks -o ./out --rootfs-dir - tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs - --bootimg-dir tmp/sysroots/qemux86-64/usr/share - --kernel-dir tmp/deploy/images/qemux86-64 - --native-sysroot tmp/sysroots/x86_64-linux - - Creating image(s)... - - Info: The new image(s) can be found here: - out/build/test-201507211313-sda.direct - - The following build artifacts were used to create the image(s): - ROOTFS_DIR: tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs - BOOTIMG_DIR: tmp/sysroots/qemux86-64/usr/share - KERNEL_DIR: tmp/deploy/images/qemux86-64 - NATIVE_SYSROOT: tmp/sysroots/x86_64-linux - - The image(s) were created using OE kickstart file: - ./test.wks - - Here is a content of test.wks: - - part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 - part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024 - - bootloader --timeout=0 --append="rootwait rootfstype=ext3 video=vesafb vga=0x318 console=tty0" - - - Finally, here's an example of the actual partition language - commands used to generate the mkefidisk image i.e. these are the - contents of the mkefidisk.wks OE kickstart file: - - # short-description: Create an EFI disk image - # long-description: Creates a partitioned EFI disk image that the user - # can directly dd to boot media. - - part /boot --source bootimg-efi --ondisk sda --fstype=efi --active - - part / --source rootfs --ondisk sda --fstype=ext3 --label platform - - part swap --ondisk sda --size 44 --label swap1 --fstype=swap - - bootloader --timeout=10 --append="rootwait console=ttyPCH0,115200" - - You can get a complete listing and description of all the - kickstart commands available for use in .wks files from 'wic help - kickstart'. -""" - -wic_kickstart_help = """ - -NAME - wic kickstart - wic kickstart reference - -DESCRIPTION - This section provides the definitive reference to the wic - kickstart language. It also provides documentation on the list of - --source plugins available for use from the 'part' command (see - the 'Platform-specific Plugins' section below). - - The current wic implementation supports only the basic kickstart - partitioning commands: partition (or part for short) and - bootloader. - - The following is a listing of the commands, their syntax, and - meanings. The commands are based on the Fedora kickstart - documentation but with modifications to reflect wic capabilities. - - https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#part-or-partition - https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#bootloader - - Commands - - * 'part' or 'partition' - - This command creates a partition on the system and uses the - following syntax: - - part [] - - The is where the partition will be mounted and - must take of one of the following forms: - - /: For example: /, /usr, or /home - - swap: The partition will be used as swap space. - - If a is not specified the partition will be created - but will not be mounted. - - Partitions with a specified will be automatically mounted. - This is achieved by wic adding entries to the fstab during image - generation. In order for a valid fstab to be generated one of the - --ondrive, --ondisk, --use-uuid or --use-label partition options must - be used for each partition that specifies a mountpoint. Note that with - --use-{uuid,label} and non-root , including swap, the mount - program must understand the PARTUUID or LABEL syntax. This currently - excludes the busybox versions of these applications. - - - The following are supported 'part' options: - - --size: The minimum partition size. Specify an integer value - such as 500. Multipliers k, M ang G can be used. If - not specified, the size is in MB. - You do not need this option if you use --source. - - --fixed-size: Exact partition size. Value format is the same - as for --size option. This option cannot be - specified along with --size. If partition data - is larger than --fixed-size and error will be - raised when assembling disk image. - - --source: This option is a wic-specific option that names the - source of the data that will populate the - partition. The most common value for this option - is 'rootfs', but can be any value which maps to a - valid 'source plugin' (see 'wic help plugins'). - - If '--source rootfs' is used, it tells the wic - command to create a partition as large as needed - and to fill it with the contents of the root - filesystem pointed to by the '-r' wic command-line - option (or the equivalent rootfs derived from the - '-e' command-line option). The filesystem type - that will be used to create the partition is driven - by the value of the --fstype option specified for - the partition (see --fstype below). - - If --source ' is used, it tells the - wic command to create a partition as large as - needed and to fill with the contents of the - partition that will be generated by the specified - plugin name using the data pointed to by the '-r' - wic command-line option (or the equivalent rootfs - derived from the '-e' command-line option). - Exactly what those contents and filesystem type end - up being are dependent on the given plugin - implementation. - - If --source option is not used, the wic command - will create empty partition. --size parameter has - to be used to specify size of empty partition. - - --sourceparams: This option is specific to wic. Supply additional - parameters to the source plugin in - key1=value1,key2 format. - - --ondisk or --ondrive: Forces the partition to be created on - a particular disk. - - --fstype: Sets the file system type for the partition. These - apply to partitions created using '--source rootfs' (see - --source above). Valid values are: - - vfat - msdos - ext2 - ext3 - ext4 - btrfs - squashfs - erofs - swap - none - - --fsoptions: Specifies a free-form string of options to be - used when mounting the filesystem. This string - will be copied into the /etc/fstab file of the - installed system and should be enclosed in - quotes. If not specified, the default string is - "defaults". - - --fspassno: Specifies the order in which filesystem checks are done - at boot time by fsck. See fs_passno parameter of - fstab(5). This parameter will be copied into the - /etc/fstab file of the installed system. If not - specified the default value of "0" will be used. - - --label label: Specifies the label to give to the filesystem - to be made on the partition. If the given - label is already in use by another filesystem, - a new label is created for the partition. - - --use-label: This option is specific to wic. It makes wic to use the - label in /etc/fstab to specify a partition. If the - --use-label and --use-uuid are used at the same time, - we prefer the uuid because it is less likely to cause - name confliction. We don't support using this parameter - on the root partition since it requires an initramfs to - parse this value and we do not currently support that. - - --active: Marks the partition as active. - - --align (in KBytes): This option is specific to wic and says - to start a partition on an x KBytes - boundary. - - --offset: This option is specific to wic that says to place a partition - at exactly the specified offset. If the partition cannot be - placed at the specified offset, the image build will fail. - Specify as an integer value optionally followed by one of the - units s/S for 512 byte sector, k/K for kibibyte, M for - mebibyte and G for gibibyte. The default unit if none is - given is k. - - --no-table: This option is specific to wic. Space will be - reserved for the partition and it will be - populated but it will not be added to the - partition table. It may be useful for - bootloaders. - - --exclude-path: This option is specific to wic. It excludes the given - relative path from the resulting image. If the path - ends with a slash, only the content of the directory - is omitted, not the directory itself. This option only - has an effect with the rootfs source plugin. - - --include-path: This option is specific to wic. It adds the contents - of the given path or a rootfs to the resulting image. - The option contains two fields, the origin and the - destination. When the origin is a rootfs, it follows - the same logic as the rootfs-dir argument and the - permissions and owners are kept. When the origin is a - path, it is relative to the directory in which wic is - running not the rootfs itself so use of an absolute - path is recommended, and the owner and group is set to - root:root. If no destination is given it is - automatically set to the root of the rootfs. This - option only has an effect with the rootfs source - plugin. - - --change-directory: This option is specific to wic. It changes to the - given directory before copying the files. This - option is useful when we want to split a rootfs in - multiple partitions and we want to keep the right - permissions and usernames in all the partitions. - - --no-fstab-update: This option is specific to wic. It does not update the - '/etc/fstab' stock file for the given partition. - - --extra-filesystem-space: This option is specific to wic. It adds extra - space after the space filled by the content - of the partition. The final size can go - beyond the size specified by --size. - By default, 10MB. This option cannot be used - with --fixed-size option. - - --extra-partition-space: This option is specific to wic. It adds extra - empty space after the space filled by the - filesystem. With --fixed-size, the extra - partition space is removed from the filesystem - size. Otherwise (with or without --size flag), - the extra partition space is added to the final - paritition size. The default value is 0MB. - - --overhead-factor: This option is specific to wic. The - size of the partition is multiplied by - this factor. It has to be greater than or - equal to 1. The default value is 1.3. - This option cannot be used with --fixed-size - option. - - --part-name: This option is specific to wic. It specifies name for GPT partitions. - - --part-type: This option is specific to wic. It specifies partition - type GUID for GPT partitions. - List of partition type GUIDS can be found here: - http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs - - --use-uuid: This option is specific to wic. It makes wic to generate - random globally unique identifier (GUID) for the partition - and use it in bootloader configuration to specify root partition. - - --uuid: This option is specific to wic. It specifies partition UUID. - It's useful if preconfigured partition UUID is added to kernel command line - in bootloader configuration before running wic. In this case .wks file can - be generated or modified to set preconfigured parition UUID using this option. - - --fsuuid: This option is specific to wic. It specifies filesystem UUID. - It's useful if preconfigured filesystem UUID is added to kernel command line - in bootloader configuration before running wic. In this case .wks file can - be generated or modified to set preconfigured filesystem UUID using this option. - - --system-id: This option is specific to wic. It specifies partition system id. It's useful - for the harware that requires non-default partition system ids. The parameter - in one byte long hex number either with 0x prefix or without it. - - --mkfs-extraopts: This option specifies extra options to pass to mkfs utility. - NOTE, that wic uses default options for some filesystems, for example - '-S 512' for mkfs.fat or '-F -i 8192' for mkfs.ext. Those options will - not take effect when --mkfs-extraopts is used. This should be taken into - account when using --mkfs-extraopts. - - --type: This option is specific to wic. Valid values are 'primary', - 'logical'. For msdos partition tables, this option specifies - the partition type. - - --hidden: This option is specific to wic. This option sets the - RequiredPartition bit (bit 0) on GPT partitions. - - --mbr: This option is specific to wic. This option is used with the - gpt-hybrid partition type that uses both a GPT partition and - an MBR header. Partitions with this flag will be included in - this MBR header. - - * bootloader - - This command allows the user to specify various bootloader - options. The following are supported 'bootloader' options: - - --timeout: Specifies the number of seconds before the - bootloader times out and boots the default option. - - --append: Specifies kernel parameters. These will be added to - bootloader command-line - for example, the syslinux - APPEND or grub kernel command line. - - --configfile: Specifies a user defined configuration file for - the bootloader. This file must be located in the - canned-wks folder or could be the full path to the - file. Using this option will override any other - bootloader option. - - --ptable: Specifies the partition table format. Valid values are - 'msdos', 'gpt', 'gpt-hybrid'. - - --source: Specifies the source plugin. If not specified, the - --source value will be copied from the partition that has - /boot as mountpoint. - - Note that bootloader functionality and boot partitions are - implemented by the various --source plugins that implement - bootloader functionality; the bootloader command essentially - provides a means of modifying bootloader configuration. - - * include - - This command allows the user to include the content of .wks file - into original .wks file. - - Command uses the following syntax: - - include - - The is either path to the file or its name. If name is - specified wic will try to find file in the directories with canned - .wks files. - -""" - -wic_help_help = """ -NAME - wic help - display a help topic - -DESCRIPTION - Specify a help topic to display it. Topics are shown above. -""" - - -wic_help = """ -Creates a customized OpenEmbedded image. - -Usage: wic [--version] - wic help [COMMAND or TOPIC] - wic COMMAND [ARGS] - - usage 1: Returns the current version of Wic - usage 2: Returns detailed help for a COMMAND or TOPIC - usage 3: Executes COMMAND - - -COMMAND: - - list - List available canned images and source plugins - ls - List contents of partitioned image or partition - rm - Remove files or directories from the vfat or ext* partitions - help - Show help for a wic COMMAND or TOPIC - write - Write an image to a device - cp - Copy files and directories to the vfat or ext* partitions - create - Create a new OpenEmbedded image - - -TOPIC: - overview - Presents an overall overview of Wic - plugins - Presents an overview and API for Wic plugins - kickstart - Presents a Wic kickstart file reference - - -Examples: - - $ wic --version - - Returns the current version of Wic - - - $ wic help cp - - Returns the SYNOPSIS and DESCRIPTION for the Wic "cp" command. - - - $ wic list images - - Returns the list of canned images (i.e. *.wks files located in - the /scripts/lib/wic/canned-wks directory. - - - $ wic create mkefidisk -e core-image-minimal - - Creates an EFI disk image from artifacts used in a previous - core-image-minimal build in standard BitBake locations - (e.g. Cooked Mode). - -""" diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py deleted file mode 100644 index 4ccd70dc555a..000000000000 --- a/scripts/lib/wic/ksparser.py +++ /dev/null @@ -1,322 +0,0 @@ -#!/usr/bin/env python3 -# -# Copyright (c) 2016 Intel, Inc. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This module provides parser for kickstart format -# -# AUTHORS -# Ed Bartosh (at] linux.intel.com> - -"""Kickstart parser module.""" - -import os -import shlex -import logging -import re -import uuid - -from argparse import ArgumentParser, ArgumentError, ArgumentTypeError - -from wic.engine import find_canned -from wic.partition import Partition -from wic.misc import get_bitbake_var - -logger = logging.getLogger('wic') - -__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}") - -def expand_line(line): - while True: - m = __expand_var_regexp__.search(line) - if not m: - return line - key = m.group()[2:-1] - val = get_bitbake_var(key) - if val is None: - logger.warning("cannot expand variable %s" % key) - return line - line = line[:m.start()] + val + line[m.end():] - -class KickStartError(Exception): - """Custom exception.""" - pass - -class KickStartParser(ArgumentParser): - """ - This class overwrites error method to throw exception - instead of producing usage message(default argparse behavior). - """ - def error(self, message): - raise ArgumentError(None, message) - -def sizetype(default, size_in_bytes=False): - def f(arg): - """ - Custom type for ArgumentParser - Converts size string in [S|s|K|k|M|G] format into the integer value - """ - try: - suffix = default - size = int(arg) - except ValueError: - try: - suffix = arg[-1:] - size = int(arg[:-1]) - except ValueError: - raise ArgumentTypeError("Invalid size: %r" % arg) - - - if size_in_bytes: - if suffix == 's' or suffix == 'S': - return size * 512 - mult = 1024 - else: - mult = 1 - - if suffix == "k" or suffix == "K": - return size * mult - if suffix == "M": - return size * mult * 1024 - if suffix == "G": - return size * mult * 1024 * 1024 - - raise ArgumentTypeError("Invalid size: %r" % arg) - return f - -def overheadtype(arg): - """ - Custom type for ArgumentParser - Converts overhead string to float and checks if it's bigger than 1.0 - """ - try: - result = float(arg) - except ValueError: - raise ArgumentTypeError("Invalid value: %r" % arg) - - if result < 1.0: - raise ArgumentTypeError("Overhead factor should be > 1.0" % arg) - - return result - -def cannedpathtype(arg): - """ - Custom type for ArgumentParser - Tries to find file in the list of canned wks paths - """ - scripts_path = os.path.abspath(os.path.dirname(__file__) + '../../..') - result = find_canned(scripts_path, arg) - if not result: - raise ArgumentTypeError("file not found: %s" % arg) - return result - -def systemidtype(arg): - """ - Custom type for ArgumentParser - Checks if the argument sutisfies system id requirements, - i.e. if it's one byte long integer > 0 - """ - error = "Invalid system type: %s. must be hex "\ - "between 0x1 and 0xFF" % arg - try: - result = int(arg, 16) - except ValueError: - raise ArgumentTypeError(error) - - if result <= 0 or result > 0xff: - raise ArgumentTypeError(error) - - return arg - -class KickStart(): - """Kickstart parser implementation.""" - - DEFAULT_EXTRA_FILESYSTEM_SPACE = 10*1024 - DEFAULT_OVERHEAD_FACTOR = 1.3 - - def __init__(self, confpath): - - self.partitions = [] - self.bootloader = None - self.lineno = 0 - self.partnum = 0 - - parser = KickStartParser() - subparsers = parser.add_subparsers() - - part = subparsers.add_parser('part') - part.add_argument('mountpoint', nargs='?') - part.add_argument('--active', action='store_true') - part.add_argument('--align', type=int) - part.add_argument('--offset', type=sizetype("K", True)) - part.add_argument('--exclude-path', nargs='+') - part.add_argument('--include-path', nargs='+', action='append') - part.add_argument('--change-directory') - part.add_argument('--extra-filesystem-space', '--extra-space', type=sizetype("M")) - part.add_argument('--extra-partition-space', type=sizetype("M")) - part.add_argument('--fsoptions', dest='fsopts') - part.add_argument('--fspassno', dest='fspassno') - part.add_argument('--fstype', default='vfat', - choices=('ext2', 'ext3', 'ext4', 'btrfs', - 'squashfs', 'vfat', 'msdos', 'erofs', - 'swap', 'none')) - part.add_argument('--mkfs-extraopts', default='') - part.add_argument('--label') - part.add_argument('--use-label', action='store_true') - part.add_argument('--no-table', action='store_true') - part.add_argument('--ondisk', '--ondrive', dest='disk', default='sda') - part.add_argument("--overhead-factor", type=overheadtype) - part.add_argument('--part-name') - part.add_argument('--part-type') - part.add_argument('--rootfs-dir') - part.add_argument('--type', default='primary', - choices = ('primary', 'logical')) - part.add_argument('--hidden', action='store_true') - - # --size and --fixed-size cannot be specified together; options - # ----extra-filesystem-space and --overhead-factor should also raise a - # parser error, but since nesting mutually exclusive groups does not work, - # ----extra-filesystem-space/--overhead-factor are handled later - sizeexcl = part.add_mutually_exclusive_group() - sizeexcl.add_argument('--size', type=sizetype("M"), default=0) - sizeexcl.add_argument('--fixed-size', type=sizetype("M"), default=0) - - part.add_argument('--source') - part.add_argument('--sourceparams') - part.add_argument('--system-id', type=systemidtype) - part.add_argument('--use-uuid', action='store_true') - part.add_argument('--uuid') - part.add_argument('--fsuuid') - part.add_argument('--no-fstab-update', action='store_true') - part.add_argument('--mbr', action='store_true') - - bootloader = subparsers.add_parser('bootloader') - bootloader.add_argument('--append') - bootloader.add_argument('--configfile') - bootloader.add_argument('--ptable', choices=('msdos', 'gpt', 'gpt-hybrid'), - default='msdos') - bootloader.add_argument('--diskid') - bootloader.add_argument('--timeout', type=int) - bootloader.add_argument('--source') - - include = subparsers.add_parser('include') - include.add_argument('path', type=cannedpathtype) - - self._parse(parser, confpath) - if not self.bootloader: - logger.warning('bootloader config not specified, using defaults\n') - self.bootloader = bootloader.parse_args([]) - - def _parse(self, parser, confpath): - """ - Parse file in .wks format using provided parser. - """ - with open(confpath) as conf: - lineno = 0 - for line in conf: - line = line.strip() - lineno += 1 - if line and line[0] != '#': - line = expand_line(line) - try: - line_args = shlex.split(line) - parsed = parser.parse_args(line_args) - except ArgumentError as err: - raise KickStartError('%s:%d: %s' % \ - (confpath, lineno, err)) - if line.startswith('part'): - # SquashFS does not support filesystem UUID - if parsed.fstype == 'squashfs': - if parsed.fsuuid: - err = "%s:%d: SquashFS does not support UUID" \ - % (confpath, lineno) - raise KickStartError(err) - if parsed.label: - err = "%s:%d: SquashFS does not support LABEL" \ - % (confpath, lineno) - raise KickStartError(err) - # erofs does not support filesystem labels - if parsed.fstype == 'erofs' and parsed.label: - err = "%s:%d: erofs does not support LABEL" % (confpath, lineno) - raise KickStartError(err) - if parsed.fstype == 'msdos' or parsed.fstype == 'vfat': - if parsed.fsuuid: - if parsed.fsuuid.upper().startswith('0X'): - if len(parsed.fsuuid) > 10: - err = "%s:%d: fsuuid %s given in wks kickstart file " \ - "exceeds the length limit for %s filesystem. " \ - "It should be in the form of a 32 bit hexadecimal" \ - "number (for example, 0xABCD1234)." \ - % (confpath, lineno, parsed.fsuuid, parsed.fstype) - raise KickStartError(err) - elif len(parsed.fsuuid) > 8: - err = "%s:%d: fsuuid %s given in wks kickstart file " \ - "exceeds the length limit for %s filesystem. " \ - "It should be in the form of a 32 bit hexadecimal" \ - "number (for example, 0xABCD1234)." \ - % (confpath, lineno, parsed.fsuuid, parsed.fstype) - raise KickStartError(err) - if parsed.use_label and not parsed.label: - err = "%s:%d: Must set the label with --label" \ - % (confpath, lineno) - raise KickStartError(err) - if not parsed.extra_partition_space: - parsed.extra_partition_space = 0 - # using ArgumentParser one cannot easily tell if option - # was passed as argument, if said option has a default - # value; --overhead-factor/--extra-filesystem-space - # cannot be used with --fixed-size, so at least detect - # when these were passed with non-0 values ... - if parsed.fixed_size: - if parsed.overhead_factor or parsed.extra_filesystem_space: - err = "%s:%d: arguments --overhead-factor and "\ - "--extra-filesystem-space not "\ - "allowed with argument --fixed-size" \ - % (confpath, lineno) - raise KickStartError(err) - else: - # ... and provide defaults if not using - # --fixed-size iff given option was not used - # (again, one cannot tell if option was passed but - # with value equal to 0) - if not parsed.overhead_factor: - parsed.overhead_factor = self.DEFAULT_OVERHEAD_FACTOR - if not parsed.extra_filesystem_space: - parsed.extra_filesystem_space = self.DEFAULT_EXTRA_FILESYSTEM_SPACE - - self.partnum += 1 - self.partitions.append(Partition(parsed, self.partnum)) - elif line.startswith('include'): - self._parse(parser, parsed.path) - elif line.startswith('bootloader'): - if not self.bootloader: - self.bootloader = parsed - # Concatenate the strings set in APPEND - append_var = get_bitbake_var("APPEND") - if append_var: - self.bootloader.append = ' '.join(filter(None, \ - (self.bootloader.append, append_var))) - if parsed.diskid: - if parsed.ptable == "msdos": - try: - self.bootloader.diskid = int(parsed.diskid, 0) - except ValueError: - err = "with --ptbale msdos only 32bit integers " \ - "are allowed for --diskid. %s could not " \ - "be parsed" % self.ptable - raise KickStartError(err) - else: - try: - self.bootloader.diskid = uuid.UUID(parsed.diskid) - except ValueError: - err = "with --ptable %s only valid uuids are " \ - "allowed for --diskid. %s could not be " \ - "parsed" % (parsed.ptable, parsed.diskid) - raise KickStartError(err) - - else: - err = "%s:%d: more than one bootloader specified" \ - % (confpath, lineno) - raise KickStartError(err) diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py deleted file mode 100644 index 310e6367a2ad..000000000000 --- a/scripts/lib/wic/misc.py +++ /dev/null @@ -1,272 +0,0 @@ -# -# Copyright (c) 2013, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This module provides a place to collect various wic-related utils -# for the OpenEmbedded Image Tools. -# -# AUTHORS -# Tom Zanussi -# -"""Miscellaneous functions.""" - -import logging -import os -import re -import subprocess -import shutil - -from collections import defaultdict - -from wic import WicError - -logger = logging.getLogger('wic') - -# executable -> recipe pairs for exec_native_cmd -NATIVE_RECIPES = {"bmaptool": "bmaptool", - "dumpe2fs": "e2fsprogs", - "grub-mkimage": "grub-efi", - "isohybrid": "syslinux", - "mcopy": "mtools", - "mdel" : "mtools", - "mdeltree" : "mtools", - "mdir" : "mtools", - "mkdosfs": "dosfstools", - "mkisofs": "cdrtools", - "mkfs.btrfs": "btrfs-tools", - "mkfs.erofs": "erofs-utils", - "mkfs.ext2": "e2fsprogs", - "mkfs.ext3": "e2fsprogs", - "mkfs.ext4": "e2fsprogs", - "mkfs.vfat": "dosfstools", - "mksquashfs": "squashfs-tools", - "mkswap": "util-linux", - "mmd": "mtools", - "parted": "parted", - "sfdisk": "util-linux", - "sgdisk": "gptfdisk", - "syslinux": "syslinux", - "tar": "tar" - } - -def runtool(cmdln_or_args): - """ wrapper for most of the subprocess calls - input: - cmdln_or_args: can be both args and cmdln str (shell=True) - return: - rc, output - """ - if isinstance(cmdln_or_args, list): - cmd = cmdln_or_args[0] - shell = False - else: - import shlex - cmd = shlex.split(cmdln_or_args)[0] - shell = True - - sout = subprocess.PIPE - serr = subprocess.STDOUT - - try: - process = subprocess.Popen(cmdln_or_args, stdout=sout, - stderr=serr, shell=shell) - sout, serr = process.communicate() - # combine stdout and stderr, filter None out and decode - out = ''.join([out.decode('utf-8') for out in [sout, serr] if out]) - except OSError as err: - if err.errno == 2: - # [Errno 2] No such file or directory - raise WicError('Cannot run command: %s, lost dependency?' % cmd) - else: - raise # relay - - return process.returncode, out - -def _exec_cmd(cmd_and_args, as_shell=False): - """ - Execute command, catching stderr, stdout - - Need to execute as_shell if the command uses wildcards - """ - logger.debug("_exec_cmd: %s", cmd_and_args) - args = cmd_and_args.split() - logger.debug(args) - - if as_shell: - ret, out = runtool(cmd_and_args) - else: - ret, out = runtool(args) - out = out.strip() - if ret != 0: - raise WicError("_exec_cmd: %s returned '%s' instead of 0\noutput: %s" % \ - (cmd_and_args, ret, out)) - - logger.debug("_exec_cmd: output for %s (rc = %d): %s", - cmd_and_args, ret, out) - - return ret, out - - -def exec_cmd(cmd_and_args, as_shell=False): - """ - Execute command, return output - """ - return _exec_cmd(cmd_and_args, as_shell)[1] - -def find_executable(cmd, paths): - recipe = cmd - if recipe in NATIVE_RECIPES: - recipe = NATIVE_RECIPES[recipe] - provided = get_bitbake_var("ASSUME_PROVIDED") - if provided and "%s-native" % recipe in provided: - return True - - return shutil.which(cmd, path=paths) - -def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""): - """ - Execute native command, catching stderr, stdout - - Need to execute as_shell if the command uses wildcards - - Always need to execute native commands as_shell - """ - # The reason -1 is used is because there may be "export" commands. - args = cmd_and_args.split(';')[-1].split() - logger.debug(args) - - if pseudo: - cmd_and_args = pseudo + cmd_and_args - - hosttools_dir = get_bitbake_var("HOSTTOOLS_DIR") - target_sys = get_bitbake_var("TARGET_SYS") - - native_paths = "%s/sbin:%s/usr/sbin:%s/usr/bin:%s/usr/bin/%s:%s/bin:%s" % \ - (native_sysroot, native_sysroot, - native_sysroot, native_sysroot, target_sys, - native_sysroot, hosttools_dir) - - native_cmd_and_args = "export PATH=%s:$PATH;%s" % \ - (native_paths, cmd_and_args) - logger.debug("exec_native_cmd: %s", native_cmd_and_args) - - # If the command isn't in the native sysroot say we failed. - if find_executable(args[0], native_paths): - ret, out = _exec_cmd(native_cmd_and_args, True) - else: - ret = 127 - out = "can't find native executable %s in %s" % (args[0], native_paths) - - prog = args[0] - # shell command-not-found - if ret == 127 \ - or (pseudo and ret == 1 and out == "Can't find '%s' in $PATH." % prog): - msg = "A native program %s required to build the image "\ - "was not found (see details above).\n\n" % prog - recipe = NATIVE_RECIPES.get(prog) - if recipe: - msg += "Please make sure wic-tools have %s-native in its DEPENDS, "\ - "build it with 'bitbake wic-tools' and try again.\n" % recipe - else: - msg += "Wic failed to find a recipe to build native %s. Please "\ - "file a bug against wic.\n" % prog - raise WicError(msg) - - return ret, out - -BOOTDD_EXTRA_SPACE = 16384 - -class BitbakeVars(defaultdict): - """ - Container for Bitbake variables. - """ - def __init__(self): - defaultdict.__init__(self, dict) - - # default_image and vars_dir attributes should be set from outside - self.default_image = None - self.vars_dir = None - - def _parse_line(self, line, image, matcher=re.compile(r"^([a-zA-Z0-9\-_+./~]+)=(.*)")): - """ - Parse one line from bitbake -e output or from .env file. - Put result key-value pair into the storage. - """ - if "=" not in line: - return - match = matcher.match(line) - if not match: - return - key, val = match.groups() - self[image][key] = val.strip('"') - - def get_var(self, var, image=None, cache=True): - """ - Get bitbake variable from 'bitbake -e' output or from .env file. - This is a lazy method, i.e. it runs bitbake or parses file only when - only when variable is requested. It also caches results. - """ - if not image: - image = self.default_image - - if image not in self: - if image and self.vars_dir: - fname = os.path.join(self.vars_dir, image + '.env') - if os.path.isfile(fname): - # parse .env file - with open(fname) as varsfile: - for line in varsfile: - self._parse_line(line, image) - else: - print("Couldn't get bitbake variable from %s." % fname) - print("File %s doesn't exist." % fname) - return - else: - # Get bitbake -e output - cmd = "bitbake -e" - if image: - cmd += " %s" % image - - log_level = logger.getEffectiveLevel() - logger.setLevel(logging.INFO) - ret, lines = _exec_cmd(cmd) - logger.setLevel(log_level) - - if ret: - logger.error("Couldn't get '%s' output.", cmd) - logger.error("Bitbake failed with error:\n%s\n", lines) - return - - # Parse bitbake -e output - for line in lines.split('\n'): - self._parse_line(line, image) - - # Make first image a default set of variables - if cache: - images = [key for key in self if key] - if len(images) == 1: - self[None] = self[image] - - result = self[image].get(var) - if not cache: - self.pop(image, None) - - return result - -# Create BB_VARS singleton -BB_VARS = BitbakeVars() - -def get_bitbake_var(var, image=None, cache=True): - """ - Provide old get_bitbake_var API by wrapping - get_var method of BB_VARS singleton. - """ - if var == "WIC_SECTOR_SIZE": - env_val = os.environ.get("WIC_SECTOR_SIZE") - if env_val is not None: - logger.warning("DEPRECATED: Using WIC_SECTOR_SIZE from environment; prefer --sector-size to avoid surprises.") - return env_val - - return BB_VARS.get_var(var, image, cache) diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py deleted file mode 100644 index df6e3271649b..000000000000 --- a/scripts/lib/wic/partition.py +++ /dev/null @@ -1,607 +0,0 @@ -# -# Copyright (c) 2013-2016 Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This module provides the OpenEmbedded partition object definitions. -# -# AUTHORS -# Tom Zanussi -# Ed Bartosh (at] linux.intel.com> - -import logging -import os -import uuid - -from wic import WicError -from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var -from wic.pluginbase import PluginMgr - -logger = logging.getLogger('wic') - -class Partition(): - - def __init__(self, args, lineno): - self.args = args - self.active = args.active - self.align = args.align - self.disk = args.disk - self.device = None - self.extra_filesystem_space = args.extra_filesystem_space - self.extra_partition_space = args.extra_partition_space - self.exclude_path = args.exclude_path - self.include_path = args.include_path - self.change_directory = args.change_directory - self.fsopts = args.fsopts - self.fspassno = args.fspassno - self.fstype = args.fstype - self.label = args.label - self.use_label = args.use_label - self.mkfs_extraopts = args.mkfs_extraopts - self.mountpoint = args.mountpoint - self.no_table = args.no_table - self.num = None - self.offset = args.offset - self.overhead_factor = args.overhead_factor - self.part_name = args.part_name - self.part_type = args.part_type - self.rootfs_dir = args.rootfs_dir - self.size = args.size - self.fixed_size = args.fixed_size - self.source = args.source - self.sourceparams = args.sourceparams - self.system_id = args.system_id - self.use_uuid = args.use_uuid - self.uuid = args.uuid - self.fsuuid = args.fsuuid - self.type = args.type - self.no_fstab_update = args.no_fstab_update - self.updated_fstab_path = None - self.has_fstab = False - self.update_fstab_in_rootfs = False - self.hidden = args.hidden - self.mbr = args.mbr - - self.lineno = lineno - self.source_file = "" - self.sector_size = 512 - - def _mkdosfs_extraopts(self): - """ - Build mkdosfs extra options ensuring the CLI sector size is applied. - """ - extraopts = self.mkfs_extraopts or '' - tokens = [] - skip_next = False - for tok in extraopts.split(): - if skip_next: - skip_next = False - continue - if tok == '-S': - skip_next = True - continue - if tok.startswith('-S'): - continue - tokens.append(tok) - tokens.extend(['-S', str(self.sector_size)]) - return ' '.join(tokens).strip() - - def _mkfs_ext_extraopts(self, base_opts): - """ - Build mkfs.ext* extra options ensuring the CLI sector size is applied. - """ - extraopts = self.mkfs_extraopts or base_opts - # Only add an explicit block size when a non-default sector size is requested. - if self.sector_size and self.sector_size != 512: - tokens = [] - skip_next = False - for tok in extraopts.split(): - if skip_next: - skip_next = False - continue - if tok == '-b': - skip_next = True - continue - if tok.startswith('-b'): - continue - tokens.append(tok) - tokens.extend(['-b', str(self.sector_size)]) - return ' '.join(tokens).strip() - return extraopts - - def get_extra_block_count(self, current_blocks): - """ - The --size param is reflected in self.size (in kB), and we already - have current_blocks (1k) blocks, calculate and return the - number of (1k) blocks we need to add to get to --size, 0 if - we're already there or beyond. - """ - logger.debug("Requested partition size for %s: %d", - self.mountpoint, self.size) - - if not self.size: - return 0 - - requested_blocks = self.size - - logger.debug("Requested blocks %d, current_blocks %d", - requested_blocks, current_blocks) - - if requested_blocks > current_blocks: - return requested_blocks - current_blocks - else: - return 0 - - def get_rootfs_size(self, actual_rootfs_size=0): - """ - Calculate the required size of rootfs taking into consideration - --size/--fixed-size and --extra-partition-space flags as well as overhead - and extra space, as specified in kickstart file. Raises an error - if the `actual_rootfs_size` is larger than fixed-size rootfs. - """ - if self.fixed_size: - rootfs_size = self.fixed_size - self.extra_partition_space - if actual_rootfs_size > rootfs_size: - raise WicError("Actual rootfs size (%d kB) is larger than " - "allowed size %d kB" % - (actual_rootfs_size, rootfs_size)) - else: - extra_blocks = self.get_extra_block_count(actual_rootfs_size) - if extra_blocks < self.extra_filesystem_space: - extra_blocks = self.extra_filesystem_space - - rootfs_size = actual_rootfs_size + extra_blocks - rootfs_size = int(rootfs_size * self.overhead_factor) - - logger.debug("Added %d extra blocks to %s to get to %d total blocks", - extra_blocks, self.mountpoint, rootfs_size) - - return rootfs_size - - @property - def disk_size(self): - """ - Obtain on-disk size of partition taking into consideration - --size/--fixed-size and --extra-partition-space options. - - """ - return self.fixed_size if self.fixed_size else self.size + self.extra_partition_space - - @property - def fs_size(self): - """ - Obtain on-disk size of filesystem inside the partition taking into - consideration --size/--fixed-size and --extra-partition-space options. - """ - return self.fixed_size - self.extra_partition_space if self.fixed_size else self.size - - def prepare(self, creator, cr_workdir, oe_builddir, rootfs_dir, - bootimg_dir, kernel_dir, native_sysroot, updated_fstab_path): - """ - Prepare content for individual partitions, depending on - partition command parameters. - """ - # capture the sector size requested on the CLI for mkdosfs invocations - self.sector_size = getattr(creator, 'sector_size', 512) or 512 - self.updated_fstab_path = updated_fstab_path - if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"): - self.update_fstab_in_rootfs = True - - if not self.source: - if self.fstype == "none" or self.no_table: - return - if not self.size and not self.fixed_size: - raise WicError("The %s partition has a size of zero. Please " - "specify a non-zero --size/--fixed-size for that " - "partition." % self.mountpoint) - - if self.fstype == "swap": - self.prepare_swap_partition(cr_workdir, oe_builddir, - native_sysroot) - self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype) - else: - if self.fstype in ('squashfs', 'erofs'): - raise WicError("It's not possible to create empty %s " - "partition '%s'" % (self.fstype, self.mountpoint)) - - rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label, - self.lineno, self.fstype) - if os.path.isfile(rootfs): - os.remove(rootfs) - - prefix = "ext" if self.fstype.startswith("ext") else self.fstype - method = getattr(self, "prepare_empty_partition_" + prefix) - method(rootfs, oe_builddir, native_sysroot) - self.source_file = rootfs - return - - plugins = PluginMgr.get_plugins('source') - - # Don't support '-' in plugin names - self.source = self.source.replace("-", "_") - - if self.source not in plugins: - raise WicError("The '%s' --source specified for %s doesn't exist.\n\t" - "See 'wic list source-plugins' for a list of available" - " --sources.\n\tSee 'wic help plugins' for " - "details on adding a new source plugin." % - (self.source, self.mountpoint)) - - srcparams_dict = {} - if self.sourceparams: - # Split sourceparams string of the form key1=val1[,key2=val2,...] - # into a dict. Also accepts valueless keys i.e. without = - splitted = self.sourceparams.split(',') - srcparams_dict = dict((par.split('=', 1) + [None])[:2] for par in splitted if par) - - plugin = plugins[self.source] - plugin.do_configure_partition(self, srcparams_dict, creator, - cr_workdir, oe_builddir, bootimg_dir, - kernel_dir, native_sysroot) - plugin.do_stage_partition(self, srcparams_dict, creator, - cr_workdir, oe_builddir, bootimg_dir, - kernel_dir, native_sysroot) - plugin.do_prepare_partition(self, srcparams_dict, creator, - cr_workdir, oe_builddir, bootimg_dir, - kernel_dir, rootfs_dir, native_sysroot) - plugin.do_post_partition(self, srcparams_dict, creator, - cr_workdir, oe_builddir, bootimg_dir, - kernel_dir, rootfs_dir, native_sysroot) - - # further processing required Partition.size to be an integer, make - # sure that it is one - if not isinstance(self.size, int): - raise WicError("Partition %s internal size is not an integer. " - "This a bug in source plugin %s and needs to be fixed." % - (self.mountpoint, self.source)) - - if self.fixed_size and self.size + self.extra_partition_space > self.fixed_size: - raise WicError("File system image of partition %s is " - "larger (%d kB + %d kB extra part space) than its allowed size %d kB" % - (self.mountpoint, self.size, self.extra_partition_space, self.fixed_size)) - - def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir, - native_sysroot, real_rootfs = True, pseudo_dir = None): - """ - Prepare content for a rootfs partition i.e. create a partition - and fill it from a /rootfs dir. - - Currently handles ext2/3/4, btrfs, vfat and squashfs. - """ - - rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label, - self.lineno, self.fstype) - if os.path.isfile(rootfs): - os.remove(rootfs) - - p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" % native_sysroot) - if (pseudo_dir): - # Canonicalize the ignore paths. This corresponds to - # calling oe.path.canonicalize(), which is used in bitbake.conf. - include_paths = [rootfs_dir] + (get_bitbake_var("PSEUDO_INCLUDE_PATHS") or "").split(",") - canonical_paths = [] - for path in include_paths: - if "$" not in path: - trailing_slash = path.endswith("/") and "/" or "" - canonical_paths.append(os.path.realpath(path) + trailing_slash) - include_paths = ",".join(canonical_paths) - - pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix - pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir - pseudo += "export PSEUDO_PASSWD=%s;" % rootfs_dir - pseudo += "export PSEUDO_NOSYMLINKEXP=1;" - pseudo += "export PSEUDO_INCLUDE_PATHS=%s;" % include_paths - pseudo += "%s " % get_bitbake_var("FAKEROOTCMD") - else: - pseudo = None - - if not self.size and real_rootfs: - # The rootfs size is not set in .ks file so try to get it - # from bitbake variable - rsize_bb = get_bitbake_var('ROOTFS_SIZE') - rdir = get_bitbake_var('IMAGE_ROOTFS') - if rsize_bb and (rdir == rootfs_dir or (rootfs_dir.split('/')[-2] == "tmp-wic" and rootfs_dir.split('/')[-1][:6] == "rootfs")): - # Bitbake variable ROOTFS_SIZE is calculated in - # Image._get_rootfs_size method from meta/lib/oe/image.py - # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, - # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE - self.size = int(round(float(rsize_bb))) - else: - # Bitbake variable ROOTFS_SIZE is not defined so compute it - # from the rootfs_dir size using the same logic found in - # get_rootfs_size() from meta/classes/image.bbclass - du_cmd = "du -ks %s" % rootfs_dir - out = exec_cmd(du_cmd) - self.size = int(out.split()[0]) - - prefix = "ext" if self.fstype.startswith("ext") else self.fstype - method = getattr(self, "prepare_rootfs_" + prefix) - method(rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot, pseudo) - self.source_file = rootfs - - # get the rootfs size in the right units for kickstart (kB) - du_cmd = "du --apparent-size -Lks %s" % rootfs - out = exec_cmd(du_cmd) - self.size = int(out.split()[0]) - - def prepare_rootfs_ext(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, - native_sysroot, pseudo): - """ - Prepare content for an ext2/3/4 rootfs partition. - """ - du_cmd = "du -ks %s" % rootfs_dir - out = exec_cmd(du_cmd) - actual_rootfs_size = int(out.split()[0]) - - rootfs_size = self.get_rootfs_size(actual_rootfs_size) - - with open(rootfs, 'w') as sparse: - os.ftruncate(sparse.fileno(), rootfs_size * 1024) - - extraopts = self._mkfs_ext_extraopts("-F -i 8192") - - # use hash_seed to generate reproducible ext4 images - (extraopts, pseudo) = self.get_hash_seed_ext4(extraopts, pseudo) - - label_str = "" - if self.label: - label_str = "-L %s" % self.label - - mkfs_cmd = "mkfs.%s %s %s %s -U %s -d %s" % \ - (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir) - exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) - - if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update: - debugfs_script_path = os.path.join(cr_workdir, "debugfs_script") - with open(debugfs_script_path, "w") as f: - f.write("cd etc\n") - f.write("rm fstab\n") - f.write("write %s fstab\n" % (self.updated_fstab_path)) - debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs) - exec_native_cmd(debugfs_cmd, native_sysroot) - - mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) - exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) - - if os.getenv('SOURCE_DATE_EPOCH'): - sde_time = hex(int(os.getenv('SOURCE_DATE_EPOCH'))) - debugfs_script_path = os.path.join(cr_workdir, "debugfs_script") - files = [] - for root, dirs, others in os.walk(rootfs_dir): - base = root.replace(rootfs_dir, "").rstrip(os.sep) - files += [ "/" if base == "" else base ] - files += [ base + "/" + n for n in dirs + others ] - with open(debugfs_script_path, "w") as f: - f.write("set_current_time %s\n" % (sde_time)) - if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update: - f.write("set_inode_field /etc/fstab mtime %s\n" % (sde_time)) - f.write("set_inode_field /etc/fstab mtime_extra 0\n") - for file in set(files): - for time in ["atime", "ctime", "crtime"]: - f.write("set_inode_field \"%s\" %s %s\n" % (file, time, sde_time)) - f.write("set_inode_field \"%s\" %s_extra 0\n" % (file, time)) - for time in ["wtime", "mkfs_time", "lastcheck"]: - f.write("set_super_value %s %s\n" % (time, sde_time)) - for time in ["mtime", "first_error_time", "last_error_time"]: - f.write("set_super_value %s 0\n" % (time)) - debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs) - exec_native_cmd(debugfs_cmd, native_sysroot) - - self.check_for_Y2038_problem(rootfs, native_sysroot) - - def get_hash_seed_ext4(self, extraopts, pseudo): - if os.getenv('SOURCE_DATE_EPOCH'): - sde_time = int(os.getenv('SOURCE_DATE_EPOCH')) - if pseudo: - pseudo = "export E2FSPROGS_FAKE_TIME=%s;%s " % (sde_time, pseudo) - else: - pseudo = "export E2FSPROGS_FAKE_TIME=%s; " % sde_time - - # Set hash_seed to generate deterministic directory indexes - namespace = uuid.UUID("e7429877-e7b3-4a68-a5c9-2f2fdf33d460") - if self.fsuuid: - namespace = uuid.UUID(self.fsuuid) - hash_seed = str(uuid.uuid5(namespace, str(sde_time))) - extraopts += " -E hash_seed=%s" % hash_seed - - return (extraopts, pseudo) - - def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, - native_sysroot, pseudo): - """ - Prepare content for a btrfs rootfs partition. - """ - du_cmd = "du -ks %s" % rootfs_dir - out = exec_cmd(du_cmd) - actual_rootfs_size = int(out.split()[0]) - - rootfs_size = self.get_rootfs_size(actual_rootfs_size) - - with open(rootfs, 'w') as sparse: - os.ftruncate(sparse.fileno(), rootfs_size * 1024) - - label_str = "" - if self.label: - label_str = "-L %s" % self.label - - mkfs_cmd = "mkfs.%s -b %d -r %s %s %s -U %s %s" % \ - (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, - self.mkfs_extraopts, self.fsuuid, rootfs) - exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) - - def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, - native_sysroot, pseudo): - """ - Prepare content for a msdos/vfat rootfs partition. - """ - du_cmd = "du --apparent-size -ks %s" % rootfs_dir - out = exec_cmd(du_cmd) - blocks = int(out.split()[0]) - - rootfs_size = self.get_rootfs_size(blocks) - - label_str = "-n boot" - if self.label: - label_str = "-n %s" % self.label - - size_str = "" - - extraopts = self._mkdosfs_extraopts() - - dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \ - (label_str, self.fsuuid, size_str, extraopts, rootfs, - rootfs_size) - exec_native_cmd(dosfs_cmd, native_sysroot) - - mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir) - exec_native_cmd(mcopy_cmd, native_sysroot) - - if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update: - mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path) - exec_native_cmd(mcopy_cmd, native_sysroot) - - chmod_cmd = "chmod 644 %s" % rootfs - exec_cmd(chmod_cmd) - - prepare_rootfs_vfat = prepare_rootfs_msdos - - def prepare_rootfs_squashfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, - native_sysroot, pseudo): - """ - Prepare content for a squashfs rootfs partition. - """ - extraopts = self.mkfs_extraopts or '-noappend' - squashfs_cmd = "mksquashfs %s %s %s" % \ - (rootfs_dir, rootfs, extraopts) - exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo) - - def prepare_rootfs_erofs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, - native_sysroot, pseudo): - """ - Prepare content for a erofs rootfs partition. - """ - extraopts = self.mkfs_extraopts or '' - erofs_cmd = "mkfs.erofs %s -U %s %s %s" % \ - (extraopts, self.fsuuid, rootfs, rootfs_dir) - exec_native_cmd(erofs_cmd, native_sysroot, pseudo=pseudo) - - def prepare_empty_partition_none(self, rootfs, oe_builddir, native_sysroot): - pass - - def prepare_empty_partition_ext(self, rootfs, oe_builddir, - native_sysroot): - """ - Prepare an empty ext2/3/4 partition. - """ - size = self.fs_size - with open(rootfs, 'w') as sparse: - os.ftruncate(sparse.fileno(), size * 1024) - - extraopts = self._mkfs_ext_extraopts("-i 8192") - - # use hash_seed to generate reproducible ext4 images - (extraopts, pseudo) = self.get_hash_seed_ext4(extraopts, None) - - label_str = "" - if self.label: - label_str = "-L %s" % self.label - - mkfs_cmd = "mkfs.%s -F %s %s -U %s %s" % \ - (self.fstype, extraopts, label_str, self.fsuuid, rootfs) - exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) - - self.check_for_Y2038_problem(rootfs, native_sysroot) - - def prepare_empty_partition_btrfs(self, rootfs, oe_builddir, - native_sysroot): - """ - Prepare an empty btrfs partition. - """ - size = self.fs_size - with open(rootfs, 'w') as sparse: - os.ftruncate(sparse.fileno(), size * 1024) - - label_str = "" - if self.label: - label_str = "-L %s" % self.label - - mkfs_cmd = "mkfs.%s -b %d %s -U %s %s %s" % \ - (self.fstype, self.size * 1024, label_str, self.fsuuid, - self.mkfs_extraopts, rootfs) - exec_native_cmd(mkfs_cmd, native_sysroot) - - def prepare_empty_partition_msdos(self, rootfs, oe_builddir, - native_sysroot): - """ - Prepare an empty vfat partition. - """ - blocks = self.fs_size - - label_str = "-n boot" - if self.label: - label_str = "-n %s" % self.label - - size_str = "" - - extraopts = self._mkdosfs_extraopts() - - dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \ - (label_str, self.fsuuid, extraopts, size_str, rootfs, - blocks) - - exec_native_cmd(dosfs_cmd, native_sysroot) - - chmod_cmd = "chmod 644 %s" % rootfs - exec_cmd(chmod_cmd) - - prepare_empty_partition_vfat = prepare_empty_partition_msdos - - def prepare_swap_partition(self, cr_workdir, oe_builddir, native_sysroot): - """ - Prepare a swap partition. - """ - path = "%s/fs.%s" % (cr_workdir, self.fstype) - - with open(path, 'w') as sparse: - os.ftruncate(sparse.fileno(), self.size * 1024) - - label_str = "" - if self.label: - label_str = "-L %s" % self.label - - mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path) - exec_native_cmd(mkswap_cmd, native_sysroot) - - def check_for_Y2038_problem(self, rootfs, native_sysroot): - """ - Check if the filesystem is affected by the Y2038 problem - (Y2038 problem = 32 bit time_t overflow in January 2038) - """ - def get_err_str(part): - err = "The {} filesystem {} has no Y2038 support." - if part.mountpoint: - args = [part.fstype, "mounted at %s" % part.mountpoint] - elif part.label: - args = [part.fstype, "labeled '%s'" % part.label] - elif part.part_name: - args = [part.fstype, "in partition '%s'" % part.part_name] - else: - args = [part.fstype, "in partition %s" % part.num] - return err.format(*args) - - # ext2 and ext3 are always affected by the Y2038 problem - if self.fstype in ["ext2", "ext3"]: - logger.warn(get_err_str(self)) - return - - ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) - - # if ext4 is affected by the Y2038 problem depends on the inode size - for line in out.splitlines(): - if line.startswith("Inode size:"): - size = int(line.split(":")[1].strip()) - if size < 256: - logger.warn("%s Inodes (of size %d) are too small." % - (get_err_str(self), size)) - break diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py deleted file mode 100644 index 640da292d3be..000000000000 --- a/scripts/lib/wic/pluginbase.py +++ /dev/null @@ -1,144 +0,0 @@ -#!/usr/bin/env python3 -# -# Copyright (c) 2011 Intel, Inc. -# -# SPDX-License-Identifier: GPL-2.0-only -# - -__all__ = ['ImagerPlugin', 'SourcePlugin'] - -import os -import logging -import types - -from collections import defaultdict -import importlib -import importlib.util - -from wic import WicError -from wic.misc import get_bitbake_var - -PLUGIN_TYPES = ["imager", "source"] - -SCRIPTS_PLUGIN_DIR = ["scripts/lib/wic/plugins", "lib/wic/plugins"] - -logger = logging.getLogger('wic') - -PLUGINS = defaultdict(dict) - -class PluginMgr: - _plugin_dirs = [] - - @classmethod - def get_plugins(cls, ptype): - """Get dictionary of : pairs.""" - if ptype not in PLUGIN_TYPES: - raise WicError('%s is not valid plugin type' % ptype) - - # collect plugin directories - if not cls._plugin_dirs: - cls._plugin_dirs = [os.path.join(os.path.dirname(__file__), 'plugins')] - layers = get_bitbake_var("BBLAYERS") or '' - for layer_path in layers.split(): - for script_plugin_dir in SCRIPTS_PLUGIN_DIR: - path = os.path.join(layer_path, script_plugin_dir) - path = os.path.abspath(os.path.expanduser(path)) - if path not in cls._plugin_dirs and os.path.isdir(path): - cls._plugin_dirs.append(path) - - if ptype not in PLUGINS: - # load all ptype plugins - for pdir in cls._plugin_dirs: - ppath = os.path.join(pdir, ptype) - if os.path.isdir(ppath): - for fname in os.listdir(ppath): - if fname.endswith('.py'): - mname = fname[:-3] - mpath = os.path.join(ppath, fname) - logger.debug("loading plugin module %s", mpath) - spec = importlib.util.spec_from_file_location(mname, mpath) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - - return PLUGINS.get(ptype) - -class PluginMeta(type): - def __new__(cls, name, bases, attrs): - class_type = type.__new__(cls, name, bases, attrs) - if 'name' in attrs: - PLUGINS[class_type.wic_plugin_type][attrs['name']] = class_type - - return class_type - -class ImagerPlugin(metaclass=PluginMeta): - wic_plugin_type = "imager" - - def do_create(self): - raise WicError("Method %s.do_create is not implemented" % - self.__class__.__name__) - -class SourcePlugin(metaclass=PluginMeta): - wic_plugin_type = "source" - """ - The methods that can be implemented by --source plugins. - - Any methods not implemented in a subclass inherit these. - """ - - @classmethod - def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir, - bootimg_dir, kernel_dir, native_sysroot): - """ - Called after all partitions have been prepared and assembled into a - disk image. This provides a hook to allow finalization of a - disk image e.g. to write an MBR to it. - """ - logger.debug("SourcePlugin: do_install_disk: disk: %s", disk_name) - - @classmethod - def do_stage_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Special content staging hook called before do_prepare_partition(), - normally empty. - - Typically, a partition will just use the passed-in parame e.g - straight bootimg_dir, etc, but in some cases, things need to - be more tailored e.g. to use a deploy dir + /boot, etc. This - hook allows those files to be staged in a customized fashion. - Not that get_bitbake_var() allows you to acces non-standard - variables that you might want to use for this. - """ - logger.debug("SourcePlugin: do_stage_partition: part: %s", part) - - @classmethod - def do_configure_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Called before do_prepare_partition(), typically used to create - custom configuration files for a partition, for example - syslinux or grub config files. - """ - logger.debug("SourcePlugin: do_configure_partition: part: %s", part) - - @classmethod - def do_prepare_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, rootfs_dir, - native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - """ - logger.debug("SourcePlugin: do_prepare_partition: part: %s", part) - - @classmethod - def do_post_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, rootfs_dir, - native_sysroot): - """ - Called after the partition is created. It is useful to add post - operations e.g. security signing the partition. - """ - logger.debug("SourcePlugin: do_post_partition: part: %s", part) diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py deleted file mode 100644 index 3adc6eee6280..000000000000 --- a/scripts/lib/wic/plugins/imager/direct.py +++ /dev/null @@ -1,702 +0,0 @@ -# -# Copyright (c) 2013, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This implements the 'direct' imager plugin class for 'wic' -# -# AUTHORS -# Tom Zanussi -# - -import logging -import os -import random -import shutil -import tempfile -import uuid - -from time import strftime - -from oe.path import copyhardlinktree - -from wic import WicError -from wic.filemap import sparse_copy -from wic.ksparser import KickStart, KickStartError -from wic.pluginbase import PluginMgr, ImagerPlugin -from wic.misc import get_bitbake_var, exec_cmd, exec_native_cmd - -logger = logging.getLogger('wic') - -class DirectPlugin(ImagerPlugin): - """ - Install a system into a file containing a partitioned disk image. - - An image file is formatted with a partition table, each partition - created from a rootfs or other OpenEmbedded build artifact and dd'ed - into the virtual disk. The disk image can subsequently be dd'ed onto - media and used on actual hardware. - """ - name = 'direct' - - def __init__(self, wks_file, rootfs_dir, bootimg_dir, kernel_dir, - native_sysroot, oe_builddir, options): - try: - self.ks = KickStart(wks_file) - except KickStartError as err: - raise WicError(str(err)) - - # parse possible 'rootfs=name' items - self.rootfs_dir = dict(rdir.split('=') for rdir in rootfs_dir.split(' ')) - self.bootimg_dir = bootimg_dir - self.kernel_dir = kernel_dir - self.native_sysroot = native_sysroot - self.oe_builddir = oe_builddir - - self.debug = options.debug - self.outdir = options.outdir - self.compressor = options.compressor - self.bmap = options.bmap - self.no_fstab_update = options.no_fstab_update - self.updated_fstab_path = None - - self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0], - strftime("%Y%m%d%H%M")) - self.workdir = self.setup_workdir(options.workdir) - self._image = None - self.ptable_format = self.ks.bootloader.ptable - self.parts = self.ks.partitions - self.sector_size = options.sector_size or 512 - - # as a convenience, set source to the boot partition source - # instead of forcing it to be set via bootloader --source - for part in self.parts: - if not self.ks.bootloader.source and part.mountpoint == "/boot": - self.ks.bootloader.source = part.source - break - - image_path = self._full_path(self.workdir, self.parts[0].disk, "direct") - self._image = PartitionedImage(image_path, self.ptable_format, self.ks.bootloader.diskid, - self.parts, self.native_sysroot, - options.extra_space, self.sector_size) - - def setup_workdir(self, workdir): - if workdir: - if os.path.exists(workdir): - raise WicError("Internal workdir '%s' specified in wic arguments already exists!" % (workdir)) - - os.makedirs(workdir) - return workdir - else: - return tempfile.mkdtemp(dir=self.outdir, prefix='tmp.wic.') - - def do_create(self): - """ - Plugin entry point. - """ - try: - self.create() - self.assemble() - self.finalize() - self.print_info() - finally: - self.cleanup() - - def update_fstab(self, image_rootfs): - """Assume partition order same as in wks""" - if not image_rootfs: - return - - fstab_path = image_rootfs + "/etc/fstab" - if not os.path.isfile(fstab_path): - return - - with open(fstab_path) as fstab: - fstab_lines = fstab.readlines() - - updated = False - for part in self.parts: - if not part.realnum or not part.mountpoint \ - or part.mountpoint == "/" or not (part.mountpoint.startswith('/') or part.mountpoint == "swap"): - continue - - if part.use_uuid: - if part.fsuuid: - # FAT UUID is different from others - if len(part.fsuuid) == 10: - device_name = "UUID=%s-%s" % \ - (part.fsuuid[2:6], part.fsuuid[6:]) - else: - device_name = "UUID=%s" % part.fsuuid - else: - device_name = "PARTUUID=%s" % part.uuid - elif part.use_label: - device_name = "LABEL=%s" % part.label - else: - # mmc device partitions are named mmcblk0p1, mmcblk0p2.. - prefix = 'p' if part.disk.startswith('mmcblk') else '' - device_name = "/dev/%s%s%d" % (part.disk, prefix, part.realnum) - - opts = part.fsopts if part.fsopts else "defaults" - passno = part.fspassno if part.fspassno else "0" - line = "\t".join([device_name, part.mountpoint, part.fstype, - opts, "0", passno]) + "\n" - - fstab_lines.append(line) - updated = True - - if updated: - self.updated_fstab_path = os.path.join(self.workdir, "fstab") - with open(self.updated_fstab_path, "w") as f: - f.writelines(fstab_lines) - if os.getenv('SOURCE_DATE_EPOCH'): - fstab_time = int(os.getenv('SOURCE_DATE_EPOCH')) - os.utime(self.updated_fstab_path, (fstab_time, fstab_time)) - - def _full_path(self, path, name, extention): - """ Construct full file path to a file we generate. """ - return os.path.join(path, "%s-%s.%s" % (self.name, name, extention)) - - # - # Actual implemention - # - def create(self): - """ - For 'wic', we already have our build artifacts - we just create - filesystems from the artifacts directly and combine them into - a partitioned image. - """ - if not self.no_fstab_update: - self.update_fstab(self.rootfs_dir.get("ROOTFS_DIR")) - - for part in self.parts: - # get rootfs size from bitbake variable if it's not set in .ks file - if not part.size: - # and if rootfs name is specified for the partition - image_name = self.rootfs_dir.get(part.rootfs_dir) - if image_name and os.path.sep not in image_name: - # Bitbake variable ROOTFS_SIZE is calculated in - # Image._get_rootfs_size method from meta/lib/oe/image.py - # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, - # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE - rsize_bb = get_bitbake_var('ROOTFS_SIZE', image_name) - if rsize_bb: - part.size = int(round(float(rsize_bb))) - - self._image.prepare(self) - self._image.layout_partitions() - self._image.create() - - def assemble(self): - """ - Assemble partitions into disk image - """ - self._image.assemble() - - def finalize(self): - """ - Finalize the disk image. - - For example, prepare the image to be bootable by e.g. - creating and installing a bootloader configuration. - """ - source_plugin = self.ks.bootloader.source - disk_name = self.parts[0].disk - if source_plugin: - # Don't support '-' in plugin names - source_plugin = source_plugin.replace("-", "_") - plugin = PluginMgr.get_plugins('source')[source_plugin] - plugin.do_install_disk(self._image, disk_name, self, self.workdir, - self.oe_builddir, self.bootimg_dir, - self.kernel_dir, self.native_sysroot) - - full_path = self._image.path - # Generate .bmap - if self.bmap: - logger.debug("Generating bmap file for %s", disk_name) - python = os.path.join(self.native_sysroot, 'usr/bin/python3-native/python3') - bmaptool = os.path.join(self.native_sysroot, 'usr/bin/bmaptool') - exec_native_cmd("%s %s create %s -o %s.bmap" % \ - (python, bmaptool, full_path, full_path), self.native_sysroot) - # Compress the image - if self.compressor: - logger.debug("Compressing disk %s with %s", disk_name, self.compressor) - exec_cmd("%s %s" % (self.compressor, full_path)) - - def print_info(self): - """ - Print the image(s) and artifacts used, for the user. - """ - msg = "The new image(s) can be found here:\n" - - extension = "direct" + {"gzip": ".gz", - "bzip2": ".bz2", - "xz": ".xz", - None: ""}.get(self.compressor) - full_path = self._full_path(self.outdir, self.parts[0].disk, extension) - msg += ' %s\n\n' % full_path - - msg += 'The following build artifacts were used to create the image(s):\n' - for part in self.parts: - if part.rootfs_dir is None: - continue - if part.mountpoint == '/': - suffix = ':' - else: - suffix = '["%s"]:' % (part.mountpoint or part.label) - rootdir = part.rootfs_dir - msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20), rootdir) - - msg += ' BOOTIMG_DIR: %s\n' % self.bootimg_dir - msg += ' KERNEL_DIR: %s\n' % self.kernel_dir - msg += ' NATIVE_SYSROOT: %s\n' % self.native_sysroot - - logger.info(msg) - - @property - def rootdev(self): - """ - Get root device name to use as a 'root' parameter - in kernel command line. - - Assume partition order same as in wks - """ - for part in self.parts: - if part.mountpoint == "/": - if part.uuid: - return "PARTUUID=%s" % part.uuid - elif part.label and self.ptable_format != 'msdos': - return "PARTLABEL=%s" % part.label - else: - suffix = 'p' if part.disk.startswith('mmcblk') else '' - return "/dev/%s%s%-d" % (part.disk, suffix, part.realnum) - - def cleanup(self): - if self._image: - self._image.cleanup() - - # Move results to the output dir - if not os.path.exists(self.outdir): - os.makedirs(self.outdir) - - for fname in os.listdir(self.workdir): - path = os.path.join(self.workdir, fname) - if os.path.isfile(path): - shutil.move(path, os.path.join(self.outdir, fname)) - - # remove work directory when it is not in debugging mode - if not self.debug: - shutil.rmtree(self.workdir, ignore_errors=True) - -# Overhead of the MBR partitioning scheme (just one sector) -MBR_OVERHEAD = 1 - -# Overhead of the GPT partitioning scheme -GPT_OVERHEAD = 34 - -class PartitionedImage(): - """ - Partitioned image in a file. - """ - - def __init__(self, path, ptable_format, disk_id, partitions, native_sysroot=None, extra_space=0, - sector_size=512): - self.path = path # Path to the image file - self.numpart = 0 # Number of allocated partitions - self.realpart = 0 # Number of partitions in the partition table - self.primary_part_num = 0 # Number of primary partitions (msdos) - self.extendedpart = 0 # Create extended partition before this logical partition (msdos) - self.extended_size_sec = 0 # Size of exteded partition (msdos) - self.logical_part_cnt = 0 # Number of total logical paritions (msdos) - self.offset = 0 # Offset of next partition (in sectors) - self.min_size = 0 # Minimum required disk size to fit - # all partitions (in bytes) - self.ptable_format = ptable_format # Partition table format - # Disk system identifier - if disk_id and ptable_format in ('gpt', 'gpt-hybrid'): - self.disk_guid = disk_id - elif os.getenv('SOURCE_DATE_EPOCH'): - self.disk_guid = uuid.UUID(int=int(os.getenv('SOURCE_DATE_EPOCH'))) - else: - self.disk_guid = uuid.uuid4() - - if disk_id and ptable_format == 'msdos': - self.identifier = disk_id - elif os.getenv('SOURCE_DATE_EPOCH'): - self.identifier = random.Random(int(os.getenv('SOURCE_DATE_EPOCH'))).randint(1, 0xffffffff) - else: - self.identifier = random.SystemRandom().randint(1, 0xffffffff) - - self.partitions = partitions - self.partimages = [] - # Size of a sector used in calculations - self.sector_size = sector_size or 512 - - self.native_sysroot = native_sysroot - num_real_partitions = len([p for p in self.partitions if not p.no_table]) - self.extra_space = extra_space - - # calculate the real partition number, accounting for partitions not - # in the partition table and logical partitions - realnum = 0 - for part in self.partitions: - if part.no_table: - part.realnum = 0 - else: - realnum += 1 - if self.ptable_format == 'msdos' and realnum > 3 and num_real_partitions > 4: - part.realnum = realnum + 1 - continue - part.realnum = realnum - - # generate parition and filesystem UUIDs - for part in self.partitions: - if not part.uuid and part.use_uuid: - if self.ptable_format in ('gpt', 'gpt-hybrid'): - part.uuid = str(uuid.uuid4()) - else: # msdos partition table - part.uuid = '%08x-%02d' % (self.identifier, part.realnum) - if not part.fsuuid: - if part.fstype == 'vfat' or part.fstype == 'msdos': - part.fsuuid = '0x' + str(uuid.uuid4())[:8].upper() - else: - part.fsuuid = str(uuid.uuid4()) - else: - #make sure the fsuuid for vfat/msdos align with format 0xYYYYYYYY - if part.fstype == 'vfat' or part.fstype == 'msdos': - if part.fsuuid.upper().startswith("0X"): - part.fsuuid = '0x' + part.fsuuid.upper()[2:].rjust(8,"0") - else: - part.fsuuid = '0x' + part.fsuuid.upper().rjust(8,"0") - - def prepare(self, imager): - """Prepare an image. Call prepare method of all image partitions.""" - for part in self.partitions: - # need to create the filesystems in order to get their - # sizes before we can add them and do the layout. - part.prepare(imager, imager.workdir, imager.oe_builddir, - imager.rootfs_dir, imager.bootimg_dir, - imager.kernel_dir, imager.native_sysroot, - imager.updated_fstab_path) - - # Converting kB to sectors for parted - part.size_sec = part.disk_size * 1024 // self.sector_size - - def layout_partitions(self): - """ Layout the partitions, meaning calculate the position of every - partition on the disk. The 'ptable_format' parameter defines the - partition table format and may be "msdos". """ - - logger.debug("Assigning %s partitions to disks", self.ptable_format) - - # The number of primary and logical partitions. Extended partition and - # partitions not listed in the table are not included. - num_real_partitions = len([p for p in self.partitions if not p.no_table]) - - # Go through partitions in the order they are added in .ks file - for num in range(len(self.partitions)): - part = self.partitions[num] - - if self.ptable_format == 'msdos' and part.part_name: - raise WicError("setting custom partition name is not " \ - "implemented for msdos partitions") - - if self.ptable_format == 'msdos' and part.part_type: - # The --part-type can also be implemented for MBR partitions, - # in which case it would map to the 1-byte "partition type" - # filed at offset 3 of the partition entry. - raise WicError("setting custom partition type is not " \ - "implemented for msdos partitions") - - if part.mbr and self.ptable_format != 'gpt-hybrid': - raise WicError("Partition may only be included in MBR with " \ - "a gpt-hybrid partition table") - - # Get the disk where the partition is located - self.numpart += 1 - if not part.no_table: - self.realpart += 1 - - if self.numpart == 1: - if self.ptable_format == "msdos": - overhead = MBR_OVERHEAD - elif self.ptable_format in ("gpt", "gpt-hybrid"): - overhead = GPT_OVERHEAD - - # Skip one sector required for the partitioning scheme overhead - self.offset += overhead - - if self.ptable_format == "msdos": - if self.primary_part_num > 3 or \ - (self.extendedpart == 0 and self.primary_part_num >= 3 and num_real_partitions > 4): - part.type = 'logical' - # Reserve a sector for EBR for every logical partition - # before alignment is performed. - if part.type == 'logical': - self.offset += 2 - - align_sectors = 0 - if part.align: - # If not first partition and we do have alignment set we need - # to align the partition. - # FIXME: This leaves a empty spaces to the disk. To fill the - # gaps we could enlargea the previous partition? - - # Calc how much the alignment is off. - align_sectors = self.offset % (part.align * 1024 // self.sector_size) - - if align_sectors: - # If partition is not aligned as required, we need - # to move forward to the next alignment point - align_sectors = (part.align * 1024 // self.sector_size) - align_sectors - - logger.debug("Realignment for %s%s with %s sectors, original" - " offset %s, target alignment is %sK.", - part.disk, self.numpart, align_sectors, - self.offset, part.align) - - # increase the offset so we actually start the partition on right alignment - self.offset += align_sectors - - if part.offset is not None: - offset = part.offset // self.sector_size - - if offset * self.sector_size != part.offset: - raise WicError("Could not place %s%s at offset %d with sector size %d" % (part.disk, self.numpart, part.offset, self.sector_size)) - - delta = offset - self.offset - if delta < 0: - raise WicError("Could not place %s%s at offset %d: next free sector is %d (delta: %d)" % (part.disk, self.numpart, part.offset, self.offset, delta)) - - logger.debug("Skipping %d sectors to place %s%s at offset %dK", - delta, part.disk, self.numpart, part.offset) - - self.offset = offset - - part.start = self.offset - self.offset += part.size_sec - - if not part.no_table: - part.num = self.realpart - else: - part.num = 0 - - if self.ptable_format == "msdos" and not part.no_table: - if part.type == 'logical': - self.logical_part_cnt += 1 - part.num = self.logical_part_cnt + 4 - if self.extendedpart == 0: - # Create extended partition as a primary partition - self.primary_part_num += 1 - self.extendedpart = part.num - else: - self.extended_size_sec += align_sectors - self.extended_size_sec += part.size_sec + 2 - else: - self.primary_part_num += 1 - part.num = self.primary_part_num - - logger.debug("Assigned %s to %s%d, sectors range %d-%d size %d " - "sectors (%d bytes).", part.mountpoint, part.disk, - part.num, part.start, self.offset - 1, part.size_sec, - part.size_sec * self.sector_size) - - # Once all the partitions have been layed out, we can calculate the - # minumim disk size - self.min_size = self.offset - if self.ptable_format in ("gpt", "gpt-hybrid"): - self.min_size += GPT_OVERHEAD - - self.min_size *= self.sector_size - self.min_size += self.extra_space - - def _create_partition(self, device, parttype, fstype, start, size): - """ Create a partition on an image described by the 'device' object. """ - - # Start is included to the size so we need to substract one from the end. - end = start + size - 1 - logger.debug("Added '%s' partition, sectors %d-%d, size %d sectors", - parttype, start, end, size) - - cmd = "export PARTED_SECTOR_SIZE=%d; parted -s %s unit s mkpart %s" % \ - (self.sector_size, device, parttype) - if fstype: - cmd += " %s" % fstype - cmd += " %d %d" % (start, end) - - return exec_native_cmd(cmd, self.native_sysroot) - - def _write_identifier(self, device, identifier): - logger.debug("Set disk identifier %x", identifier) - with open(device, 'r+b') as img: - img.seek(0x1B8) - img.write(identifier.to_bytes(4, 'little')) - - def _make_disk(self, device, ptable_format, min_size): - logger.debug("Creating sparse file %s", device) - with open(device, 'w') as sparse: - os.ftruncate(sparse.fileno(), min_size) - - logger.debug("Initializing partition table for %s", device) - exec_native_cmd("export PARTED_SECTOR_SIZE=%d; parted -s %s mklabel %s" % - (self.sector_size, device, ptable_format), self.native_sysroot) - - def _write_disk_guid(self): - if self.ptable_format in ('gpt', 'gpt-hybrid'): - logger.debug("Set disk guid %s", self.disk_guid) - sfdisk_cmd = "sfdisk --sector-size %s --disk-id %s %s" % \ - (self.sector_size, self.path, self.disk_guid) - exec_native_cmd(sfdisk_cmd, self.native_sysroot) - - def create(self): - self._make_disk(self.path, - "gpt" if self.ptable_format == "gpt-hybrid" else self.ptable_format, - self.min_size) - - self._write_identifier(self.path, self.identifier) - self._write_disk_guid() - - if self.ptable_format == "gpt-hybrid": - mbr_path = self.path + ".mbr" - self._make_disk(mbr_path, "msdos", self.min_size) - self._write_identifier(mbr_path, self.identifier) - - logger.debug("Creating partitions") - - hybrid_mbr_part_num = 0 - - for part in self.partitions: - if part.num == 0: - continue - - if self.ptable_format == "msdos" and part.num == self.extendedpart: - # Create an extended partition (note: extended - # partition is described in MBR and contains all - # logical partitions). The logical partitions save a - # sector for an EBR just before the start of a - # partition. The extended partition must start one - # sector before the start of the first logical - # partition. This way the first EBR is inside of the - # extended partition. Since the extended partitions - # starts a sector before the first logical partition, - # add a sector at the back, so that there is enough - # room for all logical partitions. - self._create_partition(self.path, "extended", - None, part.start - 2, - self.extended_size_sec) - - if part.fstype == "swap": - parted_fs_type = "linux-swap" - elif part.fstype == "vfat": - parted_fs_type = "fat32" - elif part.fstype == "msdos": - parted_fs_type = "fat16" - if not part.system_id: - part.system_id = '0x6' # FAT16 - else: - # Type for ext2/ext3/ext4/btrfs - parted_fs_type = "ext2" - - # Boot ROM of OMAP boards require vfat boot partition to have an - # even number of sectors. - if part.mountpoint == "/boot" and part.fstype in ["vfat", "msdos"] \ - and part.size_sec % 2: - logger.debug("Subtracting one sector from '%s' partition to " - "get even number of sectors for the partition", - part.mountpoint) - part.size_sec -= 1 - - self._create_partition(self.path, part.type, - parted_fs_type, part.start, part.size_sec) - - if self.ptable_format == "gpt-hybrid" and part.mbr: - hybrid_mbr_part_num += 1 - if hybrid_mbr_part_num > 4: - raise WicError("Extended MBR partitions are not supported in hybrid MBR") - self._create_partition(mbr_path, "primary", - parted_fs_type, part.start, part.size_sec) - - if self.ptable_format in ("gpt", "gpt-hybrid") and (part.part_name or part.label): - partition_label = part.part_name if part.part_name else part.label - logger.debug("partition %d: set name to %s", - part.num, partition_label) - exec_native_cmd("sfdisk --sector-size %s --part-label %s %d %s" % \ - (self.sector_size, self.path, part.num, - partition_label), self.native_sysroot) - if part.part_type: - logger.debug("partition %d: set type UID to %s", - part.num, part.part_type) - exec_native_cmd("sfdisk --sector-size %s --part-type %s %d %s" % \ - (self.sector_size, self.path, part.num, - part.part_type), self.native_sysroot) - - if part.uuid and self.ptable_format in ("gpt", "gpt-hybrid"): - logger.debug("partition %d: set UUID to %s", - part.num, part.uuid) - exec_native_cmd("sfdisk --sector-size %s --part-uuid %s %d %s" % \ - (self.sector_size, self.path, part.num, part.uuid), - self.native_sysroot) - - if part.active: - flag_name = "legacy_boot" if self.ptable_format in ('gpt', 'gpt-hybrid') else "boot" - logger.debug("Set '%s' flag for partition '%s' on disk '%s'", - flag_name, part.num, self.path) - exec_native_cmd("export PARTED_SECTOR_SIZE=%d; parted -s %s set %d %s on" % \ - (self.sector_size, self.path, part.num, flag_name), - self.native_sysroot) - if self.ptable_format == 'gpt-hybrid' and part.mbr: - exec_native_cmd("export PARTED_SECTOR_SIZE=%d; parted -s %s set %d %s on" % \ - (self.sector_size, mbr_path, hybrid_mbr_part_num, "boot"), - self.native_sysroot) - if part.system_id: - exec_native_cmd("sfdisk --sector-size %s --part-type %s %s %s" % \ - (self.sector_size, self.path, part.num, part.system_id), - self.native_sysroot) - - if part.hidden and self.ptable_format == "gpt": - logger.debug("Set hidden attribute for partition '%s' on disk '%s'", - part.num, self.path) - exec_native_cmd("sfdisk --sector-size %s --part-attrs %s %s RequiredPartition" % \ - (self.sector_size, self.path, part.num), - self.native_sysroot) - - if self.ptable_format == "gpt-hybrid": - # Write a protective GPT partition - hybrid_mbr_part_num += 1 - if hybrid_mbr_part_num > 4: - raise WicError("Extended MBR partitions are not supported in hybrid MBR") - - # parted cannot directly create a protective GPT partition, so - # create with an arbitrary type, then change it to the correct type - # with sfdisk - self._create_partition(mbr_path, "primary", "fat32", 1, GPT_OVERHEAD) - exec_native_cmd("sfdisk --sector-size %s --part-type %s %d 0xee" % \ - (self.sector_size, mbr_path, hybrid_mbr_part_num), - self.native_sysroot) - - # Copy hybrid MBR - with open(mbr_path, "rb") as mbr_file: - with open(self.path, "r+b") as image_file: - mbr = mbr_file.read(512) - image_file.write(mbr) - - def cleanup(self): - pass - - def assemble(self): - logger.debug("Installing partitions") - - for part in self.partitions: - source = part.source_file - if source: - # install source_file contents into a partition - sparse_copy(source, self.path, seek=part.start * self.sector_size) - - logger.debug("Installed %s in partition %d, sectors %d-%d, " - "size %d sectors", source, part.num, part.start, - part.start + part.size_sec - 1, part.size_sec) - - partimage = self.path + '.p%d' % part.num - os.rename(source, partimage) - self.partimages.append(partimage) diff --git a/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py b/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py deleted file mode 100644 index 4279ddded83a..000000000000 --- a/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py +++ /dev/null @@ -1,213 +0,0 @@ -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 2 as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License along -# with this program; if not, write to the Free Software Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -# -# DESCRIPTION -# This implements the 'bootimg_biosplusefi' source plugin class for 'wic' -# -# AUTHORS -# William Bourque - -import types - -from wic.pluginbase import SourcePlugin -from importlib.machinery import SourceFileLoader - -class BootimgBiosPlusEFIPlugin(SourcePlugin): - """ - Create MBR + EFI boot partition - - This plugin creates a boot partition that contains both - legacy BIOS and EFI content. It will be able to boot from both. - This is useful when managing PC fleet with some older machines - without EFI support. - - Note it is possible to create an image that can boot from both - legacy BIOS and EFI by defining two partitions : one with arg - --source bootimg_efi and another one with --source bootimg_pcbios. - However, this method has the obvious downside that it requires TWO - partitions to be created on the storage device. - Both partitions will also be marked as "bootable" which does not work on - most BIOS, has BIOS often uses the "bootable" flag to determine - what to boot. If you have such a BIOS, you need to manually remove the - "bootable" flag from the EFI partition for the drive to be bootable. - Having two partitions also seems to confuse wic : the content of - the first partition will be duplicated into the second, even though it - will not be used at all. - - Also, unlike "isoimage_isohybrid" that also does BIOS and EFI, this plugin - allows you to have more than only a single rootfs partitions and does - not turn the rootfs into an initramfs RAM image. - - This plugin is made to put everything into a single /boot partition so it - does not have the limitations listed above. - - The plugin is made so it does tries not to reimplement what's already - been done in other plugins; as such it imports "bootimg_pcbios" - and "bootimg_efi". - Plugin "bootimg_pcbios" is used to generate legacy BIOS boot. - Plugin "bootimg_efi" is used to generate the UEFI boot. Note that it - requires a --sourceparams argument to know which loader to use; refer - to "bootimg_efi" code/documentation for the list of loader. - - Imports are handled with "SourceFileLoader" from importlib as it is - otherwise very difficult to import module that has hyphen "-" in their - filename. - The SourcePlugin() methods used in the plugins (do_install_disk, - do_configure_partition, do_prepare_partition) are then called on both, - beginning by "bootimg_efi". - - Plugin options, such as "--sourceparams" can still be passed to a - plugin, as long they does not cause issue in the other plugin. - - Example wic configuration: - part /boot --source bootimg_biosplusefi --sourceparams="loader=grub-efi"\\ - --ondisk sda --label os_boot --active --align 1024 --use-uuid - """ - - name = 'bootimg_biosplusefi' - - __PCBIOS_MODULE_NAME = "bootimg_pcbios" - __EFI_MODULE_NAME = "bootimg_efi" - - __imgEFIObj = None - __imgBiosObj = None - - @classmethod - def __init__(cls): - """ - Constructor (init) - """ - - # XXX - # For some reasons, __init__ constructor is never called. - # Something to do with how pluginbase works? - cls.__instanciateSubClasses() - - @classmethod - def __instanciateSubClasses(cls): - """ - - """ - - # Import bootimg_pcbios (class name "BootimgPcbiosPlugin") - modulePath = os.path.join(os.path.dirname(os.path.realpath(__file__)), - cls.__PCBIOS_MODULE_NAME + ".py") - loader = SourceFileLoader(cls.__PCBIOS_MODULE_NAME, modulePath) - mod = types.ModuleType(loader.name) - loader.exec_module(mod) - cls.__imgBiosObj = mod.BootimgPcbiosPlugin() - - # Import bootimg_efi (class name "BootimgEFIPlugin") - modulePath = os.path.join(os.path.dirname(os.path.realpath(__file__)), - cls.__EFI_MODULE_NAME + ".py") - loader = SourceFileLoader(cls.__EFI_MODULE_NAME, modulePath) - mod = types.ModuleType(loader.name) - loader.exec_module(mod) - cls.__imgEFIObj = mod.BootimgEFIPlugin() - - @classmethod - def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir, - bootimg_dir, kernel_dir, native_sysroot): - """ - Called after all partitions have been prepared and assembled into a - disk image. - """ - - if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ): - cls.__instanciateSubClasses() - - cls.__imgEFIObj.do_install_disk( - disk, - disk_name, - creator, - workdir, - oe_builddir, - bootimg_dir, - kernel_dir, - native_sysroot) - - cls.__imgBiosObj.do_install_disk( - disk, - disk_name, - creator, - workdir, - oe_builddir, - bootimg_dir, - kernel_dir, - native_sysroot) - - @classmethod - def do_configure_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Called before do_prepare_partition() - """ - - if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ): - cls.__instanciateSubClasses() - - cls.__imgEFIObj.do_configure_partition( - part, - source_params, - creator, - cr_workdir, - oe_builddir, - bootimg_dir, - kernel_dir, - native_sysroot) - - cls.__imgBiosObj.do_configure_partition( - part, - source_params, - creator, - cr_workdir, - oe_builddir, - bootimg_dir, - kernel_dir, - native_sysroot) - - @classmethod - def do_prepare_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - """ - - if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ): - cls.__instanciateSubClasses() - - cls.__imgEFIObj.do_prepare_partition( - part, - source_params, - creator, - cr_workdir, - oe_builddir, - bootimg_dir, - kernel_dir, - rootfs_dir, - native_sysroot) - - cls.__imgBiosObj.do_prepare_partition( - part, - source_params, - creator, - cr_workdir, - oe_builddir, - bootimg_dir, - kernel_dir, - rootfs_dir, - native_sysroot) diff --git a/scripts/lib/wic/plugins/source/bootimg_efi.py b/scripts/lib/wic/plugins/source/bootimg_efi.py deleted file mode 100644 index 864d6898fc9d..000000000000 --- a/scripts/lib/wic/plugins/source/bootimg_efi.py +++ /dev/null @@ -1,436 +0,0 @@ -# -# Copyright (c) 2014, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This implements the 'bootimg_efi' source plugin class for 'wic' -# -# AUTHORS -# Tom Zanussi -# - -import logging -import os -import tempfile -import shutil -import re - -from glob import glob - -from wic import WicError -from wic.engine import get_custom_config -from wic.pluginbase import SourcePlugin -from wic.misc import (exec_cmd, exec_native_cmd, - get_bitbake_var, BOOTDD_EXTRA_SPACE) - -logger = logging.getLogger('wic') - -class BootimgEFIPlugin(SourcePlugin): - """ - Create EFI boot partition. - This plugin supports GRUB 2 and systemd-boot bootloaders. - """ - - name = 'bootimg_efi' - - @classmethod - def _copy_additional_files(cls, hdddir, initrd, dtb): - bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not bootimg_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - if initrd: - initrds = initrd.split(';') - for rd in initrds: - cp_cmd = "cp -v -p %s/%s %s" % (bootimg_dir, rd, hdddir) - out = exec_cmd(cp_cmd, True) - logger.debug("initrd files:\n%s" % (out)) - else: - logger.debug("Ignoring missing initrd") - - if dtb: - if ';' in dtb: - raise WicError("Only one DTB supported, exiting") - cp_cmd = "cp -v -p %s/%s %s" % (bootimg_dir, dtb, hdddir) - out = exec_cmd(cp_cmd, True) - logger.debug("dtb files:\n%s" % (out)) - - @classmethod - def do_configure_grubefi(cls, hdddir, creator, cr_workdir, source_params): - """ - Create loader-specific (grub-efi) config - """ - configfile = creator.ks.bootloader.configfile - custom_cfg = None - if configfile: - custom_cfg = get_custom_config(configfile) - if custom_cfg: - # Use a custom configuration for grub - grubefi_conf = custom_cfg - logger.debug("Using custom configuration file " - "%s for grub.cfg", configfile) - else: - raise WicError("configfile is specified but failed to " - "get it from %s." % configfile) - - initrd = source_params.get('initrd') - dtb = source_params.get('dtb') - - cls._copy_additional_files(hdddir, initrd, dtb) - - if not custom_cfg: - # Create grub configuration using parameters from wks file - bootloader = creator.ks.bootloader - title = source_params.get('title') - - grubefi_conf = "" - grubefi_conf += "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n" - grubefi_conf += "default=boot\n" - grubefi_conf += "timeout=%s\n" % bootloader.timeout - grubefi_conf += "menuentry '%s'{\n" % (title if title else "boot") - - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - label = source_params.get('label') - label_conf = "root=%s" % creator.rootdev - if label: - label_conf = "LABEL=%s" % label - - grubefi_conf += "linux /%s %s rootwait %s\n" \ - % (kernel, label_conf, bootloader.append) - - if initrd: - initrds = initrd.split(';') - grubefi_conf += "initrd" - for rd in initrds: - grubefi_conf += " /%s" % rd - grubefi_conf += "\n" - - if dtb: - grubefi_conf += "devicetree /%s\n" % dtb - - grubefi_conf += "}\n" - - logger.debug("Writing grubefi config %s/hdd/boot/EFI/BOOT/grub.cfg", - cr_workdir) - cfg = open("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir, "w") - cfg.write(grubefi_conf) - cfg.close() - - @classmethod - def do_configure_systemdboot(cls, hdddir, creator, cr_workdir, source_params): - """ - Create loader-specific systemd-boot/gummiboot config. Unified Kernel Image (uki) - support is done in image recipe with uki.bbclass and only systemd-boot loader config - and ESP partition structure is created here. - """ - # detect uki.bbclass usage - image_classes = get_bitbake_var("IMAGE_CLASSES").split() - unified_image = False - if "uki" in image_classes: - unified_image = True - - install_cmd = "install -d %s/loader" % hdddir - exec_cmd(install_cmd) - - install_cmd = "install -d %s/loader/entries" % hdddir - exec_cmd(install_cmd) - - bootloader = creator.ks.bootloader - loader_conf = "" - - # 5 seconds is a sensible default timeout - loader_conf += "timeout %d\n" % (bootloader.timeout or 5) - - logger.debug("Writing systemd-boot config " - "%s/hdd/boot/loader/loader.conf", cr_workdir) - cfg = open("%s/hdd/boot/loader/loader.conf" % cr_workdir, "w") - cfg.write(loader_conf) - logger.debug("loader.conf:\n%s" % (loader_conf)) - cfg.close() - - initrd = source_params.get('initrd') - dtb = source_params.get('dtb') - if not unified_image: - cls._copy_additional_files(hdddir, initrd, dtb) - - configfile = creator.ks.bootloader.configfile - custom_cfg = None - boot_conf = "" - if configfile: - custom_cfg = get_custom_config(configfile) - if custom_cfg: - # Use a custom configuration for systemd-boot - boot_conf = custom_cfg - logger.debug("Using custom configuration file " - "%s for systemd-boots's boot.conf", configfile) - else: - raise WicError("configfile is specified but failed to " - "get it from %s.", configfile) - else: - # Create systemd-boot configuration using parameters from wks file - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - title = source_params.get('title') - - boot_conf += "title %s\n" % (title if title else "boot") - boot_conf += "linux /%s\n" % kernel - - label = source_params.get('label') - label_conf = "LABEL=Boot root=%s" % creator.rootdev - if label: - label_conf = "LABEL=%s" % label - - boot_conf += "options %s %s\n" % \ - (label_conf, bootloader.append) - - if initrd: - initrds = initrd.split(';') - for rd in initrds: - boot_conf += "initrd /%s\n" % rd - - if dtb: - boot_conf += "devicetree /%s\n" % dtb - - if not unified_image: - logger.debug("Writing systemd-boot config " - "%s/hdd/boot/loader/entries/boot.conf", cr_workdir) - cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w") - cfg.write(boot_conf) - logger.debug("boot.conf:\n%s" % (boot_conf)) - cfg.close() - - - @classmethod - def do_configure_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Called before do_prepare_partition(), creates loader-specific config - """ - hdddir = "%s/hdd/boot" % cr_workdir - - install_cmd = "install -d %s/EFI/BOOT" % hdddir - exec_cmd(install_cmd) - - try: - if source_params['loader'] == 'grub-efi': - cls.do_configure_grubefi(hdddir, creator, cr_workdir, source_params) - elif source_params['loader'] == 'systemd-boot': - cls.do_configure_systemdboot(hdddir, creator, cr_workdir, source_params) - elif source_params['loader'] == 'uefi-kernel': - pass - else: - raise WicError("unrecognized bootimg_efi loader: %s" % source_params['loader']) - except KeyError: - raise WicError("bootimg_efi requires a loader, none specified") - - if get_bitbake_var("IMAGE_EFI_BOOT_FILES") is None: - logger.debug('No boot files defined in IMAGE_EFI_BOOT_FILES') - else: - boot_files = None - for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), (None, None)): - if fmt: - var = fmt % id - else: - var = "" - - boot_files = get_bitbake_var("IMAGE_EFI_BOOT_FILES" + var) - if boot_files: - break - - logger.debug('Boot files: %s', boot_files) - - # list of tuples (src_name, dst_name) - deploy_files = [] - for src_entry in re.findall(r'[\w;\-\.\+/\*]+', boot_files): - if ';' in src_entry: - dst_entry = tuple(src_entry.split(';')) - if not dst_entry[0] or not dst_entry[1]: - raise WicError('Malformed boot file entry: %s' % src_entry) - else: - dst_entry = (src_entry, src_entry) - - logger.debug('Destination entry: %r', dst_entry) - deploy_files.append(dst_entry) - - cls.install_task = []; - for deploy_entry in deploy_files: - src, dst = deploy_entry - if '*' in src: - # by default install files under their basename - entry_name_fn = os.path.basename - if dst != src: - # unless a target name was given, then treat name - # as a directory and append a basename - entry_name_fn = lambda name: \ - os.path.join(dst, - os.path.basename(name)) - - srcs = glob(os.path.join(kernel_dir, src)) - - logger.debug('Globbed sources: %s', ', '.join(srcs)) - for entry in srcs: - src = os.path.relpath(entry, kernel_dir) - entry_dst_name = entry_name_fn(entry) - cls.install_task.append((src, entry_dst_name)) - else: - cls.install_task.append((src, dst)) - - @classmethod - def do_prepare_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, prepare content for an EFI (grub) boot partition. - """ - if not kernel_dir: - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not kernel_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - staging_kernel_dir = kernel_dir - - hdddir = "%s/hdd/boot" % cr_workdir - - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - if source_params.get('create-unified-kernel-image') == "true": - raise WicError("create-unified-kernel-image is no longer supported. Please use uki.bbclass.") - - if source_params.get('install-kernel-into-boot-dir') != 'false': - install_cmd = "install -v -p -m 0644 %s/%s %s/%s" % \ - (staging_kernel_dir, kernel, hdddir, kernel) - out = exec_cmd(install_cmd) - logger.debug("Installed kernel files:\n%s" % out) - - if get_bitbake_var("IMAGE_EFI_BOOT_FILES"): - for src_path, dst_path in cls.install_task: - install_cmd = "install -v -p -m 0644 -D %s %s" \ - % (os.path.join(kernel_dir, src_path), - os.path.join(hdddir, dst_path)) - out = exec_cmd(install_cmd) - logger.debug("Installed IMAGE_EFI_BOOT_FILES:\n%s" % out) - - try: - if source_params['loader'] == 'grub-efi': - shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir, - "%s/grub.cfg" % cr_workdir) - for mod in [x for x in os.listdir(kernel_dir) if x.startswith("grub-efi-")]: - cp_cmd = "cp -v -p %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[9:]) - exec_cmd(cp_cmd, True) - shutil.move("%s/grub.cfg" % cr_workdir, - "%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir) - elif source_params['loader'] == 'systemd-boot': - for mod in [x for x in os.listdir(kernel_dir) if x.startswith("systemd-")]: - cp_cmd = "cp -v -p %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[8:]) - out = exec_cmd(cp_cmd, True) - logger.debug("systemd-boot files:\n%s" % out) - elif source_params['loader'] == 'uefi-kernel': - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if not kernel: - raise WicError("Empty KERNEL_IMAGETYPE") - target = get_bitbake_var("TARGET_SYS") - if not target: - raise WicError("Empty TARGET_SYS") - - if re.match("x86_64", target): - kernel_efi_image = "bootx64.efi" - elif re.match('i.86', target): - kernel_efi_image = "bootia32.efi" - elif re.match('aarch64', target): - kernel_efi_image = "bootaa64.efi" - elif re.match('arm', target): - kernel_efi_image = "bootarm.efi" - else: - raise WicError("UEFI stub kernel is incompatible with target %s" % target) - - for mod in [x for x in os.listdir(kernel_dir) if x.startswith(kernel)]: - cp_cmd = "cp -v -p %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, kernel_efi_image) - out = exec_cmd(cp_cmd, True) - logger.debug("uefi-kernel files:\n%s" % out) - else: - raise WicError("unrecognized bootimg_efi loader: %s" % - source_params['loader']) - - # must have installed at least one EFI bootloader - out = glob(os.path.join(hdddir, 'EFI', 'BOOT', 'boot*.efi')) - logger.debug("Installed EFI loader files:\n%s" % out) - if not out: - raise WicError("No EFI loaders installed to ESP partition. Check that grub-efi, systemd-boot or similar is installed.") - - except KeyError: - raise WicError("bootimg_efi requires a loader, none specified") - - startup = os.path.join(kernel_dir, "startup.nsh") - if os.path.exists(startup): - cp_cmd = "cp -v -p %s %s/" % (startup, hdddir) - out = exec_cmd(cp_cmd, True) - logger.debug("startup files:\n%s" % out) - - for paths in part.include_path or []: - for path in paths: - cp_cmd = "cp -v -p -r %s %s/" % (path, hdddir) - out = exec_cmd(cp_cmd, True) - logger.debug("include_path files:\n%s" % out) - - du_cmd = "du --apparent-size -ks %s" % hdddir - out = exec_cmd(du_cmd) - blocks = int(out.split()[0]) - - extra_blocks = part.get_extra_block_count(blocks) - - if extra_blocks < BOOTDD_EXTRA_SPACE: - extra_blocks = BOOTDD_EXTRA_SPACE - - blocks += extra_blocks - - logger.debug("Added %d extra blocks to %s to get to %d total blocks", - extra_blocks, part.mountpoint, blocks) - - # required for compatibility with certain devices expecting file system - # block count to be equal to partition block count - if blocks < part.fixed_size: - blocks = part.fixed_size - logger.debug("Overriding %s to %d total blocks for compatibility", - part.mountpoint, blocks) - - # dosfs image, created by mkdosfs - bootimg = "%s/boot.img" % cr_workdir - - label = part.label if part.label else "ESP" - - sector_size = getattr(creator, 'sector_size', 512) or 512 - dosfs_cmd = "mkdosfs -v -n %s -i %s -S %d -C %s %d" % \ - (label, part.fsuuid, sector_size, bootimg, blocks) - exec_native_cmd(dosfs_cmd, native_sysroot) - logger.debug("mkdosfs:\n%s" % (str(out))) - - mcopy_cmd = "mcopy -v -p -i %s -s %s/* ::/" % (bootimg, hdddir) - out = exec_native_cmd(mcopy_cmd, native_sysroot) - logger.debug("mcopy:\n%s" % (str(out))) - - chmod_cmd = "chmod 644 %s" % bootimg - exec_cmd(chmod_cmd) - - du_cmd = "du --apparent-size -Lks %s" % bootimg - out = exec_cmd(du_cmd) - bootimg_size = out.split()[0] - - part.size = int(bootimg_size) - part.source_file = bootimg diff --git a/scripts/lib/wic/plugins/source/bootimg_partition.py b/scripts/lib/wic/plugins/source/bootimg_partition.py deleted file mode 100644 index cc121a78f0a7..000000000000 --- a/scripts/lib/wic/plugins/source/bootimg_partition.py +++ /dev/null @@ -1,162 +0,0 @@ -# -# Copyright OpenEmbedded Contributors -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This implements the 'bootimg_partition' source plugin class for -# 'wic'. The plugin creates an image of boot partition, copying over -# files listed in IMAGE_BOOT_FILES bitbake variable. -# -# AUTHORS -# Maciej Borzecki -# - -import logging -import os -import re - -from oe.bootfiles import get_boot_files - -from wic import WicError -from wic.engine import get_custom_config -from wic.pluginbase import SourcePlugin -from wic.misc import exec_cmd, get_bitbake_var - -logger = logging.getLogger('wic') - -class BootimgPartitionPlugin(SourcePlugin): - """ - Create an image of boot partition, copying over files - listed in IMAGE_BOOT_FILES bitbake variable. - """ - - name = 'bootimg_partition' - image_boot_files_var_name = 'IMAGE_BOOT_FILES' - - @classmethod - def do_configure_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Called before do_prepare_partition(), create u-boot specific boot config - """ - hdddir = "%s/boot.%d" % (cr_workdir, part.lineno) - install_cmd = "install -d %s" % hdddir - exec_cmd(install_cmd) - - if not kernel_dir: - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not kernel_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - boot_files = None - for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), (None, None)): - if fmt: - var = fmt % id - else: - var = "" - - boot_files = get_bitbake_var(cls.image_boot_files_var_name + var) - if boot_files is not None: - break - - if boot_files is None: - raise WicError('No boot files defined, %s unset for entry #%d' % (cls.image_boot_files_var_name, part.lineno)) - - logger.debug('Boot files: %s', boot_files) - - cls.install_task = get_boot_files(kernel_dir, boot_files) - if source_params.get('loader') != "u-boot": - return - - configfile = cr.ks.bootloader.configfile - custom_cfg = None - if configfile: - custom_cfg = get_custom_config(configfile) - if custom_cfg: - # Use a custom configuration for extlinux.conf - extlinux_conf = custom_cfg - logger.debug("Using custom configuration file " - "%s for extlinux.conf", configfile) - else: - raise WicError("configfile is specified but failed to " - "get it from %s." % configfile) - - if not custom_cfg: - # The kernel types supported by the sysboot of u-boot - kernel_types = ["zImage", "Image", "fitImage", "uImage", "vmlinux"] - has_dtb = False - fdt_dir = '/' - kernel_name = None - - # Find the kernel image name, from the highest precedence to lowest - for image in kernel_types: - for task in cls.install_task: - src, dst = task - if re.match(image, src): - kernel_name = os.path.join('/', dst) - break - if kernel_name: - break - - for task in cls.install_task: - src, dst = task - # We suppose that all the dtb are in the same directory - if re.search(r'\.dtb', src) and fdt_dir == '/': - has_dtb = True - fdt_dir = os.path.join(fdt_dir, os.path.dirname(dst)) - break - - if not kernel_name: - raise WicError('No kernel file found') - - # Compose the extlinux.conf - extlinux_conf = "default Yocto\n" - extlinux_conf += "label Yocto\n" - extlinux_conf += " kernel %s\n" % kernel_name - if has_dtb: - extlinux_conf += " fdtdir %s\n" % fdt_dir - bootloader = cr.ks.bootloader - extlinux_conf += "append root=%s rootwait %s\n" \ - % (cr.rootdev, bootloader.append if bootloader.append else '') - - install_cmd = "install -d %s/extlinux/" % hdddir - exec_cmd(install_cmd) - cfg = open("%s/extlinux/extlinux.conf" % hdddir, "w") - cfg.write(extlinux_conf) - cfg.close() - - - @classmethod - def do_prepare_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, does the following: - - sets up a vfat partition - - copies all files listed in IMAGE_BOOT_FILES variable - """ - hdddir = "%s/boot.%d" % (cr_workdir, part.lineno) - - if not kernel_dir: - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not kernel_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - logger.debug('Kernel dir: %s', bootimg_dir) - - - for task in cls.install_task: - src_path, dst_path = task - logger.debug('Install %s as %s', src_path, dst_path) - install_cmd = "install -m 0644 -D %s %s" \ - % (os.path.join(kernel_dir, src_path), - os.path.join(hdddir, dst_path)) - exec_cmd(install_cmd) - - logger.debug('Prepare boot partition using rootfs in %s', hdddir) - part.prepare_rootfs(cr_workdir, oe_builddir, hdddir, - native_sysroot, False) diff --git a/scripts/lib/wic/plugins/source/bootimg_pcbios.py b/scripts/lib/wic/plugins/source/bootimg_pcbios.py deleted file mode 100644 index 32edac47fa09..000000000000 --- a/scripts/lib/wic/plugins/source/bootimg_pcbios.py +++ /dev/null @@ -1,484 +0,0 @@ -# -# Copyright (c) 2014, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This implements the 'bootimg_pcbios' source plugin class for 'wic' -# -# AUTHORS -# Tom Zanussi -# - -import logging -import os -import re -import shutil - -from glob import glob -from wic import WicError -from wic.engine import get_custom_config -from wic.pluginbase import SourcePlugin -from wic.misc import (exec_cmd, exec_native_cmd, - get_bitbake_var, BOOTDD_EXTRA_SPACE) - -logger = logging.getLogger('wic') - -class BootimgPcbiosPlugin(SourcePlugin): - """ - Creates boot partition that is legacy BIOS firmare bootable with - MBR/MSDOS as partition table format. Plugin will install caller - selected bootloader directly to resulting wic image. - - Supported Bootloaders: - * syslinux (default) - * grub - - ****************** Wic Plugin Depends/Vars ****************** - WKS_FILE_DEPENDS = "grub-native grub" - WKS_FILE_DEPENDS = "syslinux-native syslinux" - - # Optional variables - # GRUB_MKIMAGE_FORMAT_PC - Used to define target platform. - # GRUB_PREFIX_PATH - Used to define which directory - # grub config and modules are going - # to reside in. - GRUB_PREFIX_PATH = '/boot/grub2' # Default: /boot/grub - GRUB_MKIMAGE_FORMAT_PC = 'i386-pc' # Default: i386-pc - - WICVARS:append = "\ - GRUB_PREFIX_PATH \ - GRUB_MKIMAGE_FORMAT_PC \ - " - ****************** Wic Plugin Depends/Vars ****************** - - - **************** Example kickstart Legacy Bios Grub Boot **************** - part boot --label bios_boot --fstype ext4 --offset 1024 --fixed-size 78M - --source bootimg_pcbios --sourceparams="loader-bios=grub" --active - - part roots --label rootfs --fstype ext4 --source rootfs --use-uuid - bootloader --ptable msdos --source bootimg_pcbios - **************** Example kickstart Legacy Bios Grub Boot **************** - - - *************** Example kickstart Legacy Bios Syslinux Boot **************** - part /boot --source bootimg_pcbios --sourceparams="loader-bios=syslinux" - --ondisk sda --label boot --fstype vfat --align 1024 --active - - part roots --label rootfs --fstype ext4 --source rootfs --use-uuid - bootloader --ptable msdos --source bootimg_pcbios - """ - - name = 'bootimg_pcbios' - - # Variable required for do_install_disk - loader = '' - - @classmethod - def _get_bootimg_dir(cls, bootimg_dir, dirname): - """ - Check if dirname exists in default bootimg_dir or in STAGING_DIR. - """ - staging_datadir = get_bitbake_var("STAGING_DATADIR") - for result in (bootimg_dir, staging_datadir): - if os.path.exists("%s/%s" % (result, dirname)): - return result - - # STAGING_DATADIR is expanded with MLPREFIX if multilib is enabled - # but dependency syslinux is still populated to original STAGING_DATADIR - nonarch_datadir = re.sub('/[^/]*recipe-sysroot', '/recipe-sysroot', staging_datadir) - if os.path.exists(os.path.join(nonarch_datadir, dirname)): - return nonarch_datadir - - raise WicError("Couldn't find correct bootimg_dir, exiting") - - @classmethod - def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir, - bootimg_dir, kernel_dir, native_sysroot): - full_path = creator._full_path(workdir, disk_name, "direct") - logger.debug("Installing MBR on disk %s as %s with size %s bytes", - disk_name, full_path, disk.min_size) - - if cls.loader == 'grub': - cls._do_install_grub(creator, kernel_dir, - native_sysroot, full_path) - elif cls.loader == 'syslinux': - cls._do_install_syslinux(creator, bootimg_dir, - native_sysroot, full_path) - else: - raise WicError("boot loader some how not specified check do_prepare_partition") - - @classmethod - def do_configure_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - try: - if source_params['loader-bios'] == 'grub': - cls._do_configure_grub(part, creator, cr_workdir) - elif source_params['loader-bios'] == 'syslinux': - cls._do_configure_syslinux(part, creator, cr_workdir) - else: - raise WicError("unrecognized bootimg_pcbios loader: %s" % source_params['loader-bios']) - except KeyError: - cls._do_configure_syslinux(part, creator, cr_workdir) - - @classmethod - def do_prepare_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - try: - if source_params['loader-bios'] == 'grub': - cls._do_prepare_grub(part, cr_workdir, oe_builddir, - kernel_dir, rootfs_dir, native_sysroot) - elif source_params['loader-bios'] == 'syslinux': - cls._do_prepare_syslinux(part, creator, cr_workdir, bootimg_dir, - kernel_dir, native_sysroot) - else: - raise WicError("unrecognized bootimg_pcbios loader: %s" % source_params['loader-bios']) - - # Required by do_install_disk - cls.loader = source_params['loader-bios'] - except KeyError: - # Required by do_install_disk - cls.loader = 'syslinux' - cls._do_prepare_syslinux(part, creator, cr_workdir, bootimg_dir, - kernel_dir, native_sysroot) - - @classmethod - def _get_staging_libdir(cls): - """ - For unknown reasons when running test with poky - STAGING_LIBDIR gets unset when wic create is executed. - Bellow is a hack to determine what STAGING_LIBDIR should - be if not specified. - """ - - staging_libdir = get_bitbake_var('STAGING_LIBDIR') - staging_dir_target = get_bitbake_var('STAGING_DIR_TARGET') - - if not staging_libdir: - staging_libdir = '%s/usr/lib64' % staging_dir_target - if not os.path.isdir(staging_libdir): - staging_libdir = '%s/usr/lib32' % staging_dir_target - if not os.path.isdir(staging_libdir): - staging_libdir = '%s/usr/lib' % staging_dir_target - - return staging_libdir - - @classmethod - def _get_bootloader_config(cls, bootloader, loader): - custom_cfg = None - - if bootloader.configfile: - custom_cfg = get_custom_config(bootloader.configfile) - if custom_cfg: - logger.debug("Using custom configuration file %s " - "for %s.cfg", bootloader.configfile, - loader) - return custom_cfg - else: - raise WicError("configfile is specified but failed to " - "get it from %s." % bootloader.configfile) - return custom_cfg - - @classmethod - def _do_configure_syslinux(cls, part, creator, cr_workdir): - """ - Called before do_prepare_partition(), creates syslinux config - """ - - hdddir = "%s/hdd/boot" % cr_workdir - - install_cmd = "install -d %s" % hdddir - exec_cmd(install_cmd) - - bootloader = creator.ks.bootloader - syslinux_conf = cls._get_bootloader_config(bootloader, 'syslinux') - - if not syslinux_conf: - # Create syslinux configuration using parameters from wks file - splash = os.path.join(hdddir, "/splash.jpg") - if os.path.exists(splash): - splashline = "menu background splash.jpg" - else: - splashline = "" - - # Set a default timeout if none specified to avoid - # 'None' being the value placed within the configuration - # file. - if not bootloader.timeout: - bootloader.timeout = 500 - - # Set a default kernel params string if none specified - # to avoid 'None' being the value placed within the - # configuration file. - if not bootloader.append: - bootloader.append = "rootwait console=ttyS0,115200 console=tty0" - - syslinux_conf = "" - syslinux_conf += "PROMPT 0\n" - syslinux_conf += "TIMEOUT " + str(bootloader.timeout) + "\n" - syslinux_conf += "\n" - syslinux_conf += "ALLOWOPTIONS 1\n" - syslinux_conf += "SERIAL 0 115200\n" - syslinux_conf += "\n" - if splashline: - syslinux_conf += "%s\n" % splashline - syslinux_conf += "DEFAULT boot\n" - syslinux_conf += "LABEL boot\n" - - kernel = "/" + get_bitbake_var("KERNEL_IMAGETYPE") - syslinux_conf += "KERNEL " + kernel + "\n" - - syslinux_conf += "APPEND label=boot root=%s %s\n" % \ - (creator.rootdev, bootloader.append) - - logger.debug("Writing syslinux config %s/syslinux.cfg", hdddir) - cfg = open("%s/hdd/boot/syslinux.cfg" % cr_workdir, "w") - cfg.write(syslinux_conf) - cfg.close() - - @classmethod - def _do_prepare_syslinux(cls, part, creator, cr_workdir, bootimg_dir, - kernel_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, prepare content for legacy bios boot partition. - """ - bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux') - - staging_kernel_dir = kernel_dir - - hdddir = "%s/hdd/boot" % cr_workdir - - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - cmds = ("install -m 0644 %s/%s %s/%s" % - (staging_kernel_dir, kernel, hdddir, get_bitbake_var("KERNEL_IMAGETYPE")), - "install -m 444 %s/syslinux/ldlinux.sys %s/ldlinux.sys" % - (bootimg_dir, hdddir), - "install -m 0644 %s/syslinux/vesamenu.c32 %s/vesamenu.c32" % - (bootimg_dir, hdddir), - "install -m 444 %s/syslinux/libcom32.c32 %s/libcom32.c32" % - (bootimg_dir, hdddir), - "install -m 444 %s/syslinux/libutil.c32 %s/libutil.c32" % - (bootimg_dir, hdddir)) - - for install_cmd in cmds: - exec_cmd(install_cmd) - - du_cmd = "du --apparent-size -ks %s" % hdddir - out = exec_cmd(du_cmd) - blocks = int(out.split()[0]) - - extra_blocks = part.get_extra_block_count(blocks) - - if extra_blocks < BOOTDD_EXTRA_SPACE: - extra_blocks = BOOTDD_EXTRA_SPACE - - blocks += extra_blocks - - logger.debug("Added %d extra blocks to %s to get to %d total blocks", - extra_blocks, part.mountpoint, blocks) - - # dosfs image, created by mkdosfs - bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno) - - label = part.label if part.label else "boot" - - sector_size = getattr(creator, 'sector_size', 512) or 512 - dosfs_cmd = "mkdosfs -n %s -i %s -S %d -C %s %d" % \ - (label, part.fsuuid, sector_size, bootimg, blocks) - exec_native_cmd(dosfs_cmd, native_sysroot) - - mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) - exec_native_cmd(mcopy_cmd, native_sysroot) - - syslinux_cmd = "syslinux %s" % bootimg - exec_native_cmd(syslinux_cmd, native_sysroot) - - chmod_cmd = "chmod 644 %s" % bootimg - exec_cmd(chmod_cmd) - - du_cmd = "du --apparent-size -Lks %s" % bootimg - out = exec_cmd(du_cmd) - bootimg_size = out.split()[0] - - part.size = int(bootimg_size) - part.source_file = bootimg - - @classmethod - def _do_install_syslinux(cls, creator, bootimg_dir, - native_sysroot, full_path): - """ - Called after all partitions have been prepared and assembled into a - disk image. In this case, we install the MBR. - """ - - bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux') - mbrfile = "%s/syslinux/" % bootimg_dir - if creator.ptable_format == 'msdos': - mbrfile += "mbr.bin" - elif creator.ptable_format == 'gpt': - mbrfile += "gptmbr.bin" - else: - raise WicError("Unsupported partition table: %s" % - creator.ptable_format) - - if not os.path.exists(mbrfile): - raise WicError("Couldn't find %s. If using the -e option, do you " - "have the right MACHINE set in local.conf? If not, " - "is the bootimg_dir path correct?" % mbrfile) - - dd_cmd = "dd if=%s of=%s conv=notrunc" % (mbrfile, full_path) - exec_cmd(dd_cmd, native_sysroot) - - @classmethod - def _do_configure_grub(cls, part, creator, cr_workdir): - hdddir = "%s/hdd" % cr_workdir - bootloader = creator.ks.bootloader - - grub_conf = cls._get_bootloader_config(bootloader, 'grub') - - grub_prefix_path = get_bitbake_var('GRUB_PREFIX_PATH') - if not grub_prefix_path: - grub_prefix_path = '/boot/grub' - - grub_path = "%s/%s" %(hdddir, grub_prefix_path) - install_cmd = "install -d %s" % grub_path - exec_cmd(install_cmd) - - if not grub_conf: - # Set a default timeout if none specified to avoid - # 'None' being the value placed within the configuration - # file. - if not bootloader.timeout: - bootloader.timeout = 500 - - # Set a default kernel params string if none specified - # to avoid 'None' being the value placed within the - # configuration file. - if not bootloader.append: - bootloader.append = "rootwait rootfstype=%s " % (part.fstype) - bootloader.append += "console=ttyS0,115200 console=tty0" - - kernel = "/boot/" + get_bitbake_var("KERNEL_IMAGETYPE") - - grub_conf = 'serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n' - grub_conf += 'set gfxmode=auto\n' - grub_conf += 'set gfxpayload=keep\n\n' - grub_conf += 'set default=0\n\n' - grub_conf += '# Boot automatically after %d secs.\n' % (bootloader.timeout) - grub_conf += 'set timeout=%d\n\n' % (bootloader.timeout) - grub_conf += 'menuentry \'default\' {\n' - grub_conf += '\tsearch --no-floppy --set=root --file %s\n' % (kernel) - grub_conf += '\tprobe --set partuuid --part-uuid ($root)\n' - grub_conf += '\tlinux %s root=PARTUUID=$partuuid %s\n}\n' % \ - (kernel, bootloader.append) - - logger.debug("Writing grub config %s/grub.cfg", grub_path) - cfg = open("%s/grub.cfg" % grub_path, "w") - cfg.write(grub_conf) - cfg.close() - - @classmethod - def _do_prepare_grub(cls, part, cr_workdir, oe_builddir, - kernel_dir, rootfs_dir, native_sysroot): - """ - 1. Generate embed.cfg that'll later be embedded into core.img. - So, that core.img knows where to search for grub.cfg. - 2. Generate core.img or grub stage 1.5. - 3. Copy modules into partition. - 4. Create partition rootfs file. - """ - - hdddir = "%s/hdd" % cr_workdir - - copy_types = [ '*.mod', '*.o', '*.lst' ] - - builtin_modules = 'boot linux ext2 fat serial part_msdos part_gpt \ - normal multiboot probe biosdisk msdospart configfile search loadenv test' - - staging_libdir = cls._get_staging_libdir() - - grub_format = get_bitbake_var('GRUB_MKIMAGE_FORMAT_PC') - if not grub_format: - grub_format = 'i386-pc' - - grub_prefix_path = get_bitbake_var('GRUB_PREFIX_PATH') - if not grub_prefix_path: - grub_prefix_path = '/boot/grub' - - grub_path = "%s/%s" %(hdddir, grub_prefix_path) - core_img = '%s/grub-bios-core.img' % (kernel_dir) - grub_mods_path = '%s/grub/%s' % (staging_libdir, grub_format) - - # Generate embedded grub config - embed_cfg_str = 'search.file %s/grub.cfg root\n' % (grub_prefix_path) - embed_cfg_str += 'set prefix=($root)%s\n' % (grub_prefix_path) - embed_cfg_str += 'configfile ($root)%s/grub.cfg\n' % (grub_prefix_path) - cfg = open('%s/embed.cfg' % (kernel_dir), 'w+') - cfg.write(embed_cfg_str) - cfg.close() - - # core.img doesn't get included into boot partition - # it's later dd onto the resulting wic image. - grub_mkimage = 'grub-mkimage \ - --prefix=%s \ - --format=%s \ - --config=%s/embed.cfg \ - --directory=%s \ - --output=%s %s' % \ - (grub_prefix_path, grub_format, kernel_dir, - grub_mods_path, core_img, builtin_modules) - exec_native_cmd(grub_mkimage, native_sysroot) - - # Copy grub modules - install_dir = '%s/%s/%s' % (hdddir, grub_prefix_path, grub_format) - os.makedirs(install_dir, exist_ok=True) - - for ctype in copy_types: - files = glob('%s/grub/%s/%s' % \ - (staging_libdir, grub_format, ctype)) - for file in files: - shutil.copy2(file, install_dir, follow_symlinks=True) - - # Create boot partition - logger.debug('Prepare partition using rootfs in %s', hdddir) - part.prepare_rootfs(cr_workdir, oe_builddir, hdddir, - native_sysroot, False) - - @classmethod - def _do_install_grub(cls, creator, kernel_dir, - native_sysroot, full_path): - core_img = '%s/grub-bios-core.img' % (kernel_dir) - - staging_libdir = cls._get_staging_libdir() - - grub_format = get_bitbake_var('GRUB_MKIMAGE_FORMAT_PC') - if not grub_format: - grub_format = 'i386-pc' - - boot_img = '%s/grub/%s/boot.img' % (staging_libdir, grub_format) - if not os.path.exists(boot_img): - raise WicError("Couldn't find %s. Did you include " - "do_image_wic[depends] += \"grub:do_populate_sysroot\" " - "in your image recipe" % boot_img) - - # Install boot.img or grub stage 1 - dd_cmd = "dd if=%s of=%s conv=notrunc bs=1 seek=0 count=440" % (boot_img, full_path) - exec_cmd(dd_cmd, native_sysroot) - - if creator.ptable_format == 'msdos': - # Install core.img or grub stage 1.5 - dd_cmd = "dd if=%s of=%s conv=notrunc bs=1 seek=512" % (core_img, full_path) - exec_cmd(dd_cmd, native_sysroot) - else: - raise WicError("Unsupported partition table: %s" % - creator.ptable_format) diff --git a/scripts/lib/wic/plugins/source/empty.py b/scripts/lib/wic/plugins/source/empty.py deleted file mode 100644 index 4178912377d4..000000000000 --- a/scripts/lib/wic/plugins/source/empty.py +++ /dev/null @@ -1,89 +0,0 @@ -# -# Copyright OpenEmbedded Contributors -# -# SPDX-License-Identifier: MIT -# - -# The empty wic plugin is used to create unformatted empty partitions for wic -# images. -# To use it you must pass "empty" as argument for the "--source" parameter in -# the wks file. For example: -# part foo --source empty --ondisk sda --size="1024" --align 1024 -# -# The plugin supports writing zeros to the start of the -# partition. This is useful to overwrite old content like -# filesystem signatures which may be re-recognized otherwise. -# This feature can be enabled with -# '--sourceparams="[fill|size=[S|s|K|k|M|G]][,][bs=[S|s|K|k|M|G]]"' -# Conflicting or missing options throw errors. - -import logging -import os - -from wic import WicError -from wic.ksparser import sizetype -from wic.pluginbase import SourcePlugin - -logger = logging.getLogger('wic') - -class EmptyPartitionPlugin(SourcePlugin): - """ - Populate unformatted empty partition. - - The following sourceparams are supported: - - fill - Fill the entire partition with zeros. Requires '--fixed-size' option - to be set. - - size=[S|s|K|k|M|G] - Set the first N bytes of the partition to zero. Default unit is 'K'. - - bs=[S|s|K|k|M|G] - Write at most N bytes at a time during source file creation. - Defaults to '1M'. Default unit is 'K'. - """ - - name = 'empty' - - @classmethod - def do_prepare_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - """ - get_byte_count = sizetype('K', True) - size = 0 - - if 'fill' in source_params and 'size' in source_params: - raise WicError("Conflicting source parameters 'fill' and 'size' specified, exiting.") - - # Set the size of the zeros to be written to the partition - if 'fill' in source_params: - if part.fixed_size == 0: - raise WicError("Source parameter 'fill' only works with the '--fixed-size' option, exiting.") - size = get_byte_count(part.fixed_size) - elif 'size' in source_params: - size = get_byte_count(source_params['size']) - - if size == 0: - # Nothing to do, create empty partition - return - - if 'bs' in source_params: - bs = get_byte_count(source_params['bs']) - else: - bs = get_byte_count('1M') - - # Create a binary file of the requested size filled with zeros - source_file = os.path.join(cr_workdir, 'empty-plugin-zeros%s.bin' % part.lineno) - if not os.path.exists(os.path.dirname(source_file)): - os.makedirs(os.path.dirname(source_file)) - - quotient, remainder = divmod(size, bs) - with open(source_file, 'wb') as file: - for _ in range(quotient): - file.write(bytearray(bs)) - file.write(bytearray(remainder)) - - part.size = (size + 1024 - 1) // 1024 # size in KB rounded up - part.source_file = source_file diff --git a/scripts/lib/wic/plugins/source/extra_partition.py b/scripts/lib/wic/plugins/source/extra_partition.py deleted file mode 100644 index 0d49e2bc7cbb..000000000000 --- a/scripts/lib/wic/plugins/source/extra_partition.py +++ /dev/null @@ -1,146 +0,0 @@ -import logging -import os -import re - -from glob import glob - -from wic import WicError -from wic.pluginbase import SourcePlugin -from wic.misc import exec_cmd, get_bitbake_var - -logger = logging.getLogger('wic') - -class ExtraPartitionPlugin(SourcePlugin): - """ - Populates an extra partition with files listed in the IMAGE_EXTRA_PARTITION_FILES - BitBake variable. Files should be deployed to the DEPLOY_DIR_IMAGE directory. - - The plugin supports: - - Glob pattern matching for file selection. - - File renaming. - - Suffixes to specify the target partition (by params-name, label, UUID, - or partname), enabling multiple extra partitions to coexist. - - For example: - - IMAGE_EXTRA_PARTITION_FILES_name-randomname = "bar.conf;foo.conf" - IMAGE_EXTRA_PARTITION_FILES_label-foo = "bar.conf;foo.conf" - IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf" - IMAGE_EXTRA_PARTITION_FILES = "foo/*" - WICVARS:append = "\ - IMAGE_EXTRA_PARTITION_FILES_name-randomname \ - IMAGE_EXTRA_PARTITION_FILES_label-foo \ - IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \ - " - - """ - - name = 'extra_partition' - image_extra_partition_files_var_name = 'IMAGE_EXTRA_PARTITION_FILES' - - @classmethod - def do_configure_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Called before do_prepare_partition(), list the files to copy - """ - extradir = "%s/extra.%d" % (cr_workdir, part.lineno) - install_cmd = "install -d %s" % extradir - exec_cmd(install_cmd) - - if not kernel_dir: - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not kernel_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - extra_files = None - for (fmt, part_id) in ( - ("_name-%s", source_params.get("name")), - ("_uuid-%s", part.uuid), - ("_label-%s", part.label), - ("_part-name-%s", part.part_name), - (None, None) - ): - if fmt is None: - var = "" - elif part_id is not None: - var = fmt % part_id - else: - continue - - logger.debug("Looking for extra files in %s" % cls.image_extra_partition_files_var_name + var) - extra_files = get_bitbake_var(cls.image_extra_partition_files_var_name + var) - if extra_files is not None: - break - - if extra_files is None: - raise WicError('No extra files defined, %s unset for entry #%d' % (cls.image_extra_partition_files_var_name, part.lineno)) - - logger.info('Extra files: %s', extra_files) - - # list of tuples (src_name, dst_name) - deploy_files = [] - for src_entry in re.findall(r'[\w;\-\./\*]+', extra_files): - if ';' in src_entry: - dst_entry = tuple(src_entry.split(';')) - if not dst_entry[0] or not dst_entry[1]: - raise WicError('Malformed extra file entry: %s' % src_entry) - else: - dst_entry = (src_entry, src_entry) - - logger.debug('Destination entry: %r', dst_entry) - deploy_files.append(dst_entry) - - cls.install_task = []; - for deploy_entry in deploy_files: - src, dst = deploy_entry - if '*' in src: - # by default install files under their basename - entry_name_fn = os.path.basename - if dst != src: - # unless a target name was given, then treat name - # as a directory and append a basename - entry_name_fn = lambda name: \ - os.path.join(dst, - os.path.basename(name)) - - srcs = glob(os.path.join(kernel_dir, src)) - - logger.debug('Globbed sources: %s', ', '.join(srcs)) - for entry in srcs: - src = os.path.relpath(entry, kernel_dir) - entry_dst_name = entry_name_fn(entry) - cls.install_task.append((src, entry_dst_name)) - else: - cls.install_task.append((src, dst)) - - - @classmethod - def do_prepare_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, we copies all files listed in IMAGE_EXTRA_PARTITION_FILES variable. - """ - extradir = "%s/extra.%d" % (cr_workdir, part.lineno) - - if not kernel_dir: - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not kernel_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - for task in cls.install_task: - src_path, dst_path = task - logger.debug('Install %s as %s', src_path, dst_path) - install_cmd = "install -m 0644 -D %s %s" \ - % (os.path.join(kernel_dir, src_path), - os.path.join(extradir, dst_path)) - exec_cmd(install_cmd) - - logger.debug('Prepare extra partition using rootfs in %s', extradir) - part.prepare_rootfs(cr_workdir, oe_builddir, extradir, - native_sysroot, False) - diff --git a/scripts/lib/wic/plugins/source/isoimage_isohybrid.py b/scripts/lib/wic/plugins/source/isoimage_isohybrid.py deleted file mode 100644 index 9195ef5f3184..000000000000 --- a/scripts/lib/wic/plugins/source/isoimage_isohybrid.py +++ /dev/null @@ -1,464 +0,0 @@ -# -# Copyright OpenEmbedded Contributors -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This implements the 'isoimage_isohybrid' source plugin class for 'wic' -# -# AUTHORS -# Mihaly Varga - -import glob -import logging -import os -import re -import shutil - -from wic import WicError -from wic.engine import get_custom_config -from wic.pluginbase import SourcePlugin -from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var - -logger = logging.getLogger('wic') - -class IsoImagePlugin(SourcePlugin): - """ - Create a bootable ISO image - - This plugin creates a hybrid, legacy and EFI bootable ISO image. The - generated image can be used on optical media as well as USB media. - - Legacy boot uses syslinux and EFI boot uses grub or gummiboot (not - implemented yet) as bootloader. The plugin creates the directories required - by bootloaders and populates them by creating and configuring the - bootloader files. - - Example kickstart file: - part /boot --source isoimage_isohybrid --sourceparams="loader=grub-efi, \\ - image_name= IsoImage" --ondisk cd --label LIVECD - bootloader --timeout=10 --append=" " - - In --sourceparams "loader" specifies the bootloader used for booting in EFI - mode, while "image_name" specifies the name of the generated image. In the - example above, wic creates an ISO image named IsoImage-cd.direct (default - extension added by direct imeger plugin) and a file named IsoImage-cd.iso - """ - - name = 'isoimage_isohybrid' - - @classmethod - def do_configure_syslinux(cls, creator, cr_workdir): - """ - Create loader-specific (syslinux) config - """ - splash = os.path.join(cr_workdir, "ISO/boot/splash.jpg") - if os.path.exists(splash): - splashline = "menu background splash.jpg" - else: - splashline = "" - - bootloader = creator.ks.bootloader - - syslinux_conf = "" - syslinux_conf += "PROMPT 0\n" - syslinux_conf += "TIMEOUT %s \n" % (bootloader.timeout or 10) - syslinux_conf += "\n" - syslinux_conf += "ALLOWOPTIONS 1\n" - syslinux_conf += "SERIAL 0 115200\n" - syslinux_conf += "\n" - if splashline: - syslinux_conf += "%s\n" % splashline - syslinux_conf += "DEFAULT boot\n" - syslinux_conf += "LABEL boot\n" - - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - syslinux_conf += "KERNEL /" + kernel + "\n" - syslinux_conf += "APPEND initrd=/initrd LABEL=boot %s\n" \ - % bootloader.append - - logger.debug("Writing syslinux config %s/ISO/isolinux/isolinux.cfg", - cr_workdir) - - with open("%s/ISO/isolinux/isolinux.cfg" % cr_workdir, "w") as cfg: - cfg.write(syslinux_conf) - - @classmethod - def do_configure_grubefi(cls, part, creator, target_dir): - """ - Create loader-specific (grub-efi) config - """ - configfile = creator.ks.bootloader.configfile - if configfile: - grubefi_conf = get_custom_config(configfile) - if grubefi_conf: - logger.debug("Using custom configuration file %s for grub.cfg", - configfile) - else: - raise WicError("configfile is specified " - "but failed to get it from %s", configfile) - else: - splash = os.path.join(target_dir, "splash.jpg") - if os.path.exists(splash): - splashline = "menu background splash.jpg" - else: - splashline = "" - - bootloader = creator.ks.bootloader - - grubefi_conf = "" - grubefi_conf += "serial --unit=0 --speed=115200 --word=8 " - grubefi_conf += "--parity=no --stop=1\n" - grubefi_conf += "default=boot\n" - grubefi_conf += "timeout=%s\n" % (bootloader.timeout or 10) - grubefi_conf += "\n" - grubefi_conf += "search --set=root --label %s " % part.label - grubefi_conf += "\n" - grubefi_conf += "menuentry 'boot'{\n" - - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - grubefi_conf += "linux /%s rootwait %s\n" \ - % (kernel, bootloader.append) - grubefi_conf += "initrd /initrd \n" - grubefi_conf += "}\n" - - if splashline: - grubefi_conf += "%s\n" % splashline - - cfg_path = os.path.join(target_dir, "grub.cfg") - logger.debug("Writing grubefi config %s", cfg_path) - - with open(cfg_path, "w") as cfg: - cfg.write(grubefi_conf) - - @staticmethod - def _build_initramfs_path(rootfs_dir, cr_workdir): - """ - Create path for initramfs image - """ - - initrd = get_bitbake_var("INITRD_LIVE") or get_bitbake_var("INITRD") - if not initrd: - initrd_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not initrd_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting.") - - image_name = get_bitbake_var("IMAGE_BASENAME") - if not image_name: - raise WicError("Couldn't find IMAGE_BASENAME, exiting.") - - image_type = get_bitbake_var("INITRAMFS_FSTYPES") - if not image_type: - raise WicError("Couldn't find INITRAMFS_FSTYPES, exiting.") - - machine = os.path.basename(initrd_dir) - - pattern = '%s/%s*%s.%s' % (initrd_dir, image_name, machine, image_type) - files = glob.glob(pattern) - if files: - initrd = files[0] - - if not initrd or not os.path.exists(initrd): - # Create initrd from rootfs directory - initrd = "%s/initrd.cpio.gz" % cr_workdir - initrd_dir = "%s/INITRD" % cr_workdir - shutil.copytree("%s" % rootfs_dir, \ - "%s" % initrd_dir, symlinks=True) - - if os.path.isfile("%s/init" % rootfs_dir): - shutil.copy2("%s/init" % rootfs_dir, "%s/init" % initrd_dir) - elif os.path.lexists("%s/init" % rootfs_dir): - os.symlink(os.readlink("%s/init" % rootfs_dir), \ - "%s/init" % initrd_dir) - elif os.path.isfile("%s/sbin/init" % rootfs_dir): - shutil.copy2("%s/sbin/init" % rootfs_dir, \ - "%s" % initrd_dir) - elif os.path.lexists("%s/sbin/init" % rootfs_dir): - os.symlink(os.readlink("%s/sbin/init" % rootfs_dir), \ - "%s/init" % initrd_dir) - else: - raise WicError("Couldn't find or build initrd, exiting.") - - exec_cmd("cd %s && find . | cpio -o -H newc -R root:root >%s/initrd.cpio " \ - % (initrd_dir, cr_workdir), as_shell=True) - exec_cmd("gzip -f -9 %s/initrd.cpio" % cr_workdir, as_shell=True) - shutil.rmtree(initrd_dir) - - return initrd - - @classmethod - def do_configure_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - native_sysroot): - """ - Called before do_prepare_partition(), creates loader-specific config - """ - isodir = "%s/ISO/" % cr_workdir - - if os.path.exists(isodir): - shutil.rmtree(isodir) - - install_cmd = "install -d %s " % isodir - exec_cmd(install_cmd) - - # Overwrite the name of the created image - logger.debug(source_params) - if 'image_name' in source_params and \ - source_params['image_name'].strip(): - creator.name = source_params['image_name'].strip() - logger.debug("The name of the image is: %s", creator.name) - - @staticmethod - def _install_payload(source_params, iso_dir): - """ - Copies contents of payload directory (as specified in 'payload_dir' param) into iso_dir - """ - - if source_params.get('payload_dir'): - payload_dir = source_params['payload_dir'] - - logger.debug("Payload directory: %s", payload_dir) - shutil.copytree(payload_dir, iso_dir, symlinks=True, dirs_exist_ok=True) - - @classmethod - def do_prepare_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, prepare content for a bootable ISO image. - """ - - isodir = "%s/ISO" % cr_workdir - - cls._install_payload(source_params, isodir) - - if part.rootfs_dir is None: - if not 'ROOTFS_DIR' in rootfs_dir: - raise WicError("Couldn't find --rootfs-dir, exiting.") - rootfs_dir = rootfs_dir['ROOTFS_DIR'] - else: - if part.rootfs_dir in rootfs_dir: - rootfs_dir = rootfs_dir[part.rootfs_dir] - elif part.rootfs_dir: - rootfs_dir = part.rootfs_dir - else: - raise WicError("Couldn't find --rootfs-dir=%s connection " - "or it is not a valid path, exiting." % - part.rootfs_dir) - - if not os.path.isdir(rootfs_dir): - rootfs_dir = get_bitbake_var("IMAGE_ROOTFS") - if not os.path.isdir(rootfs_dir): - raise WicError("Couldn't find IMAGE_ROOTFS, exiting.") - - part.rootfs_dir = rootfs_dir - deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - img_iso_dir = get_bitbake_var("ISODIR") - - # Remove the temporary file created by part.prepare_rootfs() - if os.path.isfile(part.source_file): - os.remove(part.source_file) - - # Support using a different initrd other than default - if source_params.get('initrd'): - initrd = source_params['initrd'] - if not deploy_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - cp_cmd = "cp %s/%s %s" % (deploy_dir, initrd, cr_workdir) - exec_cmd(cp_cmd) - else: - # Prepare initial ramdisk - initrd = "%s/initrd" % deploy_dir - if not os.path.isfile(initrd): - initrd = "%s/initrd" % img_iso_dir - if not os.path.isfile(initrd): - initrd = cls._build_initramfs_path(rootfs_dir, cr_workdir) - - install_cmd = "install -m 0644 %s %s/initrd" % (initrd, isodir) - exec_cmd(install_cmd) - - # Remove the temporary file created by _build_initramfs_path function - if os.path.isfile("%s/initrd.cpio.gz" % cr_workdir): - os.remove("%s/initrd.cpio.gz" % cr_workdir) - - kernel = get_bitbake_var("KERNEL_IMAGETYPE") - if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": - if get_bitbake_var("INITRAMFS_IMAGE"): - kernel = "%s-%s.bin" % \ - (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - - install_cmd = "install -m 0644 %s/%s %s/%s" % \ - (kernel_dir, kernel, isodir, kernel) - exec_cmd(install_cmd) - - #Create bootloader for efi boot - try: - target_dir = "%s/EFI/BOOT" % isodir - if os.path.exists(target_dir): - shutil.rmtree(target_dir) - - os.makedirs(target_dir) - - if source_params['loader'] == 'grub-efi': - # Builds bootx64.efi/bootia32.efi if ISODIR didn't exist or - # didn't contains it - target_arch = get_bitbake_var("TARGET_SYS") - if not target_arch: - raise WicError("Coludn't find target architecture") - - if re.match("x86_64", target_arch): - grub_src_image = "grub-efi-bootx64.efi" - grub_dest_image = "bootx64.efi" - elif re.match('i.86', target_arch): - grub_src_image = "grub-efi-bootia32.efi" - grub_dest_image = "bootia32.efi" - else: - raise WicError("grub-efi is incompatible with target %s" % - target_arch) - - grub_target = os.path.join(target_dir, grub_dest_image) - if not os.path.isfile(grub_target): - grub_src = os.path.join(deploy_dir, grub_src_image) - if not os.path.exists(grub_src): - raise WicError("Grub loader %s is not found in %s. " - "Please build grub-efi first" % (grub_src_image, deploy_dir)) - shutil.copy(grub_src, grub_target) - - if not os.path.isfile(os.path.join(target_dir, "boot.cfg")): - cls.do_configure_grubefi(part, creator, target_dir) - - else: - raise WicError("unrecognized bootimg_efi loader: %s" % - source_params['loader']) - except KeyError: - raise WicError("bootimg_efi requires a loader, none specified") - - # Create efi.img that contains bootloader files for EFI booting - # if ISODIR didn't exist or didn't contains it - if os.path.isfile("%s/efi.img" % img_iso_dir): - install_cmd = "install -m 0644 %s/efi.img %s/efi.img" % \ - (img_iso_dir, isodir) - exec_cmd(install_cmd) - else: - # Default to 100 blocks of extra space for file system overhead - esp_extra_blocks = int(source_params.get('esp_extra_blocks', '100')) - - du_cmd = "du --apparent-size -ks %s/EFI" % isodir - out = exec_cmd(du_cmd) - blocks = int(out.split()[0]) - blocks += esp_extra_blocks - logger.debug("Added 100 extra blocks to %s to get to %d " - "total blocks", part.mountpoint, blocks) - - # dosfs image for EFI boot - bootimg = "%s/efi.img" % isodir - - esp_label = source_params.get('esp_label', 'EFIimg') - - sector_size = getattr(creator, 'sector_size', 512) or 512 - dosfs_cmd = "mkfs.vfat -n '%s' -S %d -C %s %d" % \ - (esp_label, sector_size, bootimg, blocks) - exec_native_cmd(dosfs_cmd, native_sysroot) - - mmd_cmd = "mmd -i %s ::/EFI" % bootimg - exec_native_cmd(mmd_cmd, native_sysroot) - - mcopy_cmd = "mcopy -i %s -s %s/EFI/* ::/EFI/" \ - % (bootimg, isodir) - exec_native_cmd(mcopy_cmd, native_sysroot) - - chmod_cmd = "chmod 644 %s" % bootimg - exec_cmd(chmod_cmd) - - # Prepare files for legacy boot - syslinux_dir = get_bitbake_var("STAGING_DATADIR") - if not syslinux_dir: - raise WicError("Couldn't find STAGING_DATADIR, exiting.") - - if os.path.exists("%s/isolinux" % isodir): - shutil.rmtree("%s/isolinux" % isodir) - - install_cmd = "install -d %s/isolinux" % isodir - exec_cmd(install_cmd) - - cls.do_configure_syslinux(creator, cr_workdir) - - install_cmd = "install -m 444 %s/syslinux/ldlinux.sys " % syslinux_dir - install_cmd += "%s/isolinux/ldlinux.sys" % isodir - exec_cmd(install_cmd) - - install_cmd = "install -m 444 %s/syslinux/isohdpfx.bin " % syslinux_dir - install_cmd += "%s/isolinux/isohdpfx.bin" % isodir - exec_cmd(install_cmd) - - install_cmd = "install -m 644 %s/syslinux/isolinux.bin " % syslinux_dir - install_cmd += "%s/isolinux/isolinux.bin" % isodir - exec_cmd(install_cmd) - - install_cmd = "install -m 644 %s/syslinux/ldlinux.c32 " % syslinux_dir - install_cmd += "%s/isolinux/ldlinux.c32" % isodir - exec_cmd(install_cmd) - - #create ISO image - iso_img = "%s/tempiso_img.iso" % cr_workdir - iso_bootimg = "isolinux/isolinux.bin" - iso_bootcat = "isolinux/boot.cat" - efi_img = "efi.img" - - mkisofs_cmd = "mkisofs -V %s " % part.label - mkisofs_cmd += "-o %s -U " % iso_img - mkisofs_cmd += "-J -joliet-long -r -iso-level 2 -b %s " % iso_bootimg - mkisofs_cmd += "-c %s -no-emul-boot -boot-load-size 4 " % iso_bootcat - mkisofs_cmd += "-boot-info-table -eltorito-alt-boot " - mkisofs_cmd += "-eltorito-platform 0xEF -eltorito-boot %s " % efi_img - mkisofs_cmd += "-no-emul-boot %s " % isodir - - logger.debug("running command: %s", mkisofs_cmd) - exec_native_cmd(mkisofs_cmd, native_sysroot) - - shutil.rmtree(isodir) - - du_cmd = "du --apparent-size -Lks %s" % iso_img - out = exec_cmd(du_cmd) - isoimg_size = int(out.split()[0]) - - part.size = isoimg_size - part.source_file = iso_img - - @classmethod - def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir, - bootimg_dir, kernel_dir, native_sysroot): - """ - Called after all partitions have been prepared and assembled into a - disk image. In this case, we insert/modify the MBR using isohybrid - utility for booting via BIOS from disk storage devices. - """ - - iso_img = "%s.p1" % disk.path - full_path = creator._full_path(workdir, disk_name, "direct") - full_path_iso = creator._full_path(workdir, disk_name, "iso") - - isohybrid_cmd = "isohybrid -u %s" % iso_img - logger.debug("running command: %s", isohybrid_cmd) - exec_native_cmd(isohybrid_cmd, native_sysroot) - - # Replace the image created by direct plugin with the one created by - # mkisofs command. This is necessary because the iso image created by - # mkisofs has a very specific MBR is system area of the ISO image, and - # direct plugin adds and configures an another MBR. - logger.debug("Replaceing the image created by direct plugin\n") - os.remove(disk.path) - shutil.copy2(iso_img, full_path_iso) - shutil.copy2(full_path_iso, full_path) diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py deleted file mode 100644 index 698413b2215a..000000000000 --- a/scripts/lib/wic/plugins/source/rawcopy.py +++ /dev/null @@ -1,115 +0,0 @@ -# -# Copyright OpenEmbedded Contributors -# -# SPDX-License-Identifier: GPL-2.0-only -# - -import logging -import os -import signal -import subprocess - -from wic import WicError -from wic.pluginbase import SourcePlugin -from wic.misc import exec_cmd, get_bitbake_var -from wic.filemap import sparse_copy - -logger = logging.getLogger('wic') - -class RawCopyPlugin(SourcePlugin): - """ - Populate partition content from raw image file. - """ - - name = 'rawcopy' - - @staticmethod - def do_image_label(fstype, dst, label): - # don't create label when fstype is none - if fstype == 'none': - return - - if fstype.startswith('ext'): - cmd = 'tune2fs -L %s %s' % (label, dst) - elif fstype in ('msdos', 'vfat'): - cmd = 'dosfslabel %s %s' % (dst, label) - elif fstype == 'btrfs': - cmd = 'btrfs filesystem label %s %s' % (dst, label) - elif fstype == 'swap': - cmd = 'mkswap -L %s %s' % (label, dst) - elif fstype in ('squashfs', 'erofs'): - raise WicError("It's not possible to update a %s " - "filesystem label '%s'" % (fstype, label)) - else: - raise WicError("Cannot update filesystem label: " - "Unknown fstype: '%s'" % (fstype)) - - exec_cmd(cmd) - - @staticmethod - def do_image_uncompression(src, dst, workdir): - def subprocess_setup(): - # Python installs a SIGPIPE handler by default. This is usually not what - # non-Python subprocesses expect. - # SIGPIPE errors are known issues with gzip/bash - signal.signal(signal.SIGPIPE, signal.SIG_DFL) - - extension = os.path.splitext(src)[1] - decompressor = { - ".bz2": "bzip2", - ".gz": "gzip", - ".xz": "xz", - ".zst": "zstd -f", - }.get(extension) - if not decompressor: - raise WicError("Not supported compressor filename extension: %s" % extension) - cmd = "%s -dc %s > %s" % (decompressor, src, dst) - subprocess.call(cmd, preexec_fn=subprocess_setup, shell=True, cwd=workdir) - - @classmethod - def do_prepare_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - """ - if not kernel_dir: - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") - if not kernel_dir: - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") - - logger.debug('Kernel dir: %s', kernel_dir) - - if 'file' not in source_params: - raise WicError("No file specified") - - if 'unpack' in source_params: - img = os.path.join(kernel_dir, source_params['file']) - src = os.path.join(cr_workdir, os.path.splitext(source_params['file'])[0]) - RawCopyPlugin.do_image_uncompression(img, src, cr_workdir) - else: - src = os.path.join(kernel_dir, source_params['file']) - - dst = os.path.join(cr_workdir, "%s.%s" % (os.path.basename(source_params['file']), part.lineno)) - - if not os.path.exists(os.path.dirname(dst)): - os.makedirs(os.path.dirname(dst)) - - if 'skip' in source_params: - sparse_copy(src, dst, skip=int(source_params['skip'])) - else: - sparse_copy(src, dst) - - # get the size in the right units for kickstart (kB) - du_cmd = "du --apparent-size -Lks %s" % dst - out = exec_cmd(du_cmd) - filesize = int(out.split()[0]) - - if filesize > part.size: - part.size = filesize - - if part.label: - RawCopyPlugin.do_image_label(part.fstype, dst, part.label) - - part.source_file = dst diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py deleted file mode 100644 index 06fce06bb156..000000000000 --- a/scripts/lib/wic/plugins/source/rootfs.py +++ /dev/null @@ -1,236 +0,0 @@ -# -# Copyright (c) 2014, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION -# This implements the 'rootfs' source plugin class for 'wic' -# -# AUTHORS -# Tom Zanussi -# Joao Henrique Ferreira de Freitas -# - -import logging -import os -import shutil -import sys - -from oe.path import copyhardlinktree -from pathlib import Path - -from wic import WicError -from wic.pluginbase import SourcePlugin -from wic.misc import get_bitbake_var, exec_native_cmd - -logger = logging.getLogger('wic') - -class RootfsPlugin(SourcePlugin): - """ - Populate partition content from a rootfs directory. - """ - - name = 'rootfs' - - @staticmethod - def __validate_path(cmd, rootfs_dir, path): - if os.path.isabs(path): - logger.error("%s: Must be relative: %s" % (cmd, path)) - sys.exit(1) - - # Disallow climbing outside of parent directory using '..', - # because doing so could be quite disastrous (we will delete the - # directory, or modify a directory outside OpenEmbedded). - full_path = os.path.abspath(os.path.join(rootfs_dir, path)) - if not full_path.startswith(os.path.realpath(rootfs_dir)): - logger.error("%s: Must point inside the rootfs: %s" % (cmd, path)) - sys.exit(1) - - return full_path - - @staticmethod - def __get_rootfs_dir(rootfs_dir): - if rootfs_dir and os.path.isdir(rootfs_dir): - return os.path.realpath(rootfs_dir) - - image_rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", rootfs_dir) - if not os.path.isdir(image_rootfs_dir): - raise WicError("No valid artifact IMAGE_ROOTFS from image " - "named %s has been found at %s, exiting." % - (rootfs_dir, image_rootfs_dir)) - - return os.path.realpath(image_rootfs_dir) - - @staticmethod - def __get_pseudo(native_sysroot, rootfs, pseudo_dir): - pseudo = "export PSEUDO_PREFIX=%s/usr;" % native_sysroot - pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir - pseudo += "export PSEUDO_PASSWD=%s;" % rootfs - pseudo += "export PSEUDO_NOSYMLINKEXP=1;" - pseudo += "%s " % get_bitbake_var("FAKEROOTCMD") - return pseudo - - @classmethod - def do_prepare_partition(cls, part, source_params, cr, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - krootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, prepare content for legacy bios boot partition. - """ - if part.rootfs_dir is None: - if not 'ROOTFS_DIR' in krootfs_dir: - raise WicError("Couldn't find --rootfs-dir, exiting") - - rootfs_dir = krootfs_dir['ROOTFS_DIR'] - else: - if part.rootfs_dir in krootfs_dir: - rootfs_dir = krootfs_dir[part.rootfs_dir] - elif part.rootfs_dir: - rootfs_dir = part.rootfs_dir - else: - raise WicError("Couldn't find --rootfs-dir=%s connection or " - "it is not a valid path, exiting" % part.rootfs_dir) - - part.rootfs_dir = cls.__get_rootfs_dir(rootfs_dir) - part.has_fstab = os.path.exists(os.path.join(part.rootfs_dir, "etc/fstab")) - pseudo_dir = os.path.join(part.rootfs_dir, "../pseudo") - if not os.path.lexists(pseudo_dir): - pseudo_dir = os.path.join(cls.__get_rootfs_dir(None), '../pseudo') - - if not os.path.lexists(pseudo_dir): - logger.warn("%s folder does not exist. " - "Usernames and permissions will be invalid " % pseudo_dir) - pseudo_dir = None - - new_rootfs = None - new_pseudo = None - # Handle excluded paths. - if part.exclude_path or part.include_path or part.change_directory or part.update_fstab_in_rootfs: - # We need a new rootfs directory we can safely modify without - # interfering with other tasks. Copy to workdir. - new_rootfs = os.path.realpath(os.path.join(cr_workdir, "rootfs%d" % part.lineno)) - - if os.path.lexists(new_rootfs): - shutil.rmtree(os.path.join(new_rootfs)) - - if part.change_directory: - cd = part.change_directory - if cd[-1] == '/': - cd = cd[:-1] - orig_dir = cls.__validate_path("--change-directory", part.rootfs_dir, cd) - else: - orig_dir = part.rootfs_dir - copyhardlinktree(orig_dir, new_rootfs) - - # Convert the pseudo directory to its new location - if (pseudo_dir): - new_pseudo = os.path.realpath( - os.path.join(cr_workdir, "pseudo%d" % part.lineno)) - if os.path.lexists(new_pseudo): - shutil.rmtree(new_pseudo) - os.mkdir(new_pseudo) - shutil.copy(os.path.join(pseudo_dir, "files.db"), - os.path.join(new_pseudo, "files.db")) - - pseudo_cmd = "%s -B -m %s -M %s" % (cls.__get_pseudo(native_sysroot, - new_rootfs, - new_pseudo), - orig_dir, new_rootfs) - exec_native_cmd(pseudo_cmd, native_sysroot) - - for in_path in part.include_path or []: - #parse arguments - include_path = in_path[0] - if len(in_path) > 2: - logger.error("'Invalid number of arguments for include-path") - sys.exit(1) - if len(in_path) == 2: - path = in_path[1] - else: - path = None - - # Pack files to be included into a tar file. - # We need to create a tar file, because that way we can keep the - # permissions from the files even when they belong to different - # pseudo enviroments. - # If we simply copy files using copyhardlinktree/copytree... the - # copied files will belong to the user running wic. - tar_file = os.path.realpath( - os.path.join(cr_workdir, "include-path%d.tar" % part.lineno)) - if os.path.isfile(include_path): - parent = os.path.dirname(os.path.realpath(include_path)) - tar_cmd = "tar c --owner=root --group=root -f %s -C %s %s" % ( - tar_file, parent, os.path.relpath(include_path, parent)) - exec_native_cmd(tar_cmd, native_sysroot) - else: - if include_path in krootfs_dir: - include_path = krootfs_dir[include_path] - include_path = cls.__get_rootfs_dir(include_path) - include_pseudo = os.path.join(include_path, "../pseudo") - if os.path.lexists(include_pseudo): - pseudo = cls.__get_pseudo(native_sysroot, include_path, - include_pseudo) - tar_cmd = "tar cf %s -C %s ." % (tar_file, include_path) - else: - pseudo = None - tar_cmd = "tar c --owner=root --group=root -f %s -C %s ." % ( - tar_file, include_path) - exec_native_cmd(tar_cmd, native_sysroot, pseudo) - - #create destination - if path: - destination = cls.__validate_path("--include-path", new_rootfs, path) - Path(destination).mkdir(parents=True, exist_ok=True) - else: - destination = new_rootfs - - #extract destination - untar_cmd = "tar xf %s -C %s" % (tar_file, destination) - if new_pseudo: - pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo) - else: - pseudo = None - exec_native_cmd(untar_cmd, native_sysroot, pseudo) - os.remove(tar_file) - - for orig_path in part.exclude_path or []: - path = orig_path - - full_path = cls.__validate_path("--exclude-path", new_rootfs, path) - - if not os.path.lexists(full_path): - continue - - if new_pseudo: - pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo) - else: - pseudo = None - if path.endswith(os.sep): - # Delete content only. - for entry in os.listdir(full_path): - full_entry = os.path.join(full_path, entry) - rm_cmd = "rm -rf %s" % (full_entry) - exec_native_cmd(rm_cmd, native_sysroot, pseudo) - else: - # Delete whole directory. - rm_cmd = "rm -rf %s" % (full_path) - exec_native_cmd(rm_cmd, native_sysroot, pseudo) - - # Update part.has_fstab here as fstab may have been added or - # removed by the above modifications. - part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab")) - if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update: - fstab_path = os.path.join(new_rootfs, "etc/fstab") - # Assume that fstab should always be owned by root with fixed permissions - install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path) - if new_pseudo: - pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo) - else: - pseudo = None - exec_native_cmd(install_cmd, native_sysroot, pseudo) - - part.prepare_rootfs(cr_workdir, oe_builddir, - new_rootfs or part.rootfs_dir, native_sysroot, - pseudo_dir = new_pseudo or pseudo_dir) diff --git a/scripts/wic b/scripts/wic deleted file mode 100755 index 7006efe5e76c..000000000000 --- a/scripts/wic +++ /dev/null @@ -1,644 +0,0 @@ -#!/usr/bin/env python3 -# -# Copyright (c) 2013, Intel Corporation. -# -# SPDX-License-Identifier: GPL-2.0-only -# -# DESCRIPTION 'wic' is the OpenEmbedded Image Creator that users can -# use to generate bootable images. Invoking it without any arguments -# will display help screens for the 'wic' command and list the -# available 'wic' subcommands. Invoking a subcommand without any -# arguments will likewise display help screens for the specified -# subcommand. Please use that interface for detailed help. -# -# AUTHORS -# Tom Zanussi -# -__version__ = "0.2.0" - -# Python Standard Library modules -import os -import sys -import argparse -import logging -import subprocess -import shutil - -from collections import namedtuple - -# External modules -scripts_path = os.path.dirname(os.path.realpath(__file__)) -lib_path = scripts_path + '/lib' -sys.path.insert(0, lib_path) -import scriptpath -scriptpath.add_oe_lib_path() - -# Check whether wic is running within eSDK environment -sdkroot = scripts_path -if os.environ.get('SDKTARGETSYSROOT'): - while sdkroot != '' and sdkroot != os.sep: - if os.path.exists(os.path.join(sdkroot, '.devtoolbase')): - # Set BUILDDIR for wic to work within eSDK - os.environ['BUILDDIR'] = sdkroot - # .devtoolbase only exists within eSDK - # If found, initialize bitbake path for eSDK environment and append to PATH - sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin') - os.environ['PATH'] += ":" + sdkroot - break - sdkroot = os.path.dirname(sdkroot) - -bitbake_exe = shutil.which('bitbake') -if bitbake_exe: - bitbake_path = scriptpath.add_bitbake_lib_path() - import bb - -from wic import WicError -from wic.misc import get_bitbake_var, BB_VARS -from wic import engine -from wic import help as hlp - - -def wic_logger(): - """Create and convfigure wic logger.""" - logger = logging.getLogger('wic') - logger.setLevel(logging.INFO) - - handler = logging.StreamHandler() - - formatter = logging.Formatter('%(levelname)s: %(message)s') - handler.setFormatter(formatter) - - logger.addHandler(handler) - - return logger - -logger = wic_logger() - -def rootfs_dir_to_args(krootfs_dir): - """ - Get a rootfs_dir dict and serialize to string - """ - rootfs_dir = '' - for key, val in krootfs_dir.items(): - rootfs_dir += ' ' - rootfs_dir += '='.join([key, val]) - return rootfs_dir.strip() - - -class RootfsArgAction(argparse.Action): - def __init__(self, **kwargs): - super().__init__(**kwargs) - - def __call__(self, parser, namespace, value, option_string=None): - if not "rootfs_dir" in vars(namespace) or \ - not type(namespace.__dict__['rootfs_dir']) is dict: - namespace.__dict__['rootfs_dir'] = {} - - if '=' in value: - (key, rootfs_dir) = value.split('=') - else: - key = 'ROOTFS_DIR' - rootfs_dir = value - - namespace.__dict__['rootfs_dir'][key] = rootfs_dir - - -def _apply_sector_size_default(args): - """ - Populate args.sector_size. - Prefer --sector-size if given, WIC_SECTOR_SIZE if not, otherwise fall back to 512. - """ - if not hasattr(args, "sector_size"): - return - - BB_VARS.vars_dir = args.vars_dir - try: - tmp_val = get_bitbake_var("WIC_SECTOR_SIZE", args.image_name) - if tmp_val: - try: - env_val = int(tmp_val) - except ValueError: - raise WicError("Invalid WIC_SECTOR_SIZE value '%s'; please provide an integer or use --sector-size." % tmp_val) - - if args.sector_size is not None: - logger.warning("WIC_SECTOR_SIZE (%d) and --sector-size (%d) were both provided; --sector-size used.", env_val, args.sector_size) - logger.warning("DEPRECATED: WIC_SECTOR_SIZE is deprecated and will be removed, use the --sector-size command-line argument instead.") - return - else: - logger.warning("WIC_SECTOR_SIZE (%s) provided but --sector-size not provided; use --sector-size to avoid confusion.", env_val) - logger.warning("DEPRECATED: WIC_SECTOR_SIZE is deprecated and will be removed, use the --sector-size command-line argument instead.") - args.sector_size = env_val - return - except: - pass - - if args.sector_size is not None: - return - args.sector_size = 512 - - -def wic_create_subcommand(options, usage_str): - """ - Command-line handling for image creation. The real work is done - by image.engine.wic_create() - """ - if options.build_rootfs and not bitbake_exe: - raise WicError("Can't build rootfs as bitbake is not in the $PATH") - - if not options.image_name: - missed = [] - for val, opt in [(options.rootfs_dir, 'rootfs-dir'), - (options.bootimg_dir, 'bootimg-dir'), - (options.kernel_dir, 'kernel-dir'), - (options.native_sysroot, 'native-sysroot')]: - if not val: - missed.append(opt) - if missed: - raise WicError("The following build artifacts are not specified: %s" % - ", ".join(missed)) - - if options.image_name: - BB_VARS.default_image = options.image_name - else: - options.build_check = False - - if options.vars_dir: - BB_VARS.vars_dir = options.vars_dir - - if options.build_check and not engine.verify_build_env(): - raise WicError("Couldn't verify build environment, exiting") - - if options.debug: - logger.setLevel(logging.DEBUG) - - if options.image_name: - if options.build_rootfs: - argv = ["bitbake", options.image_name] - if options.debug: - argv.append("--debug") - - logger.info("Building rootfs...\n") - subprocess.check_call(argv) - - rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", options.image_name) - kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE", options.image_name) - bootimg_dir = get_bitbake_var("STAGING_DATADIR", options.image_name) - - native_sysroot = options.native_sysroot - if options.vars_dir and not native_sysroot: - native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", options.image_name) - else: - if options.build_rootfs: - raise WicError("Image name is not specified, exiting. " - "(Use -e/--image-name to specify it)") - native_sysroot = options.native_sysroot - - if options.kernel_dir: - kernel_dir = options.kernel_dir - - if not options.vars_dir and (not native_sysroot or not os.path.isdir(native_sysroot)): - logger.info("Building wic-tools...\n") - subprocess.check_call(["bitbake", "wic-tools"]) - native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", "wic-tools") - - if not native_sysroot: - raise WicError("Unable to find the location of the native tools sysroot") - - wks_file = options.wks_file - - if not wks_file.endswith(".wks"): - wks_file = engine.find_canned_image(scripts_path, wks_file) - if not wks_file: - raise WicError("No image named %s found, exiting. (Use 'wic list images' " - "to list available images, or specify a fully-qualified OE " - "kickstart (.wks) filename)" % options.wks_file) - - if not options.image_name: - rootfs_dir = '' - if 'ROOTFS_DIR' in options.rootfs_dir: - rootfs_dir = options.rootfs_dir['ROOTFS_DIR'] - bootimg_dir = options.bootimg_dir - kernel_dir = options.kernel_dir - native_sysroot = options.native_sysroot - if rootfs_dir and not os.path.isdir(rootfs_dir): - raise WicError("--rootfs-dir (-r) not found, exiting") - if not os.path.isdir(bootimg_dir): - raise WicError("--bootimg-dir (-b) not found, exiting") - if not os.path.isdir(kernel_dir): - raise WicError("--kernel-dir (-k) not found, exiting") - if not os.path.isdir(native_sysroot): - raise WicError("--native-sysroot (-n) not found, exiting") - else: - not_found = not_found_dir = "" - if not os.path.isdir(rootfs_dir): - (not_found, not_found_dir) = ("rootfs-dir", rootfs_dir) - elif not os.path.isdir(kernel_dir): - (not_found, not_found_dir) = ("kernel-dir", kernel_dir) - elif not os.path.isdir(native_sysroot): - (not_found, not_found_dir) = ("native-sysroot", native_sysroot) - if not_found: - if not not_found_dir: - not_found_dir = "Completely missing artifact - wrong image (.wks) used?" - logger.info("Build artifacts not found, exiting.") - logger.info(" (Please check that the build artifacts for the machine") - logger.info(" selected in local.conf actually exist and that they") - logger.info(" are the correct artifacts for the image (.wks file)).\n") - raise WicError("The artifact that couldn't be found was %s:\n %s" % (not_found, not_found_dir)) - - krootfs_dir = options.rootfs_dir - if krootfs_dir is None: - krootfs_dir = {} - krootfs_dir['ROOTFS_DIR'] = rootfs_dir - - rootfs_dir = rootfs_dir_to_args(krootfs_dir) - - logger.info("Creating image(s)...\n") - engine.wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, - native_sysroot, options) - - -def wic_list_subcommand(args, usage_str): - """ - Command-line handling for listing available images. - The real work is done by image.engine.wic_list() - """ - if not engine.wic_list(args, scripts_path): - raise WicError("Bad list arguments, exiting") - - -def wic_ls_subcommand(args, usage_str): - """ - Command-line handling for list content of images. - The real work is done by engine.wic_ls() - """ - - if args.image_name: - BB_VARS.default_image = args.image_name - - if args.vars_dir: - BB_VARS.vars_dir = args.vars_dir - - engine.wic_ls(args, args.native_sysroot) - -def wic_cp_subcommand(args, usage_str): - """ - Command-line handling for copying files/dirs to images. - The real work is done by engine.wic_cp() - """ - if args.image_name: - BB_VARS.default_image = args.image_name - - if args.vars_dir: - BB_VARS.vars_dir = args.vars_dir - - engine.wic_cp(args, args.native_sysroot) - -def wic_rm_subcommand(args, usage_str): - """ - Command-line handling for removing files/dirs from images. - The real work is done by engine.wic_rm() - """ - if args.image_name: - BB_VARS.default_image = args.image_name - - if args.vars_dir: - BB_VARS.vars_dir = args.vars_dir - - engine.wic_rm(args, args.native_sysroot) - -def wic_write_subcommand(args, usage_str): - """ - Command-line handling for writing images. - The real work is done by engine.wic_write() - """ - if args.image_name: - BB_VARS.default_image = args.image_name - - if args.vars_dir: - BB_VARS.vars_dir = args.vars_dir - - engine.wic_write(args, args.native_sysroot) - -def wic_help_subcommand(args, usage_str): - """ - Command-line handling for help subcommand to keep the current - structure of the function definitions. - """ - pass - - -def wic_help_topic_subcommand(usage_str, help_str): - """ - Display function for help 'sub-subcommands'. - """ - print(help_str) - return - - -wic_help_topic_usage = """ -""" - -helptopics = { - "plugins": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_plugins_help], - "overview": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_overview_help], - "kickstart": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_kickstart_help], - "create": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_create_help], - "ls": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_ls_help], - "cp": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_cp_help], - "rm": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_rm_help], - "write": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_write_help], - "list": [wic_help_topic_subcommand, - wic_help_topic_usage, - hlp.wic_list_help] -} - - -def wic_init_parser_create(subparser): - subparser.add_argument("wks_file") - - subparser.add_argument("-o", "--outdir", dest="outdir", default='.', - help="name of directory to create image in") - subparser.add_argument("-w", "--workdir", - help="temporary workdir to use for intermediate files") - subparser.add_argument("-e", "--image-name", dest="image_name", - help="name of the image to use the artifacts from " - "e.g. core-image-sato") - subparser.add_argument("-r", "--rootfs-dir", action=RootfsArgAction, - help="path to the /rootfs dir to use as the " - ".wks rootfs source") - subparser.add_argument("-b", "--bootimg-dir", dest="bootimg_dir", - help="path to the dir containing the boot artifacts " - "(e.g. /EFI or /syslinux dirs) to use as the " - ".wks bootimg source") - subparser.add_argument("-k", "--kernel-dir", dest="kernel_dir", - help="path to the dir containing the kernel to use " - "in the .wks bootimg") - subparser.add_argument("-n", "--native-sysroot", dest="native_sysroot", - help="path to the native sysroot containing the tools " - "to use to build the image") - subparser.add_argument("-s", "--skip-build-check", dest="build_check", - action="store_false", default=True, help="skip the build check") - subparser.add_argument("-f", "--build-rootfs", action="store_true", help="build rootfs") - subparser.add_argument("-c", "--compress-with", choices=("gzip", "bzip2", "xz"), - dest='compressor', - help="compress image with specified compressor") - subparser.add_argument("-m", "--bmap", action="store_true", help="generate .bmap") - subparser.add_argument("--no-fstab-update" ,action="store_true", - help="Do not change fstab file.") - subparser.add_argument("-v", "--vars", dest='vars_dir', - help="directory with .env files that store " - "bitbake variables") - subparser.add_argument("-D", "--debug", dest="debug", action="store_true", - default=False, help="output debug information") - subparser.add_argument("-i", "--imager", dest="imager", - default="direct", help="the wic imager plugin") - subparser.add_argument("--extra-space", type=int, dest="extra_space", - default=0, help="additional free disk space to add to the image") - subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, - help="sector size in bytes (default: 512)") - return - - -def wic_init_parser_list(subparser): - subparser.add_argument("list_type", - help="can be 'images' or 'source-plugins' " - "to obtain a list. " - "If value is a valid .wks image file") - subparser.add_argument("help_for", default=[], nargs='*', - help="If 'list_type' is a valid .wks image file " - "this value can be 'help' to show the help information " - "defined inside the .wks file") - return - -def imgtype(arg): - """ - Custom type for ArgumentParser - Converts path spec to named tuple: (image, partition, path) - """ - image = arg - part = path = None - if ':' in image: - image, part = image.split(':') - if '/' in part: - part, path = part.split('/', 1) - if not path: - path = '/' - - if not os.path.isfile(image): - err = "%s is not a regular file or symlink" % image - raise argparse.ArgumentTypeError(err) - - return namedtuple('ImgType', 'image part path')(image, part, path) - -def wic_init_parser_ls(subparser): - subparser.add_argument("path", type=imgtype, - help="image spec: [:[]]") - subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, - help="sector size in bytes (default: 512)") - subparser.add_argument("-n", "--native-sysroot", - help="path to the native sysroot containing the tools") - subparser.add_argument("-e", "--image-name", dest="image_name", - help="name of the image to use the artifacts from " - "e.g. core-image-sato") - subparser.add_argument("-v", "--vars", dest='vars_dir', - help="directory with .env files that store " - "bitbake variables") - -def imgpathtype(arg): - img = imgtype(arg) - if img.part is None: - raise argparse.ArgumentTypeError("partition number is not specified") - return img - -def wic_init_parser_cp(subparser): - subparser.add_argument("src", - help="image spec: :[] or ") - subparser.add_argument("dest", - help="image spec: :[] or ") - subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, - help="sector size in bytes (default: 512)") - subparser.add_argument("-n", "--native-sysroot", - help="path to the native sysroot containing the tools") - subparser.add_argument("-e", "--image-name", dest="image_name", - help="name of the image to use the artifacts from " - "e.g. core-image-sato") - subparser.add_argument("-v", "--vars", dest='vars_dir', - help="directory with .env files that store " - "bitbake variables") - -def wic_init_parser_rm(subparser): - subparser.add_argument("path", type=imgpathtype, - help="path: :") - subparser.add_argument("--sector-size", dest="sector_size", type=int, default=None, - help="sector size in bytes (default: 512)") - subparser.add_argument("-n", "--native-sysroot", - help="path to the native sysroot containing the tools") - subparser.add_argument("-r", dest="recursive_delete", action="store_true", default=False, - help="remove directories and their contents recursively, " - " this only applies to ext* partition") - subparser.add_argument("-e", "--image-name", dest="image_name", - help="name of the image to use the artifacts from " - "e.g. core-image-sato") - subparser.add_argument("-v", "--vars", dest='vars_dir', - help="directory with .env files that store " - "bitbake variables") - -def expandtype(rules): - """ - Custom type for ArgumentParser - Converts expand rules to the dictionary {: size} - """ - if rules == 'auto': - return {} - result = {} - for rule in rules.split(','): - try: - part, size = rule.split(':') - except ValueError: - raise argparse.ArgumentTypeError("Incorrect rule format: %s" % rule) - - if not part.isdigit(): - raise argparse.ArgumentTypeError("Rule '%s': partition number must be integer" % rule) - - # validate size - multiplier = 1 - for suffix, mult in [('K', 1024), ('M', 1024 * 1024), ('G', 1024 * 1024 * 1024)]: - if size.upper().endswith(suffix): - multiplier = mult - size = size[:-1] - break - if not size.isdigit(): - raise argparse.ArgumentTypeError("Rule '%s': size must be integer" % rule) - - result[int(part)] = int(size) * multiplier - - return result - -def wic_init_parser_write(subparser): - subparser.add_argument("image", - help="path to the wic image") - subparser.add_argument("target", - help="target file or device") - subparser.add_argument("-e", "--expand", type=expandtype, - help="expand rules: auto or :[,:]") - subparser.add_argument("-n", "--native-sysroot", - help="path to the native sysroot containing the tools") - subparser.add_argument("--image-name", dest="image_name", - help="name of the image to use the artifacts from " - "e.g. core-image-sato") - subparser.add_argument("-v", "--vars", dest='vars_dir', - help="directory with .env files that store " - "bitbake variables") - -def wic_init_parser_help(subparser): - helpparsers = subparser.add_subparsers(dest='help_topic', help=hlp.wic_usage) - for helptopic in helptopics: - helpparsers.add_parser(helptopic, help=helptopics[helptopic][2]) - return - - -subcommands = { - "create": [wic_create_subcommand, - hlp.wic_create_usage, - hlp.wic_create_help, - wic_init_parser_create], - "list": [wic_list_subcommand, - hlp.wic_list_usage, - hlp.wic_list_help, - wic_init_parser_list], - "ls": [wic_ls_subcommand, - hlp.wic_ls_usage, - hlp.wic_ls_help, - wic_init_parser_ls], - "cp": [wic_cp_subcommand, - hlp.wic_cp_usage, - hlp.wic_cp_help, - wic_init_parser_cp], - "rm": [wic_rm_subcommand, - hlp.wic_rm_usage, - hlp.wic_rm_help, - wic_init_parser_rm], - "write": [wic_write_subcommand, - hlp.wic_write_usage, - hlp.wic_write_help, - wic_init_parser_write], - "help": [wic_help_subcommand, - wic_help_topic_usage, - hlp.wic_help_help, - wic_init_parser_help] -} - - -def init_parser(parser): - parser.add_argument("--version", action="version", - version="%(prog)s {version}".format(version=__version__)) - parser.add_argument("-D", "--debug", dest="debug", action="store_true", - default=False, help="output debug information") - - subparsers = parser.add_subparsers(dest='command', help=hlp.wic_usage) - for subcmd in subcommands: - subparser = subparsers.add_parser(subcmd, help=subcommands[subcmd][2]) - subcommands[subcmd][3](subparser) - -class WicArgumentParser(argparse.ArgumentParser): - def format_help(self): - return hlp.wic_help - -def main(argv): - parser = WicArgumentParser( - description="wic version %s" % __version__) - - init_parser(parser) - - args = parser.parse_args(argv) - - if args.debug: - logger.setLevel(logging.DEBUG) - - _apply_sector_size_default(args) - - if "command" in vars(args): - if args.command == "help": - if args.help_topic is None: - parser.print_help() - elif args.help_topic in helptopics: - hlpt = helptopics[args.help_topic] - hlpt[0](hlpt[1], hlpt[2]) - return 0 - - # validate wic cp src and dest parameter to identify which one of it is - # image and cast it into imgtype - if args.command == "cp": - if ":" in args.dest: - args.dest = imgtype(args.dest) - elif ":" in args.src: - args.src = imgtype(args.src) - else: - raise argparse.ArgumentTypeError("no image or partition number specified.") - - return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands) - - -if __name__ == "__main__": - try: - sys.exit(main(sys.argv[1:])) - except WicError as err: - print() - logger.error(err) - sys.exit(1) From patchwork Mon Feb 23 21:50:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70DB1EEC29A for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6360.1771883429738537222 for ; Mon, 23 Feb 2026 13:50:29 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=j/60Dmu9; spf=pass (domain: gmail.com, ip: 209.85.219.45, mailfrom: twoerner@gmail.com) Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-89577f866d6so61573336d6.0 for ; Mon, 23 Feb 2026 13:50:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883428; x=1772488228; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=RF3uxXUk8lnfdOyMK/QowLhBUlO9fU7Q9Dw1rsZ3npc=; b=j/60Dmu9PuHA2fuFn2prhymK8BIyVZ5R8t384LzDoPuYovTZUytVzZHtjmpIBdDOHI dYcvud6MmrhdCsP2UWO91G/Lscud2rA84aB0oRMcOt8MV2+gaXYy3gpw/8VX7uvm1/KG ZFu6+7d23Z3aHkNJx+qpZfQMWT26Y7KhOXUPMORJWbzLmwEgqznJKAxlcKpglgAx61HD i88Iq9BtZjKoGrfQilQ8XBjZOUqe2IWVLcIEUzyzv4mTCeaznQ7ue3VwSOOiKVP3Cddz YAGHU4N44bNdAL6SeIXuc+cOjO/V/tIIvZw070YeawLui8xkg7/uJKbiF13DWut1Ryqs Vgdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883428; x=1772488228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=RF3uxXUk8lnfdOyMK/QowLhBUlO9fU7Q9Dw1rsZ3npc=; b=U7JKox0VEgGYhdwGzLyASkvFdaLs5XYeShRKay9DxG8ImwsXg7QG2iAffqtARb19l1 UJYznK6ruJnhlHVAWWy1qVKA+3h+fuuSeLJY143D/gz3JOWeRoBgiqRWbTOau23xLFvn e3dj4Z5UehpKdk/5TSVzegZk0542RufSIUKS7+U8Mktn3SZ7i8NZSm521o3fXD/IXOxW 3BPqsxewtZRSmVkbKcgrOqmMsRyR2dFvsvagSo7hevVjKpuorKvrJm7sPWYDcWcxmZUY Muc+6CHGoQWTr/kW0lu0lP82X81MqfaK+1Q4iWGuiyh4Q/XUrpXCfnWaVZl0+UvS3pdH QiGg== X-Gm-Message-State: AOJu0YwsbtbuJLwMaEvFL3i0LRxamrRjTg90VUtMl7J9YVrgj74ejXln pS/hr0j34AJ8K12+eRr1I4TIS0+BjqByd6KX0UnORKLOwNhy6LlcVQf1gM09Gp+W X-Gm-Gg: ATEYQzy5Vd1LGcJPq7C0vKoXX2tINHNvscGsCdeePzV3RXK3szbKjJwnz6YXzF4aSB5 6exTyExZjYJv47DHCYOT6UCMzVeASaKH0sVozfGm6uEaMev9xW7U3y8nbuDLStVJRYXPS3+SThF YATnYdArwR3ZaH0CBQTXU+1mQlxW981f2Ggvh5IEQ7tgnxwx7IiRXdLK7dLFaZHWj79PGJdVEzV CAROjodlbeIrQyKhQN/d90I82A9vIhq3Wft76QF0fZL4WI9h6nZ3m5lZKb0m54AKmCncCf4BYWF Tf6L9TYmIwoWCrIBPbThP5raUXqGkW2VCEreKs+yjGUgUJJv9dKbwK0LvzRRw/2EsRCQb4skAVc efGWm+wAdV7iSzNUZvPeG+52C654OLFaXIyxZ78o2mnaRg9Cyfnlnu6vSUzB1kmApuJcBtxj7Kc +Zx1BYB94ELdh2O34mLnYI9LZPDSuzt2NR69bkHjfHOcjn84sTqvM5k2J2Uvo+6F8WbQ== X-Received: by 2002:ad4:5e8c:0:b0:896:f4db:82c0 with SMTP id 6a1803df08f44-89965acdc2dmr222479806d6.1.1771883427968; Mon, 23 Feb 2026 13:50:27 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:27 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 04/10] wic: provide oe-core wks files Date: Mon, 23 Feb 2026 16:50:02 -0500 Message-ID: <20260223215008.2062721-5-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231704 By convention, wic will look for a "wic" directory in the top-level of all BBLAYERS in order to find *wks files. oe-core was the only exception; it stored its *wks files in scripts/lib/wic/canned-wks. Take the *wks, *wks.in, *inc, and *cfg files that were in the old wks location and place them in a "wic" top-level directory in the oe-core layer, thereby following the convention of every other layer. Signed-off-by: Trevor Woerner --- changes in v5 - rebase with master - split patchset out again v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large patch changes in v2: - none --- meta/classes-recipe/image_types_wic.bbclass | 2 +- meta/wic/common.wks.inc | 3 +++ meta/wic/directdisk-bootloader-config.cfg | 27 +++++++++++++++++++++ meta/wic/directdisk-bootloader-config.wks | 8 ++++++ meta/wic/directdisk-gpt.wks | 10 ++++++++ meta/wic/directdisk-multi-rootfs.wks | 23 ++++++++++++++++++ meta/wic/directdisk.wks | 8 ++++++ meta/wic/efi-bootdisk.wks.in | 3 +++ meta/wic/efi-uki-bootdisk.wks.in | 3 +++ meta/wic/mkdisk-ufs.wks | 5 ++++ meta/wic/mkefidisk.wks | 11 +++++++++ meta/wic/mkhybridiso.wks | 7 ++++++ meta/wic/qemuloongarch.wks | 3 +++ meta/wic/qemuriscv.wks | 3 +++ meta/wic/qemux86-directdisk.wks | 8 ++++++ meta/wic/sdimage-bootpart.wks | 6 +++++ meta/wic/systemd-bootdisk.wks | 11 +++++++++ 17 files changed, 140 insertions(+), 1 deletion(-) create mode 100644 meta/wic/common.wks.inc create mode 100644 meta/wic/directdisk-bootloader-config.cfg create mode 100644 meta/wic/directdisk-bootloader-config.wks create mode 100644 meta/wic/directdisk-gpt.wks create mode 100644 meta/wic/directdisk-multi-rootfs.wks create mode 100644 meta/wic/directdisk.wks create mode 100644 meta/wic/efi-bootdisk.wks.in create mode 100644 meta/wic/efi-uki-bootdisk.wks.in create mode 100644 meta/wic/mkdisk-ufs.wks create mode 100644 meta/wic/mkefidisk.wks create mode 100644 meta/wic/mkhybridiso.wks create mode 100644 meta/wic/qemuloongarch.wks create mode 100644 meta/wic/qemuriscv.wks create mode 100644 meta/wic/qemux86-directdisk.wks create mode 100644 meta/wic/sdimage-bootpart.wks create mode 100644 meta/wic/systemd-bootdisk.wks diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass index 675aa9751367..f2fd00e6d7ec 100644 --- a/meta/classes-recipe/image_types_wic.bbclass +++ b/meta/classes-recipe/image_types_wic.bbclass @@ -45,7 +45,7 @@ inherit_defer ${@bb.utils.contains('INITRAMFS_IMAGE_BUNDLE', '1', 'kernel-artifa WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks" WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks" -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}" WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) or ''}" def wks_search(files, search_path): diff --git a/meta/wic/common.wks.inc b/meta/wic/common.wks.inc new file mode 100644 index 000000000000..4a440ddafe64 --- /dev/null +++ b/meta/wic/common.wks.inc @@ -0,0 +1,3 @@ +# This file is included into 3 canned wks files from this directory +part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 +part / --source rootfs --use-uuid --fstype=ext4 --label platform --align 1024 diff --git a/meta/wic/directdisk-bootloader-config.cfg b/meta/wic/directdisk-bootloader-config.cfg new file mode 100644 index 000000000000..c58e74a853cb --- /dev/null +++ b/meta/wic/directdisk-bootloader-config.cfg @@ -0,0 +1,27 @@ +# This is an example configuration file for syslinux. +TIMEOUT 50 +ALLOWOPTIONS 1 +SERIAL 0 115200 +PROMPT 0 + +UI vesamenu.c32 +menu title Select boot options +menu tabmsg Press [Tab] to edit, [Return] to select + +DEFAULT Graphics console boot + +LABEL Graphics console boot +KERNEL /vmlinuz +APPEND label=boot rootwait + +LABEL Serial console boot +KERNEL /vmlinuz +APPEND label=boot rootwait console=ttyS0,115200 + +LABEL Graphics console install +KERNEL /vmlinuz +APPEND label=install rootwait + +LABEL Serial console install +KERNEL /vmlinuz +APPEND label=install rootwait console=ttyS0,115200 diff --git a/meta/wic/directdisk-bootloader-config.wks b/meta/wic/directdisk-bootloader-config.wks new file mode 100644 index 000000000000..3529e05c876e --- /dev/null +++ b/meta/wic/directdisk-bootloader-config.wks @@ -0,0 +1,8 @@ +# short-description: Create a 'pcbios' direct disk image with custom bootloader config +# long-description: Creates a partitioned legacy BIOS disk image that the user +# can directly dd to boot media. The bootloader configuration source is a user file. + +include common.wks.inc + +bootloader --configfile="directdisk-bootloader-config.cfg" + diff --git a/meta/wic/directdisk-gpt.wks b/meta/wic/directdisk-gpt.wks new file mode 100644 index 000000000000..cb640056f199 --- /dev/null +++ b/meta/wic/directdisk-gpt.wks @@ -0,0 +1,10 @@ +# short-description: Create a 'pcbios' direct disk image +# long-description: Creates a partitioned legacy BIOS disk image that the user +# can directly dd to boot media. + + +part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 +part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid + +bootloader --ptable gpt --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" + diff --git a/meta/wic/directdisk-multi-rootfs.wks b/meta/wic/directdisk-multi-rootfs.wks new file mode 100644 index 000000000000..4fd1999ffb69 --- /dev/null +++ b/meta/wic/directdisk-multi-rootfs.wks @@ -0,0 +1,23 @@ +# short-description: Create multi rootfs image using rootfs plugin +# long-description: Creates a partitioned disk image with two rootfs partitions +# using rootfs plugin. +# +# Partitions can use either +# - indirect rootfs references to image recipe(s): +# wic create directdisk-multi-indirect-recipes -e core-image-minimal \ +# --rootfs-dir rootfs1=core-image-minimal +# --rootfs-dir rootfs2=core-image-minimal-dev +# +# - or paths to rootfs directories: +# wic create directdisk-multi-rootfs \ +# --rootfs-dir rootfs1=tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/ +# --rootfs-dir rootfs2=tmp/work/qemux86_64-poky-linux/core-image-minimal-dev/1.0-r0/rootfs/ +# +# - or any combinations of -r and --rootfs command line options + +part /boot --source bootimg_pcbios --ondisk sda --label boot --active --align 1024 +part / --source rootfs --rootfs-dir=rootfs1 --ondisk sda --fstype=ext4 --label platform --align 1024 +part /rescue --source rootfs --rootfs-dir=rootfs2 --ondisk sda --fstype=ext4 --label secondary --align 1024 + +bootloader --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" + diff --git a/meta/wic/directdisk.wks b/meta/wic/directdisk.wks new file mode 100644 index 000000000000..8c8e06b02c66 --- /dev/null +++ b/meta/wic/directdisk.wks @@ -0,0 +1,8 @@ +# short-description: Create a 'pcbios' direct disk image +# long-description: Creates a partitioned legacy BIOS disk image that the user +# can directly dd to boot media. + +include common.wks.inc + +bootloader --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" + diff --git a/meta/wic/efi-bootdisk.wks.in b/meta/wic/efi-bootdisk.wks.in new file mode 100644 index 000000000000..5211972955a5 --- /dev/null +++ b/meta/wic/efi-bootdisk.wks.in @@ -0,0 +1,3 @@ +bootloader --ptable gpt +part /boot --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/boot --fstype=vfat --label boot --active --align 1024 --use-uuid --overhead-factor 1.2 +part / --source rootfs --fstype=ext4 --label root --align 1024 --exclude-path boot/ diff --git a/meta/wic/efi-uki-bootdisk.wks.in b/meta/wic/efi-uki-bootdisk.wks.in new file mode 100644 index 000000000000..cac0fa32cdab --- /dev/null +++ b/meta/wic/efi-uki-bootdisk.wks.in @@ -0,0 +1,3 @@ +bootloader --ptable gpt --timeout=5 +part /boot --source bootimg_efi --sourceparams="loader=${EFI_PROVIDER}" --label boot --active --align 1024 --use-uuid --part-name="ESP" --part-type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B --overhead-factor=1 +part / --source rootfs --fstype=ext4 --label root --align 1024 --exclude-path boot/ diff --git a/meta/wic/mkdisk-ufs.wks b/meta/wic/mkdisk-ufs.wks new file mode 100644 index 000000000000..dfd2f3854890 --- /dev/null +++ b/meta/wic/mkdisk-ufs.wks @@ -0,0 +1,5 @@ +bootloader --ptable gpt + +part /boot --source rootfs --fstype=vfat --label boot --change-directory boot/ +part / --source rootfs --fstype=ext4 --label root --exclude-path boot/ +part swap --fstype=swap --label swap --size 1M diff --git a/meta/wic/mkefidisk.wks b/meta/wic/mkefidisk.wks new file mode 100644 index 000000000000..16dfe76dfe27 --- /dev/null +++ b/meta/wic/mkefidisk.wks @@ -0,0 +1,11 @@ +# short-description: Create an EFI disk image +# long-description: Creates a partitioned EFI disk image that the user +# can directly dd to boot media. + +part /boot --source bootimg_efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 + +part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid + +part swap --ondisk sda --size 44 --label swap1 --fstype=swap + +bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=${KERNEL_CONSOLE} console=tty0" diff --git a/meta/wic/mkhybridiso.wks b/meta/wic/mkhybridiso.wks new file mode 100644 index 000000000000..c3a030e5b40e --- /dev/null +++ b/meta/wic/mkhybridiso.wks @@ -0,0 +1,7 @@ +# short-description: Create a hybrid ISO image +# long-description: Creates an EFI and legacy bootable hybrid ISO image +# which can be used on optical media as well as USB media. + +part /boot --source isoimage_isohybrid --sourceparams="loader=grub-efi,image_name=HYBRID_ISO_IMG" --ondisk cd --label HYBRIDISO + +bootloader --timeout=15 --append="" diff --git a/meta/wic/qemuloongarch.wks b/meta/wic/qemuloongarch.wks new file mode 100644 index 000000000000..8465c7a8c0a2 --- /dev/null +++ b/meta/wic/qemuloongarch.wks @@ -0,0 +1,3 @@ +# short-description: Create qcow2 image for LoongArch QEMU machines + +part / --source rootfs --fstype=ext4 --label root --align 4096 --size 5G diff --git a/meta/wic/qemuriscv.wks b/meta/wic/qemuriscv.wks new file mode 100644 index 000000000000..12c68b706917 --- /dev/null +++ b/meta/wic/qemuriscv.wks @@ -0,0 +1,3 @@ +# short-description: Create qcow2 image for RISC-V QEMU machines + +part / --source rootfs --fstype=ext4 --label root --align 4096 --size 5G diff --git a/meta/wic/qemux86-directdisk.wks b/meta/wic/qemux86-directdisk.wks new file mode 100644 index 000000000000..808997611a4a --- /dev/null +++ b/meta/wic/qemux86-directdisk.wks @@ -0,0 +1,8 @@ +# short-description: Create a qemu machine 'pcbios' direct disk image +# long-description: Creates a partitioned legacy BIOS disk image that the user +# can directly use to boot a qemu machine. + +include common.wks.inc + +bootloader --timeout=0 --append="rw oprofile.timer=1 rootfstype=ext4 console=tty console=ttyS0 " + diff --git a/meta/wic/sdimage-bootpart.wks b/meta/wic/sdimage-bootpart.wks new file mode 100644 index 000000000000..f9f8044f7dc4 --- /dev/null +++ b/meta/wic/sdimage-bootpart.wks @@ -0,0 +1,6 @@ +# short-description: Create SD card image with a boot partition +# long-description: Creates a partitioned SD card image. Boot files +# are located in the first vfat partition. + +part /boot --source bootimg_partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 +part / --source rootfs --ondisk mmcblk0 --fstype=ext4 --label root --align 4 diff --git a/meta/wic/systemd-bootdisk.wks b/meta/wic/systemd-bootdisk.wks new file mode 100644 index 000000000000..3fb2c0e35f3a --- /dev/null +++ b/meta/wic/systemd-bootdisk.wks @@ -0,0 +1,11 @@ +# short-description: Create an EFI disk image with systemd-boot +# long-description: Creates a partitioned EFI disk image that the user +# can directly dd to boot media. The selected bootloader is systemd-boot. + +part /boot --source bootimg_efi --sourceparams="loader=systemd-boot" --ondisk sda --label msdos --active --align 1024 --use-uuid + +part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid + +part swap --ondisk sda --size 44 --label swap1 --fstype=swap --use-uuid + +bootloader --ptable gpt --timeout=5 --append="rootwait rootfstype=ext4 console=ttyS0,115200 console=tty0" From patchwork Mon Feb 23 21:50:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62AF7EEC299 for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6458.1771883430650578162 for ; Mon, 23 Feb 2026 13:50:30 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=VIpIIYA1; spf=pass (domain: gmail.com, ip: 209.85.222.172, mailfrom: twoerner@gmail.com) Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-8cb38e6d164so589682885a.3 for ; Mon, 23 Feb 2026 13:50:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883429; x=1772488229; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=AsseCcXms625CqCeLs0bxdhQqq9xplOJDSdw5hf+SKM=; b=VIpIIYA1LgERaoAfVBro0i1xj/vy/CS7pvArVvUzm7YYs6iJjltotSk/TTAgS1gA5J cpqKroFDjfIAYeyrfqxIIZcL0wA7DhtgyYS+nbq/s8X7hQM1dQyDq6RVjvqus4xPWQ/2 QaUw8meMO6/nfCUBoxmwbwmcBImXR8pqYswRvswvzciX1plbDXWP+9ElozOvliItLs/n LO3KM4Svf6wcBO3+3tWJgLVMGRYRMgn5pmtC+QYvoJkt+g5Lj0HSWE3i3VRYj7vitgvU AAl0TXouugo9KovYmNDmpacSrcScJzYYp1ntRZg0Uol81yw7/JL3fRgDszsqkyzO9eOH jSvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883429; x=1772488229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=AsseCcXms625CqCeLs0bxdhQqq9xplOJDSdw5hf+SKM=; b=NfxcuYQAjJSXReA0SZGjEgHpGtEWruEvLnMcjHMOP2mUXmmBF94EU8lTYIotKTWDMP ACv1/tE6EtTkL6WHwbBvqKiAtSPAURH4Cg1ksyPsx+Wti36IRaIdTtXqJIP3X//2QmPW l+xWX2S/U2OxF/o2Nd9JZ72gh0sA2116cZXZobQePpRo50Q7z3UxHA8Dwxcl19Qp932G 57XzQxgYbYFa2pj1fDiQZ4Sr5iiRbbu65VcQGO+QaegWTr8Gwn2Z2UlHWwP8kV26aPFy fhEHyd2WcAlVVRHEUVXV0Qujts6HhE62LT2Kjrn72xBlJp9Nj54U3WRCr94CFVyiWoXS HLRg== X-Gm-Message-State: AOJu0Yxh1wzURuWxHzEK30AE2ri/VOWBZEMwuvHJepGQCz+xu0U6F8m8 mV8wRddC8U8f4qXFDl1YxKYNNqDnZIH7BkphKN+JcC/VunqG7YPrJglAS4AmXVOK X-Gm-Gg: AZuq6aJxX9fKSrGdBAneMqwWDv1zWtuRWdvC6xfpYMbXsreKtuYIBFCu4UKuxphGyf9 mgtdbn6z9mqAPYhAn7LhrmJo+Hh8ICorkhZcCNcmo6d9vc71M+oKnc0J31W11ACTqzFDEndj5Y6 RODtULtoKbizSCC1NJFfxQYGASPUjN5sjnSIG/f3FMoMNOqKJMPYQIQy+nnBjEXJIkKoRm35yYv XiI+cizpzmdiKfsL85P2WkKT9duCmD+XoarE4F9/xf3mtQwCuzfrbbcNMGMILdxQDYjGLjRAUp5 Knz9p0GDKd4cnkllOxaShjSB321FJJahpeY/zCRZp4Brpjh/6C5tpt/Oc0rYeyg/h3wOdcLF6Th qyQIjKUkhJdjlhzCVUiRcJvAjvyjlRzLiGq78/CTjWO30c0A029SWR54R37QWj6htI00w3eKtle GP4jqUDackBiqOg5acID7o0bJQ0ThTYovYGKNy8cjhlEgYpgv+CJmxSoPzvSCag6aPEw== X-Received: by 2002:a05:620a:4692:b0:8bb:7e56:f204 with SMTP id af79cd13be357-8cb8c9f9371mr1277930585a.20.1771883428839; Mon, 23 Feb 2026 13:50:28 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:28 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 05/10] wic: add recipe Date: Mon, 23 Feb 2026 16:50:03 -0500 Message-ID: <20260223215008.2062721-6-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231705 Add a recipe to pull in the newly-externalized wic utility. Add wic to the list of native wic-tools, and make it a dependency of the image bbclass for wic images. Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - split patch set back out into smaller patches - do not add TOPDIR and WIC_SECTOR_SIZE to image_types_wic.bbclass if a user wants them, they can add them to WICVARS in their config v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large patch changes in v2: - add wic as a native dependency of the image_types_wic.bbclass --- meta/classes-recipe/image_types_wic.bbclass | 4 ++-- meta/conf/distro/include/maintainers.inc | 3 ++- meta/recipes-core/meta/wic-tools.bb | 1 + meta/recipes-support/wic/wic_0.3.0.bb | 18 ++++++++++++++++++ 4 files changed, 23 insertions(+), 3 deletions(-) create mode 100644 meta/recipes-support/wic/wic_0.3.0.bb diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass index f2fd00e6d7ec..3a4c497de19a 100644 --- a/meta/classes-recipe/image_types_wic.bbclass +++ b/meta/classes-recipe/image_types_wic.bbclass @@ -111,14 +111,14 @@ do_image_wic[cleandirs] = "${WORKDIR}/build-wic" USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}" WKS_FILE_CHECKSUM = "${@wks_checksums(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) if '${USING_WIC}' else ''}" do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}" -do_image_wic[depends] += "${@' '.join('%s-native:do_populate_sysroot' % r for r in ('parted', 'gptfdisk', 'dosfstools', 'mtools'))}" +do_image_wic[depends] += "${@' '.join('%s-native:do_populate_sysroot' % r for r in ('wic', 'parted', 'gptfdisk', 'dosfstools', 'mtools'))}" # We ensure all artfacts are deployed (e.g virtual/bootloader) do_image_wic[recrdeptask] += "do_deploy" do_image_wic[deptask] += "do_image_complete" WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}' -WKS_FILE_DEPENDS_DEFAULT += "bmaptool-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native" +WKS_FILE_DEPENDS_DEFAULT += "wic-native bmaptool-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native" # Unified kernel images need objcopy WKS_FILE_DEPENDS_DEFAULT += "virtual/cross-binutils" WKS_FILE_DEPENDS_BOOTLOADERS = "" diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc index b5ab35d92a06..46bd48117910 100644 --- a/meta/conf/distro/include/maintainers.inc +++ b/meta/conf/distro/include/maintainers.inc @@ -884,7 +884,8 @@ RECIPE_MAINTAINER:pn-weston = "Denys Dmytriyenko " RECIPE_MAINTAINER:pn-weston-init = "Denys Dmytriyenko " RECIPE_MAINTAINER:pn-wget = "Yi Zhao " RECIPE_MAINTAINER:pn-which = "Unassigned " -RECIPE_MAINTAINER:pn-wic-tools = "Unassigned " +RECIPE_MAINTAINER:pn-wic = "Trevor Woerner " +RECIPE_MAINTAINER:pn-wic-tools = "Trevor Woerner " RECIPE_MAINTAINER:pn-wireless-regdb = "Unassigned " RECIPE_MAINTAINER:pn-wpa-supplicant = "Unassigned " RECIPE_MAINTAINER:pn-wpebackend-fdo = "Unassigned " diff --git a/meta/recipes-core/meta/wic-tools.bb b/meta/recipes-core/meta/wic-tools.bb index fa983ed5f12c..45fb873dd6bd 100644 --- a/meta/recipes-core/meta/wic-tools.bb +++ b/meta/recipes-core/meta/wic-tools.bb @@ -3,6 +3,7 @@ SUMMARY = "A meta recipe to build native tools used by wic." LICENSE = "MIT" DEPENDS = "\ + wic-native \ parted-native gptfdisk-native dosfstools-native \ mtools-native bmaptool-native grub-native cdrtools-native \ btrfs-tools-native squashfs-tools-native pseudo-native \ diff --git a/meta/recipes-support/wic/wic_0.3.0.bb b/meta/recipes-support/wic/wic_0.3.0.bb new file mode 100644 index 000000000000..d2a2e282b2f0 --- /dev/null +++ b/meta/recipes-support/wic/wic_0.3.0.bb @@ -0,0 +1,18 @@ +SUMMARY = "OpenEmbedded Image Creator (wic) standalone CLI" +HOMEPAGE = "https://git.yoctoproject.org/wic" +LICENSE = "GPL-2.0-only" +LIC_FILES_CHKSUM = "file://LICENSE;md5=4ee23c52855c222cba72583d301d2338" + +SRC_URI = "git://git.yoctoproject.org/wic.git;branch=master;protocol=https" +SRCREV = "7e7ede0806b43503d13d96a1eb4c091e2cd527ad" + +inherit python_hatchling + +RDEPENDS:${PN} += " \ + python3-core \ + python3-json \ + python3-logging \ + python3-misc \ + " + +BBCLASSEXTEND = "native nativesdk" From patchwork Mon Feb 23 21:50:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81652 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60416EEC298 for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6459.1771883431398831856 for ; Mon, 23 Feb 2026 13:50:31 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=BU7bB9BB; spf=pass (domain: gmail.com, ip: 209.85.219.48, mailfrom: twoerner@gmail.com) Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-896f4627dffso65251386d6.0 for ; Mon, 23 Feb 2026 13:50:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883430; x=1772488230; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=TJFAz92psE2gQvr4gA+JgW9zcTH5/wSQx1lNRmt/5qI=; b=BU7bB9BBm5ZeveT65E09q2uWD7AcWuoAhgvNNl6Tyah9ZZ/O7VGzlIAideerptY0wU wCCZbrzesXR7ncEvWuGCiOhDSw4SkafSCtW5IP66Lo9oNHHRhwHjATT4iB79xmwTyTmx cJc4QT7uSgMegvLjgr349tYwILZT6FUq1nHWUMUuE4PPNdoxeNfTU5XgJcBHZ9LmcPoM m0AgyUHgUuTv1C7imQ88RujZPSwCv2slYe/kJbtTBgR/z+VzeJ3UyyUzSIE66QERXJ1j tHxvnvlOci+4A8pWui1vKTCXeLxPJVPCQ2XFvvjbJcG7P23OL9g1ZBQeVn4HBHV01/wc 6GIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883430; x=1772488230; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=TJFAz92psE2gQvr4gA+JgW9zcTH5/wSQx1lNRmt/5qI=; b=CI1l0go3U1c57awRNjA1h8F/LbCNhahTKZO2XQ9anj6M1e/lX4Q1eUBIeFZuSz7UtY mnrInr6Ruc8tD25MKZ8B9VwpsMrHnYvBYU+07iHZGXLI7YcJNOMfcgWb/2DJwWJ9aLqt Qz64ARSB2xdmzw58pRAYfwjV9d6f69iHZIXEyVrwakWHnbphOw6PyUZugED4ll+8eryY We37Epy5QMgqgHOBEWMnD8yvKorAu0aL8rysn5au4nyYFiKo8JVICJheIOrks2hnko4L T/rPX45qJoQqF2PM1LKNfhRX5dnTHX/XrTB3h55LVGHWrQG0VDdA6vpWmVF9eA8ZT0LT 74UQ== X-Gm-Message-State: AOJu0YzvoqOEsConmYSykXByboPkzj9/rRzqN7+n7LKthSfw/2nHU1DL 8KtP0WMQd8X3ECKN2A+vHW0RIkay+6wckMSo/4OAgNxrWowP1/fiR7cp+OgKYwMQ X-Gm-Gg: ATEYQzyXWyKc/VWANVd7VSVgmQAcEwOsZGSOLe1LjlUvhl0DuiYDS6mGfafyRgpfT5K 7eiak+z5GoBybl5DbI/1JzcdGUUXaA10aajTeVdQTiP0kT9nJ7ps3ErXIWL2FnSIM91hJy82bm9 F4bjPkW6HUTLYr6oCoOsKGqHOL6Ve8ETBWo8+YcksZt7X8ualUyr6mys7x35+hAJXTe3a9HnRMD /9qEowRZQXrTxeePXJ7oi9zS69ANqruP94JWeGBeBzV97eU2xcExP8+8mWBjrrunnKOZlceTQUL FS7HCfTztyTCw6W2ltVjUzL22em7IS5zjqjiK5jLbgab8aKWbDNmxLb+fkuwdpg+vfX1Z8E9eXN FAY4CN7AI3xl9ac6wl9/9i1puc59dAdcNVySqaVT4MCXLDxOFsl/J8/+vZw8JULdPMo76GglDZE lictik8yNcHNh7dehGZHLPGClGN3ajw8uYY2AgqABOW1ycI9ZYfuAXmrPqVHktVezDmw== X-Received: by 2002:a05:6214:21ea:b0:895:4c45:a615 with SMTP id 6a1803df08f44-89979d360a5mr145948836d6.44.1771883429734; Mon, 23 Feb 2026 13:50:29 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:29 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 06/10] oe-selftest/cases/wic.py: update WicTestCase Date: Mon, 23 Feb 2026 16:50:04 -0500 Message-ID: <20260223215008.2062721-7-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231706 The wic oe-selftest defines its own class, WicTestCase, for handling setup, teardown, and other pieces needed to run the individual wic oe-selftests. Now that wic is a tool external to oe-core, it, and the tools on which it depends, need to be built natively before the test can be run, need to be made available to the tests, and need to remain available while the test is running. Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - split back out into smaller patches v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large patch changes in v2: - none --- meta/lib/oeqa/selftest/cases/wic.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 7ad61fa700f7..e9c28baf51eb 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -57,12 +57,22 @@ class WicTestCase(OESelftestTestCase): image_is_ready = False wicenv_cache = {} + oldpath = "" + envfile = "" def setUpLocal(self): """This code is executed before each test method.""" self.resultdir = os.path.join(self.builddir, "wic-tmp") super(WicTestCase, self).setUpLocal() + config = 'DEPENDS:pn-core-image-minimal += " %s"' % get_bb_var("DEPENDS", "wic-tools") + self.append_config(config) + bitbake('wic-tools core-image-minimal core-image-minimal-mtdutils') + WicTestCase.oldpath = os.environ['PATH'] + os.environ['PATH'] = get_bb_var("PATH", "wic-tools") + ":" + os.environ['PATH'] + image = 'core-image-minimal' + WicTestCase.envfile = os.path.join(self._get_image_env_path(image), image) + '.env' + # Do this here instead of in setUpClass as the base setUp does some # clean up which can result in the native tools built earlier in # setUpClass being unavailable. @@ -78,6 +88,7 @@ class WicTestCase(OESelftestTestCase): """Remove resultdir as it may contain images.""" rmtree(self.resultdir, ignore_errors=True) super(WicTestCase, self).tearDownLocal() + os.environ['PATH'] = WicTestCase.oldpath def _get_image_env_path(self, image): """Generate and obtain the path to .env""" From patchwork Mon Feb 23 21:50:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C48EEEC29F for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qv1-f66.google.com (mail-qv1-f66.google.com [209.85.219.66]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6362.1771883433829964960 for ; Mon, 23 Feb 2026 13:50:33 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=VsyoPvUk; spf=pass (domain: gmail.com, ip: 209.85.219.66, mailfrom: twoerner@gmail.com) Received: by mail-qv1-f66.google.com with SMTP id 6a1803df08f44-899a2f4cdddso1664386d6.2 for ; Mon, 23 Feb 2026 13:50:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883432; x=1772488232; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ziPaPy0MUJCJqPh5lLEbYWtw+3gQanHJ8DGciOzeTUY=; b=VsyoPvUk86M7QoDFJY7JK6x2teLeE5kDRk/pR2WuKNbFN6zQyro2ABPADY1EAtVe38 rAIsKiUx0JQdg5oLB16tN8gtaUXQusabGAAbmik2WnNyqWT950q8QIpTw973oNysAjwq XM4YAW9ayV1qTQ8kYCkECC0jhnOZleQeMfwPHDWmCON5lokkl0/G5HFj3mRfsje7+WA7 +A0lEMSm1D5nbUlFAZX3FhwOlrv6HbjAEZCdF+Hrr4/qayctwP4NkpTPWKbdxxsOfcIC AqGMahT5BkkwhZ8uYRHUMMeOlOBMba5pkFfOxnr59anKxrEcqcqk1Pm7Tym6nKVAi85s 4DbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883432; x=1772488232; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ziPaPy0MUJCJqPh5lLEbYWtw+3gQanHJ8DGciOzeTUY=; b=kLh7UGjH0z5HzDCGKTwCUPa5q2Ehgf4qq3RE+j8YWrW1gb0Rovrg6SlQnwXWmMJmfe g5ASp0aZMU5zXXvmY/B+um3IC3sH1fg6yDaGEO66hLW4xp0pBKr8ZI1LlaagyqT8krUU RBi+4sLBJCTP+Bxj8mQzPfMrLyMNnkkfzg9b7eLcaiZlDJBEYwaw1lu3PyrEmojb2t4y 4/pIvNi3pMvNIE5t1Rtxe1CRlarZYSk+4CtngDoh3A/aWf6LU9yOtJzXAMpLJjyHSfM/ y90LDQs+HrX0okWzVJ63qZ9QFEX8DFxusH8C8NQ5j3zgBnTp2FQHHgnWXUaHLLfsf/98 foFQ== X-Gm-Message-State: AOJu0YwiaAILsbmbGih/nrrPKjhYxNGKyaWYpMwR/vYlVwDiykmDOjEs M/WjIyuZVI0Twtb/oHNqkEHX+KkHDUNdk7x7HohQM/mYCT/biUgtKrd40izveocLmOo= X-Gm-Gg: ATEYQzzP8c68Ng/GOIGLod9Ssaddxe7MepDaiz3G0Wa1yEqiyUKhmqTuJuOTNDLfX+P Uv76ZN4DvS58fxHiawu/OaHpxwjMTXcqHkTo/BUNxZQPHAZKtotU6K4n/px3ptOCFuQC7TFySy/ IrnMhbtwuMlkGzRE9M0AffYezi6eaoCum2Qk2xx97rxhPlczgeyK3pY1GAETJ/3auL/ePiDxqod ezjRHPaxFwZhjt6NPjnOrt7GymguItbNkdHXcLgh1GisGJkt2OOOAvXkUr3k8lF1Ipx+gF+Xhuw 8jUoE4lS3PY+NdAZe1sfR6Z5sEjIgXKlcR8b2V16jrIqdnGpAqWqnHIuQ263BDg5/tRL5ubLT7t tjIhRcWVTItsdqaTti6grnJYMH0zkO8QYPszwGvAz5aG8bwIGmoLlPp7uWfxVeEoNVDpC41pZJl 9/QH2YF20qyD6hhQVicAWbCZn9NvcnCfPHro12NrQNa8rCdqJN8LPdf0LVDb6oxpSWJg== X-Received: by 2002:a05:6214:da9:b0:895:3ec:9e6c with SMTP id 6a1803df08f44-89979c5612amr154503566d6.1.1771883431993; Mon, 23 Feb 2026 13:50:31 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:30 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 07/10] oe-selftest/cases/wic.py: oe-selftest -r wic.CLITests -> PASS Date: Mon, 23 Feb 2026 16:50:05 -0500 Message-ID: <20260223215008.2062721-8-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231708 Changes for the wic.CLITests oe-selftests to pass now that wic is a stand-alone app and no longer part of oe-core. Note that the base class for these tests had to change to WicTestCase so that the wic-native tool would be built and available. Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - split back out into smaller patches - now that the sector-size re-implementation is ahead of this patch, various wic sub commands (ls, rm, cp) no longer need a --vars file v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large patch changes in v2: - none --- meta/lib/oeqa/selftest/cases/wic.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index e9c28baf51eb..22163ccdb3e2 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -99,7 +99,7 @@ class WicTestCase(OESelftestTestCase): WicTestCase.wicenv_cache[image] = os.path.join(stdir, machine, 'imgdata') return WicTestCase.wicenv_cache[image] -class CLITests(OESelftestTestCase): +class CLITests(WicTestCase): def test_version(self): """Test wic --version""" runCmd('wic --version') @@ -139,18 +139,18 @@ class CLITests(OESelftestTestCase): def test_list_images(self): """Test wic list images""" - runCmd('wic list images') + runCmd('wic list images -v %s' % self.envfile) def test_list_source_plugins(self): """Test wic list source-plugins""" - runCmd('wic list source-plugins') + runCmd('wic list source-plugins -v %s' % self.envfile) def test_listed_images_help(self): """Test wic listed images help""" - output = runCmd('wic list images').output + output = runCmd('wic list images -v %s' % self.envfile).output imagelist = [line.split()[0] for line in output.splitlines()] for image in imagelist: - runCmd('wic list %s help' % image) + runCmd('wic list -v %s %s help' % (self.envfile, image)) def test_unsupported_subcommand(self): """Test unsupported subcommand""" From patchwork Mon Feb 23 21:50:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3841DEEC28E for ; Mon, 23 Feb 2026 21:50:35 +0000 (UTC) Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6363.1771883434612333677 for ; Mon, 23 Feb 2026 13:50:34 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Z1c7/El4; spf=pass (domain: gmail.com, ip: 209.85.222.194, mailfrom: twoerner@gmail.com) Received: by mail-qk1-f194.google.com with SMTP id af79cd13be357-8cb40030be5so476535885a.3 for ; Mon, 23 Feb 2026 13:50:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883433; x=1772488233; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=nAVKkGzqrZxomZbwKbD9gtEZA4l0KtJcNPiQ2/98/VY=; b=Z1c7/El4xauowY3tbB1/C++jUQd2UIEMAftyMShZlTt1nniLGYzG9haxSUL2zq2s3m iU3NDZSO5xHKd4VfNdcSlvJJNCbcweiXIqTRL17bc3i45w8aPu24jD8KVg1eqA5Gj0+N oDuwvK0g91bksYuPUODYIkHOad9Q7/SPVma/ISjwuMAoAy8VKaXqB+f23LYiErVxkbaH jlQ27MYSHRe7EMrEubyv4FFquEI7pbJ3x4v1NGqLK6Au4QY3J2h9vVRgC87IzRses8oK c6utGE5tl5JtGllm2Ak49zHT1H+2UdlwRjoH6ddsrfvyTtmBf9bYXFXBfAoCqMN+GhaG wPeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883433; x=1772488233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nAVKkGzqrZxomZbwKbD9gtEZA4l0KtJcNPiQ2/98/VY=; b=tVTeNPgZfiySbA+GXW0QFYgEZ8s8+nvJHvlmpMg3t0xNmf3lP76OASwdjsYKXX+6tS kfJe5OHk/E18bDcre20VdkJ6wdqFr8NpOj4hVNp0dOAaEYUAxrISCH95vpVQ+qYWCzFM NYT3oFDvSfNAmYKsI54QH+x3RiPUTraG5aQyDizYtRpxvJW2C0f0y0x3j35sUUWqgZR3 AaZQFvyJJQdS501nVZpZvtYHtEVlM/77uhhLmop9GRt5Fgz0N7FmUd9d5ILZGgrobtsJ ZBzXWjlolDtyXyNJBA6uML7XFLdAvC8CIBdpX1MiAa3rlDRIUem1rM6nGREpZPZpXbyz unyw== X-Gm-Message-State: AOJu0YwNgycG9EqFkisgOFUlpWyALKWQHMKDb8gjP1Ne00hf3HgkQtOd r1O+8vOarPg7tgTYPqlYJd946uVn2sUtDLspk4ysE7N0by1RJsxMR2QKKaw6p/vVSFw= X-Gm-Gg: AZuq6aIH8/i0kE2nscvyjUCZqzrF4CWVA0uDq7Josi4m4l04FffVkX6OO0tzwAzBHiL VdHSIubCDkNGHHmvv9GyKPE1Gbmb9MrThNbnh6FPz9BTSJMwoIHKQ43ofeW0Tq7NcnQT0CO6emv nidJh7Z0vVsKTipUCjjWOHzLCbrTvq2HcPZuFv7gM7gWMuQt2CvkIXXJO0IT3aaPVUgni1ZNXff 7WDgDLNX/Br/YfdNdclElvM1am4OsqILFkdbZoXB0tU1eMtI5Wao7uxcjZ9htgNtcezGJEbwFDu E0fJtkyhuGkNH3YrNXpSMzjO1+Vf86JsuNHF0pguCyvKWXb5lN0ucR10WQJbMr6OayRWodp0TkG 3sMXyP4vjqx2RAzke2YlpI4zGdFBOn4gTH8LpfjGaLFFJ5jP4LmcSTpalA6OCRhObr4KejsK+oB O0Bh3LNaS38F2I5PgDz5suhbZ3JhI7STzN7wnfcgpUNP9IihjIt30++aX6SFoddhcClA== X-Received: by 2002:a05:620a:1aa9:b0:8b2:ea2b:923c with SMTP id af79cd13be357-8cb8c9e680bmr1281698585a.14.1771883432896; Mon, 23 Feb 2026 13:50:32 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:32 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 08/10] oe-selftest/cases/wic.py: oe-selftest -r wic.ModifyTests -> PASS Date: Mon, 23 Feb 2026 16:50:06 -0500 Message-ID: <20260223215008.2062721-9-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231709 Changes for the wic.ModifyTests oe-selftests to pass now that wic is a stand-alone app and no longer part of oe-core. Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - re-split out in to individual, smaller patches - now that the sector-size re-implementation is ahead of this patch, various wic subcommands (ls, cp, rm) no longer need a --vars file v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large patch changes in v2: - none --- meta/lib/oeqa/selftest/cases/wic.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 22163ccdb3e2..8a6e51abb968 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -1962,9 +1962,9 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" class ModifyTests(WicTestCase): def test_wic_ls(self): """Test listing image content using 'wic ls'""" - runCmd("wic create wictestdisk " + runCmd("wic create wictestdisk -v %s " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s" % (self.envfile, self.resultdir)) images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) @@ -1980,9 +1980,9 @@ class ModifyTests(WicTestCase): def test_wic_cp(self): """Test copy files and directories to the the wic image.""" - runCmd("wic create wictestdisk " + runCmd("wic create wictestdisk -v %s " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s" % (self.envfile, self.resultdir)) images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) @@ -2026,9 +2026,9 @@ class ModifyTests(WicTestCase): def test_wic_rm(self): """Test removing files and directories from the the wic image.""" - runCmd("wic create mkefidisk " + runCmd("wic create mkefidisk -v %s " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s" % (self.envfile, self.resultdir)) images = glob(os.path.join(self.resultdir, "mkefidisk-*.direct")) self.assertEqual(1, len(images)) @@ -2054,9 +2054,9 @@ class ModifyTests(WicTestCase): def test_wic_ls_ext(self): """Test listing content of the ext partition using 'wic ls'""" - runCmd("wic create wictestdisk " + runCmd("wic create wictestdisk -v %s " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s" % (self.envfile, self.resultdir)) images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) @@ -2069,9 +2069,9 @@ class ModifyTests(WicTestCase): def test_wic_cp_ext(self): """Test copy files and directories to the ext partition.""" - runCmd("wic create wictestdisk " + runCmd("wic create wictestdisk -v %s " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s" % (self.envfile, self.resultdir)) images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) @@ -2169,9 +2169,9 @@ class ModifyTests(WicTestCase): def test_wic_rm_ext(self): """Test removing files from the ext partition.""" - runCmd("wic create mkefidisk " + runCmd("wic create mkefidisk -v %s " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s" % (self.envfile, self.resultdir)) images = glob(os.path.join(self.resultdir, "mkefidisk-*.direct")) self.assertEqual(1, len(images)) From patchwork Mon Feb 23 21:50:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83004EEC299 for ; Mon, 23 Feb 2026 21:50:45 +0000 (UTC) Received: from mail-qv1-f68.google.com (mail-qv1-f68.google.com [209.85.219.68]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6463.1771883436812968358 for ; Mon, 23 Feb 2026 13:50:37 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=VWviK+hE; spf=pass (domain: gmail.com, ip: 209.85.219.68, mailfrom: twoerner@gmail.com) Received: by mail-qv1-f68.google.com with SMTP id 6a1803df08f44-896f8feee14so58616276d6.0 for ; Mon, 23 Feb 2026 13:50:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883435; x=1772488235; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=/qJfQT8ZTVQRIl5J+6LXKVOV0QUX1bsqdo3xc6Wi9sQ=; b=VWviK+hEdyg/C50ZYgyoDMaY8I40/B7Fu3qCmdxNpu82Kmu5ap1vpZQS85D+5dufif FfrZ0BPnyky5IAf0Vt71sEEma71SOq1ZRaEvbEP3AnC4kXqx8/ZaHKmoYjti/lywfOay dv2ObInib9z5+C2H/ZLG2iHbsI2g58QBifjfniCSK7BzfL1EuTYzbxjesz6YlU6gZLbB A/T5Lg3nFFljPj6OTct35GYMUGklO03CBjhcU2gjWs2a/waOYV2tHfXJkFwBS3bwSA6F j2v1E2x5mJAcSpdFYDv24m36lfsu//bUyc07y4T7rqgQvEMna4m7NG+H28d7BbIDrzm3 xYSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883435; x=1772488235; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/qJfQT8ZTVQRIl5J+6LXKVOV0QUX1bsqdo3xc6Wi9sQ=; b=N3Og9Zo2qXs9wU7+20uc6noc1MH/junrpxpmBpqgot0fEAzyvVPR7Ry5xvrPpNhD9y 1lGC9llk4O4LufbO2Qi60weSC8MRF19PuetClz7DRt6jWfD/oM2kFYqMenGSavRJLEdC OLAR6Wc5FzaBY/7c8gpo+s7QYmhXZdU47Vd+K7O2uYmjqkP0o8feMv4lojkMYl5WMnMp RuuUPhkCSgCzH1d6tEpRU05LxgSAxFszN9OvvELuowF8JDF4hb18VeL2Ggs4CgHakfhb a7bHgjhlUAZlrfYTbrabfQDD6ld1JGqvP6fEae1SG3AFQXsdu/hM7va6CEPij5h6VOX0 d20A== X-Gm-Message-State: AOJu0Yw+4mWyx8nK8kSCyHe1kIs6VlSCElftDoNT0RftoQeYil/Hm/eP WQPuOdfNsXtHbQxk8uQwYqXc+/iiy/RtgkmxnLVi+kQfq0nW5FuqLVwRXLpbqg+Elzg= X-Gm-Gg: ATEYQzzqHYtsTH28oMV6ApguVB1JIF5nJvTyAM55W7rAWvE5L4ECbF3eJ/HW+Bu2uza 7MSf8QoblYN7TC+GrXEEJhOwZmmb1rQXnFQ+sJCtoErPoKxnJSaQFJ1IXgKhFDJBawgdRemuN7x bZMJWveQ9cZ+WlQT5yhdjnIajInJW+PIj/y3k9BvCbP6RjnqxipaTfEbP0rEFyPcRZwrvHlisd1 aBsH7sSRHLfRdNAGWPiWKHv+lR76KYjGSbyydOl14lVvwJJw/Uc7+0Q4TGkvTiUdLXAo20nonW8 3r2xlQLyFRDUeoGOZInifo9gHqkiTGFBPlVO+VlXTERuNbciQoGc1I8ms5EuhdtaxfKQfA78JV1 krnfEDQlPExBQFmVjW6xChvjAYBghYj/jl7791qJxfFZ20o1otIt0DvEmGxqcRPyLoCn2AI8mZe WeXJLOjXxq12cw5GI93n0Eq1HSSkXTbOwL/MWKOpTU3jZgdrh3oMr67744ISwITyG5Mg== X-Received: by 2002:a05:6214:2685:b0:895:4b79:83bd with SMTP id 6a1803df08f44-89979e1b255mr153408816d6.7.1771883434778; Mon, 23 Feb 2026 13:50:34 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:33 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 09/10] oe-selftest/cases/wic.py: oe-selftest -r wic.Wic -> PASS Date: Mon, 23 Feb 2026 16:50:07 -0500 Message-ID: <20260223215008.2062721-10-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231710 Changes for the wic.Wic oe-selftests to pass now that wic is a stand-alone app and no longer part of oe-core. Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - split back out into multiple, smaller patches - now that the sector-size re-implementation is ahead of this patch, several wic subcommands (ls, cp, rm) no longer need a --vars file v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large, atomic commit changes in v2: - none --- meta/lib/oeqa/selftest/cases/wic.py | 136 ++++++++++++++++------------ 1 file changed, 76 insertions(+), 60 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 8a6e51abb968..355c680f80ba 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -99,6 +99,12 @@ class WicTestCase(OESelftestTestCase): WicTestCase.wicenv_cache[image] = os.path.join(stdir, machine, 'imgdata') return WicTestCase.wicenv_cache[image] + def _create_image_env_file(self, image): + bitbake('%s -c rootfs_wicenv' % image) + stagingdir = get_bb_var('STAGING_DIR', image) + machine = self.td['MACHINE'] + return os.path.join(stagingdir, machine, 'imgdata', image) + '.env' + class CLITests(WicTestCase): def test_version(self): """Test wic --version""" @@ -175,8 +181,8 @@ class Wic(WicTestCase): with TemporaryDirectory() as tmpdir: img = 'core-image-minimal' # build the image using the WKS file - cmd = "wic create %s -e %s -o %s" % ( - wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % ( + wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join( @@ -208,7 +214,7 @@ class Wic(WicTestCase): with TemporaryDirectory() as tmpdir: img = 'core-image-minimal' # build the image using the WKS file - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*.direct" % wksname)) @@ -252,7 +258,7 @@ class Wic(WicTestCase): bitbake(img) self.remove_config(config) - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] @@ -271,14 +277,14 @@ class Wic(WicTestCase): def test_build_image_name(self): """Test wic create wictestdisk --image-name=core-image-minimal""" - cmd = "wic create wictestdisk --image-name=core-image-minimal -o %s" % self.resultdir + cmd = "wic create wictestdisk --image-name=core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join (self.resultdir, "wictestdisk-*.direct")))) @skipIfNotArch(['i586', 'i686', 'x86_64']) def test_gpt_image(self): """Test creation of core-image-minimal with gpt table and UUID boot""" - cmd = "wic create directdisk-gpt --image-name core-image-minimal -o %s" % self.resultdir + cmd = "wic create directdisk-gpt --image-name core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-*.direct")))) @@ -291,7 +297,7 @@ class Wic(WicTestCase): self.append_config(config) bitbake('core-image-minimal core-image-minimal-initramfs') self.remove_config(config) - cmd = "wic create mkhybridiso --image-name core-image-minimal -o %s" % self.resultdir + cmd = "wic create mkhybridiso --image-name core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "HYBRID_ISO_IMG-*.direct")))) self.assertEqual(1, len(glob(os.path.join (self.resultdir, "HYBRID_ISO_IMG-*.iso")))) @@ -299,14 +305,14 @@ class Wic(WicTestCase): @skipIfNotArch(['i586', 'i686', 'x86_64']) def test_qemux86_directdisk(self): """Test creation of qemux-86-directdisk image""" - cmd = "wic create qemux86-directdisk -e core-image-minimal -o %s" % self.resultdir + cmd = "wic create qemux86-directdisk -e core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "qemux86-directdisk-*direct")))) @skipIfNotArch(['i586', 'i686', 'x86_64', 'aarch64']) def test_mkefidisk(self): """Test creation of mkefidisk image""" - cmd = "wic create mkefidisk -e core-image-minimal -o %s" % self.resultdir + cmd = "wic create mkefidisk -e core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "mkefidisk-*direct")))) @@ -317,7 +323,7 @@ class Wic(WicTestCase): self.append_config(config) bitbake('core-image-minimal') self.remove_config(config) - cmd = "wic create directdisk-bootloader-config -e core-image-minimal -o %s" % self.resultdir + cmd = "wic create directdisk-bootloader-config -e core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-bootloader-config-*direct")))) @@ -328,15 +334,16 @@ class Wic(WicTestCase): self.append_config(config) bitbake('core-image-minimal') self.remove_config(config) - cmd = "wic create systemd-bootdisk -e core-image-minimal -o %s" % self.resultdir + cmd = "wic create systemd-bootdisk -e core-image-minimal -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "systemd-bootdisk-*direct")))) def test_efi_bootpart(self): """Test creation of efi-bootpart image""" - cmd = "wic create mkefidisk -e core-image-minimal -o %s" % self.resultdir kimgtype = get_bb_var('KERNEL_IMAGETYPE', 'core-image-minimal') self.append_config('IMAGE_EFI_BOOT_FILES = "%s;kernel"\n' % kimgtype) + envfile = self._create_image_env_file('core-image-minimal') + cmd = "wic create mkefidisk -e core-image-minimal -o %s -v %s" % (self.resultdir, envfile) runCmd(cmd) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') images = glob(os.path.join(self.resultdir, "mkefidisk-*.direct")) @@ -345,9 +352,10 @@ class Wic(WicTestCase): def test_sdimage_bootpart(self): """Test creation of sdimage-bootpart image""" - cmd = "wic create sdimage-bootpart -e core-image-minimal -o %s" % self.resultdir kimgtype = get_bb_var('KERNEL_IMAGETYPE', 'core-image-minimal') - self.write_config('IMAGE_BOOT_FILES = "%s"\n' % kimgtype) + self.append_config('IMAGE_BOOT_FILES = "%s"\n' % kimgtype) + envfile = self._create_image_env_file('core-image-minimal') + cmd = "wic create sdimage-bootpart -e core-image-minimal -o %s -v %s" % (self.resultdir, envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "sdimage-bootpart-*direct")))) @@ -361,7 +369,7 @@ class Wic(WicTestCase): self.append_config(config) bitbake('core-image-minimal') self.remove_config(config) - cmd = "wic create directdisk -e core-image-minimal" + cmd = "wic create directdisk -e core-image-minimal -v %s" % self.envfile runCmd(cmd) self.assertEqual(1, len(glob("directdisk-*.direct"))) @@ -374,55 +382,58 @@ class Wic(WicTestCase): 'core-image-minimal')) bbvars = {key.lower(): value for key, value in bb_vars.items()} bbvars['resultdir'] = self.resultdir + bbvars['envfile'] = self.envfile runCmd("wic create directdisk " "-b %(staging_datadir)s " "-k %(deploy_dir_image)s " "-n %(recipe_sysroot_native)s " "-r %(image_rootfs)s " - "-o %(resultdir)s" % bbvars) + "-o %(resultdir)s " + "-v %(envfile)s" % bbvars) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-*.direct")))) def test_compress_gzip(self): """Test compressing an image with gzip""" runCmd("wic create wictestdisk " "--image-name core-image-minimal " - "-c gzip -o %s" % self.resultdir) + "-c gzip -o %s -v %s" % (self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct.gz")))) def test_compress_bzip2(self): """Test compressing an image with bzip2""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " - "-c bzip2 -o %s" % self.resultdir) + "-c bzip2 -o %s -v %s" % (self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct.bz2")))) def test_compress_xz(self): """Test compressing an image with xz""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " - "--compress-with=xz -o %s" % self.resultdir) + "--compress-with=xz -o %s -v %s" % (self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct.xz")))) def test_wrong_compressor(self): """Test how wic breaks if wrong compressor is provided""" self.assertEqual(2, runCmd("wic create wictestdisk " "--image-name=core-image-minimal " - "-c wrong -o %s" % self.resultdir, + "-c wrong -o %s -v %s" % (self.resultdir, self.envfile), ignore_status=True).status) def test_debug_short(self): """Test -D option""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " - "-D -o %s" % self.resultdir) + "-D -o %s -v %s" % (self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "tmp.wic*")))) def test_debug_long(self): """Test --debug option""" runCmd("wic create wictestdisk " + "--vars %s " "--image-name=core-image-minimal " - "--debug -o %s" % self.resultdir) + "--debug -o %s" % (self.envfile, self.resultdir)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "tmp.wic*")))) @@ -430,30 +441,32 @@ class Wic(WicTestCase): """Test -s option""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " - "-s -o %s" % self.resultdir) + "-s -o %s -v %s" % (self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) def test_skip_build_check_long(self): """Test --skip-build-check option""" runCmd("wic create wictestdisk " + "--vars %s " "--image-name=core-image-minimal " "--skip-build-check " - "--outdir %s" % self.resultdir) + "--outdir %s" % (self.envfile, self.resultdir)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) def test_build_rootfs_short(self): """Test -f option""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " - "-f -o %s" % self.resultdir) + "-f -o %s -v %s" % (self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) def test_build_rootfs_long(self): """Test --build-rootfs option""" runCmd("wic create wictestdisk " + "--vars %s " "--image-name=core-image-minimal " "--build-rootfs " - "--outdir %s" % self.resultdir) + "--outdir %s" % (self.envfile, self.resultdir)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) # TODO this doesn't have to be x86-specific @@ -461,10 +474,11 @@ class Wic(WicTestCase): def test_rootfs_indirect_recipes(self): """Test usage of rootfs plugin with rootfs recipes""" runCmd("wic create directdisk-multi-rootfs " + "--vars %s " "--image-name=core-image-minimal " "--rootfs rootfs1=core-image-minimal " "--rootfs rootfs2=core-image-minimal " - "--outdir %s" % self.resultdir) + "--outdir %s" % (self.envfile, self.resultdir)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-multi-rootfs*.direct")))) # TODO this doesn't have to be x86-specific @@ -478,7 +492,9 @@ class Wic(WicTestCase): bbvars = {key.lower(): value for key, value in bb_vars.items()} bbvars['wks'] = "directdisk-multi-rootfs" bbvars['resultdir'] = self.resultdir + bbvars['envfile'] = self.envfile runCmd("wic create %(wks)s " + "--vars %(envfile)s " "--bootimg-dir=%(staging_datadir)s " "--kernel-dir=%(deploy_dir_image)s " "--native-sysroot=%(recipe_sysroot_native)s " @@ -503,8 +519,8 @@ part /usr --source rootfs --ondisk mmcblk0 --fstype=ext4 --rootfs-dir %s/usr part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --rootfs-dir %s/usr part /mnt --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/whoami --rootfs-dir %s/usr""" % (rootfs_dir, rootfs_dir, rootfs_dir)) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) os.remove(wks_file) wicout = glob(os.path.join(self.resultdir, "%s-*direct" % 'temp')) @@ -641,8 +657,8 @@ part /mnt --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/whoa part /part1 --source rootfs --ondisk mmcblk0 --fstype=ext4 part /part2 --source rootfs --ondisk mmcblk0 --fstype=ext4 --include-path %s""" % (include_path)) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] part2 = glob(os.path.join(self.resultdir, 'temp-*.direct.p2'))[0] @@ -678,8 +694,8 @@ part /part2 --source rootfs --ondisk mmcblk0 --fstype=ext4 --include-path %s""" wks.write(""" part / --source rootfs --fstype=ext4 --include-path %s --include-path core-image-minimal-mtdutils export/""" % (include_path)) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] @@ -703,22 +719,22 @@ part / --source rootfs --fstype=ext4 --include-path %s --include-path core-imag # Absolute argument. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils /export") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) # Argument pointing to parent directory. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils ././..") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) # 3 Argument pointing to parent directory. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ dummy") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) def test_exclude_path_errors(self): @@ -728,15 +744,15 @@ part / --source rootfs --fstype=ext4 --include-path %s --include-path core-imag # Absolute argument. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path /usr") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) # Argument pointing to parent directory. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path ././..") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) def test_permissions(self): @@ -771,8 +787,8 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc wks_file = os.path.join(include_path, 'temp.wks') with open(wks_file, 'w') as wks: wks.write(test) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) for part in glob(os.path.join(self.resultdir, 'temp-*.direct.p*')): res = runCmd("debugfs -R 'ls -p' %s" % (part), stderr=subprocess.PIPE) @@ -808,8 +824,8 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc wks_file = os.path.join(include_path, 'temp.wks') with open(wks_file, 'w') as wks: wks.write("part /etc --source rootfs --fstype=ext4 --change-directory=etc") - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] @@ -827,8 +843,8 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc # Absolute argument. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --fstype=ext4 --change-directory /usr") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) # Argument pointing to parent directory. @@ -858,8 +874,8 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc wks.writelines(['part / --source rootfs --fstype=ext4 --label rootfs\n', 'part /mnt/p2 --source rootfs --rootfs-dir=core-image-minimal ', '--fstype=ext4 --label p2 --no-fstab-update\n']) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) part_fstab_md5sum = [] for i in range(1, 3): @@ -884,15 +900,15 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc # Absolute argument. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --fstype=ext4 --no-fstab-update /etc") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) # Argument pointing to parent directory. with open(wks_file, 'w') as wks: wks.write("part / --source rootfs --fstype=ext4 --no-fstab-update ././..") - self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir), ignore_status=True).status) + self.assertNotEqual(0, runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile), ignore_status=True).status) os.remove(wks_file) def test_extra_space(self): @@ -900,7 +916,7 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc extraspace = 1024**3 runCmd("wic create wictestdisk " "--image-name core-image-minimal " - "--extra-space %i -o %s" % (extraspace ,self.resultdir)) + "--extra-space %i -o %s -v %s" % (extraspace ,self.resultdir, self.envfile)) wicout = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(wicout)) size = os.path.getsize(wicout[0]) @@ -913,7 +929,7 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc # Absolute argument. with open(wks_file, 'w') as wks: wks.write("part testspace --no-table --fixed-size 16k --offset 4080k") - runCmd("wic create %s --image-name core-image-minimal -o %s" % (wks_file, self.resultdir)) + runCmd("wic create %s --image-name core-image-minimal -o %s -v %s" % (wks_file, self.resultdir, self.envfile)) wicout = glob(os.path.join(self.resultdir, "*.*")) @@ -934,8 +950,8 @@ part / --source rootfs --fstype=ext4 part / --source rootfs --fstype=ext4 --hidden bootloader --ptable gpt""") - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) + runCmd("wic create %s -e core-image-minimal -o %s -v %s" \ + % (wks_file, self.resultdir, self.envfile)) wicout = os.path.join(self.resultdir, "*.direct") result = runCmd("%s/usr/sbin/sfdisk --part-attrs %s 1" % (sysroot, wicout)) @@ -961,7 +977,7 @@ bootloader --ptable gpt""") 'part --fstype ext4 --source rootfs --label rofs-a\n', 'part --fstype ext4 --source rootfs --use-uuid\n']) wks.flush() - cmd = "wic create %s -e core-image-minimal -o %s --sector-size 4096" % (wks.name, self.resultdir) + cmd = "wic create %s -e core-image-minimal -o %s --sector-size 4096 -v %s" % (wks.name, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] images = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) From patchwork Mon Feb 23 21:50:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 81657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C796EEC292 for ; Mon, 23 Feb 2026 21:50:45 +0000 (UTC) Received: from mail-qv1-f65.google.com (mail-qv1-f65.google.com [209.85.219.65]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6367.1771883437425134723 for ; Mon, 23 Feb 2026 13:50:37 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Pfm8e/38; spf=pass (domain: gmail.com, ip: 209.85.219.65, mailfrom: twoerner@gmail.com) Received: by mail-qv1-f65.google.com with SMTP id 6a1803df08f44-8947404b367so56349246d6.3 for ; Mon, 23 Feb 2026 13:50:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1771883436; x=1772488236; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=MaF9303L4M3TErsjrEnhkCN66JogbFNs00A98nfrq9I=; b=Pfm8e/38OZ/KDGhQN+P+33NSU/FJeah7sjJY4JpZebOV/Ii48dacVeRWh8ePrqTiFN tr7G7jk4Q8sKaSnPu1foqjgtXOhSJ8gG1fAbQ+N9qNnPecIkRCXMAejjpY38jQ8M1CcF +kRm4EOBJEXPKApVVBvmV41QgtICSeB07YcF5uMD+ljiNchX8Ia7Uy7wrXvh3bNbCl6U czK/23ddDQ/VWpFSMmR/3noWNPTiOIDxjgMUzCFL02u0N624Pc/oMQozHouzF8sisMo8 Po/NjFzOoRi8gYsRClku/kLQqaUbCG8yOawO9YL4VFAVUYG0CxKrfBfo2WusAxIaqBLm 51UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771883436; x=1772488236; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MaF9303L4M3TErsjrEnhkCN66JogbFNs00A98nfrq9I=; b=NqKN9v9sPp4rWjELGwbC8c3bs3R1tmEYGzIsofjFv/QOJ3V5nDw/8JsxounsTukxZc Nw97guNawqDutZAAih67A0KwdoH5ZVaAHIS5nRi3dEsny00IfSkQ+MmDmt78N32CEKx0 JVAoYqdvfmJiY29XFT+Glm+XGZX9iq3z7brvEG1XUxRJ2cNGqSRgXTp7A8QaP7k799SQ IopLmM9JEo5FV37yggC42XXh/uQLfpNpsWj39zeLuscH5G3PFH+IYu9OJobkNtY1XngK 1Z2xdsKqvr1kasxUDaCNYH3q9iFcIVyhYhQpZSOKZhi/IHwrHO1UNi0ye0Ia5hhxxvM5 mHCA== X-Gm-Message-State: AOJu0YyxesV+L/IGdi+fFg8ffq3h6lNp8LHl6xxCEvcpsafjGNNKppdO bK/n8IV6GBXRXhraugKYr7put+CSzluJr/oa3CcL1S1j8MDr7CH/hLiInphaUbgUf90= X-Gm-Gg: ATEYQzzdcP5YhOhW1DpwKeIvC87/7yRWmT4e20EV+roW/hmxu1P3IZPYkMY08Rf+pHB 6H+SpsJ4yNJQp2P4XPzxyAY/cLyU4MjAfSae9lPuJ2B7sk8GRo6y4z5UPVH2wsLQ6/h3YGSD8wI wZ5Zh/ZUDU47ncpOmnM5U2BpUDYu+NVr4O7l6qqcE/Io5ew5RzehhMQXusIv7oKlOySMHD7PPcZ mFFriBjwrKaBOUpdiOrHoeNPJnli76DhRq135esg/odHlG0uvIxtqcl4b99Tv6sh9Zw040RblDj KkumSJs8E3p8/P9jJnuiB0cD2oBIl5ZvWAagTSg7Esv4bnlppRy08namAI5urvt5aMS4jWSFWFR Jx21LS3j02kb7F7HrctofCDYRwwjv423CPDlh7AZN+yGRFTGay8OiSzAT5CNSpQGBD/T8StgzuH 2Ktx6NF+ejnauMKKOVxWPOeDWDjjFHM0QlGSTI8z1BmLi+2IYFsT8Opnvo7SZL/yTCAg== X-Received: by 2002:a05:6214:27c9:b0:894:8211:b025 with SMTP id 6a1803df08f44-89979c71ac0mr157124656d6.18.1771883435749; Mon, 23 Feb 2026 13:50:35 -0800 (PST) Received: from localhost.localdomain (pppoe-209-91-167-254.vianet.ca. [209.91.167.254]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8997c691113sm93811606d6.1.2026.02.23.13.50.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 13:50:35 -0800 (PST) From: Trevor Woerner To: openembedded-core@lists.openembedded.org Subject: [PATCH v5 10/10] oe-selftest/cases/wic.py: oe-selftest -r wic.Wic2 -> PASS Date: Mon, 23 Feb 2026 16:50:08 -0500 Message-ID: <20260223215008.2062721-11-twoerner@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260223215008.2062721-1-twoerner@gmail.com> References: <20260223215008.2062721-1-twoerner@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 23 Feb 2026 21:50:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/231711 Changes for the wic.Wic2 oe-selftests to pass now that wic is a stand-alone app and no longer part of oe-core. NOTE: the wic.Wic2.test_sparse_copy test was deleted since it tests code that is internal to wic itself; which is no longer possible now that wic is not part of oe-core. Signed-off-by: Trevor Woerner --- changes in v5: - rebase with master - re-split up into multiple, smaller patches - now that the sector-size re-implementation is ahead of this patch, various wic subcommands (ls, cp, rm) no longer need a --vars file - combine qemu and non-qemu patches together (were separate patches) v4: skipped to align/combine with other patch set changes in v3: - squashed into 1 large commit changes in v2: - none --- meta/lib/oeqa/selftest/cases/wic.py | 70 ++++++++++------------------- 1 file changed, 24 insertions(+), 46 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 355c680f80ba..ea1742dab110 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -1005,14 +1005,14 @@ class Wic2(WicTestCase): def test_bmap_short(self): """Test generation of .bmap file -m option""" - cmd = "wic create wictestdisk -e core-image-minimal -m -o %s" % self.resultdir + cmd = "wic create wictestdisk -e core-image-minimal -m -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct.bmap")))) def test_bmap_long(self): """Test generation of .bmap file --bmap option""" - cmd = "wic create wictestdisk -e core-image-minimal --bmap -o %s" % self.resultdir + cmd = "wic create wictestdisk -e core-image-minimal --bmap -o %s -v %s" % (self.resultdir, self.envfile) runCmd(cmd) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct.bmap")))) @@ -1048,9 +1048,9 @@ class Wic2(WicTestCase): native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools") runCmd("wic create wictestdisk " - "--image-name=%s -v %s -n %s -o %s" + "--image-name=%s -v %s -n %s -o %s -v %s" % (image, imgenvdir, native_sysroot, - self.resultdir)) + self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) def test_image_vars_dir_long(self): @@ -1063,9 +1063,10 @@ class Wic2(WicTestCase): "--image-name=%s " "--vars %s " "--native-sysroot %s " - "--outdir %s" + "--outdir %s " + "--vars %s" % (image, imgenvdir, native_sysroot, - self.resultdir)) + self.resultdir, self.envfile)) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) # TODO this test could also work on aarch64 @@ -1075,6 +1076,7 @@ class Wic2(WicTestCase): config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "wic-image-minimal"\n'\ 'MACHINE_FEATURES:append = " efi"\n' image_recipe_append = """ +DEPENDS:append = " wic-native" do_image_wic[postfuncs] += "run_wic_cmd" run_wic_cmd() { echo "test" >> ${WORKDIR}/test.wic-cp @@ -1115,6 +1117,7 @@ run_wic_cmd() { 'MACHINE_FEATURES:append = " efi"\n' self.append_config(config) image_recipe_append = """ +DEPENDS:append = " wic-native" do_image_wic[postfuncs] += "run_wic_cmd" run_wic_cmd() { echo "test" >> ${WORKDIR}/test.wic-cp @@ -1179,7 +1182,7 @@ run_wic_cmd() { return wkspath def _get_wic(self, wkspath, ignore_status=False): - p = runCmd("wic create %s -e core-image-minimal -o %s" % (wkspath, self.resultdir), + p = runCmd("wic create %s -e core-image-minimal -o %s -v %s" % (wkspath, self.resultdir, self.envfile), ignore_status=ignore_status) if p.status: @@ -1420,7 +1423,7 @@ run_wic_cmd() { wks.write('part / --source rawcopy --sourceparams="file=%s.%s%s"\n'\ % (bb_vars['IMAGE_LINK_NAME'], fstype, params)) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, image, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, image, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -1513,7 +1516,7 @@ run_wic_cmd() { 'part / --source rootfs --fstype=ext4 --align 1024 --use-uuid\n'\ 'bootloader --timeout=0 --append="console=ttyS0,115200n8"\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*.direct" % wksname)) @@ -1525,6 +1528,7 @@ run_wic_cmd() { config = 'IMAGE_EFI_BOOT_FILES="/etc/fstab;testfile"\nIMAGE_FSTYPES = "wic"\nWKS_FILE = "test_uefikernel.wks"\nMACHINE_FEATURES:append = " efi"\n' self.append_config(config) bitbake('core-image-minimal') + envfile = self._create_image_env_file('core-image-minimal') self.remove_config(config) img = 'core-image-minimal' @@ -1533,7 +1537,7 @@ run_wic_cmd() { 'part / --source rootfs --fstype=ext4 --align 1024 --use-uuid\n'\ 'bootloader --timeout=0 --append="console=ttyS0,115200n8"\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*.direct" % wksname)) @@ -1678,6 +1682,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" " """) self.append_config(config) + self.envfile = self._create_image_env_file('core-image-minimal') deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1748,7 +1753,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" 'part emptyext2 --fstype ext2 --size 1M\n', 'part emptybtrfs --fstype btrfs --size 150M\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -1760,7 +1765,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" wks.writelines(['part / --fstype ext3 --source rootfs --system-id 0xFF '\ '--overhead-factor 1.2 --size 100k\n']) wks.flush() - cmd = "wic create %s -e core-image-minimal -o %s" % (wks.name, self.resultdir) + cmd = "wic create %s -e core-image-minimal -o %s -v %s" % (wks.name, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -1769,41 +1774,14 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" def test_image_bootpart_globbed(self): """Test globbed sources with image-bootpart plugin""" img = "core-image-minimal" - cmd = "wic create sdimage-bootpart -e %s -o %s" % (img, self.resultdir) config = 'IMAGE_BOOT_FILES = "%s*"' % get_bb_var('KERNEL_IMAGETYPE', img) self.append_config(config) + envfile = self._create_image_env_file(img) + cmd = "wic create sdimage-bootpart -e %s -o %s -v %s" % (img, self.resultdir, envfile) runCmd(cmd) self.remove_config(config) self.assertEqual(1, len(glob(os.path.join(self.resultdir, "sdimage-bootpart-*direct")))) - def test_sparse_copy(self): - """Test sparse_copy with FIEMAP and SEEK_HOLE filemap APIs""" - libpath = os.path.join(self.td['COREBASE'], 'scripts', 'lib', 'wic') - sys.path.insert(0, libpath) - from filemap import FilemapFiemap, FilemapSeek, sparse_copy, ErrorNotSupp - with NamedTemporaryFile("w", suffix=".wic-sparse") as sparse: - src_name = sparse.name - src_size = 1024 * 10 - sparse.truncate(src_size) - # write one byte to the file - with open(src_name, 'r+b') as sfile: - sfile.seek(1024 * 4) - sfile.write(b'\x00') - dest = sparse.name + '.out' - # copy src file to dest using different filemap APIs - for api in (FilemapFiemap, FilemapSeek, None): - if os.path.exists(dest): - os.unlink(dest) - try: - sparse_copy(sparse.name, dest, api=api) - except ErrorNotSupp: - continue # skip unsupported API - dest_stat = os.stat(dest) - self.assertEqual(dest_stat.st_size, src_size) - # 8 blocks is 4K (physical sector size) - self.assertEqual(dest_stat.st_blocks, 8) - os.unlink(dest) - def test_mkfs_extraopts(self): """Test wks option --mkfs-extraopts for empty and not empty partitions""" img = 'core-image-minimal' @@ -1817,7 +1795,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" 'part emptyext2 --fstype ext2 --size 1M --mkfs-extraopts "-D -F -i 8192"\n', 'part emptybtrfs --fstype btrfs --size 100M --mkfs-extraopts "--mixed -K"\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -1916,7 +1894,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" 'part empty --source empty --sourceparams="size=2048k,bs=512K" --ondisk sda --size 4M --align 1024\n' ]) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] wicout = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -1934,7 +1912,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" with NamedTemporaryFile("w", suffix=".wks") as wks: wks.writelines(['part empty --source empty --sourceparams="fill" --ondisk sda --size 1M\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) result = runCmd(cmd, ignore_status=True) self.assertIn("Source parameter 'fill' only works with the '--fixed-size' option, exiting.", result.output) self.assertNotEqual(0, result.status) @@ -1948,7 +1926,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" 'part /boot --size=100M --active --fstype=ext4 --label boot\n' 'part / --source rootfs --fstype=ext4 --label root\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) @@ -1966,7 +1944,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" 'part /boot --size=100M --active --fstype=ext4 --label boot\n' 'part / --source rootfs --fstype=ext4 --label root\n']) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s -v %s" % (wks.name, img, self.resultdir, self.envfile) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname))