From patchwork Fri Jul 25 14:30:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: lamine.rehahlia@smile.fr X-Patchwork-Id: 67469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D04F1C87FCA for ; Fri, 25 Jul 2025 14:30:45 +0000 (UTC) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mx.groups.io with SMTP id smtpd.web11.20818.1753453838671340099 for ; Fri, 25 Jul 2025 07:30:39 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=T2MkBb+s; spf=pass (domain: smile.fr, ip: 209.85.128.43, mailfrom: lamine.rehahlia@smile.fr) Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4563cfac2d2so23317315e9.3 for ; Fri, 25 Jul 2025 07:30:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1753453837; x=1754058637; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=/AT3RgF3UX1KjAoDAwVNHIhrvt33kUjDUaNFBqYquGc=; b=T2MkBb+s2Nu4yQ5M/+xIM1tq7CnyWfmLw8Fy/II9lFhmBmDOhqBxtzj19DGIKK+jDT 3GdyIYoGBaLsxy5EPES0bcHgObTjxaIkXvrEfHjP0d4mvpH7xnatu6oYkO7KFSTppGnz /WUj2VGb7fMc9owjcPX75BfkboGaXzOG/WwqY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753453837; x=1754058637; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/AT3RgF3UX1KjAoDAwVNHIhrvt33kUjDUaNFBqYquGc=; b=m+P1oxQmVaO9Vkmcr/Hy/HplMSAr4gkiWu0aNwqUr1qjvsCfMif743XOE2/JQ/c7Zy sSVz8z9FtG/KfHCnTSBojoJpm3rbYHgVdagBaewFOBq1oO5zmnCkA4xEUt9dx4I5JO8H QTFXyZuMDi5ouQhCNF+eGqZkQ/JZQicLQUMtiFPy0ZLYn0uiHdr8yLzoLDuHkBpmRcQK GJRszeYT4SifHXAcO5iCbO9t0Xya5ZrEsWYWAzRh46OmH41DqX42gZ6gyaUgdweLx3wt K80+xRS9T3rVuWd6/bsIWdwdxlL5dHEaHa22VuwcsiPJIaANDHOlfSZWwKSVk7eFZdmU CsJA== X-Gm-Message-State: AOJu0YxsiezbMdZaPJhCop/R51Lo9NIjhv5qWJTOz9fG1lYU+2gTV6hy C7TfmjAEiiO3hpDP2K5b07Y8gCO+QWD1656QqeiysNh8/n18uVQZfJfA0P+nQ/84IICfyLPVOvv 1Xeajrg== X-Gm-Gg: ASbGncsJ4AGRqUm2eEglxvvJF2xYqPtVvjUy/su/SQwwW/plthaiyLtHjcVpBEsDRlT VLHaPV2yk7GnctPXnyV8a0GW5Zm5RtiX2hfjl+TtBfQLCbH1INGYFvTJJQ0uYUiH75ZswDjZrSs kCSy3LLrQViFuEt0U9fbPwcCPyWhriwFICyr5ld7U3tgAIA8wKj+JG9b3g/oIH0kD5OBthJttZz C1TOssIx2iORHrZyk35fa8vozf4pO2RBOzYrDTwcqfbxZp74y+MgQjoOWQJ5CsJC0BcToH46J6G bps0Ua40neKvvLzibHUpC09LGoTntWeWZ+aOU8XK5NsZu8o8JRGSU2leLjt78Y4YP4vqzBXfAH2 u6LBmIbf91HXCC5g0GsKtVwJzYi2Pm37M X-Google-Smtp-Source: AGHT+IGbAzdXlTNer5LOrJutcl0K55GXiWpuQOSGjsGeEyF3xoCLYjTMN9wnUU1DbGpjWwGm0MK9Eg== X-Received: by 2002:a05:600c:a0c:b0:456:26ad:46d2 with SMTP id 5b1f17b1804b1-458762fb401mr19123635e9.6.1753453836752; Fri, 25 Jul 2025 07:30:36 -0700 (PDT) Received: from FRSMI24-GAIN.. ([2a01:e0a:98e:c1d0:8bd1:eafc:f0ae:9e4f]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-458705376d2sm55692225e9.3.2025.07.25.07.30.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 07:30:36 -0700 (PDT) From: lamine.rehahlia@smile.fr To: openembedded-core@lists.openembedded.org Cc: Lamine REHAHLIA Subject: [PATCH v2] runqemu: Add support for running compressed .zst rootfs images Date: Fri, 25 Jul 2025 16:30:19 +0200 Message-ID: <20250725143019.2189728-1-lamine.rehahlia@smile.fr> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 25 Jul 2025 14:30:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/220914 From: Lamine REHAHLIA Enhance runqemu to detect and decompress .zst-compressed rootfs images (e.g. ext4.zst, wic.zst) automatically. If a decompressed image already exists in the original directory, it will be reused to avoid overwriting build artifacts. Otherwise, the image is decompressed and removed after the QEMU session ends. This allows runqemu to be used seamlessly with compressed image formats generated by the build system or during releases. Note: support for .zst images is only available when snapshot mode is enabled IMPORTANT: This patch assumes that the original directory of the .zst-compressed image is writable. If, for some reason, the path passed from CI or another system to the script is read-only, the decompression step will fail when trying to write the uncompressed image to the same directory. Signed-off-by: Lamine REHAHLIA --- scripts/runqemu | 40 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/scripts/runqemu b/scripts/runqemu index 3d77046972..30954ba332 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -376,7 +376,45 @@ class BaseConfig(object): re.search('fitImage', p) or re.search('uImage', p): self.kernel = p elif os.path.isfile(p) and ('-image-' in os.path.basename(p) or '.rootfs.' in os.path.basename(p)): - self.rootfs = p + + # Decompress ZST image if needed + if p.endswith('.zst'): + # Ensure snapshot mode is active before allowing decompression. + if not self.snapshot: + raise RunQemuError(".zst images are only supported when snapshot mode is enabled. ") + # Get the real path to the image to avoid issues when a symbolic link is passed. + # This ensures we always operate on the actual file. + image_path = os.path.realpath(p) + # Extract target filename by removing .zst + image_dir = os.path.dirname(image_path) + uncompressed_name = os.path.basename(image_path).replace(".zst", "") + uncompressed_path = os.path.join(image_dir, uncompressed_name) + + # If the decompressed image already exists (e.g., in the deploy directory), + # we use it directly to avoid overwriting artifacts generated by the build system. + # This prevents redundant decompression and preserves build outputs. + if os.path.exists(uncompressed_path): + logger.warning(f"Found existing decompressed image: {uncompressed_path}, Using it directly.") + else: + logger.info(f"Decompressing {p} to {uncompressed_path}") + # Ensure the 'zstd' tool is installed before attempting to decompress. + if not shutil.which('zstd'): + raise RunQemuError(f"'zstd' is required to decompress {p} but was not found in PATH") + try: + with open(uncompressed_path, 'wb') as out_file: + subprocess.check_call(['zstd', '-d', '-c', image_path], stdout=out_file) + except subprocess.CalledProcessError as e: + self.cleanup_files.append(uncompressed_path) + raise RunQemuError(f"Failed to decompress {p}: {e}") + + # Mark for deletion at the end + self.cleanup_files.append(uncompressed_path) + + # Use the decompressed image as the rootfs + self.rootfs = uncompressed_path + + else: + self.rootfs = p # Check filename against self.fstypes can handle .cpio.gz, # otherwise, its type would be "gz", which is incorrect. fst = ""