From patchwork Fri Jul 25 12:38:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: lamine.rehahlia@smile.fr X-Patchwork-Id: 67465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4708AC87FCE for ; Fri, 25 Jul 2025 12:38:35 +0000 (UTC) Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by mx.groups.io with SMTP id smtpd.web10.18200.1753447111526800283 for ; Fri, 25 Jul 2025 05:38:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile.fr header.s=google header.b=fLG/ZDls; spf=pass (domain: smile.fr, ip: 209.85.128.44, mailfrom: lamine.rehahlia@smile.fr) Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-45600581226so21572625e9.1 for ; Fri, 25 Jul 2025 05:38:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile.fr; s=google; t=1753447110; x=1754051910; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iXe9DPa2glyuDTg7v+UhHd1ZvzxOppZV9Hu/BcfHeKc=; b=fLG/ZDls4tBL4fNH2L8AL8QYE/qm5DMM2Da0UAbURzDZ6ufq1Shb0/BGO31d3DX4gQ vkLOI6HGA+MTXKsejV0lGq/zPF8Z4sTfWuwGBFvfRJV206TjfbnzCZQC414riPb2rbI8 tsYBRPGNvpRmaHbyaN6+lgmmpUtqTEa12yZN0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753447110; x=1754051910; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iXe9DPa2glyuDTg7v+UhHd1ZvzxOppZV9Hu/BcfHeKc=; b=l4YauRyuwzN021jdldZ1lthh7i/5IViSkCoH8ztfuYjv42ga6U8nzQjrPBQXy0vr/4 7uKWL/woRepgzNbB1KDHikeqrIeddIlvJ2oRsKxVwCmj0k/jV4jn20HG013K9I+qegVa NF3uwiHI/KctlXKbrD+ed2MBPMM0HRWSR37OcymfwdBn5AKAV4xTT3lpeTU5Ny6lCDCA rdHvFk+qdh0jpQY1kERInBDMvjWoibUON6cirWuNw8B/yW5vHXT+BKaHzgj8zh8DFwCn VuF+6GCyR7tcR7D1L0YeRL1KM4cT24pexJKYl/8zNfnehLe6kDMs0ESDy6BsWbxzizQm +HVQ== X-Gm-Message-State: AOJu0YxelCUnHRUgwGZCtVmuOJWmylNtJ/MzChCglubPk3eP5G7FE5fS s5v+6ZU16E70oZ7bCFGSuwvY1nty7TIXssViLdF2bFy0gdqFZGNeO01q+3cVYHxpi7F4hS0P1Ez JtNAa7g== X-Gm-Gg: ASbGnctk1rN+W0RMBFbYyRcM+iG2TyNmxcKb+0jhu5xlibxjaEAOhG39eu08JkAcVHb HAvaj6dNqgKtqKqHPdHn/Jr0x3izNAU1ps4Xg7BRY4cbJVlAhb0lrA7/gd7JVrOVH1tUrAL+H8e ET3nlwDZtxBJcOI4g1EbN1rOcjF7d1hk8krVRr0lGDaio4y4Hw0EH1nWoR/63hQcla9Z69ZSUuP LGCY3AeJ/ergszYrAPHrlGPc3HBow0yLjSA4ByaQVQd67SanPGiTCjHtKmNUtjhSGAkB/oPF/+i 0AxbQBeoMoH+B2FkTjZvRNYqLKwR2B4ZJjq96DZQpm9TDc2ZcFcysg2Y4Vr+ZTvCBvZX2y6Ir6s FQXlKbddCxx3Vcc71OOFz+9mBnGl6NuDi X-Google-Smtp-Source: AGHT+IEkAVVacHV6KUfACzMWyiU15dmvjqqPQ/9OMlgmAtKP1erqRuMfNThCLIrxb7KRF89KH1iRzw== X-Received: by 2002:a05:600c:a00b:b0:456:1ac8:cace with SMTP id 5b1f17b1804b1-458763184demr16656375e9.12.1753447109438; Fri, 25 Jul 2025 05:38:29 -0700 (PDT) Received: from FRSMI24-GAIN.. ([2a01:e0a:98e:c1d0:8bd1:eafc:f0ae:9e4f]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b76fcad7fdsm5080669f8f.43.2025.07.25.05.38.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 05:38:29 -0700 (PDT) From: lamine.rehahlia@smile.fr To: openembedded-core@lists.openembedded.org Cc: Lamine REHAHLIA Subject: [PATCH 2/2] runqemu: Add support for running compressed .zst rootfs images Date: Fri, 25 Jul 2025 14:38:11 +0200 Message-ID: <20250725123811.1718097-2-lamine.rehahlia@smile.fr> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250725123811.1718097-1-lamine.rehahlia@smile.fr> References: <20250725123811.1718097-1-lamine.rehahlia@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 25 Jul 2025 12:38:35 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/220909 From: Lamine REHAHLIA Enhance runqemu to detect and decompress .zst-compressed rootfs images (e.g. ext4.zst, wic.zst) automatically. If a decompressed image already exists in the original directory, it will be reused to avoid overwriting build artifacts. Otherwise, the image is decompressed and removed after the QEMU session ends. This allows runqemu to be used seamlessly with compressed image formats generated by the build system or during releases. IMPORTANT: This patch assumes that the original directory of the .zst-compressed image is writable. If, for some reason, the path passed from CI or another system to the script is read-only, the decompression step will fail when trying to write the uncompressed image to the same directory. Signed-off-by: Lamine REHAHLIA --- scripts/runqemu | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/scripts/runqemu b/scripts/runqemu index 6c263d6b2a..b6af619f20 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -376,7 +376,42 @@ class BaseConfig(object): re.search('fitImage', p) or re.search('uImage', p): self.kernel = p elif os.path.isfile(p) and ('-image-' in os.path.basename(p) or '.rootfs.' in os.path.basename(p)): - self.rootfs = p + + # Decompress ZST image if needed + if p.endswith('.zst'): + # Get the real path to the image to avoid issues when a symbolic link is passed. + # This ensures we always operate on the actual file. + image_path = os.path.realpath(p) + # Extract target filename by removing .zst + image_dir = os.path.dirname(image_path) + uncompressed_name = os.path.basename(image_path).replace(".zst", "") + uncompressed_path = os.path.join(image_dir, uncompressed_name) + + # If the decompressed image already exists (e.g., in the deploy directory), + # we use it directly to avoid overwriting artifacts generated by the build system. + # This prevents redundant decompression and preserves build outputs. + if os.path.exists(uncompressed_path): + logger.warning(f"Found existing decompressed image: {uncompressed_path}, Using it directly.") + else: + logger.info(f"Decompressing {p} to {uncompressed_path}") + # Ensure the 'zstd' tool is installed before attempting to decompress. + if not shutil.which('zstd'): + raise RunQemuError(f"'zstd' is required to decompress {p} but was not found in PATH") + try: + with open(uncompressed_path, 'wb') as out_file: + subprocess.check_call(['zstd', '-d', '-c', image_path], stdout=out_file) + except subprocess.CalledProcessError as e: + self.cleanup_files.append(uncompressed_path) + raise RunQemuError(f"Failed to decompress {p}: {e}") + + # Mark for deletion at the end + self.cleanup_files.append(uncompressed_path) + + # Use the decompressed image as the rootfs + self.rootfs = uncompressed_path + + else: + self.rootfs = p # Check filename against self.fstypes can handle .cpio.gz, # otherwise, its type would be "gz", which is incorrect. fst = ""