From patchwork Wed Sep 25 21:37:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brennan Coslett X-Patchwork-Id: 49616 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5891CCF9FA for ; Wed, 25 Sep 2024 21:38:01 +0000 (UTC) Subject: URL encoding/decoding bug with files like "serialgetty@.service" or "sshd@.service" with archiver.bbclass #bitbake #kirkstone #poky To: bitbake-devel@lists.openembedded.org From: "Brennan Coslett" X-Originating-Location: Denver, Colorado, US (165.225.11.61) X-Originating-Platform: Linux Chrome 129 User-Agent: GROUPS.IO Web Poster MIME-Version: 1.0 Date: Wed, 25 Sep 2024 14:37:58 -0700 Message-ID: List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 25 Sep 2024 21:38:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16602 Hey All, I believe there is a bug with how bitbake handles systemd template files when archiving source, as the files are ending up in the rootfs. I was working on getting a script working to package up the tmp/deploy/source directory and noticed that while the archiver was creating tmp/deploy/sources/aarch64-shasta-linux/systemd-serialgetty-1.0-r5/ there wasn't anything inside it. I think this is the base case for the bug as it has a SRC_URI of "file://serial-getty@.service" vs some of the other packages which include these template systemd files that end in @.service contain a ton of other files (I verified the same issue shows up with openssh). I assume that it will happen with all of these files: ``` $ find layers/ meta-* -name '*@*' layers/meta-openembedded/meta-networking/recipes-support/nuttcp/nuttcp/nuttcp@.service layers/meta-openembedded/meta-networking/recipes-support/openvpn/openvpn/openvpn@.service layers/poky/meta/recipes-connectivity/openssh/openssh/sshd@.service layers/poky/meta/recipes-connectivity/dhcpcd/files/dhcpcd@.service layers/poky/meta/recipes-connectivity/ppp/ppp/ppp@.service layers/poky/meta/recipes-core/dropbear/dropbear/dropbear@.service layers/poky/meta/recipes-core/systemd/systemd-serialgetty/serial-getty@.service ``` I was able to add some additional debug prints in archiver.bbclass and tried to dig further but the insides of `fetch2` were a bit over my head. ``` ``` This was tested with kirkstone, (this commit specifically e938b18b5342bd28eadb44ad39dbf1f5cf5be09b): ``` DEBUG: systemd-serialgetty-1.0-r5 do_ar_original: BRENNAN full SRC_URI ['file://serial-getty@.service'] DEBUG: systemd-serialgetty-1.0-r5 do_ar_original: BRENNAN ['file://.service'] ``` It then can't find that file and so then the archiver doesn't package anything up. Hopefully bitbake-devel is the right mailing list for folks with more knowledge of this stuff? Thanks, Brennan Coslett diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass index 4a5865d7b5..daf7b4605b 100644 --- a/meta/classes/archiver.bbclass +++ b/meta/classes/archiver.bbclass @@ -181,6 +181,7 @@ python do_ar_original() { ar_outdir = d.getVar('ARCHIVER_OUTDIR') bb.note('Archiving the original source...') urls = d.getVar("SRC_URI").split() +    bb.debug(1, "BRENNAN full SRC_URI %s" % urls) # destsuffix (git fetcher) and subdir (everything else) are allowed to be # absolute paths (for example, destsuffix=${S}/foobar). # That messes with unpacking inside our tmpdir below, because the fetchers @@ -198,7 +199,7 @@ python do_ar_original() { del decoded[5][param] encoded = bb.fetch2.encodeurl(decoded) urls[i] = encoded - +    bb.debug(1, "BRENNAN %s" % urls) # Cleanup SRC_URI before call bb.fetch2.Fetch() since now SRC_URI is in the # variable "urls", otherwise there might be errors like: # The SRCREV_FORMAT variable must be set when multiple SCMs are used