From patchwork Thu Jul 4 16:23:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enguerrand de Ribaucourt X-Patchwork-Id: 46040 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 167CDC30653 for ; Thu, 4 Jul 2024 16:23:48 +0000 (UTC) Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by mx.groups.io with SMTP id smtpd.web11.14167.1720110221618473113 for ; Thu, 04 Jul 2024 09:23:42 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b=lPdbrbDx; spf=pass (domain: savoirfairelinux.com, ip: 208.88.110.44, mailfrom: enguerrand.de-ribaucourt@savoirfairelinux.com) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 876FF9C5763; Thu, 4 Jul 2024 12:23:40 -0400 (EDT) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10032) with ESMTP id HE7lWiMZ7XjZ; Thu, 4 Jul 2024 12:23:39 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id AF5B59C5794; Thu, 4 Jul 2024 12:23:39 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.savoirfairelinux.com AF5B59C5794 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=savoirfairelinux.com; s=DFC430D2-D198-11EC-948E-34200CB392D2; t=1720110219; bh=j/U0qvHgxP65CxULR8l/S9mx7FiuqQQE6DBnWuFIXMU=; h=From:To:Date:Message-Id:MIME-Version; b=lPdbrbDxKZ5rlVKNRSpL0bDt5VKHZPQzv/kILSGyW+l5OdT4LATzh0HI4RZnOGJNw rZWofFL6GP+m02s1qByl9NkIOFzVrlMTxEke9TDEFJMJIE9i0/gqcGMECotPBV68cb yH4fKy4gtqdMhNSdsDOWQReakDlOGmcQwyy+8wtEYE6X1BT0ErdxqrBoF4bJJM/wwW aMtNF0o9rmJre2TuGmFGFMKyXmfVn2GQ3tx2sSWL7Vq2DLLGaL2uS6/3FmKc10BO3K f08Hnk1yncnjmic8hXVHigWtGox+aQAh6WTcIqSOhegtD07s/NxVCw37Bgbkx4di2a QMPqjwDr7nQXQ== X-Virus-Scanned: amavis at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10026) with ESMTP id Z8adDCX5SWie; Thu, 4 Jul 2024 12:23:39 -0400 (EDT) Received: from sfl-deribaucourt.rennes.sfl (80-15-101-118.ftth.fr.orangecustomers.net [80.15.101.118]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id E73119C5763; Thu, 4 Jul 2024 12:23:38 -0400 (EDT) From: Enguerrand de Ribaucourt To: bitbake-devel@lists.openembedded.org Cc: tanguy.raufflet@savoirfairelinux.com, richard.purdie@linuxfoundation.org, Enguerrand de Ribaucourt Subject: [PATCH 1/2] bitbake: fetch2: npmsw: fix fetching git revisions not on master Date: Thu, 4 Jul 2024 18:23:32 +0200 Message-Id: <20240704162333.1497107-1-enguerrand.de-ribaucourt@savoirfairelinux.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 04 Jul 2024 16:23:48 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16389 The NPM package.json documentation[1] states that git URLs may contain a commit-ish suffix to specify a specific revision. When running `npm install`, this revision will be looked for on any branch of the repository. The bitbake implementation however translates the URL stored in package.json into a git URL to be fetch by the bitbake git fetcher. The bitbake fetcher git.py, enforces the branch to be master by default. If the revision specified in the package.json is not on the master branch, the fetch will fail while the package.json is valid. To fix this, append the ";nobranch=1" suffix to the revision in the git URL to be fetched. This will make the bitbake git fetcher ignore the branch and respect the behavior of `npm install``. This can be tested with the following command: $ devtool add --npm-dev https://github.com/seapath/cockpit-cluster-dashboard.git -B version Which points to a project which has a package.json with a git URL: ```json "devDependencies": { "cockpit-repo": "git+https://github.com/cockpit-project/cockpit.git#d34cabacb8e5e1e028c7eea3d6e3b606d862b8ac" } ``` In this repo, the specified revision is on the "main" branch, which would fail without this fix. [1] https://docs.npmjs.com/cli/v10/configuring-npm/package-json#git-urls-as-dependencies Co-authored-by: Tanguy Raufflet Signed-off-by: Tanguy Raufflet Signed-off-by: Enguerrand de Ribaucourt --- bitbake/lib/bb/fetch2/npmsw.py | 1 + 1 file changed, 1 insertion(+) diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py index ff5f8dc755..018e0ad546 100644 --- a/bitbake/lib/bb/fetch2/npmsw.py +++ b/bitbake/lib/bb/fetch2/npmsw.py @@ -184,6 +184,7 @@ class NpmShrinkWrap(FetchMethod): uri = URI("git://" + str(groups["url"])) uri.params["protocol"] = str(groups["protocol"]) uri.params["rev"] = str(groups["rev"]) + uri.params["nobranch"] = "1" uri.params["destsuffix"] = destsuffix url = str(uri) From patchwork Thu Jul 4 16:23:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enguerrand de Ribaucourt X-Patchwork-Id: 46039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17C20C41513 for ; Thu, 4 Jul 2024 16:23:48 +0000 (UTC) Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by mx.groups.io with SMTP id smtpd.web10.14230.1720110221961795309 for ; Thu, 04 Jul 2024 09:23:42 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b=ratpSoRM; spf=pass (domain: savoirfairelinux.com, ip: 208.88.110.44, mailfrom: enguerrand.de-ribaucourt@savoirfairelinux.com) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 4ABA09C57D2; Thu, 4 Jul 2024 12:23:41 -0400 (EDT) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10032) with ESMTP id jMiToq93O8H5; Thu, 4 Jul 2024 12:23:40 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 8BFA99C5C6F; Thu, 4 Jul 2024 12:23:40 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.savoirfairelinux.com 8BFA99C5C6F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=savoirfairelinux.com; s=DFC430D2-D198-11EC-948E-34200CB392D2; t=1720110220; bh=xoS5t+T7O5m1fzVdpWs5yqbgUURDaFF8MDl2PM00L9s=; h=From:To:Date:Message-Id:MIME-Version; b=ratpSoRMhKbUfv0awwjcL6U/dXQf/qRNDQ+ipba3MlCtSJfMLZAA7RGMll50a7eT3 9CWyhX+m1Mpd8wDA04FdIYwgDBv6M8LzBlEY7OjBOx4CvFbtSLRSmOgGUJJwPERH47 crUlyYo1+nnPG43SYXvlFg9RlKKvYKUFiQ7NujvgHQrnQPRwFxvNZUYRcYoQnem+Ku 64OFcTMQrzMS3Mg4FeSpklDNdPFW9pasu7rypD/ZEU9cjxt+6s+1k/Uwf06cEjHIl6 RJirzyS7zCb/PIm39RLWnbKvMcH9WC+6rTlUu4DM0gbmx5eGx5SPqIKRS9HsNpqLqX R1Hr6U+DbEaFQ== X-Virus-Scanned: amavis at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10026) with ESMTP id W8l9OCG8_prn; Thu, 4 Jul 2024 12:23:40 -0400 (EDT) Received: from sfl-deribaucourt.rennes.sfl (80-15-101-118.ftth.fr.orangecustomers.net [80.15.101.118]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id CC9769C57D2; Thu, 4 Jul 2024 12:23:39 -0400 (EDT) From: Enguerrand de Ribaucourt To: bitbake-devel@lists.openembedded.org Cc: tanguy.raufflet@savoirfairelinux.com, richard.purdie@linuxfoundation.org, Enguerrand de Ribaucourt Subject: [PATCH 2/2] npm: accept unspecified versions in package.json Date: Thu, 4 Jul 2024 18:23:33 +0200 Message-Id: <20240704162333.1497107-2-enguerrand.de-ribaucourt@savoirfairelinux.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240704162333.1497107-1-enguerrand.de-ribaucourt@savoirfairelinux.com> References: <20240704162333.1497107-1-enguerrand.de-ribaucourt@savoirfairelinux.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 04 Jul 2024 16:23:48 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16390 Our current emulation mandates that the package.json contains a version field. Some packages may not provide it when they are not published to the registry. The actual `npm pack` would allow such packages, so should we. This patch adds default values to allow building such packages. For the shrinkwrap, we can actually use the resolved field which contains the exact source, including the revision, to pass integrity tests. This applies for instance to this package which doesn't declare a version: - https://github.com/cockpit-project/cockpit/blob/23701a555a5af13f998ee4c7526d27fdb5669d63/package.json#L2 Co-authored-by: Tanguy Raufflet Signed-off-by: Tanguy Raufflet Signed-off-by: Enguerrand de Ribaucourt --- bitbake/lib/bb/fetch2/npmsw.py | 2 +- meta/classes-recipe/npm.bbclass | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py index 018e0ad546..044f5b96f8 100644 --- a/bitbake/lib/bb/fetch2/npmsw.py +++ b/bitbake/lib/bb/fetch2/npmsw.py @@ -97,7 +97,7 @@ class NpmShrinkWrap(FetchMethod): integrity = params.get("integrity", None) resolved = params.get("resolved", None) - version = params.get("version", None) + version = params.get("version", params.get("resolved", None)) # Handle registry sources if is_semver(version) and integrity: diff --git a/meta/classes-recipe/npm.bbclass b/meta/classes-recipe/npm.bbclass index 91da3295f2..a73ff29be8 100644 --- a/meta/classes-recipe/npm.bbclass +++ b/meta/classes-recipe/npm.bbclass @@ -72,8 +72,10 @@ def npm_pack(env, srcdir, workdir): j = json.load(f) # base does not really matter and is for documentation purposes - # only. But the 'version' part must exist because other parts of + # only. But the 'version' part must exist because other parts of # the bbclass rely on it. + if 'version' not in j: + j['version'] = '0.0.0-unknown' base = j['name'].split('/')[-1] tarball = os.path.join(workdir, "%s-%s.tgz" % (base, j['version']));