From patchwork Mon Aug 12 12:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enguerrand de Ribaucourt X-Patchwork-Id: 47678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AF8AC52D7C for ; Mon, 12 Aug 2024 12:25:54 +0000 (UTC) Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by mx.groups.io with SMTP id smtpd.web10.44943.1723465552087597337 for ; Mon, 12 Aug 2024 05:25:52 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b=s9WglGME; spf=pass (domain: savoirfairelinux.com, ip: 208.88.110.44, mailfrom: enguerrand.de-ribaucourt@savoirfairelinux.com) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 5EA379C5D4F; Mon, 12 Aug 2024 08:25:51 -0400 (EDT) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10032) with ESMTP id A9bxW5WUb6cx; Mon, 12 Aug 2024 08:25:49 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 7F5E89C5C98; Mon, 12 Aug 2024 08:25:49 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.savoirfairelinux.com 7F5E89C5C98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=savoirfairelinux.com; s=DFC430D2-D198-11EC-948E-34200CB392D2; t=1723465549; bh=zzxcbdv9xw6FetVB3J/GbPt7URvvc3TfqrxxCQNaFuY=; h=From:To:Date:Message-Id:MIME-Version; b=s9WglGMEUEQJHXjOdu+vFcAuztFImj2ln4sXlYUT4fpt7mEOVCZ2BpFdAL4uGIt+H YY6//WNCn4jLeKzKRNTDK3opQeZiv/p4m8IrCceP5vrPMqENqvCdgdFJNVNQp+/+Yd HTwEivOxmaf+52gVjXdclb8+fvpW9RmPYvEFcJCPMD0Ry5kQOr8ESCjz8CbE9YLvsp UIKCmkpTNZFvrE6LeTQT8GMi40ShJHQCivCdbTMYkivMYYl62Jfv6TWRmicyZ9k/De l4yiUqOlHOOyeQxJDMuL50Gy1Ls9iwq2YO1gjdwnAIeNnmRMA/mapQUvnw2c/UrNQd AD56sFZre+7Ww== X-Virus-Scanned: amavis at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10026) with ESMTP id QQ2fVBffakWo; Mon, 12 Aug 2024 08:25:49 -0400 (EDT) Received: from sfl-deribaucourt.rennes.sfl (80-15-101-118.ftth.fr.orangecustomers.net [80.15.101.118]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id 8CA9B9C56BD; Mon, 12 Aug 2024 08:25:48 -0400 (EDT) From: Enguerrand de Ribaucourt To: bitbake-devel@lists.openembedded.org Cc: tanguy.raufflet@savoirfairelinux.com, richard.purdie@linuxfoundation.org, alexandre.belloni@bootlin.com, stefan.herbrechtsmeier@weidmueller.com, Enguerrand de Ribaucourt Subject: [bitbake-devel][PATCH v4 1/2] bitbake: fetch2: npmsw: fix fetching git revisions not on master Date: Mon, 12 Aug 2024 14:25:38 +0200 Message-Id: <20240812122539.573337-2-enguerrand.de-ribaucourt@savoirfairelinux.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812122539.573337-1-enguerrand.de-ribaucourt@savoirfairelinux.com> References: <20240812122539.573337-1-enguerrand.de-ribaucourt@savoirfairelinux.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 12 Aug 2024 12:25:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16465 The NPM package.json documentation[1] states that git URLs may contain a commit-ish suffix to specify a specific revision. When running `npm install`, this revision will be looked for on any branch of the repository. The bitbake implementation however translates the URL stored in package.json into a git URL to be fetch by the bitbake git fetcher. The bitbake fetcher git.py, enforces the branch to be master by default. If the revision specified in the package.json is not on the master branch, the fetch will fail while the package.json is valid. To fix this, append the ";nobranch=1" suffix to the revision in the git URL to be fetched. This will make the bitbake git fetcher ignore the branch and respect the behavior of `npm install``. This can be tested with the following command: $ devtool add --npm-dev https://github.com/seapath/cockpit-cluster-dashboard.git -B version Which points to a project which has a package.json with a git URL: ```json "devDependencies": { "cockpit-repo": "git+https://github.com/cockpit-project/cockpit.git#d34cabacb8e5e1e028c7eea3d6e3b606d862b8ac" } ``` In this repo, the specified revision is on the "main" branch, which would fail without this fix. [1] https://docs.npmjs.com/cli/v10/configuring-npm/package-json#git-urls-as-dependencies Co-authored-by: Tanguy Raufflet Signed-off-by: Tanguy Raufflet Signed-off-by: Enguerrand de Ribaucourt --- bitbake/lib/bb/fetch2/npmsw.py | 1 + 1 file changed, 1 insertion(+) diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py index b55e885d7b..d8ed9df327 100644 --- a/bitbake/lib/bb/fetch2/npmsw.py +++ b/bitbake/lib/bb/fetch2/npmsw.py @@ -184,6 +184,7 @@ class NpmShrinkWrap(FetchMethod): uri = URI("git://" + str(groups["url"])) uri.params["protocol"] = str(groups["protocol"]) uri.params["rev"] = str(groups["rev"]) + uri.params["nobranch"] = "1" uri.params["destsuffix"] = destsuffix url = str(uri) From patchwork Mon Aug 12 12:25:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Enguerrand de Ribaucourt X-Patchwork-Id: 47679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E5FEC531DC for ; Mon, 12 Aug 2024 12:25:54 +0000 (UTC) Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by mx.groups.io with SMTP id smtpd.web11.44946.1723465551625059556 for ; Mon, 12 Aug 2024 05:25:51 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b=YG7Jd06g; spf=pass (domain: savoirfairelinux.com, ip: 208.88.110.44, mailfrom: enguerrand.de-ribaucourt@savoirfairelinux.com) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id E4EC19C5480; Mon, 12 Aug 2024 08:25:50 -0400 (EDT) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10032) with ESMTP id ncgeoTGiNwKj; Mon, 12 Aug 2024 08:25:50 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 91E5B9C56BD; Mon, 12 Aug 2024 08:25:50 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.savoirfairelinux.com 91E5B9C56BD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=savoirfairelinux.com; s=DFC430D2-D198-11EC-948E-34200CB392D2; t=1723465550; bh=WsjllNXlLq/9nlH/srxN3thv5rIgJ/kZlEAKWFhWAjU=; h=From:To:Date:Message-Id:MIME-Version; b=YG7Jd06gVvnaC9GajO0/xOmQDeSdk3JNCmf281QgEaqUZaXrjyi4VbX763aLg5acr SEiIILs2xQH5oWPh9NqTP9FTNP1t6Z3AB3WL6wNenDu0kdw1/VcZakN9KKxjPMgBX/ PYwd2BI1X4ysBzEftVIE7JN5CKRICn7w6h5oAqiwGv4FNpCUjWLcay/G800ST+FnLN /RcOYHUo+1JixJYne0wm8OZiiF4PaEU+/wGdv3u162jl1F4eBlxj1dVl4IpDkzmnrY vcAk4S0cr/c7PNcpz6d/u1PtyNQy2oY5ca7W88tMCDw11HK9ap6t0wPxySFXacxYXh i9s2SAvsEkCJw== X-Virus-Scanned: amavis at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10026) with ESMTP id eO_k8Y1Y-Nxb; Mon, 12 Aug 2024 08:25:50 -0400 (EDT) Received: from sfl-deribaucourt.rennes.sfl (80-15-101-118.ftth.fr.orangecustomers.net [80.15.101.118]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id 99AE49C5480; Mon, 12 Aug 2024 08:25:49 -0400 (EDT) From: Enguerrand de Ribaucourt To: bitbake-devel@lists.openembedded.org Cc: tanguy.raufflet@savoirfairelinux.com, richard.purdie@linuxfoundation.org, alexandre.belloni@bootlin.com, stefan.herbrechtsmeier@weidmueller.com, Enguerrand de Ribaucourt Subject: [bitbake-devel][PATCH v4 2/2] npmsw: accept unspecified versions in package.json Date: Mon, 12 Aug 2024 14:25:39 +0200 Message-Id: <20240812122539.573337-3-enguerrand.de-ribaucourt@savoirfairelinux.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812122539.573337-1-enguerrand.de-ribaucourt@savoirfairelinux.com> References: <20240812122539.573337-1-enguerrand.de-ribaucourt@savoirfairelinux.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 12 Aug 2024 12:25:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/16464 Our current emulation mandates that the package.json contains a version field. Some packages may not provide it when they are not published to the registry. The actual `npm pack` would allow such packages, so should we. A first commit was required in the npm class. For the shrinkwrap part , we can actually use the resolved field which contains the exact source, including the revision, to pass integrity tests. v3: - Split bitbake npmsw.py modification in another commit Co-authored-by: Tanguy Raufflet Signed-off-by: Tanguy Raufflet Signed-off-by: Enguerrand de Ribaucourt --- bitbake/lib/bb/fetch2/npmsw.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py index d8ed9df327..a5fa598deb 100644 --- a/bitbake/lib/bb/fetch2/npmsw.py +++ b/bitbake/lib/bb/fetch2/npmsw.py @@ -97,7 +97,7 @@ class NpmShrinkWrap(FetchMethod): integrity = params.get("integrity", None) resolved = params.get("resolved", None) - version = params.get("version", None) + version = params.get("version", resolved) # Handle registry sources if is_semver(version) and integrity: