From patchwork Sat May 27 06:25:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: belouargamohamed@gmail.com X-Patchwork-Id: 24602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E8D9C7EE23 for ; Sat, 27 May 2023 06:25:39 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.web10.5827.1685168736669166023 for ; Fri, 26 May 2023 23:25:36 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@gmail.com header.s=20221208 header.b=InVTRIHO; spf=pass (domain: gmail.com, ip: 209.85.128.48, mailfrom: belouargamohamed@gmail.com) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-3f6da07ff00so15645515e9.3 for ; Fri, 26 May 2023 23:25:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685168735; x=1687760735; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=8bUpakl76o27hJf6ZRoQnS9ltw01zvaQbrcJZKpx2eI=; b=InVTRIHO2mDfCTCls2LkklMnmPjiT50g8WNyCHmVPJrfLe7SXA4z2UAr0X3tOZJgSD Tr8W+dS3L+j3kFIyvIXYLoG5j/L8uOZibRoXRnkg7sU6sbKJDzIqj6EhOq3TjallTSgz +POtxWa9GK151W6xVvBvEcfApTFYRvBbZlJZ0uS4MVX0dissYn6qA5fcduK+ZK/5EWUk M7www1yhvNryFqdQG/fwslYldw57rsfGmrCFjtXSKFjT7Q1tfspHLharbRjdimzMLYnC YFq+nZAQcGsCc136CoE7ESBhwn5ulI+25bg/r91os3CYYrNhQ057W0lJ+rIJ+JMgVqm2 d7KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685168735; x=1687760735; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8bUpakl76o27hJf6ZRoQnS9ltw01zvaQbrcJZKpx2eI=; b=i1axdj64kw9nI8zS2Sotor1E8KSWmDLOowfbhMHNmJtnSN8rzsUIjlNgji7vf7cYxD GH2ghSyPd5RrRkM7a22bLici3esRE3icnAlgSAfPZYpcruC0Gfw489fO/0LHD8A8eQDB +KdVNE7HesboLWGS9e1LoNT8gzcw49tuotGxX5ZmFGOOwiPF6SCbfp7EL688765Jr8wV 7w+ozvhhTrMs4cioINnz9MenKeX3aSGE/bc5pOK7p9Cnc3micJKbptupskg1v0MDfE83 s/UwRcamaQeBCdqxL5EkV9LSb/LtRDgPE54u1wBXEgUpxEsVhLdscnotYoHcTpZfKRYv Ktng== X-Gm-Message-State: AC+VfDxLsFiN1Fhd9054dVw1OQO6snbtcRvSg7UYp9PlBmKvOBw5PaHO FGtqsgij/Ov+EJGlosbZfwPNKoo7VUSQUWWP X-Google-Smtp-Source: ACHHUZ6ZdbrGcM8nMXHQ/LwFsRNydMogCLE1t4GyaBghKToRo8G+oreMJB2uh8zqzyFu3hKdhf3g7A== X-Received: by 2002:a05:600c:246:b0:3f4:e4a2:9f7f with SMTP id 6-20020a05600c024600b003f4e4a29f7fmr3662773wmj.36.1685168734633; Fri, 26 May 2023 23:25:34 -0700 (PDT) Received: from localhost.localdomain (2a02-842a-002e-7001-f2b0-7c97-e26b-fbea.rev.sfr.net. [2a02:842a:2e:7001:f2b0:7c97:e26b:fbea]) by smtp.gmail.com with ESMTPSA id e21-20020a05600c219500b003f4fffccd73sm7347071wme.9.2023.05.26.23.25.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 23:25:34 -0700 (PDT) From: belouargamohamed@gmail.com X-Google-Original-From: m.belouarga@technologyandstrategy.com To: openembedded-core@lists.openembedded.org Cc: e.aubineau@technologyandstrategy.com, f.lahoudere@technologyandstrategy.com, BELOUARGA Mohamed Subject: [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts Date: Sat, 27 May 2023 08:25:28 +0200 Message-Id: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 27 May 2023 06:25:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/181791 From: BELOUARGA Mohamed Npm packages do not have yocto friendly names. fore instance we can have names like "@example/npmPackage" npm fetcher has a function that convert these names to yocto friendly names. But in recipe tool we have an other function (duplicate). Signed-off-by: BELOUARGA Mohamed --- scripts/lib/recipetool/create_npm.py | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index 3394a89970..e667a4d19b 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -13,6 +13,7 @@ import sys import tempfile import bb from bb.fetch2.npm import NpmEnvironment +from bb.fetch2.npm import npm_package from bb.fetch2.npmsw import foreach_dependencies from recipetool.create import RecipeHandler from recipetool.create import get_license_md5sums @@ -30,15 +31,6 @@ def tinfoil_init(instance): class NpmRecipeHandler(RecipeHandler): """Class to handle the npm recipe creation""" - @staticmethod - def _npm_name(name): - """Generate a Yocto friendly npm name""" - name = re.sub("/", "-", name) - name = name.lower() - name = re.sub(r"[^\-a-z0-9]", "", name) - name = name.strip("-") - return name - @staticmethod def _get_registry(lines): """Get the registry value from the 'npm://registry' url""" @@ -143,7 +135,7 @@ class NpmRecipeHandler(RecipeHandler): # Handle the dependencies def _handle_dependency(name, params, deptree): - suffix = "-".join([self._npm_name(dep) for dep in deptree]) + suffix = "-".join([npm_package(dep) for dep in deptree]) destdirs = [os.path.join("node_modules", dep) for dep in deptree] destdir = os.path.join(*destdirs) packages["${PN}-" + suffix] = destdir @@ -173,7 +165,7 @@ class NpmRecipeHandler(RecipeHandler): if "name" not in data or "version" not in data: return False - extravalues["PN"] = self._npm_name(data["name"]) + extravalues["PN"] = npm_package(data["name"]) extravalues["PV"] = data["version"] if "description" in data: From patchwork Sat May 27 06:25:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: belouargamohamed@gmail.com X-Patchwork-Id: 24603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D19CC77B73 for ; Sat, 27 May 2023 06:25:39 +0000 (UTC) Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by mx.groups.io with SMTP id smtpd.web11.6055.1685168738186063924 for ; Fri, 26 May 2023 23:25:38 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@gmail.com header.s=20221208 header.b=j6aZhQYp; spf=pass (domain: gmail.com, ip: 209.85.128.41, mailfrom: belouargamohamed@gmail.com) Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-3f6d7abe934so9980485e9.2 for ; Fri, 26 May 2023 23:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685168736; x=1687760736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g2Pg2bBoqEJWl2vqD3mDnFoMzH6DPiC4LqbtV9o3gxY=; b=j6aZhQYpP/5lg0AHSwhgS6Cink8pXRPKwO3QU5VEOSz2STHR73FFqNqRNkHv0O0SwK sAzKDxt1tDtgl3YJFOKFe9rTo5PxB+CgKkzVhR8WNIhsIeL81q5JoaRYJgs9sV6EJL37 gRgPmUb0dPK5XCnv/dFNpIJpZwF57hu0j+0wcu4B6Pd5+1taTliyVpg7A/GMqyDiRYXU MLRkChslXEkEk/IV8hnibRT0nWFzMokHHZIQWzdhvazOUNZda/pfZdJIlIc38mdHeNHQ bE1/tQgUfw3acQEzIL9Q4TFudAHssD/XXI0OdEOPw5uFe3DRDpWL6CyWOIXkDhayUl1q IRsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685168736; x=1687760736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g2Pg2bBoqEJWl2vqD3mDnFoMzH6DPiC4LqbtV9o3gxY=; b=TnPzXNjwqhl036PBHKu6SFkIOZ3xZhftEH4Y+KwKVanrA6eyTlJ7Ao9L75sB6Ea3gm iphAJvho0kbzc4eUfXxXOKFwWkTV411wTF7Cckzqs/MyNWleTScO5AUdQ7pHVr8lwH1L zTgjSKcLBLkgy7ub97DuoRUqL/vQztsDZe8rs6GPlgVCjK5tuJBK8PSgZB+CYLu2YqiK 5K0lT4QHKXmgwveH3ZYjiR+a4MY9JFJ0L0x2hy7S+nuoFCjeaOXzSfeRvyYq4h9pgd/i nnMGBBbYagod9jWXfB/7J17BVjWgmKn+J6J+wq12B1mSAf++7nQzhC+KW2zvi7eeN0qL l1Hw== X-Gm-Message-State: AC+VfDyeVaE4fSLOptp8fHuG3bFY8zv6W0EkOoRrZYY5rbAjTuQj2eq3 DgCKqOZDVXmu/XIg4g5KQsTilrDRW3iQfdRk X-Google-Smtp-Source: ACHHUZ7HSEhL2SfuIBmEpEnhe4TVS4WzrTxqcZC6jnzRbq4rhTcJNkECNBRkcLm7v7x56ArAsQZsqA== X-Received: by 2002:a5d:420c:0:b0:306:31e0:964 with SMTP id n12-20020a5d420c000000b0030631e00964mr2896630wrq.55.1685168736153; Fri, 26 May 2023 23:25:36 -0700 (PDT) Received: from localhost.localdomain (2a02-842a-002e-7001-f2b0-7c97-e26b-fbea.rev.sfr.net. [2a02:842a:2e:7001:f2b0:7c97:e26b:fbea]) by smtp.gmail.com with ESMTPSA id e21-20020a05600c219500b003f4fffccd73sm7347071wme.9.2023.05.26.23.25.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 23:25:35 -0700 (PDT) From: belouargamohamed@gmail.com X-Google-Original-From: m.belouarga@technologyandstrategy.com To: openembedded-core@lists.openembedded.org Cc: e.aubineau@technologyandstrategy.com, f.lahoudere@technologyandstrategy.com, BELOUARGA Mohamed Subject: [kirkstone][PATCH 2/5] classes: npm: Handle peer dependencies for npm packages Date: Sat, 27 May 2023 08:25:29 +0200 Message-Id: <20230527062532.938477-2-m.belouarga@technologyandstrategy.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> References: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 27 May 2023 06:25:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/181792 From: BELOUARGA Mohamed NPM changed its manner to handle peer dependencies over its versions: - Before NPM 3: NPM installs automatically peer dependencies - Between NPM 3 and 7: NPM shows a warning about peer dependencies - After NPM 3: NPM reworked its manner how to handle peer dependencies The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot fetch them. in the same time peer dependencies are not direct dependencies, they should be installed as run time dependencies. Signed-off-by: BELOUARGA Mohamed --- meta/classes/npm.bbclass | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass index 8379c7b988..abc4c6f92b 100644 --- a/meta/classes/npm.bbclass +++ b/meta/classes/npm.bbclass @@ -108,6 +108,7 @@ python npm_do_configure() { import tempfile from bb.fetch2.npm import NpmEnvironment from bb.fetch2.npm import npm_unpack + from bb.fetch2.npm import npm_package from bb.fetch2.npmsw import foreach_dependencies from bb.progress import OutOfProgressHandler from oe.npm_registry import NpmRegistry @@ -168,6 +169,7 @@ python npm_do_configure() { if has_shrinkwrap_file: cached_shrinkwrap = copy.deepcopy(orig_shrinkwrap) cached_shrinkwrap.pop("dependencies", None) + cached_shrinkwrap["packages"][""].pop("peerDependencies", None) # Manage the dependencies progress = OutOfProgressHandler(d, r"^(\d+)/(\d+)$") @@ -202,6 +204,19 @@ python npm_do_configure() { if has_shrinkwrap_file: foreach_dependencies(orig_shrinkwrap, _count_dependency, dev) foreach_dependencies(orig_shrinkwrap, _cache_dependency, dev) + + # Manage Peer Dependencies + if has_shrinkwrap_file: + packages = orig_shrinkwrap.get("packages", {}) + peer_deps = packages.get("", {}).get("peerDependencies", {}) + package_runtime_dependencies = d.getVar("RDEPENDS:%s" % d.getVar("PN")) + + for peer_dep in peer_deps: + peer_dep_yocto_name = npm_package(peer_dep) + if peer_dep_yocto_name not in package_runtime_dependencies: + bb.warn(peer_dep + " is a peer dependencie that is not in RDEPENDS variable. " + + "Please add this peer dependencie to the RDEPENDS variable as %s and generate its recipe with devtool" + % peer_dep_yocto_name) # Configure the main package with tempfile.TemporaryDirectory() as tmpdir: @@ -277,6 +292,9 @@ python npm_do_compile() { args.append(("target_arch", d.getVar("NPM_ARCH"))) args.append(("build-from-source", "true")) + # Don't install peer dependencies as they should be in RDEPENDS variable + args.append(("legacy-peer-deps", "true")) + # Pack and install the main package (tarball, _) = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir) cmd = "npm install %s %s" % (shlex.quote(tarball), d.getVar("EXTRA_OENPM")) From patchwork Sat May 27 06:25:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: belouargamohamed@gmail.com X-Patchwork-Id: 24606 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D989C77B73 for ; Sat, 27 May 2023 06:25:49 +0000 (UTC) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mx.groups.io with SMTP id smtpd.web10.5829.1685168739736297783 for ; Fri, 26 May 2023 23:25:40 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@gmail.com header.s=20221208 header.b=IBiEpX3w; spf=pass (domain: gmail.com, ip: 209.85.128.53, mailfrom: belouargamohamed@gmail.com) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-3f68fc6b479so15820945e9.2 for ; Fri, 26 May 2023 23:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685168738; x=1687760738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2irKUYwyYh1dcWlnjohfgaLhlT8ND93pcpRz1tlDtpw=; b=IBiEpX3wFvlEqQ1IW24jMK6QVDnz47ZtEkC4Bt9WfmFVak0xZTwEXFm5wvX6vEyWBa FkUaUabQSRx6ApGvgtcvzvUdJ6jXWFDFQT5s5pv8kpbvkpn5bpNvF+tMbSZabF9/gcVF YZZMSU4ddQnekhY3j0k6E3VMPuFWR1YYcVLEJRsdmGw7G0vcC791hsgoKWCLoguXbMhZ /WTjLOpliFAB8XZHaVy0cca6p9KWWw48V82Q0sqJ10svzRhdG1x9Al3Y+rVCohsTjdlM YDjFyiZxht2fBFiDoRlAYeV6tq/VWga5/XvUBpRc8ZfMVLJpnpEsPn4WQy6ov8lJ9zu9 Mjxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685168738; x=1687760738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2irKUYwyYh1dcWlnjohfgaLhlT8ND93pcpRz1tlDtpw=; b=Jq/+FFfhpA6hU0tKMr1YzcvtAEBynHg/+j1T7B0VCsiTxXbpb+pRoOvsREnDD+S81u zLSDGL6GTZiaLSIGkiK7HQAAo4SrPHoH3+IdbiqSHW75jBgyAoN5qdXcnbdgxnMZDqSL YJqyuyVAfFNWRC5jUCkUfcjCBxS9kPlyFnL5sKZbAR5UyWqeIXYQ27lGzHM0UPCXv/Wc 8sXWLHlYy4mt7w/GFdo9P3o9MFqZznI5e4pbsTP5502oB8e1lc8OUPw+QNMZcqamHr1t pug6hZYAPs4CA9z+N79GLLEhRiN4M+7ELvWR2ZVsaUQSoQHtA/501dRZpaCTKamDNIFS 79sA== X-Gm-Message-State: AC+VfDwibF9gu0telf2KzOFpMlSBkzLh8ija8OxF8pwI7mnEtfDOJzDi 0i0wpffMjf0juFQ516evf4dy6SkNrx78MVN0 X-Google-Smtp-Source: ACHHUZ58xKvlAmpYKZIgEeyqP3SXyLPO0pGWldONu7rVJIqJ5V4P3XWY2+JfuYI4p0HYJhNTd8SV5Q== X-Received: by 2002:a05:600c:2111:b0:3f6:3bb:5132 with SMTP id u17-20020a05600c211100b003f603bb5132mr3096649wml.24.1685168737629; Fri, 26 May 2023 23:25:37 -0700 (PDT) Received: from localhost.localdomain (2a02-842a-002e-7001-f2b0-7c97-e26b-fbea.rev.sfr.net. [2a02:842a:2e:7001:f2b0:7c97:e26b:fbea]) by smtp.gmail.com with ESMTPSA id e21-20020a05600c219500b003f4fffccd73sm7347071wme.9.2023.05.26.23.25.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 23:25:37 -0700 (PDT) From: belouargamohamed@gmail.com X-Google-Original-From: m.belouarga@technologyandstrategy.com To: openembedded-core@lists.openembedded.org Cc: e.aubineau@technologyandstrategy.com, f.lahoudere@technologyandstrategy.com, BELOUARGA Mohamed Subject: [kirkstone][PATCH 3/5] recipetool: create: npm: Add support for the new format of the shrinkwrap file Date: Sat, 27 May 2023 08:25:30 +0200 Message-Id: <20230527062532.938477-3-m.belouarga@technologyandstrategy.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> References: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 27 May 2023 06:25:49 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/181793 From: BELOUARGA Mohamed The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly. The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format. Signed-off-by: BELOUARGA Mohamed --- scripts/lib/recipetool/create_npm.py | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index e667a4d19b..25e7ddb472 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -134,11 +134,10 @@ class NpmRecipeHandler(RecipeHandler): licfiles.append(os.path.relpath(readme, srctree)) # Handle the dependencies - def _handle_dependency(name, params, deptree): + def _handle_dependency(name, params, destdir): + deptree = destdir.split('node_modules/') suffix = "-".join([npm_package(dep) for dep in deptree]) - destdirs = [os.path.join("node_modules", dep) for dep in deptree] - destdir = os.path.join(*destdirs) - packages["${PN}-" + suffix] = destdir + packages["${PN}" + suffix] = destdir _licfiles_append_fallback_readme_files(destdir) with open(shrinkwrap_file, "r") as f: @@ -234,7 +233,7 @@ class NpmRecipeHandler(RecipeHandler): value = origvalue.replace("version=" + data["version"], "version=${PV}") value = value.replace("version=latest", "version=${PV}") values = [line.strip() for line in value.strip('\n').splitlines()] - if "dependencies" in shrinkwrap: + if "dependencies" in shrinkwrap.get("packages", {}).get("", {}): values.append(url_recipe) return values, None, 4, False From patchwork Sat May 27 06:25:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: belouargamohamed@gmail.com X-Patchwork-Id: 24605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05B26C7EE23 for ; Sat, 27 May 2023 06:25:49 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.web11.6056.1685168741072354024 for ; Fri, 26 May 2023 23:25:41 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@gmail.com header.s=20221208 header.b=lp6gheWP; spf=pass (domain: gmail.com, ip: 209.85.128.48, mailfrom: belouargamohamed@gmail.com) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-3f6e13940daso15849155e9.0 for ; Fri, 26 May 2023 23:25:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685168739; x=1687760739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i8/dDsCJrSYL8PrYUOdwWzwHq/4ZBZ2SonINWnKkMkw=; b=lp6gheWPbzqEcZOriCRXmQlQk2xmKv9BlHjzeh3yy8iEPkGL1KNJZvCu8abE6aEi24 vImHzb1tFlpLMsEKRbMpxWFqWUVE7MJ31GOZRf6IILHLb7aOKrFr7Nn7VLHslFY10TyI Dr6eLz6mU9B7A54Txb5LQpNCH4bNqhf4r2yXF7ZaG0Pcc8IQp+8xQCltWkTvALlsUUZk leUOHr35zaRi5pF2SgDJtfiFBluM6RmuJMSH1OstWGeocxCZVQsjBWSnW3cAhkluk20L XaQKpW5UB0K1hfMiAe1DGpLuheI3G/dbkIp9up+2bkoOA804LKbP+O+V6KMVTKxwZjcS kRuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685168739; x=1687760739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i8/dDsCJrSYL8PrYUOdwWzwHq/4ZBZ2SonINWnKkMkw=; b=PhXyWvdOwil/ES1mfesqPWSGsQsi/u5VlWnTCmnbw21czUFsuvCKWiwuDKg9zp7fbp woiy4ikSHijdADPKJhiDYvonaqrKXHkos+mzurjFJQb7tijfqQvoyHf/PfOGE5zbndAm q4N7xkk03OBUr8IR0BsPyp7e+2Cv6Ai9l/vtcg76DMkN+bG6wdjdSa2CtlpTrM3U5fxv RAzK6ZGlMWZEUWPDhmnLNLgMfycYFNH0+5VPd4h/8BiLCfH/08gNZgL8nIC//tpFILu8 +yNjBBp/dDnfBOolumo/ULU9AmLoJdEWlIC5r5uWzAEb91vSr+sWyDwTT7CMShApS5Ed 7laQ== X-Gm-Message-State: AC+VfDwLI33lBB/E6ykeyDt2MY3/p3BH06GK94Er6zYLuCoJA1VpCCAp lo0yd79GHlRHjqy37RqqTm5S1VWJujygz+/t X-Google-Smtp-Source: ACHHUZ7N5hy9HmgeSqJ4gDLfA6I6yOBq3MIgpn4wocdL87bfDE6ZKMeJjA7bmJ9uk35aF2ytxL+bLg== X-Received: by 2002:a05:600c:228c:b0:3f5:fbd0:94ab with SMTP id 12-20020a05600c228c00b003f5fbd094abmr3343231wmf.3.1685168739117; Fri, 26 May 2023 23:25:39 -0700 (PDT) Received: from localhost.localdomain (2a02-842a-002e-7001-f2b0-7c97-e26b-fbea.rev.sfr.net. [2a02:842a:2e:7001:f2b0:7c97:e26b:fbea]) by smtp.gmail.com with ESMTPSA id e21-20020a05600c219500b003f4fffccd73sm7347071wme.9.2023.05.26.23.25.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 23:25:38 -0700 (PDT) From: belouargamohamed@gmail.com X-Google-Original-From: m.belouarga@technologyandstrategy.com To: openembedded-core@lists.openembedded.org Cc: e.aubineau@technologyandstrategy.com, f.lahoudere@technologyandstrategy.com, BELOUARGA Mohamed Subject: [kirkstone][PATCH 4/5] recipetool: create: npm: Add support to handle peer dependencies Date: Sat, 27 May 2023 08:25:31 +0200 Message-Id: <20230527062532.938477-4-m.belouarga@technologyandstrategy.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> References: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 27 May 2023 06:25:49 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/181794 From: BELOUARGA Mohamed NPM changed its manner to handle peer dependencies over its versions: - Before NPM 3: NPM installs automatically peer dependencies - Between NPM 3 and 7: NPM shows a warning about peer dependencies - After NPM 3: NPM reworked its manner how to handle peer dependencies The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot fetch them. in the same time peer dependencies are not direct dependencies, they should be installed as run time dependencies. Signed-off-by: BELOUARGA Mohamed --- scripts/lib/recipetool/create_npm.py | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index 25e7ddb472..113a89f6a6 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -146,6 +146,23 @@ class NpmRecipeHandler(RecipeHandler): foreach_dependencies(shrinkwrap, _handle_dependency, dev) return licfiles, packages + + # Handle the peer dependencies + def _handle_peer_dependency(self, shrinkwrap_file): + """Check if package has peer dependencies and show warning if it is the case""" + with open(shrinkwrap_file, "r") as f: + shrinkwrap = json.load(f) + + packages = shrinkwrap.get("packages", {}) + peer_deps = packages.get("", {}).get("peerDependencies", {}) + + for peer_dep in peer_deps: + peer_dep_yocto_name = npm_package(peer_dep) + bb.warn(peer_dep + " is a peer dependencie of the actual package. " + + "Please add this peer dependencie to the RDEPENDS variable as %s and generate its recipe with devtool" + % peer_dep_yocto_name) + + def process(self, srctree, classes, lines_before, lines_after, handled, extravalues): """Handle the npm recipe creation""" @@ -283,6 +300,9 @@ class NpmRecipeHandler(RecipeHandler): classes.append("npm") handled.append("buildsystem") + # Check if package has peer dependencies and inform the user + self._handle_peer_dependency(shrinkwrap_file) + return True def register_recipe_handlers(handlers): From patchwork Sat May 27 06:25:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: belouargamohamed@gmail.com X-Patchwork-Id: 24604 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05A8CC77B7C for ; Sat, 27 May 2023 06:25:49 +0000 (UTC) Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by mx.groups.io with SMTP id smtpd.web11.6058.1685168742704428403 for ; Fri, 26 May 2023 23:25:43 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@gmail.com header.s=20221208 header.b=ipukR/qF; spf=pass (domain: gmail.com, ip: 209.85.128.45, mailfrom: belouargamohamed@gmail.com) Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-3f6d3f83d0cso15626645e9.2 for ; Fri, 26 May 2023 23:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685168741; x=1687760741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ykaw+olWTZ0e8jh7zgVVzcbFHBjvbKsIgcOS3GLFZk0=; b=ipukR/qFLneBwtPaqygIOiN4ErMD+frNTiQVhhZUVR49BofrZPKqcZm470Ku+snHCA wA1kk66dIYKSDtWY7+jUWdP4hSRrGt6Np5mVyZ+JNJNCyKZxoRvnN7dpJd47+bQicvqF CFoT0l0HU6v/TyLKjWIkPc/MFAotyE9LLEm4DFdvvEDHi6LuvNddZvOpu3DN09ycDbmL LwBP5WsvimKhpBswqAjYbKVS7txkExaLUZocvtWshwkYK08CVid07bQ2LEBNoYskpbkA OPchD3e61zvrVDHj4184ddlKrIWGI5/9Q01F7u+wcKjD27HYNvqjqeIz0rsFLLgl/uth HIEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685168741; x=1687760741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ykaw+olWTZ0e8jh7zgVVzcbFHBjvbKsIgcOS3GLFZk0=; b=GkjI8HEImllI3OGyW+NbSyNvocU3aeQPr4XIhuEJr1LCvOzGAgcvJ9sDiA9WNKMI0C u4GRZ2n98gq3t4nLMi/4Xq4x9hnm0+jKUcH1Br9Oc5Wi9qkUHqCWFcv7JnBqGGUkg/qN zedoaicAWXBmdVXsBR7fbgtOdCfbX7dmgHSuJ+zN9OZ5SDUQK7tM7WsWD2EDQQzOwqsA aViqjMUsSSfcn6W52ubzsZmt0lKOTyv3/36/c9cFkk/cX3sTwBh27abzieRuKYY0zcSt 3Cjxs9csyPe3J2FmmI2GgByuYm5BSn7I95B6fX7MsJMqd0iFIeWhSmE4UIKOebANDIgW RF0Q== X-Gm-Message-State: AC+VfDxi/jsI+uxrdCW4F8PEnWMhyFOlz9TIbILD9eYdbbCbhNbqPgU7 spW0nRtdPiW0/ngpBjwG60bqivyl8k3Eqk+S X-Google-Smtp-Source: ACHHUZ4iNYazPicgEZcztz+KacqOwJiV5l//udZdaRBeM6ccy3jkWfhOj1Evmf7rjNW4cUQ4OJmaRw== X-Received: by 2002:a7b:cc8e:0:b0:3f4:2b98:428f with SMTP id p14-20020a7bcc8e000000b003f42b98428fmr3377776wma.39.1685168740753; Fri, 26 May 2023 23:25:40 -0700 (PDT) Received: from localhost.localdomain (2a02-842a-002e-7001-f2b0-7c97-e26b-fbea.rev.sfr.net. [2a02:842a:2e:7001:f2b0:7c97:e26b:fbea]) by smtp.gmail.com with ESMTPSA id e21-20020a05600c219500b003f4fffccd73sm7347071wme.9.2023.05.26.23.25.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 23:25:40 -0700 (PDT) From: belouargamohamed@gmail.com X-Google-Original-From: m.belouarga@technologyandstrategy.com To: openembedded-core@lists.openembedded.org Cc: e.aubineau@technologyandstrategy.com, f.lahoudere@technologyandstrategy.com, BELOUARGA Mohamed Subject: [kirkstone][PATCH 5/5] classes: npm: Add support for the new format of the shrinkwrap file Date: Sat, 27 May 2023 08:25:32 +0200 Message-Id: <20230527062532.938477-5-m.belouarga@technologyandstrategy.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> References: <20230527062532.938477-1-m.belouarga@technologyandstrategy.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 27 May 2023 06:25:49 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/181795 From: BELOUARGA Mohamed 1 - Adapt do_configure to the new format of the shrinkwrap 2 - Remove useless function _npmsw_dependency_dict because the dictionnary is already given by npmsw:foreach_dependencies 3 - Rename arguments of callback functions Signed-off-by: BELOUARGA Mohamed --- meta/classes/npm.bbclass | 27 ++++++--------------------- 1 file changed, 6 insertions(+), 21 deletions(-) diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass index abc4c6f92b..6899a64909 100644 --- a/meta/classes/npm.bbclass +++ b/meta/classes/npm.bbclass @@ -129,22 +129,6 @@ python npm_do_configure() { sha512 = bb.utils.sha512_file(tarball) return "sha512-" + base64.b64encode(bytes.fromhex(sha512)).decode() - def _npmsw_dependency_dict(orig, deptree): - """ - Return the sub dictionary in the 'orig' dictionary corresponding to the - 'deptree' dependency tree. This function follows the shrinkwrap file - format. - """ - ptr = orig - for dep in deptree: - if "dependencies" not in ptr: - ptr["dependencies"] = {} - ptr = ptr["dependencies"] - if dep not in ptr: - ptr[dep] = {} - ptr = ptr[dep] - return ptr - # Manage the manifest file and shrinkwrap files orig_manifest_file = d.expand("${S}/package.json") orig_shrinkwrap_file = d.expand("${S}/npm-shrinkwrap.json") @@ -176,24 +160,25 @@ python npm_do_configure() { progress_total = 1 # also count the main package progress_done = 0 - def _count_dependency(name, params, deptree): + def _count_dependency(name, params, destsuffix): nonlocal progress_total progress_total += 1 - def _cache_dependency(name, params, deptree): - destsubdirs = [os.path.join("node_modules", dep) for dep in deptree] - destsuffix = os.path.join(*destsubdirs) + def _cache_dependency(name, params, destsuffix): with tempfile.TemporaryDirectory() as tmpdir: # Add the dependency to the npm cache destdir = os.path.join(d.getVar("S"), destsuffix) (tarball, pkg) = npm_pack(env, destdir, tmpdir) _npm_cache_add(tarball, pkg) # Add its signature to the cached shrinkwrap - dep = _npmsw_dependency_dict(cached_shrinkwrap, deptree) + dep = params dep["version"] = pkg['version'] dep["integrity"] = _npm_integrity(tarball) if params.get("dev", False): dep["dev"] = True + if "dependencies" not in cached_shrinkwrap: + cached_shrinkwrap["dependencies"] = {} + cached_shrinkwrap["dependencies"][name] = dep # Display progress nonlocal progress_done progress_done += 1