From patchwork Fri Nov 20 13:53:32 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105077 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6323.1795182826573577360 for ; Fri, 20 Nov 2026 05:53:46 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=GtA6tp3w; spf=pass (domain: gmail.com, ip: 209.85.128.45, mailfrom: stondo@gmail.com) Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-4a35a60499eso7217155e9.0 for ; Fri, 20 Nov 2026 05:53:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182825; x=1795787625; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=dt/B9SfL0/WeDc9OgwBpfZ7ZRxcHO8ccyD3tvyMmp3g=; b=GtA6tp3wLwbANwsA1RX81Hl8Q53qX86YG7oW8OZyohz+Od9rZ5SS36X5h32g+s3d4G uKFgc1MC6pUptpw7AV4c1//T83bKEiIrM3cqohZKuFfPco8ZQwZJHpMjoPa7Xxd9aVrA +csPfF1ul/15N6BtVkRyu95NOWKdRm3g9LNPD2YFg0LQBnOgYTBkIjpdW08CAVwjN+Ph ThPa6MYW+gS6VUNTQPW5B50MJyGXlZThFXiRPaXNEzwQq/4CjZnrtx5Fw85BpU5M82CW kfKd7e6kG9Fm/YoakhYbNS2vin4wyRl4T6g+oBEA40imso4pwhK6OUzMCfk+r5FKrIlu 8u7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182825; x=1795787625; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=dt/B9SfL0/WeDc9OgwBpfZ7ZRxcHO8ccyD3tvyMmp3g=; b=0HPHczlC03swi/FxVuhrTUq7SpM1lx2VrscPaGeXQN796Y856cr6CC4Y015yZguBPF Z0DUpiBe5AxJNkt1dHR5qmBFo4YngcRpmFTroijvPdVHhaoVLvsLdgq2r8iYaNXdbVNR b1vjjrqP/VthbJJpH8U1U+Y8SMwo4ysaq6+GYh9fJ6i6hp8XV1kVDl4ZybRRnylULcyE wzMsF7TqpFJIbaMTT1qHIkwAdH1oN6FnptTVLzi2Mh7Vyx17NPtJAJ7szEiKESx+Ypd9 h6h2qn0QgvVyB8DSYxgCvQQm6ySQzwoO3QbZRBxjflYQCuFA7064YYMWzdfV3JMiIJxB T9Kw== X-Gm-Message-State: APuU4Z+7inblwj1V0ptMa7bTVNlo5eZ/KNwfOdB59qS5LU9kDV1x5nyz nP0NQK1B5BVOY59Ws1llS/wY3hN5+99hxBgWn61YuwUFzCIj6++Byzu351XDnoL2 X-Gm-Gg: AYBFou2+m/YTVHZ/3c1zsLPbaZ5NWPAltlM0CfU84OcVphOmuTm17RlK8tPd5x1JmIb tW8DhRxmqwgbfNZcf5hqPhAzgMjJi63PanvTJPdVourThApLQhyTBiiwjledjZVuEtJMHvhgslx c82yCd/8ZJNRqqF74P9/gQPToWWf+gv2+C9RBXEPf2/nOmAqFuyB4Hxs4TRcWLlnJhT8xrEpgim 6iobvBl6q/w/DG01Jcs3/SNTLVQuc892bvFlOPW0aAf7Y3WCd6Z6qB+DZgvzlvOKN/ExnU/jiAE eSNZjjLXgVddAnkjlnbW9BPIdw8niGRI97088jt0f0iI5vSf3XcDENDcBPPddmS/tbSQiArj6WJ lPLbL8UqQD/yc7kT4eytDIeDzcAy3dR0fDc2ylwgkLNbwRz+398KrKfef9uq/DC6qCHbYqXP6to kocdBKt8SL+sxSnBCR/ZpX87kjZ+TX1njlhhbx437PxZmBcx+Q+Dj+aWgnZuZz77BvA1bvUGqr6 4a8QeStVZMVDmPjl8dUS14OWPNFcWXSMWPdD9o/NTpJDruSxu1ElEwpucjoHLEFqZdru0LUyfAM hX1OJ1bd9tnyrRaR7bgIQZ63PIuMfOlt X-Received: by 2002:a05:600c:a0d:b0:4a1:825a:56fd with SMTP id 5b1f17b1804b1-4bb88826c1fmr69214955e9.32.1795182824774; Fri, 20 Nov 2026 05:53:44 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:44 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 01/10] recipetool: create_npm: Give the fallback licenses as a string Date: Fri, 20 Nov 2026 14:53:32 +0100 Message-ID: <20261120135342.2628228-2-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:53:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250794 From: Stefano Tondo Since 727ea2769c ("license: Remove tidy_licenses()"), handle_license_vars() reads LICENSE from the extra values as a license string. The npm handler still gives a set, so recipetool create stops with "AttributeError: 'set' object has no attribute 'strip'" for each npm package that has dependencies. Join the sorted fallback licenses with " AND ". Also remove a line that sorted a copy of the set and did not use the result. Signed-off-by: Stefano Tondo --- scripts/lib/recipetool/create_npm.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index 10eca3c63e..5edf647176 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -283,10 +283,9 @@ class NpmRecipeHandler(RecipeHandler): (licfiles, packages, fallback_licenses) = self._handle_licenses(srctree, shrinkwrap_file, dev) licvalues = match_licenses(licfiles, srctree, d) split_pkg_licenses(licvalues, packages, lines_after, fallback_licenses) - fallback_licenses_flat = set(license for sublist in fallback_licenses.values() for license in sublist) - fallback_licesens_flat = sorted(fallback_licenses_flat).sort() + fallback_licenses_flat = sorted(set(license for sublist in fallback_licenses.values() for license in sublist)) extravalues["LIC_FILES_CHKSUM"] = generate_common_licenses_chksums(fallback_licenses_flat, d) - extravalues["LICENSE"] = fallback_licenses_flat + extravalues["LICENSE"] = " AND ".join(fallback_licenses_flat) classes.append("npm") handled.append("buildsystem") From patchwork Fri Nov 20 13:53:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105078 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6386.1795182827902775215 for ; Fri, 20 Nov 2026 05:53:48 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=O9yoBDyc; spf=pass (domain: gmail.com, ip: 209.85.128.42, mailfrom: stondo@gmail.com) Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-4a140e7405dso8337795e9.3 for ; Fri, 20 Nov 2026 05:53:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182826; x=1795787626; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=36PISSeuZF8VjgxufAYHZcqgrQ/dXVu9sOTMES5F2JE=; b=O9yoBDycND5pfeG/niLmQPaqe+a6Y6pUmF1cCDESFKL+IMMVzRx90yvUwYp/+0S3ZX SeYDDAmRHkzlNr6JSrT+ZwK/zw1ltYS5f93tZ8LahehFdEziNYXpyIw86Ad17Dfl1Dj1 zqE9/xeykei1qADCGSz5jeCtzWBC1g2qmdjsPlypYWWMxlrv2DTxDBr4owHeC+RkB/EX vQdlkcyq6L9A4cDzq1oHEQBfh49xWzvO6s0Slh/Q6EWtLe+2t4QkQbC+siCWE+aPrvlW U0f+pEa5/nNK1aZkhT0ND7r+ODUqmt0Y8IH6lvHBDP31HEYw6bp05eHn8IMEeOw3MpVE AKSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182826; x=1795787626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=36PISSeuZF8VjgxufAYHZcqgrQ/dXVu9sOTMES5F2JE=; b=AnolIsSPMxebmDj2Uh+EsVV3+zEkvNFY2zqzVpm1UkYUoXzyrz0W8Qc+U/s8qsKigI 0UGD15K1SSpgkSiTyGuYSEjf/VfUBHzIM5B0uKR56vMEH0h01JpOmhrSfGqpI3cvVFi+ 7MkHvjiUCvL/wcRRVrc8Ww+gE8tl5kmPHdORtARJYLXt/bSEHcfce3udbFC637POG0mU XtvcShi7mlelBLKbAOT5KEsr78YNNLWqWlYovs/rBbn0lBHWqwQxmN7MncdoChYIU/E5 Pup9DD7JrbtvWeh/6aWHbUN/l0It5AzFh5IqAq/PDhxiYOb8qG1tkpqN6Ue/MTdw+eu8 1X/Q== X-Gm-Message-State: APuU4Z+gJc4W0F7yKESV1hYJavlk6NC9PQK6IAuADFb0+UeWzUUMVXp6 gpterajQVYqjvaUu1Gl0Qqw5vVnFPQ10WoUEGQ4hFos4zWvbFdRa4jVNxkEqujyn X-Gm-Gg: AYBFou22JSvJik7RZmVYRh5pfdVwHKpSI8bWloU2GFcL+KwPIX0sB51ekEGJ7xFNopL ykhDzEzZ9DNFBuX6awiO0MNyHCMkZxpvaVTz9J+O/5AGLQy3LVr/bKwCGd+/rwaFvxR88/zqGc0 hQsUj8tdBqpDvD+ZIud04bentQkjfGXFIl+XWLx06ROS2b8qugtl5mixrRXeiGRnfbQsEVakQVd jkoJM2swDZp5HFtKlGhQydSKoasyzyepHJEmMlVp4IN71Pfmau8TV+IwO3xkUnHfdL+ZEk3N5t7 DVGS2pnhg+CuCPjknE/rW6Ywvp8Bp1e+514CV7h2x2HTr1AaAdBXfiK9k2f2HQXEVNOzOhxfoHV GBK8C/W7b5WjRJR7aqZwhsC5fyF+s1j8mEHFQhvN8E2hyHjaPLv9SRIs/eNT5YGQs3CqWF9KJOx ojitQayaSOw/G8wEJFQpb25ogMlyrLqbL2WXZUgV2SyCOWyY08VcCvRGPKpH9KearOBOnRMV2Kx 1H7k2DJuprpSi/hQc3v+OKFedmiVLCtmO5zp+qQFq7XiOupK/u8qlvsqMcHFOVKKJEUL1cXK/// Sirt2F+5pQa3 X-Received: by 2002:a05:600c:1c0f:b0:4a1:f625:2108 with SMTP id 5b1f17b1804b1-4bb8881d4a3mr75203535e9.33.1795182825830; Fri, 20 Nov 2026 05:53:45 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:45 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 02/10] recipetool: create: Get the checksum of npm packages from the registry Date: Fri, 20 Nov 2026 14:53:33 +0100 Message-ID: <20261120135342.2628228-3-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:53:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250795 From: Stefano Tondo Since bitbake 9e45e00a2 ("fetch/{npm,npmsw}: re-enable fetchers now that checksums come from SRC_URI"), an npm:// URL needs a checksum. recipetool create gets an npm:// URL without a checksum, so it fails before the fetch: MissingParameterError: ... Missing checksum for npm package The usual way, a fetch with BB_STRICT_CHECKSUM = "ignore", does not work for npm on purpose. So get the checksum from the registry ("dist.integrity") before the fetch, and add it to the URL in the recipe. For a new recipe, the registry is the only source of this checksum. Update the selftest for the checksum in SRC_URI. Signed-off-by: Stefano Tondo --- meta/lib/oeqa/selftest/cases/recipetool.py | 4 +++- scripts/lib/recipetool/create.py | 22 ++++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py b/meta/lib/oeqa/selftest/cases/recipetool.py index 0aaa090ff5..c6aa77399f 100644 --- a/meta/lib/oeqa/selftest/cases/recipetool.py +++ b/meta/lib/oeqa/selftest/cases/recipetool.py @@ -449,7 +449,9 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['HOMEPAGE'] = 'https://github.com/savoirfairelinux/node-server-example#readme' checkvars['LICENSE'] = 'BSD-3-Clause AND ISC AND MIT AND Unknown' urls = [] - urls.append('npm://registry.npmjs.org/;package=@savoirfairelinux/node-server-example;version=${PV}') + urls.append('npm://registry.npmjs.org/;package=@savoirfairelinux/node-server-example;version=${PV}' + ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7' + '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223') urls.append('npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json') checkvars['SRC_URI'] = set(urls) checkvars['S'] = '${WORKDIR}/npm' diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py index 662e7a6232..2df17dd6e4 100644 --- a/scripts/lib/recipetool/create.py +++ b/scripts/lib/recipetool/create.py @@ -413,6 +413,18 @@ def is_package(url): return True return False +def get_npm_checksum(registry, package, version): + """Get the checksum of an npm package from the registry. + + The npm fetcher needs a checksum before the download. When a new recipe + is created, the registry is the only source of this checksum. + """ + import urllib.request + from bb.fetch.npm import npm_integrity + url = "https://%s/%s/%s" % (registry.strip("/"), package, version) + with urllib.request.urlopen(url, timeout=60) as response: + return npm_integrity(json.load(response)["dist"]["integrity"]) + def create_recipe(args): import bb.process import tempfile @@ -523,6 +535,15 @@ def create_recipe(args): # Assume 'master' branch if not set if scheme in ['git', 'gitsm'] and 'branch' not in params and 'nobranch' not in params: params['branch'] = 'master' + npm_checksum = {} + if scheme == 'npm' and not any(p in params for p in bb.fetch.npm.CHECKSUM_PARMS): + try: + checksum_name, checksum = get_npm_checksum(network + path, params.get('package', ''), params.get('version', '')) + except Exception as e: + logger.error('Unable to get the checksum of %s from the registry: %s' % (srcuri, e)) + sys.exit(1) + npm_checksum[checksum_name] = checksum + params.update(npm_checksum) fetchuri = bb.fetch.encodeurl((scheme, network, path, user, passwd, params)) tmpparent = tinfoil.config_data.getVar('BASE_WORKDIR') @@ -587,6 +608,7 @@ def create_recipe(args): # Since we might have a value in srcbranch, we need to # recontruct the srcuri to include 'branch' in params. scheme, network, path, user, passwd, params = bb.fetch.decodeurl(srcuri) + params.update(npm_checksum) if scheme in ['git', 'gitsm']: params['branch'] = srcbranch or 'master' From patchwork Fri Nov 20 13:53:34 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105079 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6387.1795182829017688061 for ; Fri, 20 Nov 2026 05:53:49 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=yhxw8JqP; spf=pass (domain: gmail.com, ip: 209.85.128.50, mailfrom: stondo@gmail.com) Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-4bb72b44a97so7565855e9.0 for ; Fri, 20 Nov 2026 05:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182827; x=1795787627; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=qoktkxkMmeEsBX5a7whK/5fe/YLv/jsg29f9FuRfLZk=; b=yhxw8JqPv5+rKOoVJOvk31VJloZ4q468FhtMnG/PEMc9VtDZuadKS5WtN+nwxWQ6Nw fYvGKh2piBPa4UWS+QHs+q4E8bvDOsBHTeIwWXpn57bNyCBghcoqeBDBkCnQqkqhzW4x TDklCnB35VD7K75KmvnQBw3/1KOHuuPekPozd57vxdRDOU/OclqF5Czt0mVmsdDCaxMk 7a96FlLtydclBE+MCKshCxwp4HJTkoK+fnR1PVil5p+kq4g4OtY+GHUZSl82fmBIc/s6 rcywaUfoMFixkQCkzeIStg2aYSJb9FFy1cRkleUsYT4lVcl0M8Y53S1r7BAobVsyM7as FqcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182827; x=1795787627; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=qoktkxkMmeEsBX5a7whK/5fe/YLv/jsg29f9FuRfLZk=; b=Dsig9Ua61sHrWQhvQrreWlkePEHB+50SnYIJQWMHyUPFsMEBZdmhbm1qtmrAnwZNvq hd8Yz/Ew1+Nxnz8TZgHJqf7XsqbwF0L7piGveX8HkoU10a07lduFpxvbhBxDSw+ynKG7 a1gh4ZXlTsvx8/z5qh2M4koI9GNFNL5xEvqPYrBG6egbvWn2jeEYvNOM07QD2kzuvGb6 HFm+GRBMGhTJhi0IHsHSJOSRCwy6Taa9UfMxhua6Ir/2/ic97kp/bIy3YAHWdYh1d6MM uJB3/BlnHSbrtnm/2vleBwvbAZGmo64IySIuB8aI9cQlqe937+d/2SWU8X5992BIuJix cdTA== X-Gm-Message-State: APuU4Z+cJh2SI7Rz1wx3OfqfvOasPk1FfgZklbY2XZWzaQdIiinBNgwk LTQCpnlK5zRox9Yv7UIiPtT5eUj9ad0XUuvwTpGdqviR+HIZZ01XN93eyq/o/SCC X-Gm-Gg: AYBFou1K/Z0PWVaoUoH+qsiqnln90ate4Pe0neZoWDt1g6CKTWwS8WOLELrOZbYcJda kI7Q3RKfwUWUwmY8YiHFiyp3URHrbAiEjKtvQXCY2T2nGd29TJ10iICEH5U0ZrOBYMC4EfZwCGG S+u9BRbIDLQGBG4TG5b1e+uMo0dKAS9fUe+oGbwjomNcsHC4ZUfUtBswj8oykTpMBNzRwjj7kND 6NMdH4xULhIr7Om5Vlfcwp1WPJJ4DbTnylkSYM8P7UZaqno+7Jfoy2YpfWLTjtEhkTgW6FRaQgh +V72vRqlgmCQVO6CEPaLTqX8ZRE8HkQ1Jrnhs9KMxrZVLXYqbM9VsjizrCf4ennlL/GlsVbO3vb 4YvaEkU831pwMwFEDh9I/SC/W5yJbFcuaQfxG/R7f4JIxFEIH99Jlyonn3Cxbmb8a655/CEpXRF Xxz2iNKDMAva23VId33zemCOML8YhY+116il7VSM16LM1Va22GRUDhRjO6zZwuOOG9X0gj6m3jA gqP857WyrK3egbhlM8LajzTuZcEgCPszWlr5qLmB52viTa3QNm+dVqDSWV8UZsUeD+avDLuml4a +/NYxenGuijJ X-Received: by 2002:a05:600c:4515:b0:4bb:6f67:ca0b with SMTP id 5b1f17b1804b1-4bb888267f4mr72033495e9.29.1795182827160; Fri, 20 Nov 2026 05:53:47 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:46 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 03/10] recipetool: create_npm: Give the package fallback licenses as strings Date: Fri, 20 Nov 2026 14:53:34 +0100 Message-ID: <20261120135342.2628228-4-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:53:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250796 From: Stefano Tondo The npm handler finds the licenses of a dependency in its package.json when it has no license file. Since 727ea2769c ("license: Remove tidy_licenses()"), split_pkg_licenses() writes such a fallback value as it is. The npm handler gives a list, so the recipe gets lines like: LICENSE:${PN}-cookie-signature = "['MIT']" Join the licenses of each package with " AND ", as before that change. Signed-off-by: Stefano Tondo --- scripts/lib/recipetool/create_npm.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index 5edf647176..437cb91ad4 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -282,7 +282,8 @@ class NpmRecipeHandler(RecipeHandler): bb.note("Handling licences ...") (licfiles, packages, fallback_licenses) = self._handle_licenses(srctree, shrinkwrap_file, dev) licvalues = match_licenses(licfiles, srctree, d) - split_pkg_licenses(licvalues, packages, lines_after, fallback_licenses) + split_pkg_licenses(licvalues, packages, lines_after, + {pkg: " AND ".join(lics) for pkg, lics in fallback_licenses.items()}) fallback_licenses_flat = sorted(set(license for sublist in fallback_licenses.values() for license in sublist)) extravalues["LIC_FILES_CHKSUM"] = generate_common_licenses_chksums(fallback_licenses_flat, d) extravalues["LICENSE"] = " AND ".join(fallback_licenses_flat) From patchwork Fri Nov 20 13:53:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105080 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6389.1795182830191692025 for ; Fri, 20 Nov 2026 05:53:50 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=q3z0pOOO; spf=pass (domain: gmail.com, ip: 209.85.128.49, mailfrom: stondo@gmail.com) Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-49d05d51553so6184635e9.2 for ; Fri, 20 Nov 2026 05:53:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182828; x=1795787628; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=g+nWPS+sYOm8kcYbdd/JaIdRX6YVtZqKdNHWi7mMUy4=; b=q3z0pOOOYWP0JgM2O50qq8EfapPPCMoTdWQdNBFZiJqJCrF3wigenf9zdNfriIJIyM kp53uhVX/2A96buOJFGV1+33o7dYsXWMWaCBaUoeCLMP80uKdaUpfn8/iBuybylzFI4u vhim2iOY7qynblPnpKXProQ/gGup3ZVEhCKJUpavF3isOl/gXZz3RMhU/aH92cZIeNMI QQ3rCYUoOTZhJHMcrSPzdEWejXAmW3zm0sceGVEC9RCNz0AoDFUL74eOLKXSXWMN3fbF W3eEgt0BZPpaFBqiQFzDDHT83uJbdiY2bnVYgMjz72cakviZO0V8VMsawI7R0Ncsv0wH FhFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182828; x=1795787628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=g+nWPS+sYOm8kcYbdd/JaIdRX6YVtZqKdNHWi7mMUy4=; b=j3MMUGjLb31D3jBP97gphDxe8B51Il53xuBwkn5l7siaR6RBfq0ERiJxs4wadfwHmi 5ZYIIu17KB6xtph9ULAflbgzR+05cXgnbY8A/rA8MxrDCLl9fFLvHa2ZvrITLazyQZZA LRdyOURcrYYDmt8zlMrvbxPnZxsIM89jnzIsOQnWaXoz0+ofwTBjuvdAeRJbMRJ3pLC1 /npzJuV2uF9l19KH1M9L7sXP/OEK6UXPni3ePN7LqxCbuC4s3ZbDby2m/BErjNq/RyWu rPbggk/7eOhLHzLX0oyVfVOgISWwWTu6MpmPAWZK83OoqgV2VitdG1643/FrpXbuDtus Wnsg== X-Gm-Message-State: APuU4Z93T3LZqCv3+DRjeQzKKlW9A81Xhd1fx2iO2dwmKPhNlbawx1NX C62CULHZIF9S0WBkwP3PcbEHOLZJ5HjIAuoWJ1XGdVIUl+kJaynBb12V2v0lWJ1Q X-Gm-Gg: AYBFou3bwEj9LHmx0AHfidHPVOhMNfrUKMDKx4uaxsMkaIMBc+sbHvrL/A0Pplr2Hrx CeTA3N3nV/8XI5dHD/CnEfvdpGZBzdHk2oiScF51odedfskOI1PUVl9JFsK9zZkALRFZ9A8j2a4 lyOWCQLWJl9a7Vv71V77eMguKUk4yF+Ix88yE1xcEuxc9EIjHQv2nHlz6HU4bITgaFZathLMQ3a x2qHDUj5GZNrOLP7vIJ8Ru1KCKKTVSM/pk5cH6B3fbFOAomnswvKIaFYTj+0+K0sQrnRlGlN3rJ ayUU388ESIcDOBP7Mk/2TInhi96pgWhBK2wJe7xGCVKJYa68QZkHEfGgpQdixdZ8Pe5OXX16BWZ 5MIxe6BHOM5ejAqrR6js47/bZPL7jU8A2QqCcFkD5EQAmVbJzWWIVX889SHP9+DGVRW1Qkh64tw jhlNRYkQIm6QiZyGzOlC/fblHeRr0v4Lg1kBgSUQRrPsXAWQq8iMLUcRrnRplsbeiWJy/NhvhDV jqRyyrj1z2Qq+DdZkjvZpfM6UNvdNvm+OKmujSmZ29u66S8idBiBhZv+v0DFo8JsPKXwPZm+KNV wQQg/uuFuBg= X-Received: by 2002:a05:600c:3547:b0:4a2:a6e:80f5 with SMTP id 5b1f17b1804b1-4bb888098bfmr67461895e9.11.1795182828415; Fri, 20 Nov 2026 05:53:48 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:47 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 04/10] recipetool: create_npm: Remove the "@" of a scope from generated names Date: Fri, 20 Nov 2026 14:53:35 +0100 Message-ID: <20261120135342.2628228-5-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:53:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250797 From: Stefano Tondo Since bitbake commit 7c9573cb6, npm_package() keeps the "@" of a scoped package. This gives unique download file names. recipetool also uses npm_package() for the recipe name and the package names. So it creates "@savoirfairelinux-node-server-example_1.0.0.bb", and for a scoped dependency it writes overrides that bitbake cannot parse: LICENSE:${PN}-@-execute = "MIT" Remove the "@" from these names. The selftest expects the recipe name without it. Signed-off-by: Stefano Tondo --- scripts/lib/recipetool/create_npm.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index 437cb91ad4..50dd45c7d2 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -23,6 +23,12 @@ logger = logging.getLogger('recipetool') TINFOIL = None +def npm_recipe_name(package): + """Get a recipe or package name from an npm package name""" + # npm_package() keeps the "@" of a scope for unique download file names. + # An override cannot have it. + return npm_package(package).lstrip("@").strip("-") + def tinfoil_init(instance): """Initialize tinfoil""" global TINFOIL @@ -147,7 +153,7 @@ class NpmRecipeHandler(RecipeHandler): # Handle the dependencies def _handle_dependency(name, params, destdir): deptree = destdir.split('node_modules/') - suffix = "-".join([npm_package(dep) for dep in deptree]) + suffix = "-".join([npm_recipe_name(dep) for dep in deptree]) packages["${PN}" + suffix] = destdir _get_package_licenses(destdir, "${PN}" + suffix) @@ -171,7 +177,7 @@ class NpmRecipeHandler(RecipeHandler): peer_deps = packages.get("", {}).get("peerDependencies", {}) for peer_dep in peer_deps: - peer_dep_yocto_name = npm_package(peer_dep) + peer_dep_yocto_name = npm_recipe_name(peer_dep) bb.warn(peer_dep + " is a peer dependencie of the actual package. " + "Please add this peer dependencie to the RDEPENDS variable as %s and generate its recipe with devtool" % peer_dep_yocto_name) @@ -195,7 +201,7 @@ class NpmRecipeHandler(RecipeHandler): if "name" not in data or "version" not in data: return False - extravalues["PN"] = npm_package(data["name"]) + extravalues["PN"] = npm_recipe_name(data["name"]) extravalues["PV"] = data["version"] if "description" in data: From patchwork Fri Nov 20 13:53:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105081 Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6390.1795182833324185139 for ; Fri, 20 Nov 2026 05:53:53 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=N0OwUyLh; spf=pass (domain: gmail.com, ip: 209.85.128.46, mailfrom: stondo@gmail.com) Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-4bb6886d30aso8332805e9.3 for ; Fri, 20 Nov 2026 05:53:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182831; x=1795787631; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=c85RqZbOBafUb/rHdfV0pA5S80yvde7LFp49XKEquLY=; b=N0OwUyLhm64r9h3HmSHyd5D599BaNSbRzk2iqGMcyRUddJncDB4G6XrKic0WPuNnwJ hsr+nf7knd2CW/StkdoCSwrJzKDPT//44k+HIQqa0Xs9tn5fF3N32trx5guWbyzPlBV7 qeJAOiAm/i6NJ5HSls+zWIKOYujmrr6TIZJtu+4L6Bv7HciTPLChC418aSIFV5KeoNoj 6V/orPrJNSYnt7mG4e4U9SvOg36/+ntFNfmFyfqoikCsX762yHsnpj0ZL1zJi/n/lenB 4f8u1rb1vxq+QZnLjNaa6BDcs4THctQCR8O4gcvccv9r89F9yk6SIbpCx3PdBqPqMSGv JqaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182831; x=1795787631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=c85RqZbOBafUb/rHdfV0pA5S80yvde7LFp49XKEquLY=; b=FGeHptgWVYumee4sr4qNkpTz7tLlblHd8sv0Merqe74hX3j0vJEnHZCunOtkG4hnWf CTz7x0Cq8YnoaxbE4t/Zk3+sdEUSjoxApXM2s8BYLJVqRm+jC3zGxEp7Ihi1dJWfoixt 32f/THUrUAF5RQnxVWXIfbUPvU6oXVbPwBkeQ1g9iuHYOVSGzptqrfO4jb7oRq6N/uQQ Ul6f8Q2Bvj00QoMDJHCbGvcBzh3+sp6ab9sfam4hTsimIfx+VYKOW8ErSfVOh8atRPVt uOSL7uh2iZOWewIx3o29uPW5VbJqGltg/Zpd3tOc7v75vx3unNDnuSZtfGURiEiyiHoz acHA== X-Gm-Message-State: APuU4Z94yD/GrmK/pQBn3/tEUPJhI3kBW25hfPSQfl4EpFydVlwYiFqc VJB0eDZvCYUcMBFO5Hw5JCLvCz/7tpllckm+w4abwNHxHGNa7dtFBTqgK8CaINPx X-Gm-Gg: AYBFou1cfxfKX+xHSwZ4EAhcp1So3AAHR6RsNMr6hJw7mvDTjef8pSXVdUaq55TLcb5 IpIG69ru5crcjql1wfxZnCQEq+VmIbskAbqY8cooNTvW0VCPXoUw08F7F/8B1nKmWj7nIHU9eA4 eUtYim8uyPpzWKvwZHT3NYavIbtHDQL47WyvdYuYVubbYQ7o2f9NNox5+WDOkTv/u9pcI6Uj1vE YpzoXLTqzIWzjlEaHNRgY+lWOfaT34xzVg4FvdMCQWbxHh/mqWykf3gDHafVImbYRAkS8iO8iDm DVrZpK93aGlj/OEctcAx7M++EaXYJ6Wxeb1FHdPjAs3CBAh/lQ4m61ecfKzhlxabsrU+eyd+ep8 uD+/Q24u9Anv6G6ulQAktm3zhz24hmHltFRnB96DhPt3CD3nY+0DeU029zhDYzFI7zrYsmJJdgN pRW6mnbNZQK8zu1gI645LCTq6/wg1X354F/j2ux96pc5UTm3dKfRjRvjHemTuQ82TPggu2SWsum CCIAUjqpsBKpmIguIe2hVmSVyxUUcs0BIXQ2qy/ma25SWrZmz2c0jD7W/Y5r9tcYVfeAmnlV0xb MA9ZTSU4UwQ= X-Received: by 2002:a05:600c:64c7:b0:4bb:68b9:f610 with SMTP id 5b1f17b1804b1-4bb887ffee2mr66227375e9.9.1795182831601; Fri, 20 Nov 2026 05:53:51 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:48 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 05/10] oeqa/selftest: recipetool: Expect the npm source in UNPACKDIR Date: Fri, 20 Nov 2026 14:53:36 +0100 Message-ID: <20261120135342.2628228-6-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:54:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250798 From: Stefano Tondo recipetool writes S = "${UNPACKDIR}/npm" for an npm package, because the fetcher unpacks into UNPACKDIR. The selftest still expected "${WORKDIR}/npm". Signed-off-by: Stefano Tondo --- meta/lib/oeqa/selftest/cases/recipetool.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py b/meta/lib/oeqa/selftest/cases/recipetool.py index c6aa77399f..ba4f674b4c 100644 --- a/meta/lib/oeqa/selftest/cases/recipetool.py +++ b/meta/lib/oeqa/selftest/cases/recipetool.py @@ -454,7 +454,7 @@ class RecipetoolCreateTests(RecipetoolBase): '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223') urls.append('npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json') checkvars['SRC_URI'] = set(urls) - checkvars['S'] = '${WORKDIR}/npm' + checkvars['S'] = '${UNPACKDIR}/npm' checkvars['LICENSE:${PN}'] = 'MIT' checkvars['LICENSE:${PN}-base64'] = 'Unknown' checkvars['LICENSE:${PN}-accepts'] = 'MIT' From patchwork Fri Nov 20 13:53:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105082 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6391.1795182835328005592 for ; Fri, 20 Nov 2026 05:53:55 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=SdSe/mh5; spf=pass (domain: gmail.com, ip: 209.85.128.50, mailfrom: stondo@gmail.com) Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-4a214c5c4c0so8254225e9.3 for ; Fri, 20 Nov 2026 05:53:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182833; x=1795787633; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=T4svXfMP+7Pl3KnEXyBVA4XOxrgtbUv6S1dBfgT2lzI=; b=SdSe/mh5QZEQJYTLEKsZiMxoNyUMr0zT28LyqQETKuHNQEMBExrntaY/FFSZq1ipdH es/HMNGGyKXR9T12c4TBbSkbVf0NwNW5rUmr4grUnE2wMcjnDrNw3zLnCY/GgABXG51H 2pwEUudHiP/Ka5q2ZQdhEZrvilP4B2k1B/NhyZ89csrJJUrH8i6iw9Yya96EuKhfYe0j GRj/Vj2J3TdCB3toxZqrA08jUcCgrX7YFX2nezk67pmjOIy+a37pRxMOIzUQIULtLxAV 8OD3Zrn63hAk2QFBJwbxyTa/bWMI2PQRUZKrQnTy5RqAA2lDr24t87tK7voW/4Ph4uM3 2E8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182833; x=1795787633; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=T4svXfMP+7Pl3KnEXyBVA4XOxrgtbUv6S1dBfgT2lzI=; b=e8l+ULmgUQLJMFdj4/nWj8Mz68iVK9v71xKEK5OdLmou9RrgaXRfyG11VDFHnHxV8K eYnW+ZVVw2Ats57AUisGhSLq/Dz+SgZRVbytJQbcgxrQr0cOqjfjuvrjzQiffKPO8BTS OGb0gXCwkpBP1b0dW6tfAYiCKxky35pTmYtmh0v9SmQ3lZOtWFF9OlD0GloCXMUTRTby 7FKj5pWD0z8PF1hizAwHPZQK/VFlzKpqXpDNXedlUR6xP3r+wdtaSlhOffkdXigWp4Ao UIERrtoC5omozgBAQAlgkblW+v9DOICGGdMNg8al1wHOVvccgijoVntz2ShmjFprLbxT vLYA== X-Gm-Message-State: APuU4Z8PcPe7NM0WxVP8vJPAVYWPqNsDwEZ1GgQUXzT5agbctaXo0ylH frmEogiZFb2NEYW0esyMTDKae1L0nn82Kv1gdQc/+GYrsI/i6YsQWgt0Wr5J1+xp X-Gm-Gg: AYBFou2TOIwG++6xCwBJiY73mHO0khBiWCAUT3Lw8MTGDHPD9b93vpwg5u0+88Ekxbb pxVZZHcIlKTsJ01DovlkLqu2A82vuWh/TH3qCeepgLNII5eEvWN37eK3KxcjAWwUjvBwGyDAQMJ 7pR5D6DRuNJwEez5MJozgux6VlsMbNGeJj7PNTbEyMwVLAAZtpnef4YdNmYXwxJ28r9NOvJCtA6 JM3VjTawa4iSGJuYzCrKg0pFSvG3iT7rMgdfUfnkrdm1kZ7u3rRF2iOjnKCbdkMNaEBWyllg7gp nEsFbfqiq9qdtqRqj+mYQPwctVSjFr8m6SqelWfsFP+3Ng0/E0MhK3o2UgTe4vpG373ynE2n7le wAXySUjdA3yiEg+btaoufdCdRxjO3CLzGaxdQ3OHxgaEQJQ9QBGfH6SQwq5G7cmDLalDwlnk0xU SCvP3PsdcgI+j6/WgG9+//xZcXYofw8kDqleD4pgNJUf/ozcuo+1lPcH6Cp3IlXjy41JNiQDbwM 6QQAmOCOtK6X9p3NjjMzPmBVVaiNLeAQMbHQO29BXWmrt07dXLs/811oR432M/x/XSmPY7qGEtA afi8wm3s9K0= X-Received: by 2002:a05:600c:154c:b0:4a2:729:f7ce with SMTP id 5b1f17b1804b1-4bb8881c378mr72838085e9.8.1795182833457; Fri, 20 Nov 2026 05:53:53 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:52 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 06/10] lib/oe/npm: Add SRC_URI entries for the dependencies of a shrinkwrap file Date: Fri, 20 Nov 2026 14:53:37 +0100 Message-ID: <20261120135342.2628228-7-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:54:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250799 From: Stefano Tondo The npmsw fetcher reads npm-shrinkwrap.json to find the dependencies of an npm package. So SRC_URI does not show these dependencies, and the fetcher holds logic that belongs in the metadata. Add shrinkwrap_src_uri(), which gives one SRC_URI entry for each dependency of a shrinkwrap file: - A package from a registry gets an npm:// entry with its name, version and checksum. - Another tarball gets its URL with the checksum. - A git dependency gets a git:// entry with its revision. Each entry unpacks to the location of the dependency in the source tree, where npm.bbclass expects it. Add src_uri_differences(), which compares these entries with SRC_URI. Add tests for both functions. Signed-off-by: Stefano Tondo --- meta/lib/oe/npm.py | 99 ++++++++++++++++++ meta/lib/oeqa/selftest/cases/oelib/npm.py | 121 ++++++++++++++++++++++ 2 files changed, 220 insertions(+) create mode 100644 meta/lib/oe/npm.py create mode 100644 meta/lib/oeqa/selftest/cases/oelib/npm.py diff --git a/meta/lib/oe/npm.py b/meta/lib/oe/npm.py new file mode 100644 index 0000000000..19f547839e --- /dev/null +++ b/meta/lib/oe/npm.py @@ -0,0 +1,99 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: GPL-2.0-only +# + +"""Explicit SRC_URI entries for the dependencies of an npm shrinkwrap file.""" + +import os +import re + +import bb.fetch +import bb.utils +from bb.fetch.npm import npm_integrity, npm_localfile +from bb.fetch.npmsw import foreach_dependencies + +GIT_RESOLVED_RE = re.compile(r"^git\+(?P[a-z]+)://(?P[^#]+)#(?P[0-9a-f]+)$") + + +def shrinkwrap_src_uri(shrinkwrap, destdir, dev=False): + """Return one SRC_URI entry for each dependency of a shrinkwrap file. + + shrinkwrap is the content of npm-shrinkwrap.json (lockfileVersion 2 or + later). destdir is the directory of the main package, relative to + UNPACKDIR. Each dependency unpacks to its location in destdir (for example + destdir/node_modules/foo), where npm.bbclass expects it. dev also adds the + devDependencies. The entries are sorted by location. + + A package from a registry gets an npm:// entry, so that the fetcher knows + its name and version. Other tarballs get a plain URL and git dependencies + a git:// URL. Each entry has the checksum or revision of the shrinkwrap + file. + """ + entries = {} + + def _add(name, data, location): + package = data.get("name", name) + version = data.get("version") + resolved = data.get("resolved") + integrity = data.get("integrity") + destsuffix = os.path.join(destdir, location) + + if not resolved: + # npm leaves out "resolved" for some duplicate packages. npm.bbclass + # does not need these, as npmsw already skipped them. + if version and integrity: + return + raise ValueError("%s: no 'resolved' field" % location) + + if data.get("link") or resolved.startswith("file:"): + raise ValueError("%s: local dependencies are not supported" % location) + + match = GIT_RESOLVED_RE.match(resolved) + if match: + entries[location] = "git://%s;protocol=%s;rev=%s;nobranch=1;destsuffix=%s" % ( + match["url"], match["protocol"], match["rev"], destsuffix) + return + + if not integrity or not resolved.startswith(("https://", "http://")): + raise ValueError("%s: unsupported dependency %s" % (location, resolved)) + + checksum_name, checksum = npm_integrity(integrity) + basename = package.split("/")[-1] + suffix = "/%s/-/%s-%s.tgz" % (package, basename, version) + if resolved.startswith("https://") and resolved.endswith(suffix) and bb.utils.is_semver(version): + registry = resolved[len("https://"):-len(suffix)] + entries[location] = "npm://%s/;package=%s;version=%s;%s=%s;destsuffix=%s" % ( + registry, package, version, checksum_name, checksum, destsuffix) + else: + entries[location] = "%s;downloadfilename=%s;%s=%s;subdir=%s;striplevel=1" % ( + resolved, npm_localfile(os.path.basename(resolved)), checksum_name, checksum, destsuffix) + + foreach_dependencies(shrinkwrap, _add, dev) + return [entries[location] for location in sorted(entries)] + + +def src_uri_differences(expected, src_uri): + """Compare the expected entries with SRC_URI. + + Returns (missing, extra): the expected entries that SRC_URI does not have, + and the SRC_URI entries that unpack into a node_modules directory but are + not expected. Entries are compared after parsing, so the order of the URL + parameters does not matter. + """ + def _key(url): + scheme, host, path, user, password, parm = bb.fetch.decodeurl(url) + return (scheme, host, path, user, password, tuple(sorted(parm.items()))) + + def _unpacks_to_node_modules(url): + parm = bb.fetch.decodeurl(url)[5] + dest = parm.get("destsuffix") or parm.get("subdir") or "" + return "node_modules/" in dest + "/" + + expected_keys = {_key(url): url for url in expected} + actual_keys = {_key(url): url for url in src_uri} + missing = [url for key, url in expected_keys.items() if key not in actual_keys] + extra = [url for key, url in actual_keys.items() + if key not in expected_keys and _unpacks_to_node_modules(url)] + return missing, extra diff --git a/meta/lib/oeqa/selftest/cases/oelib/npm.py b/meta/lib/oeqa/selftest/cases/oelib/npm.py new file mode 100644 index 0000000000..df4f6aff5b --- /dev/null +++ b/meta/lib/oeqa/selftest/cases/oelib/npm.py @@ -0,0 +1,121 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +import base64 +from unittest.case import TestCase + +import oe.npm + +CHECKSUM = bytes(range(64)).hex() +INTEGRITY = "sha512-" + base64.b64encode(bytes(range(64))).decode() + + +def shrinkwrap(packages): + """Return a shrinkwrap file with the given dependencies""" + return {"lockfileVersion": 3, "packages": {"": {"name": "main"}, **packages}} + + +def registry(package, version, host="registry.npmjs.org"): + """Return a dependency from a registry""" + basename = package.split("/")[-1] + return { + "version": version, + "resolved": "https://%s/%s/-/%s-%s.tgz" % (host, package, basename, version), + "integrity": INTEGRITY, + } + + +class TestShrinkwrapSrcUri(TestCase): + def test_registry(self): + entries = oe.npm.shrinkwrap_src_uri(shrinkwrap({ + "node_modules/send": registry("send", "0.19.0"), + "node_modules/send/node_modules/ms": registry("ms", "2.1.3"), + "node_modules/@scope/pkg": registry("@scope/pkg", "1.0.0", "npm.example.com/path"), + }), "npm") + self.assertEqual(entries, [ + "npm://npm.example.com/path/;package=@scope/pkg;version=1.0.0;sha512sum=%s;destsuffix=npm/node_modules/@scope/pkg" % CHECKSUM, + "npm://registry.npmjs.org/;package=send;version=0.19.0;sha512sum=%s;destsuffix=npm/node_modules/send" % CHECKSUM, + "npm://registry.npmjs.org/;package=ms;version=2.1.3;sha512sum=%s;destsuffix=npm/node_modules/send/node_modules/ms" % CHECKSUM, + ]) + + def test_alias(self): + # An alias installs a package under another name: the "name" field + # gives the real package + dep = registry("real", "1.0.0") + dep["name"] = "real" + entries = oe.npm.shrinkwrap_src_uri(shrinkwrap({"node_modules/alias": dep}), "npm") + self.assertEqual(entries, [ + "npm://registry.npmjs.org/;package=real;version=1.0.0;sha512sum=%s;destsuffix=npm/node_modules/alias" % CHECKSUM, + ]) + + def test_tarball(self): + entries = oe.npm.shrinkwrap_src_uri(shrinkwrap({ + "node_modules/foo": { + "version": "1.0.0", + "resolved": "https://example.com/downloads/foo.tgz", + "integrity": INTEGRITY, + }, + }), "npm") + self.assertEqual(entries, [ + "https://example.com/downloads/foo.tgz;downloadfilename=npm2/foo.tgz;sha512sum=%s;subdir=npm/node_modules/foo;striplevel=1" % CHECKSUM, + ]) + + def test_git(self): + entries = oe.npm.shrinkwrap_src_uri(shrinkwrap({ + "node_modules/foo": { + "version": "1.0.0", + "resolved": "git+ssh://git@github.com/example/foo.git#0123456789abcdef0123456789abcdef01234567", + }, + }), "npm") + self.assertEqual(entries, [ + "git://git@github.com/example/foo.git;protocol=ssh;rev=0123456789abcdef0123456789abcdef01234567;nobranch=1;destsuffix=npm/node_modules/foo", + ]) + + def test_skipped(self): + dev = registry("dev", "1.0.0") + dev["dev"] = True + bundled = registry("bundled", "1.0.0") + bundled["inBundle"] = True + no_resolved = registry("dup", "1.0.0") + del no_resolved["resolved"] + packages = { + "node_modules/dev": dev, + "node_modules/bundled": bundled, + "node_modules/dup": no_resolved, + } + self.assertEqual(oe.npm.shrinkwrap_src_uri(shrinkwrap(packages), "npm"), []) + self.assertEqual(len(oe.npm.shrinkwrap_src_uri(shrinkwrap(packages), "npm", dev=True)), 1) + + def test_unsupported(self): + for dep in ({"version": "1.0.0", "resolved": "file:../foo"}, + {"link": True, "resolved": "../foo"}, + {"version": "1.0.0"}): + with self.assertRaises(ValueError): + oe.npm.shrinkwrap_src_uri(shrinkwrap({"node_modules/foo": dep}), "npm") + + +class TestSrcUriDifferences(TestCase): + EXPECTED = [ + "npm://registry.npmjs.org/;package=foo;version=1.0.0;sha512sum=%s;destsuffix=npm/node_modules/foo" % CHECKSUM, + "npm://registry.npmjs.org/;package=bar;version=2.0.0;sha512sum=%s;destsuffix=npm/node_modules/bar" % CHECKSUM, + ] + + def test_match(self): + # The order of the entries and of the URL parameters does not matter. + # Entries that do not unpack into node_modules are not extra. + src_uri = [ + "npm://registry.npmjs.org/;package=main;version=1.0.0;sha512sum=%s" % CHECKSUM, + "file://npm-shrinkwrap.json;subdir=npm", + self.EXPECTED[1], + "npm://registry.npmjs.org/;destsuffix=npm/node_modules/foo;sha512sum=%s;version=1.0.0;package=foo" % CHECKSUM, + ] + self.assertEqual(oe.npm.src_uri_differences(self.EXPECTED, src_uri), ([], [])) + + def test_missing_and_extra(self): + old_bar = "npm://registry.npmjs.org/;package=bar;version=1.0.0;sha512sum=%s;destsuffix=npm/node_modules/bar" % CHECKSUM + missing, extra = oe.npm.src_uri_differences(self.EXPECTED, [self.EXPECTED[0], old_bar]) + self.assertEqual(missing, [self.EXPECTED[1]]) + self.assertEqual(extra, [old_bar]) From patchwork Fri Nov 20 13:53:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105083 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6326.1795182837376713429 for ; Fri, 20 Nov 2026 05:53:57 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=DdICw9uj; spf=pass (domain: gmail.com, ip: 209.85.128.43, mailfrom: stondo@gmail.com) Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4a35a60499eso7219025e9.0 for ; Fri, 20 Nov 2026 05:53:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182836; x=1795787636; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=v8pbqMvWDNioJB4Ic6GYugair7N4FYILDOpwzxqIKn0=; b=DdICw9ujh5idArPXCYmV/UJ47cu5OMb/UyqhaQ2GjtFbJcgw4igtOyOTePLxhgvLWD VABIcDMkbiJdbntyu/xztOmwWIUFMA7x3dbz//mcNvM2NGu1zae0iVTZ2HERhKPGLgFg wD0YzvCT6ObcjYqANlnm0rNSi4hKrp6YtO5U2PrS63FYGpfKljPLaSXYbQegZgPdaFrR yEc86x8dSwDLYW059MN0A6lBrQ28S5CoBii/pBBrkb+kJ9KNY2kJUVYBMhPNIHqxrmhv pG6EiohqwnqXGaHLVcOxceH20Sh3/uGxOezk4XrpNqDJTuvWpDRvWDcQ2haOQgZlXLjz PZhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182836; x=1795787636; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=v8pbqMvWDNioJB4Ic6GYugair7N4FYILDOpwzxqIKn0=; b=xVvDkGYK8DEvwBTnIHeySief8k3sMWIxYySxQjBXR+/6aZA2jFPD88dcSb947oEl4L DVJUjAiXYdlm/8AGb7HUA+Hl50MlMxExdB+CS2ONH0I1mxjQKaJvCPMWyod2QMjoSqK7 ymyiuY2l3br6TirmcfN3xkM0A1JLGfQVS/ogKN8Z+B4RE6++GCKUn8aMH0dPR80DNx3s /7YRajcVG7w8hdPce9aMTVMNNdgMbun6D5fuVi/QAzse2e2yCuRqON+pD2/yM4TrVaAV Kq0cJ/jYg5rK7YiDrV1ItfHA2sISuAOCi3trXCHYRPqWKBTqteGM2vvRDj+KQexfYgAG iu0A== X-Gm-Message-State: APuU4Z+Mdf2wsDv3w1+/ZglNdWqIm3z3xwAp7FtwBANM4kztPIe33ZIS tqxK9R5QMJa+kiJbppi6GzbGoN4ZqglL6g4h9Iv5BFPZxCIY0RR7wy2WK2h/QySp X-Gm-Gg: AYBFou0xomC3AfaBo1yaNacAba8aC6A7A1+k2uxPNiXNgxpq0DNW0BBxIlNRPvv86vp oPQJuPBvVhmnPAQhi8AhiO3bxt8usTfTbOtgzFiJtVnZmBLTDdDq1/hMchCDOi0FzT+ir7OO9x7 Ttwgi3gqG2AasvNk0Seelhm1XdyJlg2qMsZHAX4VNcCGvT7GR0zBeQh18HOaTAAgKS7O5ygZpry o5acsAJl5iloAVcs1ULc7Uh8CSoggRzys+iXOtHuM2FFmFuHcAqK5AE+bhXJDPoW3x1/Ke3s0El 7h7z4Xxdo95BpSBOg36WLMeIh2/J6PwrIWDfo42wNQB283vs/n/t288L4f+x8MREBl+zXWemGl5 aKU/+aiZQAVO3PpXHWAr/kJKi9g1jeMjU7HpDLY0wwvDnc6EylJ+os2zMbb45+6SyZ9tuiMSh1F ECiBOIgzztMbMclf9DNK2Q14QTRu9fBDKnVV/kZrbPpUuhzUW5xA2wnsE25adeNhDGBSukwjT8g OV+9sBxRvH4jfHbvcJw6/P++RZGfRRzsfn5rBLrFd4yvqHKQlwuZfKdyJbffj3j/XiB+O9GPq7s 0n2JOzkIS1/ew8WpMzhroE/QSQYI2Gmg X-Received: by 2002:a05:600d:6510:10b0:4a1:811c:8078 with SMTP id 5b1f17b1804b1-4bb8881de57mr48311695e9.26.1795182835502; Fri, 20 Nov 2026 05:53:55 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:53 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 07/10] npm-update-dependencies: Add a class to list the npm dependencies Date: Fri, 20 Nov 2026 14:53:38 +0100 Message-ID: <20261120135342.2628228-8-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:54:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250800 From: Stefano Tondo Add a class with the task do_update_npm_deps. The task reads npm-shrinkwrap.json in ${S} and writes one SRC_URI entry for each dependency to ${THISDIR}/${BPN}-npm-deps.inc. The recipe requires this file. This is the same model as the cargo and Go update tasks. Then the recipe does not need npmsw://. The fetchers see each dependency as a normal SRC_URI entry with its own checksum. Signed-off-by: Stefano Tondo --- .../npm-update-dependencies.bbclass | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 meta/classes-recipe/npm-update-dependencies.bbclass diff --git a/meta/classes-recipe/npm-update-dependencies.bbclass b/meta/classes-recipe/npm-update-dependencies.bbclass new file mode 100644 index 0000000000..43b00b9b1b --- /dev/null +++ b/meta/classes-recipe/npm-update-dependencies.bbclass @@ -0,0 +1,48 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +## +## Purpose: +## This class is used to update the list of npm dependencies in SRC_URI +## by reading npm-shrinkwrap.json in the source tree. It writes one entry +## with a checksum for each dependency, so that no fetcher has to read the +## shrinkwrap file. npm.bbclass checks that SRC_URI matches the shrinkwrap +## file. +## +## To perform the update: bitbake -c update_npm_deps recipe-name + +addtask do_update_npm_deps after do_patch +do_update_npm_deps[nostamp] = "1" +do_update_npm_deps[doc] = "Update the recipe by reading npm-shrinkwrap.json and write in ${THISDIR}/${BPN}-npm-deps.inc" + +RECIPE_UPGRADE_EXTRA_TASKS += "do_update_npm_deps" + +python do_update_npm_deps() { + import json + import oe.npm + + shrinkwrap_file = d.expand("${S}/npm-shrinkwrap.json") + with open(shrinkwrap_file) as f: + shrinkwrap = json.load(f) + + destdir = os.path.relpath(d.getVar("S"), d.getVar("UNPACKDIR")) + dev = bb.utils.to_boolean(d.getVar("NPM_INSTALL_DEV"), False) + try: + entries = oe.npm.shrinkwrap_src_uri(shrinkwrap, destdir, dev) + except ValueError as e: + bb.fatal("%s: %s" % (shrinkwrap_file, e)) + + lines = ["# Autogenerated with 'bitbake -c update_npm_deps %s'" % d.getVar("PN"), ""] + lines.append('SRC_URI += " \\') + lines.extend(" %s \\" % entry for entry in entries) + lines.append('"') + + target_file = d.expand("${THISDIR}/${BPN}-npm-deps.inc") + with open(target_file, "w") as f: + f.write("\n".join(lines) + "\n") + + bb.note("Successfully updated %d npm dependencies inside %s" % (len(entries), target_file)) +} From patchwork Fri Nov 20 13:53:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105084 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.6392.1795182838765092756 for ; Fri, 20 Nov 2026 05:53:59 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=VqJAwY45; spf=pass (domain: gmail.com, ip: 209.85.128.53, mailfrom: stondo@gmail.com) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-4ba013938d7so8364465e9.2 for ; Fri, 20 Nov 2026 05:53:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182837; x=1795787637; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=dh3CddWyAc8EdAOuyBMk6UWEKR1I6aiNupgWVQ2ZM5A=; b=VqJAwY45HugSuSPDmUgg4Nzg+T5Z1EY5tWf+7Hmb1bXCf8rIu7U59GgCt4MX/AS2/A t7uyMZjyTek6cbVt9qf6UmLgK6sykdI3qCjcVel3nbEdwMPIrmHbmBTZSt3AxEavLgXB qod5SSmsKAbvYRzzCYHUkk71yi8ls0LLDVCVtQTjFg6XtiV3MltT7qiwohyA/Cycj+qM oPiNfrOAMeaacXc56STY2Tmuo/aRjGL8muP4qzgydSdzx83eTiqohCDonQKxQ2wxSK2l ongK6+FBBLyJ48ZZJSEQuM+o8dqILz/BYNqq7MCDKptq7RJzHOHYATQdD07ujkctRp/Z pG+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182837; x=1795787637; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=dh3CddWyAc8EdAOuyBMk6UWEKR1I6aiNupgWVQ2ZM5A=; b=Tengk4slhQZSobAWH1UZ+zrHBkkhhpzyaDoZ3O+ltyMXl98T7qt2AiYejvS3i0BFgF Z07vq42Wz9B2eODh+5340BwEkildMqYdC4YPZgMy2sn8Jjd7AZVYD5Ro9EjksrFElXrv cAqY1z+7nji1RKZqW17YBiRJOOMmLys/t4LeBilOCptwfKyKUXwJgrpPiRvSq4sTEf8U gsmVUhz/ZENqS3sxajIktpCjLvFwHSV6crXiE0ZBE74dhP75XqyCYh/5TMzWJYUL8z0/ spWVRZaqUYVoyZkUQ7JCqXa3oF1nKEneWu8ert98Lfx2JTr1ElyP6MoqmCMxkO2Orlpt 3iig== X-Gm-Message-State: APuU4Z/04AsOd9lUEBk4lquu5FfABe00gnO6ZvL2ZQIZf6+UZK6oVaYN SdaCe2itg3hvSodcOmWu2cm3mtiMFZOokb8djuLtByOAJWnbw5Ni3vyYQNTPyAnd X-Gm-Gg: AYBFou0V8XPYtvpzwcWDH5aZRUI+s+rD7Omyt0/glBGIEGcuhz8X2lKU+/pFEL1/5za E+PdmjD5MeJqKeANeOb/+U81WOs3bev7mK+No2SFeqg6Ew51NsO1r69KLaVFghCHMlsSK8I6j1x bQsdW3gum0uFgW5pohXUZPngmVhCHuwz6ZKMZTd+ffa5bkN5Mjs9888hVwaAyjQBWWnB/2a+4i1 KDHgmnVTgj0zJAQ3PwXDdRy6nddY7MTu/Ou4LmnSAOdEvbUrKHtgoArJUjBx7rYBBrKkh8/TqFI p3g5pDRLU0vvsbpYAAfwZm3M2lfNYtrZyqwGJ6wGble/hwZgkQEuFypxaX8murUY4ZXCHOPc//J pZbYds8MLPq4GCE7Lrpz6qOr/uETHNaRV51Zh7ZWcndQV9KHP+ZWWJRY+sFGDRgO968Voc3Dzmf Xzw3kIBc/ID2ixGin8XLaD9fnT91otUKv9FSEO6NNPAglA5onSlCaftlwp0OSGtWopQ2PWOMowq CZkH7WkO9QqN6Cksmo7VRoGVdCRxAvI3WKVXTVq9DA0ya4Lim8NsVp5TV8/kB+f1ENWY8SeAY7R i8fodxBunAE= X-Received: by 2002:a05:600c:6749:b0:4bb:8508:7fb3 with SMTP id 5b1f17b1804b1-4bb8881c3b3mr66589475e9.11.1795182836909; Fri, 20 Nov 2026 05:53:56 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:56 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 08/10] npm: Check that SRC_URI lists all dependencies of the shrinkwrap file Date: Fri, 20 Nov 2026 14:53:39 +0100 Message-ID: <20261120135342.2628228-9-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:54:02 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250801 From: Stefano Tondo When a recipe lists the npm dependencies in SRC_URI, the list can become different from npm-shrinkwrap.json, for example after an upgrade. Then npm.bbclass packs a wrong set of dependencies. If SRC_URI has no npmsw:// entry, compare SRC_URI with the shrinkwrap file in do_configure. Fail if a dependency is missing, or if an entry unpacks into node_modules but the shrinkwrap file does not have it. The error tells the user to run the update task. Do not check with externalsrc, which removes the dependencies from SRC_URI. There the external source tree must contain them. Signed-off-by: Stefano Tondo --- meta/classes-recipe/npm.bbclass | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/meta/classes-recipe/npm.bbclass b/meta/classes-recipe/npm.bbclass index 6f3261238d..e0d3d88d2b 100644 --- a/meta/classes-recipe/npm.bbclass +++ b/meta/classes-recipe/npm.bbclass @@ -111,6 +111,24 @@ def npm_pack(env, srcdir, workdir, excludes=None): return (tarball, j) +def npm_check_src_uri(d, shrinkwrap): + """Check that SRC_URI has one entry for each dependency of the shrinkwrap file""" + import oe.npm + + destdir = os.path.relpath(d.getVar("S"), d.getVar("UNPACKDIR")) + dev = bb.utils.to_boolean(d.getVar("NPM_INSTALL_DEV"), False) + try: + expected = oe.npm.shrinkwrap_src_uri(shrinkwrap, destdir, dev) + except ValueError as e: + bb.fatal("npm-shrinkwrap.json: %s" % e) + + missing, extra = oe.npm.src_uri_differences(expected, d.getVar("SRC_URI").split()) + if missing or extra: + bb.fatal("SRC_URI does not match npm-shrinkwrap.json (%d missing, %d extra entries). " + "Inherit npm-update-dependencies and run 'bitbake -c update_npm_deps %s'.\n%s" + % (len(missing), len(extra), d.getVar("PN"), + "\n".join(["missing: " + url for url in missing] + ["extra: " + url for url in extra]))) + python npm_do_configure() { """ Step one: configure the npm cache and the main npm package @@ -177,6 +195,12 @@ python npm_do_configure() { if int(orig_shrinkwrap.get("lockfileVersion", 0)) < 2: bb.fatal("%s: lockfileVersion version 2 or later is required" % orig_shrinkwrap_file) + # externalsrc removes the dependencies from SRC_URI: the external + # source tree must contain them + if not d.getVar("EXTERNALSRC") and \ + not any(url.startswith("npmsw://") for url in d.getVar("SRC_URI").split()): + npm_check_src_uri(d, orig_shrinkwrap) + cached_shrinkwrap = copy.deepcopy(orig_shrinkwrap) for package in orig_shrinkwrap["packages"]: if package != "": From patchwork Fri Nov 20 13:53:40 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105085 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6329.1795182840367305114 for ; Fri, 20 Nov 2026 05:54:00 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=GpZfRrQF; spf=pass (domain: gmail.com, ip: 209.85.128.42, mailfrom: stondo@gmail.com) Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-4bb6c7b6dbdso9123565e9.0 for ; Fri, 20 Nov 2026 05:54:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182839; x=1795787639; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=dSI+31ZgQbseRx/DeUkISc7ekuIlP5XcIMXf9pbqaEs=; b=GpZfRrQFySkAjFbT3/kUV3upfVyZI6kVKBZVReaVPTmf4Bi8gka2E7unnVylRtv5a2 OldNRwBQmH1YgB77cZFZf/jw3wwPaVwL5oPZnGeF4bsEuHU1vNM9Oicg2DIXC/4RAzXW WYDW/nFy2aoX4T7FyVzB7ucrNv6t5InxCYLLW/G9A1/R7EVn9isUsvC3ijFmyoJVsbf7 HoJVTs5ZnOBIM4LP+07gEi4QK3aBz00zKYZtK3U893LhacDft3EdWl3yH5ymsbLDXAIk S6ZM0EBqBaSRUleTu+WLH89AMqY3uzxKFdaYyUJnWw7UQJHq1ePbqAE6EK7lP4HxY0ux OYRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182839; x=1795787639; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=dSI+31ZgQbseRx/DeUkISc7ekuIlP5XcIMXf9pbqaEs=; b=t8gx3mqKRr9Dm0uIZ9Mvu+bPD3Qgua6F4IjNLW/QDRlR63PcS4ooSEIR+IlKmgewtm Ed1kYnUga5qUAWeMg5thnhS4HArD0PHQCmXrw1GaIxmRpJlx4yshoNwPSOR09e+F5AEC Xybxk2ABciXLI7nB5TOlJXqslrMOyix/wG531mfjUuAgF3zvzvgiM4YLvVBDqww3qJf4 yj/dgvOV75n2DEc6yCSKrXUUxuOisZUIGcjln97ntw7o2dPbELROkRSSjt1FztBPZkoC 6gsLtEtwzXyarbr71jO2UikHuG678CFpfB2tBxMzDiZJylvyOeBcjvas4vE3ZjYc6nQx GeIQ== X-Gm-Message-State: APuU4Z85ZG8cN8aPJV5SDiifA4wjrhDaM0pL3/I5Pm0i230hO1iSKJr3 RB2T0hGWM06b2ASr6G8Qr2sS6Z/QV+3IJ/4VqVWiCVG7SOF/mzhm2ou5XfEXPGL5 X-Gm-Gg: AYBFou0ulabMu0Tw5qJxJlYSqPeIIGaBrMojVMG2ld1AIiFuckQndkjVjCH5GCJCWJg kdr1mayWRRVG+4vmTOS/dEssnpEeCmySPIOkYauGzCke4mSecZnm0Xp5MNKK29JRJO1dTyuXiPS dvjsMAZrsQFxtg7dFtqaaiBan9zzIMSnvIBA/fwzm778D6liXuMu0ZFfPceKGV9wEg8YgiHi0lO B0DILiq0W/eF2V03Lsf5HNZZDfpn7ugQYKenwz50l81kPFVtyvSjuDBW3Jpvh78xkezJJgNs1K/ XokGEyDrU8D4ykbDcXjCJsqWottFUxEKxWfgHLObhqYm0Q1mBBPc7D7REkyYCUkcPKF3PpT7mfu GXpIwueINbgx6xc5WcZnU+Gt8/QjXGX6BDePxlBimqvuAWoYVSqUTTkhmpDKNYDj/x1bfCrHQ0Q tsmvpFTKEiVl0npiDFaJdmd3+wKIk1j0jpm5xltj9QnFHV3SDZ3g9lNrE8kYVA/6YjUTh5cro25 j/RN6havy0lSu6r6TO/yYyfc7lycOfR2UXeq9eNm5dGnrRcPjlrTB/U4R5l1nMdWPI2d363AnBP oBTy2ZIDBeg= X-Received: by 2002:a05:600c:a09:b0:4a2:1a:a129 with SMTP id 5b1f17b1804b1-4bb88829115mr69268985e9.35.1795182838489; Fri, 20 Nov 2026 05:53:58 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:57 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 09/10] recipetool: create: Clean the recipe after its tasks Date: Fri, 20 Nov 2026 14:53:40 +0100 Message-ID: <20261120135342.2628228-10-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:54:02 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250802 From: Stefano Tondo recipetool can run tasks of the new recipe, as it does for Go. These tasks leave stamps, for example for do_unpack and do_patch. devtool then builds the recipe with externalsrc, which uses other stamps, and its do_clean removes the work directory. The first stamps stay. A later build of the recipe with the same signatures trusts these stamps and does not unpack again, so the source is missing. The next commit runs do_update_npm_deps for npm recipes. Then test_devtool_add_npm followed by test_recipetool_create_npm fails: do_update_npm_deps cannot find npm-shrinkwrap.json. Run do_clean after the tasks. Signed-off-by: Stefano Tondo --- scripts/lib/recipetool/create.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py index 2df17dd6e4..b2cc92c90b 100644 --- a/scripts/lib/recipetool/create.py +++ b/scripts/lib/recipetool/create.py @@ -955,6 +955,11 @@ def create_recipe(args): for task in run_tasks: logger.info("Running task %s" % task) tinfoil.build_file_sync(outfile, task) + if run_tasks: + # Remove the work directory and the stamps of these tasks. Else a + # later build of the recipe can trust the stamps, also when devtool + # has removed the work directory in the meantime. + tinfoil.build_file_sync(outfile, "clean") if tempsrc: if args.keep_temp: From patchwork Fri Nov 20 13:53:41 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Tondo X-Patchwork-Id: 105086 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by mx.groups.io with SMTP id smtpd.msgproc02-g2.6332.1795182842366178423 for ; Fri, 20 Nov 2026 05:54:02 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20260707 header.b=RyV1pOHs; spf=pass (domain: gmail.com, ip: 209.85.221.44, mailfrom: stondo@gmail.com) Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-491971ac844so783957f8f.2 for ; Fri, 20 Nov 2026 05:54:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795182841; x=1795787641; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=FVb0RBjObLWKOF1sdMvbn7+00s1R4c3PODXt9WvlTXg=; b=RyV1pOHs8tMAQ3j+zcqNiC/E7aRzI6v5I5ykz7klQrg42mtVOxkwa+pzBpgwlq9qaz 8GQlBmPMNVAnH7mY0mgkLCdI7PHKEL8gx3v5XFh1cPOqrAxCD6pDjS4T6bSKkzwNgK8q NuwWz2eF+nfLeehjoZwfBG+CI4btVYPDsPALnKLjg6PaStQRWq+CZmG4kVLw0j9EsXKA bQI8isLjXqgMSpOlHs3Le64CE3XUX/u02c7fB+RINIkFlGzvw5OsEgMKn9t+13g+SKFI eCjOo20jYySyFfczQeomTOcmhWYqNQ4qmMa4XYeKQfBqmjsKSn4ATmORcXxxpM/gJmLE EDsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795182841; x=1795787641; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=FVb0RBjObLWKOF1sdMvbn7+00s1R4c3PODXt9WvlTXg=; b=YtcWXlol2JgJ1x1XevagBd7Up+r4b1hSqxyt7Q48b4hvhs3g6UDstCxv1W5gCTLUlB naUFvGePsF7dZ1Pls+zs0u7pno2XDhoR6mKpYZeqpRcfX4WQnCAmXTJWNi2gwOR36gaq BJRzBrj86ozzr3lJv7DPKgUHvNp9x/4njbsNGpzdU9cOXgqX1FVGssauPNvGKSXBWAiJ hWsnww58rd8XwUJlfPhEHDQIYsk0yoGJ/D5zqA9dK/l6dUHi8EG5HPVg2bRsY3pd+rTn F0r9QkUOWT4aLltSW9L6xQa/UHo6lIWTFXHB9p4De7Ei0p85tFqvor6zvDda08hALLf6 Tg5w== X-Gm-Message-State: APuU4Z+Ke6WAhJ1eVGVE3XFCbKyonnbdKB/1JIkipDp4qHWA34CkCuY1 vn8TDJdK4ZlrtQyY2tyAex7PJf8YboXREqu3ii0/lFrJxUsP3R+TJjv59BC7BBTP X-Gm-Gg: AYBFou3g+7ti0Qw27A2sOeNM4ccqLLk2/cPG1xzCSx5kcPzCA5xKzTHpijbH/lNwly5 yOtPkROa7m2LQQjZv2+FvQAxfy/2pFSS1qY5iUgh4ptGNdphLzy3Oehfbw3jmAuyJ3ctwsGb8I3 RFg0H/t8+ulrFQvMw0IWeP9kvvDv/BS+h7q12d2eB8TkezY1O3XOq9i7vLVtJ+eZW/IhgNv9v8l K4rBxEz0XFNt4K1zAum78AgwoCXgCC8FspbEEC8lo0b6L6y8RObwvFgkwAN5ACOBPRfbWqKzPsb Ar+JF7sUq4Mc3SQNkyUxWp2hInOf6RGcijoTNSpOwyZ5wRn1Qe5ab/CnGUmk52OWS87tYP9EJ9Z +ypS0mQUhJpSBemPsURxHwXCIw6bXyxhlEti7tW0v4eldvH2uY+fQ2BbrnbNttAY0CVVOsrf8+P dYcVETC3HwnX7NYEkXw+/Zcef1912BFE1045YjnY4zDu6PswJleEJPrD6Obr2KVbno8N5v+MsXy d8C8YmxOJ3/xC1YXM33EYYnJpst1XFIRWzwap+l3W1P62Rp/HQFKUaQrhBIQXllRBDca9U2EaGZ BF56WfreWD/1 X-Received: by 2002:a05:600c:674f:b0:4bb:56bb:1196 with SMTP id 5b1f17b1804b1-4bb8880fb9bmr74502185e9.23.1795182840438; Fri, 20 Nov 2026 05:54:00 -0800 (PST) Return-Path: Received: from fedora ([165.225.94.251]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb88f6b6a6sm101009695e9.5.2026.11.20.05.53.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2026 05:53:59 -0800 (PST) From: Stefano Tondo To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin , Joshua Watt , Stefan Herbrechtsmeier , Peter Marko , stefano.tondo.ext@siemens.com Subject: [PATCH 10/10] recipetool: create_npm: List the npm dependencies in SRC_URI Date: Fri, 20 Nov 2026 14:53:41 +0100 Message-ID: <20261120135342.2628228-11-stondo@gmail.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com> References: <20261120135342.2628228-1-stondo@gmail.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by lin-tor-korg-lkml-alma10-1 with HTTPS for ; Fri, 20 Nov 2026 13:54:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/250803 From: Stefano Tondo recipetool adds an npmsw:// entry without destsuffix to the recipe. Since bitbake commit 84f102954, the npmsw fetcher unpacks into the root of UNPACKDIR and not into ${S}. So npm-shrinkwrap.json is not in ${S}, and npm.bbclass silently packs no dependencies. The package then has no node_modules. Create the recipe with the explicit list of dependencies instead: - Add npm-shrinkwrap.json to SRC_URI, so that it unpacks into ${S}. - Inherit npm-update-dependencies and require ${BPN}-npm-deps.inc. - Run the update task, which writes the dependencies to the .inc file. create_recipe() gives the source subdirectory to the handlers in extravalues, so that the npm handler knows where ${S} is. Fetch the dependencies from the same list for the license scan. The selftest now checks the .inc file. Signed-off-by: Stefano Tondo --- meta/lib/oeqa/selftest/cases/recipetool.py | 16 +++++++-- scripts/lib/recipetool/create.py | 4 +++ scripts/lib/recipetool/create_npm.py | 42 +++++++++++++++------- 3 files changed, 48 insertions(+), 14 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py b/meta/lib/oeqa/selftest/cases/recipetool.py index ba4f674b4c..181d95804f 100644 --- a/meta/lib/oeqa/selftest/cases/recipetool.py +++ b/meta/lib/oeqa/selftest/cases/recipetool.py @@ -5,11 +5,13 @@ # import errno +import json import os import shutil import tempfile import urllib.parse +import oe.npm import oe.spdx_license from oeqa.utils.commands import runCmd, bitbake, get_bb_var @@ -452,16 +454,26 @@ class RecipetoolCreateTests(RecipetoolBase): urls.append('npm://registry.npmjs.org/;package=@savoirfairelinux/node-server-example;version=${PV}' ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7' '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223') - urls.append('npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json') + urls.append('file://npm-shrinkwrap.json;subdir=npm') checkvars['SRC_URI'] = set(urls) checkvars['S'] = '${UNPACKDIR}/npm' checkvars['LICENSE:${PN}'] = 'MIT' checkvars['LICENSE:${PN}-base64'] = 'Unknown' checkvars['LICENSE:${PN}-accepts'] = 'MIT' checkvars['LICENSE:${PN}-inherits'] = 'ISC' - inherits = ['npm'] + inherits = ['npm', 'npm-update-dependencies'] self._test_recipe_contents(recipefile, checkvars, inherits) + # The update task writes one SRC_URI entry for each dependency + with open(shrinkwrap) as f: + expected = oe.npm.shrinkwrap_src_uri(json.load(f), 'npm') + deps_require_file = os.path.join(temprecipe, 'savoirfairelinux-node-server-example-npm-deps.inc') + self.assertFileExists(deps_require_file) + checkvars = {'SRC_URI': set(expected)} + self._test_recipe_contents(deps_require_file, checkvars, []) + self.assertTrue(any(url.startswith('npm://registry.npmjs.org/;package=accepts;') + and url.endswith(';destsuffix=npm/node_modules/accepts') for url in expected)) + def test_recipetool_create_github(self): # Basic test to see if github URL mangling works. Deliberately use an # older release of Meson at present so we don't need a toml parser. diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py index b2cc92c90b..46a321a866 100644 --- a/scripts/lib/recipetool/create.py +++ b/scripts/lib/recipetool/create.py @@ -784,8 +784,12 @@ def create_recipe(args): classes.append('bin_package') handled.append('buildsystem') + # Handlers can read the source subdirectory (S relative to UNPACKDIR), + # when S is set to it above + extravalues['srcsubdir'] = srcsubdir if not args.binary else '' for handler in handlers: handler.process(srctree_use, classes, lines_before, lines_after, handled, extravalues) + extravalues.pop('srcsubdir', None) # native and nativesdk classes are special and must be inherited last # If present, put them at the end of the classes list diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py index 50dd45c7d2..1b03aef612 100644 --- a/scripts/lib/recipetool/create_npm.py +++ b/scripts/lib/recipetool/create_npm.py @@ -12,6 +12,7 @@ import re import sys import tempfile import bb +import oe.npm from bb.fetch.npm import NpmEnvironment from bb.fetch.npm import npm_package from bb.fetch.npmsw import foreach_dependencies @@ -257,33 +258,39 @@ class NpmRecipeHandler(RecipeHandler): extravalues.setdefault("extrafiles", {}) extravalues["extrafiles"]["npm-shrinkwrap.json"] = shrinkwrap_copy - url_local = "npmsw://%s" % shrinkwrap_file - url_recipe= "npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json" + has_dependencies = "dependencies" in shrinkwrap.get("packages", {}).get("", {}) - if dev: - url_local += ";dev=1" - url_recipe += ";dev=1" + # The shrinkwrap file must be in ${S}, where npm.bbclass reads it + srcsubdir = extravalues.get("srcsubdir") or "${BP}" + url_recipe = "file://npm-shrinkwrap.json;subdir=%s" % srcsubdir - # Add the npmsw url in the SRC_URI of the generated recipe + # Add the shrinkwrap file to the SRC_URI of the generated recipe def _handle_srcuri(varname, origvalue, op, newlines): - """Update the version value and add the 'npmsw://' url""" + """Update the version value and add the shrinkwrap file""" value = origvalue.replace("version=" + data["version"], "version=${PV}") value = value.replace("version=latest", "version=${PV}") values = [line.strip() for line in value.strip('\n').splitlines()] - if "dependencies" in shrinkwrap.get("packages", {}).get("", {}): + if has_dependencies: values.append(url_recipe) return values, None, 4, False (_, newlines) = bb.utils.edit_metadata(lines_before, ["SRC_URI"], _handle_srcuri) lines_before[:] = [line.rstrip('\n') for line in newlines] + try: + dependency_urls = oe.npm.shrinkwrap_src_uri(shrinkwrap, "", dev) + except ValueError as e: + bb.error("%s: %s" % (shrinkwrap_file, e)) + sys.exit(14) + # In order to generate correct licence checksums in the recipe the - # dependencies have to be fetched again using the npmsw url + # dependencies have to be fetched again bb.note("Fetching npm dependencies ...") bb.utils.remove(os.path.join(srctree, "node_modules"), recurse=True) - fetcher = bb.fetch.Fetch([url_local], d) - fetcher.download() - fetcher.unpack(srctree) + if dependency_urls: + fetcher = bb.fetch.Fetch(dependency_urls, d) + fetcher.download() + fetcher.unpack(srctree) bb.note("Handling licences ...") (licfiles, packages, fallback_licenses) = self._handle_licenses(srctree, shrinkwrap_file, dev) @@ -297,6 +304,17 @@ class NpmRecipeHandler(RecipeHandler): classes.append("npm") handled.append("buildsystem") + if has_dependencies: + # npm-update-dependencies writes the dependencies to + # ${BPN}-npm-deps.inc. Write a stub file, which the task replaces. + classes.append("npm-update-dependencies") + extravalues["run_tasks"] = "update_npm_deps" + stub = os.path.join(tempfile.mkdtemp(prefix="recipetool-npm-"), "npm-deps.inc") + with open(stub, "w") as f: + f.write("# FROM RECIPETOOL\n") + extravalues["extrafiles"]["../{pn}-npm-deps.inc"] = stub + lines_before.append("require ${BPN}-npm-deps.inc") + # Check if package has peer dependencies and inform the user self._handle_peer_dependency(shrinkwrap_file)