| Message ID | isVB.1662474023926890292.lmGR@lists.yoctoproject.org |
|---|---|
| State | New |
| Headers | show |
| Series | Update upstream kirkstone branches | expand |
Hi Paulo, On 9/6/22 16:20, paulo@eero.com wrote: > Hello all! > > I've been just recently working with Yocto at my new position and just ran into an issue that was already solved on Yocto's main branches for both openembedded and poky layers. > There's a problem with npm on upstream kirkstone, where the fetching of dependencies fails due to stale or wrong metadata being read. This seems to have affected quite a few people (from what I browsed online), and found a solution which is already merged to the main branches for both poky and openembedded layers, done by Enrico Sholtz. > The patches are https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.yoctoproject.org_project_oe-2Dcore_list_-3Fseries-3D4303&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=7HzKBeVbSvWreLPwsDAKWZVdW0sTf8vqJp82dKy01KWfJ9e8kzZpevt-vvV7Z2Ga&s=xKaCrxEjeUTs2WwwWY226DJ9lZmm7QsiOPK4eCRZM_0&e= for poky and the patch file attached to this message. > > I've applied these changes to kirkstone and it solved the issues as promised and explained by Enrico on https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_topic_91903472&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=7HzKBeVbSvWreLPwsDAKWZVdW0sTf8vqJp82dKy01KWfJ9e8kzZpevt-vvV7Z2Ga&s=x2rwsw6fLSrNf-kgh01BmbsrcSsW8vKxcbRuhBFDpxY&e= > > I'd like to know how would one proceed to add these changes to upstream Yocto, since it would benefit a lot of people using this new LTS version of the project. > For OpenEmbedded-core patches: https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Backporting_fixes_to_stable_releases For meta-openembedded, I do not know how Khem deals with backport requests. You could do the same and then send your patch to openembedded-devel@lists.openembedded.org as explained here: https://github.com/openembedded/meta-openembedded/tree/master/meta-oe ? You can also open a PR against the proper branch I guess. Cc'ing Khem just in case. Cheers, Quentin
Hey there Quentin! Thanks for the update, sorry for the delay in responding; got caught up in some other things here. Alright man, I'm gonna study the processes and ping some other people in this regard! Regards, Paulo Vitor Becker Ayrosa Monteiro de Andrade. On Tue, Sep 6, 2022 at 11:13 AM Quentin Schulz < quentin.schulz@theobroma-systems.com> wrote: > Hi Paulo, > > On 9/6/22 16:20, paulo@eero.com wrote: > > Hello all! > > > > I've been just recently working with Yocto at my new position and just > ran into an issue that was already solved on Yocto's main branches for both > openembedded and poky layers. > > There's a problem with npm on upstream kirkstone, where the fetching of > dependencies fails due to stale or wrong metadata being read. This seems to > have affected quite a few people (from what I browsed online), and found a > solution which is already merged to the main branches for both poky and > openembedded layers, done by Enrico Sholtz. > > The patches are > https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.yoctoproject.org_project_oe-2Dcore_list_-3Fseries-3D4303&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=7HzKBeVbSvWreLPwsDAKWZVdW0sTf8vqJp82dKy01KWfJ9e8kzZpevt-vvV7Z2Ga&s=xKaCrxEjeUTs2WwwWY226DJ9lZmm7QsiOPK4eCRZM_0&e= > for poky and the patch file attached to this message. > > > > I've applied these changes to kirkstone and it solved the issues as > promised and explained by Enrico on > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_topic_91903472&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=7HzKBeVbSvWreLPwsDAKWZVdW0sTf8vqJp82dKy01KWfJ9e8kzZpevt-vvV7Z2Ga&s=x2rwsw6fLSrNf-kgh01BmbsrcSsW8vKxcbRuhBFDpxY&e= > > > > I'd like to know how would one proceed to add these changes to upstream > Yocto, since it would benefit a lot of people using this new LTS version of > the project. > > > > For OpenEmbedded-core patches: > > https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Backporting_fixes_to_stable_releases > > For meta-openembedded, I do not know how Khem deals with backport > requests. You could do the same and then send your patch to > openembedded-devel@lists.openembedded.org as explained here: > https://github.com/openembedded/meta-openembedded/tree/master/meta-oe ? > You can also open a PR against the proper branch I guess. Cc'ing Khem > just in case. > > Cheers, > Quentin >
commit 5bffa3f2f2c698da53fa5c4ad4418fc3087fe48d
Author: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Date: Thu May 19 12:23:07 2022 +0200
nodejs-oe-cache-native: initial checkin
This implements an 'npm cache add' like functionality but allows to
specify the key of the data and sets metadata which are required to
find the data.
It is used to cache information as done during 'npm install'.
Keyformat and metadata are nodejs version specific.
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
diff --git a/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-16.14/oe-npm-cache b/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-16.14/oe-npm-cache
new file mode 100755
index 000000000..f59620764
--- /dev/null
+++ b/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-16.14/oe-npm-cache
@@ -0,0 +1,77 @@
+#!/usr/bin/env node
+
+/// Usage: oe-npm-cache <cache-dir> <type> <key> <file-name>
+/// <type> ... meta - metainformation about package
+/// tgz - tarball
+
+const process = require("node:process");
+
+module.paths.unshift("@@libdir@@/node_modules/npm/node_modules");
+
+const cacache = require('cacache')
+const fs = require('fs')
+
+// argv[0] is 'node', argv[1] is this script
+const cache_dir = process.argv[2]
+const type = process.argv[3]
+const key = process.argv[4]
+const file = process.argv[5]
+
+const data = fs.readFileSync(file)
+
+// metadata content is highly nodejs dependent; when cache entries are not
+// found, place debug statements in 'make-fetch-happen/lib/cache/policy.js'
+// (CachePolicy::satisfies())
+const xlate = {
+ 'meta': {
+ 'key_prefix': 'make-fetch-happen:request-cache:',
+ 'metadata': function() {
+ return {
+ time: Date.now(),
+ url: key,
+ reqHeaders: {
+ 'accept': 'application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*',
+ },
+ resHeaders: {
+ "content-type": "application/json",
+ "status": 200,
+ },
+ options: {
+ compress: true,
+ }
+ };
+ },
+ },
+
+ 'tgz': {
+ 'key_prefix': 'make-fetch-happen:request-cache:',
+ 'metadata': function() {
+ return {
+ time: Date.now(),
+ url: key,
+ reqHeaders: {
+ 'accept': '*/*',
+ },
+ resHeaders: {
+ "content-type": "application/octet-stream",
+ "status": 200,
+ },
+ options: {
+ compress: true,
+ },
+ };
+ },
+ },
+};
+
+const info = xlate[type];
+let opts = {}
+
+if (info.metadata) {
+ opts['metadata'] = info.metadata();
+}
+
+cacache.put(cache_dir, info.key_prefix + key, data, opts)
+ .then(integrity => {
+ console.log(`Saved content of ${key} (${file}).`);
+})
diff --git a/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_16.14.bb b/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_16.14.bb
new file mode 100644
index 000000000..a61dd5018
--- /dev/null
+++ b/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_16.14.bb
@@ -0,0 +1,21 @@
+DESCRIPTION = "OE helper for manipulating npm cache"
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/Apache-2.0;md5=89aea4e17d99a7cacdbeed46a0096b10"
+
+SRC_URI = "\
+ file://oe-npm-cache \
+"
+
+inherit native
+
+B = "${WORKDIR}/build"
+
+do_configure() {
+ sed -e 's!@@libdir@@!${libdir}!g' < '${WORKDIR}/oe-npm-cache' > '${B}/oe-npm-cache'
+}
+
+do_install() {
+ install -D -p -m 0755 ${B}/oe-npm-cache ${D}${bindir}/oe-npm-cache
+}
+
+RDEPENDS:${PN} = "nodejs-native"