| Message ID | 20240912142953.1580877-1-christli@axis.com |
|---|---|
| State | New |
| Headers | show |
| Series | go-mod.bbclass: Set GO_MOD_CACHE_DIR | expand |
Hi Christian, Christian Lindeberg via lists.openembedded.org <christian.lindeberg= axis.com@lists.openembedded.org> escreveu (quinta, 12/09/2024 à(s) 15:30): > From: Christian Lindeberg <christian.lindeberg@axis.com> > > Set the GO_MOD_CACHE_DIR variable and move the location of the module > cache to enable the use of the go module fetchers for module > dependencies. > Also, clean out the module cache before unpacking. > > Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com> > --- > meta/classes-recipe/go-mod.bbclass | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/meta/classes-recipe/go-mod.bbclass > b/meta/classes-recipe/go-mod.bbclass > index ca3a690d05..71a6712c47 100644 > --- a/meta/classes-recipe/go-mod.bbclass > +++ b/meta/classes-recipe/go-mod.bbclass > @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw" > > inherit go > > +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), > 'pkg/mod')}" > +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), > d.getVar('GO_MOD_CACHE_DIR'))}" > Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this will be equal to: GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod" export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod" in the end this can be written as: export GOMODCACHE ?= "${S}/pkg/mod" Is it correct or am I seeing something wrong? [1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16 Jose +do_unpack[cleandirs] += "${GOMODCACHE}" > + > GO_WORKDIR ?= "${GO_IMPORT}" > do_compile[dirs] += "${B}/src/${GO_WORKDIR}" > - > -export GOMODCACHE = "${B}/.mod" > - > -do_compile[cleandirs] += "${B}/.mod" > -- > 2.39.2 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#204444): > https://lists.openembedded.org/g/openembedded-core/message/204444 > Mute This Topic: https://lists.openembedded.org/mt/108413937/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On 9/12/2024 9:29 AM, Christian Lindeberg wrote: > From: Christian Lindeberg <christian.lindeberg@axis.com> > > Set the GO_MOD_CACHE_DIR variable and move the location of the module > cache to enable the use of the go module fetchers for module > dependencies. > Also, clean out the module cache before unpacking. > > Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com> > --- > meta/classes-recipe/go-mod.bbclass | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass > index ca3a690d05..71a6712c47 100644 > --- a/meta/classes-recipe/go-mod.bbclass > +++ b/meta/classes-recipe/go-mod.bbclass > @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw" > > inherit go > > +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}" > +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}" > +do_unpack[cleandirs] += "${GOMODCACHE}" > + > GO_WORKDIR ?= "${GO_IMPORT}" > do_compile[dirs] += "${B}/src/${GO_WORKDIR}" > - > -export GOMODCACHE = "${B}/.mod" > - > -do_compile[cleandirs] += "${B}/.mod" > What is the reasoning behind moving the cache dir out of the build dir ${B} and into the bitbake managed sources dir ${S} ? There might be a good reason for doing it, but it was not explained in the commit message. > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#204444): https://lists.openembedded.org/g/openembedded-core/message/204444 > Mute This Topic: https://lists.openembedded.org/mt/108413937/6551054 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [reatmon@ti.com] > -=-=-=-=-=-=-=-=-=-=-=- >
[I can’t reply inline due to the HTML-formatting.]
I believe the goal is to have pkg/mod as a subdirectory of ${S}. At the same time, ${GO_MOD_CACHE_DIR} needs to be a relative path (as it is passed to the fetcher via the subdir= parameter). However, the suggested code only works for cases where ${S} matches ${WORKDIR}/subdirectory. In case it is ${WORKDIR}/sub/directory, it will fail. I would suggest to change the code like this:
export GOMODCACHE = "${S}/pkg/mod"
GO_MOD_CACHE_DIR = "${@os.path.relpath(d.getVar('GOMODCACHE'), d.getVar('WORKDIR'))}"
do_unpack[cleandirs] += "${GOMODCACHE}"
I have verified that the above fetches a Go module into the correct directory with S set to ${WORKDIR}/git/${BPN}. I have also verified that crucible (one of the recipes in meta-oe that uses the go-mod bbclass) still builds with the above. And as a bonus I now also have a patch that modifies the crucible recipe to use the gomod fetcher (which requires the above change and also the patch I just sent to the bitbake list for the gomod fetcher). Unfortunately I will not have time to finish it until I’m back from OSS in Vienna next week.
//Peter
From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Jose Quaresma
Sent: den 12 september 2024 19:01
To: Christian Lindeberg <Christian.Lindeberg@axis.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR
Hi Christian,
Christian Lindeberg via lists.openembedded.org<http://lists.openembedded.org> <christian.lindeberg=axis.com@lists.openembedded.org<mailto:axis.com@lists.openembedded.org>> escreveu (quinta, 12/09/2024 à(s) 15:30):
From: Christian Lindeberg <christian.lindeberg@axis.com<mailto:christian.lindeberg@axis.com>>
Set the GO_MOD_CACHE_DIR variable and move the location of the module
cache to enable the use of the go module fetchers for module
dependencies.
Also, clean out the module cache before unpacking.
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com<mailto:christian.lindeberg@axis.com>>
---
meta/classes-recipe/go-mod.bbclass | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass
index ca3a690d05..71a6712c47 100644
--- a/meta/classes-recipe/go-mod.bbclass
+++ b/meta/classes-recipe/go-mod.bbclass
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
inherit go
+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}<mailto:$%7b@os.path.join(os.path.basename(d.getVar('S')),%20'pkg/mod')%7d>"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}<mailto:$%7b@os.path.join(os.path.dirname(d.getVar('S')),%20d.getVar('GO_MOD_CACHE_DIR'))%7d>"
Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this will be equal to:
GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"
in the end this can be written as:
export GOMODCACHE ?= "${S}/pkg/mod"
Is it correct or am I seeing something wrong?
[1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16
Jose
+do_unpack[cleandirs] += "${GOMODCACHE}"
+
GO_WORKDIR ?= "${GO_IMPORT}"
do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
--
2.39.2
--
Best regards,
José Quaresma
On Thu, Sep 12, 2024 at 07:01 PM, Jose Quaresma wrote: > > Hi Christian, > > Christian Lindeberg via lists.openembedded.org ( > http://lists.openembedded.org ) <christian.lindeberg= axis.com@lists.openembedded.org > > escreveu (quinta, 12/09/2024 à(s) 15:30): > >> From: Christian Lindeberg < christian.lindeberg@axis.com > >> >> Set the GO_MOD_CACHE_DIR variable and move the location of the module >> cache to enable the use of the go module fetchers for module >> dependencies. >> Also, clean out the module cache before unpacking. >> >> Signed-off-by: Christian Lindeberg < christian.lindeberg@axis.com > >> --- >> meta/classes-recipe/go-mod.bbclass | 8 ++++---- >> 1 file changed, 4 insertions(+), 4 deletions(-) >> >> diff --git a/meta/classes-recipe/go-mod.bbclass >> b/meta/classes-recipe/go-mod.bbclass >> index ca3a690d05..71a6712c47 100644 >> --- a/meta/classes-recipe/go-mod.bbclass >> +++ b/meta/classes-recipe/go-mod.bbclass >> @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw" >> >> inherit go >> >> +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), >> 'pkg/mod')}" >> +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), >> d.getVar('GO_MOD_CACHE_DIR'))}" > > > Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this > will be equal to: > > GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod" > export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod" > > in the end this can be written as: > > export GOMODCACHE ?= "${S}/pkg/mod" > > Is it correct or am I seeing something wrong? > Correct, I sent a new version with this and the suggestion from Peter Kjellerstedt. Thanks, Christian > > > [1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16 > > > Jose > > >> +do_unpack[cleandirs] += "${GOMODCACHE}" >> + >> GO_WORKDIR ?= "${GO_IMPORT}" >> do_compile[dirs] += "${B}/src/${GO_WORKDIR}" >> - >> -export GOMODCACHE = "${B}/.mod" >> - >> -do_compile[cleandirs] += "${B}/.mod" >> -- >> 2.39.2 >> >> >> >> > > > > > -- > Best regards, > > José Quaresma >
On Thu, Sep 12, 2024 at 10:14 PM, Ryan Eatmon wrote: > > On 9/12/2024 9:29 AM, Christian Lindeberg wrote: > >> From: Christian Lindeberg <christian.lindeberg@axis.com> >> >> Set the GO_MOD_CACHE_DIR variable and move the location of the module >> cache to enable the use of the go module fetchers for module >> dependencies. >> Also, clean out the module cache before unpacking. >> >> Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com> >> --- >> meta/classes-recipe/go-mod.bbclass | 8 ++++---- >> 1 file changed, 4 insertions(+), 4 deletions(-) >> >> diff --git a/meta/classes-recipe/go-mod.bbclass >> b/meta/classes-recipe/go-mod.bbclass >> index ca3a690d05..71a6712c47 100644 >> --- a/meta/classes-recipe/go-mod.bbclass >> +++ b/meta/classes-recipe/go-mod.bbclass >> @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw" >> >> inherit go >> >> +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), >> 'pkg/mod')}" >> +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), >> d.getVar('GO_MOD_CACHE_DIR'))}" >> +do_unpack[cleandirs] += "${GOMODCACHE}" >> + >> GO_WORKDIR ?= "${GO_IMPORT}" >> do_compile[dirs] += "${B}/src/${GO_WORKDIR}" >> - >> -export GOMODCACHE = "${B}/.mod" >> - >> -do_compile[cleandirs] += "${B}/.mod" > > > What is the reasoning behind moving the cache dir out of the build dir > ${B} and into the bitbake managed sources dir ${S} ? There might be a > good reason for doing it, but it was not explained in the commit message. In the review of the gomod and gomodgit fetchers concerns were raised about letting the fetchers unpack under ${GOMODCACHE} instead of relative to the unpack root directory. With the fetchers there is now a way to have the module cache part of the bitbake managed sources instead of having to resort to do_compile[network] = "1". Thanks, Christian > > > > > -- > Ryan Eatmon reatmon@ti.com > ----------------------------------------- > Texas Instruments, Inc. - LCPD - MGTS
diff --git a/meta/classes-recipe/go-mod.bbclass b/meta/classes-recipe/go-mod.bbclass index ca3a690d05..71a6712c47 100644 --- a/meta/classes-recipe/go-mod.bbclass +++ b/meta/classes-recipe/go-mod.bbclass @@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw" inherit go +GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}" +export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}" +do_unpack[cleandirs] += "${GOMODCACHE}" + GO_WORKDIR ?= "${GO_IMPORT}" do_compile[dirs] += "${B}/src/${GO_WORKDIR}" - -export GOMODCACHE = "${B}/.mod" - -do_compile[cleandirs] += "${B}/.mod"