diff mbox series

[master/kirkstone] k3r5: ensure separate TMPDIR used for this multiconfig

Message ID 20231112190743.1496752-1-denis@denix.org
State Accepted
Delegated to: Ryan Eatmon
Headers show
Series [master/kirkstone] k3r5: ensure separate TMPDIR used for this multiconfig | expand

Commit Message

Denys Dmytriyenko Nov. 12, 2023, 7:07 p.m. UTC 
From: Denys Dmytriyenko <denys@konsulko.com>

This k3r5 multiconfig builds baremetal components (and corresponding native,
nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal". The
expectation is that those components and tools will use a separate TMPDIR
to isolate from the main Linux build that uses "glibc" TCLIBC and to avoid
potential conflicts.

OE-Core "nodistro" default configuration already sets TCLIBCAPPEND facility
to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal" temp
directory for this multiconfig and "tmp-glibc" for the main Linux one. Other
distros like Arago follow this convention and even extend a bit (e.g. Arago
also adds TCMODE suffix to TMPDIR for external toolchain support separation).

But Poky (and derivative distros, like AGL or YoE) disable TCLIBCAPPEND and
result in a combined TMPDIR, leading to potential conflicts, such as:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273

And that's just the beginning, there were other conflicts observed later in
the build, e.g. during nativesdk builds - that's why we also set a unique
SDKPKGSUFFIX here.

To force a separate TMPDIR for k3r5 multiconfig, we have to directly append
a suffix to it. Multiple other options were tested in hopes of making it
slightly cleaner, but they either didn't work or were dismissed. For example,
trying to override TCLIBCAPPEND getting cleared by a distro would require
using a machine/soc-override, which doesn't have enough scope (nativesdk)
or forcing it with :forcevariable would also change the main Linux TMPDIR
and affect existing CI flows. Also, using TCLIBC itself as a suffix to add
to TMPDIR may result in getting it appended twice (tmp-baremetal-baremetal)
when normal TCLIBCAPPEND facility is used. Hence the least invasive/confusing
option is to always append "-k3r5" suffix to this multiconfig TMPDIR. That
results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while OE-Core
"nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and "tmp-glibc"
for the main).

Also note, meta-ti-bsp layer.conf sets up images and sdks to be deployed
into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY variable
that controls it is set weakly, allowing to be modified from a distro
configuration or local.conf. It means that all images and sdks can be
deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as
the final destination, by using := for immediate variable expansion:
TI_COMMON_DEPLOY := "${TMPDIR}/deploy"

Signed-off-by: Denys Dmytriyenko <denys@konsulko.com>
---
 meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++
 1 file changed, 2 insertions(+)

Comments

Jose Quaresma Nov. 13, 2023, 1:20 p.m. UTC | #1 
Denys Dmytriyenko <denis@denix.org> escreveu no dia domingo, 12/11/2023
à(s) 19:07:

> From: Denys Dmytriyenko <denys@konsulko.com>
>
> This k3r5 multiconfig builds baremetal components (and corresponding
> native,
> nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal". The
> expectation is that those components and tools will use a separate TMPDIR
> to isolate from the main Linux build that uses "glibc" TCLIBC and to avoid
> potential conflicts.
>
> OE-Core "nodistro" default configuration already sets TCLIBCAPPEND facility
> to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal" temp
> directory for this multiconfig and "tmp-glibc" for the main Linux one.
> Other
> distros like Arago follow this convention and even extend a bit (e.g. Arago
> also adds TCMODE suffix to TMPDIR for external toolchain support
> separation).
>
> But Poky (and derivative distros, like AGL or YoE) disable TCLIBCAPPEND and
> result in a combined TMPDIR, leading to potential conflicts, such as:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273
>
> And that's just the beginning, there were other conflicts observed later in
> the build, e.g. during nativesdk builds - that's why we also set a unique
> SDKPKGSUFFIX here.
>
> To force a separate TMPDIR for k3r5 multiconfig, we have to directly append
> a suffix to it. Multiple other options were tested in hopes of making it
> slightly cleaner, but they either didn't work or were dismissed. For
> example,
> trying to override TCLIBCAPPEND getting cleared by a distro would require
> using a machine/soc-override, which doesn't have enough scope (nativesdk)
> or forcing it with :forcevariable would also change the main Linux TMPDIR
> and affect existing CI flows. Also, using TCLIBC itself as a suffix to add
> to TMPDIR may result in getting it appended twice (tmp-baremetal-baremetal)
> when normal TCLIBCAPPEND facility is used. Hence the least
> invasive/confusing
> option is to always append "-k3r5" suffix to this multiconfig TMPDIR. That
> results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while OE-Core
> "nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and
> "tmp-glibc"
> for the main).
>
> Also note, meta-ti-bsp layer.conf sets up images and sdks to be deployed
> into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY variable
> that controls it is set weakly, allowing to be modified from a distro
> configuration or local.conf. It means that all images and sdks can be
> deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as
> the final destination, by using := for immediate variable expansion:
> TI_COMMON_DEPLOY := "${TMPDIR}/deploy"
>
> Signed-off-by: Denys Dmytriyenko <denys@konsulko.com>
> ---
>  meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf
> b/meta-ti-bsp/conf/multiconfig/k3r5.conf
> index 04b9a746..e36c87ed 100644
> --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf
> +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf
> @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 =
> "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}"
>
>  MACHINE:append = "-k3r5"
>  TCLIBC = "baremetal"
> +TMPDIR:append = "-k3r5"
> +
>

Thanks for fixing this long remaining issue on the kirkstone branch.

AFAIK all the collision issues are mostly fixed on oe-core master.
For me there are still sporadic failures with the SPDX bbclass.
Can this change be optional? with something like:

TMPDIRAPPEND ?= "-k3r5"
TMPDIR:append = "${TMPDIRAPPEND}"

This adds the possibility of more easily reverting this change
to being able to fix what is pending in oe-core master.

Jose


>  SDKPKGSUFFIX = "k3r5-nativesdk"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#17293):
> https://lists.yoctoproject.org/g/meta-ti/message/17293
> Mute This Topic: https://lists.yoctoproject.org/mt/102547765/5052612
> Group Owner: meta-ti+owner@lists.yoctoproject.org
> Unsubscribe:
> https://lists.yoctoproject.org/g/meta-ti/leave/11369865/5052612/926574700/xyzzy
> [quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Denys Dmytriyenko Nov. 21, 2023, 9:48 p.m. UTC | #2 
On Mon, Nov 13, 2023 at 01:20:25PM +0000, Jose Quaresma wrote:
> Denys Dmytriyenko <denis@denix.org> escreveu no dia domingo, 12/11/2023
> à(s) 19:07:
> 
> > From: Denys Dmytriyenko <denys@konsulko.com>
> >
> > This k3r5 multiconfig builds baremetal components (and corresponding
> > native,
> > nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal". The
> > expectation is that those components and tools will use a separate TMPDIR
> > to isolate from the main Linux build that uses "glibc" TCLIBC and to avoid
> > potential conflicts.
> >
> > OE-Core "nodistro" default configuration already sets TCLIBCAPPEND facility
> > to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal" temp
> > directory for this multiconfig and "tmp-glibc" for the main Linux one.
> > Other
> > distros like Arago follow this convention and even extend a bit (e.g. Arago
> > also adds TCMODE suffix to TMPDIR for external toolchain support
> > separation).
> >
> > But Poky (and derivative distros, like AGL or YoE) disable TCLIBCAPPEND and
> > result in a combined TMPDIR, leading to potential conflicts, such as:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273
> >
> > And that's just the beginning, there were other conflicts observed later in
> > the build, e.g. during nativesdk builds - that's why we also set a unique
> > SDKPKGSUFFIX here.
> >
> > To force a separate TMPDIR for k3r5 multiconfig, we have to directly append
> > a suffix to it. Multiple other options were tested in hopes of making it
> > slightly cleaner, but they either didn't work or were dismissed. For
> > example,
> > trying to override TCLIBCAPPEND getting cleared by a distro would require
> > using a machine/soc-override, which doesn't have enough scope (nativesdk)
> > or forcing it with :forcevariable would also change the main Linux TMPDIR
> > and affect existing CI flows. Also, using TCLIBC itself as a suffix to add
> > to TMPDIR may result in getting it appended twice (tmp-baremetal-baremetal)
> > when normal TCLIBCAPPEND facility is used. Hence the least
> > invasive/confusing
> > option is to always append "-k3r5" suffix to this multiconfig TMPDIR. That
> > results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while OE-Core
> > "nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and
> > "tmp-glibc"
> > for the main).
> >
> > Also note, meta-ti-bsp layer.conf sets up images and sdks to be deployed
> > into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY variable
> > that controls it is set weakly, allowing to be modified from a distro
> > configuration or local.conf. It means that all images and sdks can be
> > deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as
> > the final destination, by using := for immediate variable expansion:
> > TI_COMMON_DEPLOY := "${TMPDIR}/deploy"
> >
> > Signed-off-by: Denys Dmytriyenko <denys@konsulko.com>
> > ---
> >  meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > b/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > index 04b9a746..e36c87ed 100644
> > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 =
> > "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}"
> >
> >  MACHINE:append = "-k3r5"
> >  TCLIBC = "baremetal"
> > +TMPDIR:append = "-k3r5"
> > +
> >
> 
> Thanks for fixing this long remaining issue on the kirkstone branch.
> 
> AFAIK all the collision issues are mostly fixed on oe-core master.

Yes, there were several related fixes in oe-core master recently, correct. 
But not everything was fixes yet even in master.


> For me there are still sporadic failures with the SPDX bbclass.

And SPDX was one of the areas that seen collision fixes in master. 
Unfortunately, those fixes have very little chances to get backported 
to kirkstone. Hence this change.


> Can this change be optional? with something like:
> 
> TMPDIRAPPEND ?= "-k3r5"
> TMPDIR:append = "${TMPDIRAPPEND}"
> 
> This adds the possibility of more easily reverting this change
> to being able to fix what is pending in oe-core master.

The point of this fix was to force separate TMPDIRs to avoid potential 
breakage - I was not planning on making it optional...

The request seems to be very specific to layer/framework debugging purposes 
at which point this can be modified directly in the conf file. Allowing it to 
be disabled cleanly for a build seems dubious. But if there's a strong desire, 
I can send a v2...
Jose Quaresma Nov. 22, 2023, 12:05 p.m. UTC | #3 
Denys Dmytriyenko <denis@denix.org> escreveu no dia terça, 21/11/2023 à(s)
21:48:

> On Mon, Nov 13, 2023 at 01:20:25PM +0000, Jose Quaresma wrote:
> > Denys Dmytriyenko <denis@denix.org> escreveu no dia domingo, 12/11/2023
> > à(s) 19:07:
> >
> > > From: Denys Dmytriyenko <denys@konsulko.com>
> > >
> > > This k3r5 multiconfig builds baremetal components (and corresponding
> > > native,
> > > nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal".
> The
> > > expectation is that those components and tools will use a separate
> TMPDIR
> > > to isolate from the main Linux build that uses "glibc" TCLIBC and to
> avoid
> > > potential conflicts.
> > >
> > > OE-Core "nodistro" default configuration already sets TCLIBCAPPEND
> facility
> > > to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal"
> temp
> > > directory for this multiconfig and "tmp-glibc" for the main Linux one.
> > > Other
> > > distros like Arago follow this convention and even extend a bit (e.g.
> Arago
> > > also adds TCMODE suffix to TMPDIR for external toolchain support
> > > separation).
> > >
> > > But Poky (and derivative distros, like AGL or YoE) disable
> TCLIBCAPPEND and
> > > result in a combined TMPDIR, leading to potential conflicts, such as:
> > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273
> > >
> > > And that's just the beginning, there were other conflicts observed
> later in
> > > the build, e.g. during nativesdk builds - that's why we also set a
> unique
> > > SDKPKGSUFFIX here.
> > >
> > > To force a separate TMPDIR for k3r5 multiconfig, we have to directly
> append
> > > a suffix to it. Multiple other options were tested in hopes of making
> it
> > > slightly cleaner, but they either didn't work or were dismissed. For
> > > example,
> > > trying to override TCLIBCAPPEND getting cleared by a distro would
> require
> > > using a machine/soc-override, which doesn't have enough scope
> (nativesdk)
> > > or forcing it with :forcevariable would also change the main Linux
> TMPDIR
> > > and affect existing CI flows. Also, using TCLIBC itself as a suffix to
> add
> > > to TMPDIR may result in getting it appended twice
> (tmp-baremetal-baremetal)
> > > when normal TCLIBCAPPEND facility is used. Hence the least
> > > invasive/confusing
> > > option is to always append "-k3r5" suffix to this multiconfig TMPDIR.
> That
> > > results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while
> OE-Core
> > > "nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and
> > > "tmp-glibc"
> > > for the main).
> > >
> > > Also note, meta-ti-bsp layer.conf sets up images and sdks to be
> deployed
> > > into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY
> variable
> > > that controls it is set weakly, allowing to be modified from a distro
> > > configuration or local.conf. It means that all images and sdks can be
> > > deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as
> > > the final destination, by using := for immediate variable expansion:
> > > TI_COMMON_DEPLOY := "${TMPDIR}/deploy"
> > >
> > > Signed-off-by: Denys Dmytriyenko <denys@konsulko.com>
> > > ---
> > >  meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++
> > >  1 file changed, 2 insertions(+)
> > >
> > > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > > b/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > > index 04b9a746..e36c87ed 100644
> > > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf
> > > @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 =
> > > "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}"
> > >
> > >  MACHINE:append = "-k3r5"
> > >  TCLIBC = "baremetal"
> > > +TMPDIR:append = "-k3r5"
> > > +
> > >
> >
> > Thanks for fixing this long remaining issue on the kirkstone branch.
> >
> > AFAIK all the collision issues are mostly fixed on oe-core master.
>
> Yes, there were several related fixes in oe-core master recently, correct.
> But not everything was fixes yet even in master.
>
>
> > For me there are still sporadic failures with the SPDX bbclass.
>
> And SPDX was one of the areas that seen collision fixes in master.
> Unfortunately, those fixes have very little chances to get backported
> to kirkstone. Hence this change.
>

I'm not opposed to having that in kirkstone.
I am working on this SPDX backport but as you said it will be very hard
because some
of these backports requires sstate, siggen and also bitbake patches.


>
>
> > Can this change be optional? with something like:
> >
> > TMPDIRAPPEND ?= "-k3r5"
> > TMPDIR:append = "${TMPDIRAPPEND}"
> >
> > This adds the possibility of more easily reverting this change
> > to being able to fix what is pending in oe-core master.
>
> The point of this fix was to force separate TMPDIRs to avoid potential
> breakage - I was not planning on making it optional...
>

I've been using a separate TMPDIR directory for some time now on kirkstone
and it was quite difficult to do so
because there is no way to change multiconfig/k3r5.conf in place.
I was told several times here on this list that such a change was not
necessary.


> The request seems to be very specific to layer/framework debugging
> purposes
> at which point this can be modified directly in the conf file. Allowing it
> to
> be disabled cleanly for a build seems dubious. But if there's a strong
> desire,
> I can send a v2...
>

Having something like the above gives more freedom to change it back to a
single TMPDIR.
I prefer a v2 where the user can choose what suits your build best.

Jose


>
> --
> Denys
>
diff mbox series

Patch

diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf b/meta-ti-bsp/conf/multiconfig/k3r5.conf
index 04b9a746..e36c87ed 100644
--- a/meta-ti-bsp/conf/multiconfig/k3r5.conf
+++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf
@@ -4,4 +4,6 @@  DEPLOY_DIR_IMAGE:k3r5 = "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}"
 
 MACHINE:append = "-k3r5"
 TCLIBC = "baremetal"
+TMPDIR:append = "-k3r5"
+
 SDKPKGSUFFIX = "k3r5-nativesdk"