Message ID | 20231112190743.1496752-1-denis@denix.org |
---|---|
State | Accepted |
Delegated to: | Ryan Eatmon |
Headers | show |
Series | [master/kirkstone] k3r5: ensure separate TMPDIR used for this multiconfig | expand |
Denys Dmytriyenko <denis@denix.org> escreveu no dia domingo, 12/11/2023 à(s) 19:07: > From: Denys Dmytriyenko <denys@konsulko.com> > > This k3r5 multiconfig builds baremetal components (and corresponding > native, > nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal". The > expectation is that those components and tools will use a separate TMPDIR > to isolate from the main Linux build that uses "glibc" TCLIBC and to avoid > potential conflicts. > > OE-Core "nodistro" default configuration already sets TCLIBCAPPEND facility > to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal" temp > directory for this multiconfig and "tmp-glibc" for the main Linux one. > Other > distros like Arago follow this convention and even extend a bit (e.g. Arago > also adds TCMODE suffix to TMPDIR for external toolchain support > separation). > > But Poky (and derivative distros, like AGL or YoE) disable TCLIBCAPPEND and > result in a combined TMPDIR, leading to potential conflicts, such as: > https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273 > > And that's just the beginning, there were other conflicts observed later in > the build, e.g. during nativesdk builds - that's why we also set a unique > SDKPKGSUFFIX here. > > To force a separate TMPDIR for k3r5 multiconfig, we have to directly append > a suffix to it. Multiple other options were tested in hopes of making it > slightly cleaner, but they either didn't work or were dismissed. For > example, > trying to override TCLIBCAPPEND getting cleared by a distro would require > using a machine/soc-override, which doesn't have enough scope (nativesdk) > or forcing it with :forcevariable would also change the main Linux TMPDIR > and affect existing CI flows. Also, using TCLIBC itself as a suffix to add > to TMPDIR may result in getting it appended twice (tmp-baremetal-baremetal) > when normal TCLIBCAPPEND facility is used. Hence the least > invasive/confusing > option is to always append "-k3r5" suffix to this multiconfig TMPDIR. That > results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while OE-Core > "nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and > "tmp-glibc" > for the main). > > Also note, meta-ti-bsp layer.conf sets up images and sdks to be deployed > into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY variable > that controls it is set weakly, allowing to be modified from a distro > configuration or local.conf. It means that all images and sdks can be > deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as > the final destination, by using := for immediate variable expansion: > TI_COMMON_DEPLOY := "${TMPDIR}/deploy" > > Signed-off-by: Denys Dmytriyenko <denys@konsulko.com> > --- > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf > b/meta-ti-bsp/conf/multiconfig/k3r5.conf > index 04b9a746..e36c87ed 100644 > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 = > "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}" > > MACHINE:append = "-k3r5" > TCLIBC = "baremetal" > +TMPDIR:append = "-k3r5" > + > Thanks for fixing this long remaining issue on the kirkstone branch. AFAIK all the collision issues are mostly fixed on oe-core master. For me there are still sporadic failures with the SPDX bbclass. Can this change be optional? with something like: TMPDIRAPPEND ?= "-k3r5" TMPDIR:append = "${TMPDIRAPPEND}" This adds the possibility of more easily reverting this change to being able to fix what is pending in oe-core master. Jose > SDKPKGSUFFIX = "k3r5-nativesdk" > -- > 2.25.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#17293): > https://lists.yoctoproject.org/g/meta-ti/message/17293 > Mute This Topic: https://lists.yoctoproject.org/mt/102547765/5052612 > Group Owner: meta-ti+owner@lists.yoctoproject.org > Unsubscribe: > https://lists.yoctoproject.org/g/meta-ti/leave/11369865/5052612/926574700/xyzzy > [quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Mon, Nov 13, 2023 at 01:20:25PM +0000, Jose Quaresma wrote: > Denys Dmytriyenko <denis@denix.org> escreveu no dia domingo, 12/11/2023 > à(s) 19:07: > > > From: Denys Dmytriyenko <denys@konsulko.com> > > > > This k3r5 multiconfig builds baremetal components (and corresponding > > native, > > nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal". The > > expectation is that those components and tools will use a separate TMPDIR > > to isolate from the main Linux build that uses "glibc" TCLIBC and to avoid > > potential conflicts. > > > > OE-Core "nodistro" default configuration already sets TCLIBCAPPEND facility > > to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal" temp > > directory for this multiconfig and "tmp-glibc" for the main Linux one. > > Other > > distros like Arago follow this convention and even extend a bit (e.g. Arago > > also adds TCMODE suffix to TMPDIR for external toolchain support > > separation). > > > > But Poky (and derivative distros, like AGL or YoE) disable TCLIBCAPPEND and > > result in a combined TMPDIR, leading to potential conflicts, such as: > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273 > > > > And that's just the beginning, there were other conflicts observed later in > > the build, e.g. during nativesdk builds - that's why we also set a unique > > SDKPKGSUFFIX here. > > > > To force a separate TMPDIR for k3r5 multiconfig, we have to directly append > > a suffix to it. Multiple other options were tested in hopes of making it > > slightly cleaner, but they either didn't work or were dismissed. For > > example, > > trying to override TCLIBCAPPEND getting cleared by a distro would require > > using a machine/soc-override, which doesn't have enough scope (nativesdk) > > or forcing it with :forcevariable would also change the main Linux TMPDIR > > and affect existing CI flows. Also, using TCLIBC itself as a suffix to add > > to TMPDIR may result in getting it appended twice (tmp-baremetal-baremetal) > > when normal TCLIBCAPPEND facility is used. Hence the least > > invasive/confusing > > option is to always append "-k3r5" suffix to this multiconfig TMPDIR. That > > results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while OE-Core > > "nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and > > "tmp-glibc" > > for the main). > > > > Also note, meta-ti-bsp layer.conf sets up images and sdks to be deployed > > into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY variable > > that controls it is set weakly, allowing to be modified from a distro > > configuration or local.conf. It means that all images and sdks can be > > deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as > > the final destination, by using := for immediate variable expansion: > > TI_COMMON_DEPLOY := "${TMPDIR}/deploy" > > > > Signed-off-by: Denys Dmytriyenko <denys@konsulko.com> > > --- > > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > index 04b9a746..e36c87ed 100644 > > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 = > > "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}" > > > > MACHINE:append = "-k3r5" > > TCLIBC = "baremetal" > > +TMPDIR:append = "-k3r5" > > + > > > > Thanks for fixing this long remaining issue on the kirkstone branch. > > AFAIK all the collision issues are mostly fixed on oe-core master. Yes, there were several related fixes in oe-core master recently, correct. But not everything was fixes yet even in master. > For me there are still sporadic failures with the SPDX bbclass. And SPDX was one of the areas that seen collision fixes in master. Unfortunately, those fixes have very little chances to get backported to kirkstone. Hence this change. > Can this change be optional? with something like: > > TMPDIRAPPEND ?= "-k3r5" > TMPDIR:append = "${TMPDIRAPPEND}" > > This adds the possibility of more easily reverting this change > to being able to fix what is pending in oe-core master. The point of this fix was to force separate TMPDIRs to avoid potential breakage - I was not planning on making it optional... The request seems to be very specific to layer/framework debugging purposes at which point this can be modified directly in the conf file. Allowing it to be disabled cleanly for a build seems dubious. But if there's a strong desire, I can send a v2...
Denys Dmytriyenko <denis@denix.org> escreveu no dia terça, 21/11/2023 à(s) 21:48: > On Mon, Nov 13, 2023 at 01:20:25PM +0000, Jose Quaresma wrote: > > Denys Dmytriyenko <denis@denix.org> escreveu no dia domingo, 12/11/2023 > > à(s) 19:07: > > > > > From: Denys Dmytriyenko <denys@konsulko.com> > > > > > > This k3r5 multiconfig builds baremetal components (and corresponding > > > native, > > > nativesdk and cross tools) and sets TCLIBC accordingly to "baremetal". > The > > > expectation is that those components and tools will use a separate > TMPDIR > > > to isolate from the main Linux build that uses "glibc" TCLIBC and to > avoid > > > potential conflicts. > > > > > > OE-Core "nodistro" default configuration already sets TCLIBCAPPEND > facility > > > to automatically add a suffix to TMPDIR, resulting in "tmp-baremetal" > temp > > > directory for this multiconfig and "tmp-glibc" for the main Linux one. > > > Other > > > distros like Arago follow this convention and even extend a bit (e.g. > Arago > > > also adds TCMODE suffix to TMPDIR for external toolchain support > > > separation). > > > > > > But Poky (and derivative distros, like AGL or YoE) disable > TCLIBCAPPEND and > > > result in a combined TMPDIR, leading to potential conflicts, such as: > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=15273 > > > > > > And that's just the beginning, there were other conflicts observed > later in > > > the build, e.g. during nativesdk builds - that's why we also set a > unique > > > SDKPKGSUFFIX here. > > > > > > To force a separate TMPDIR for k3r5 multiconfig, we have to directly > append > > > a suffix to it. Multiple other options were tested in hopes of making > it > > > slightly cleaner, but they either didn't work or were dismissed. For > > > example, > > > trying to override TCLIBCAPPEND getting cleared by a distro would > require > > > using a machine/soc-override, which doesn't have enough scope > (nativesdk) > > > or forcing it with :forcevariable would also change the main Linux > TMPDIR > > > and affect existing CI flows. Also, using TCLIBC itself as a suffix to > add > > > to TMPDIR may result in getting it appended twice > (tmp-baremetal-baremetal) > > > when normal TCLIBCAPPEND facility is used. Hence the least > > > invasive/confusing > > > option is to always append "-k3r5" suffix to this multiconfig TMPDIR. > That > > > results in "tmp-k3r5" in Poky (leaving main TMPDIR as "tmp"), while > OE-Core > > > "nodistro" and Arago would end up with "tmp-baremetal-k3r5" (and > > > "tmp-glibc" > > > for the main). > > > > > > Also note, meta-ti-bsp layer.conf sets up images and sdks to be > deployed > > > into a common location outside of TMPDIRs, but TI_COMMON_DEPLOY > variable > > > that controls it is set weakly, allowing to be modified from a distro > > > configuration or local.conf. It means that all images and sdks can be > > > deployed into the main TMPDIR if one's CI flow expects tmp/deploy/ as > > > the final destination, by using := for immediate variable expansion: > > > TI_COMMON_DEPLOY := "${TMPDIR}/deploy" > > > > > > Signed-off-by: Denys Dmytriyenko <denys@konsulko.com> > > > --- > > > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > > > 1 file changed, 2 insertions(+) > > > > > > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > > b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > > index 04b9a746..e36c87ed 100644 > > > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > > @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 = > > > "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}" > > > > > > MACHINE:append = "-k3r5" > > > TCLIBC = "baremetal" > > > +TMPDIR:append = "-k3r5" > > > + > > > > > > > Thanks for fixing this long remaining issue on the kirkstone branch. > > > > AFAIK all the collision issues are mostly fixed on oe-core master. > > Yes, there were several related fixes in oe-core master recently, correct. > But not everything was fixes yet even in master. > > > > For me there are still sporadic failures with the SPDX bbclass. > > And SPDX was one of the areas that seen collision fixes in master. > Unfortunately, those fixes have very little chances to get backported > to kirkstone. Hence this change. > I'm not opposed to having that in kirkstone. I am working on this SPDX backport but as you said it will be very hard because some of these backports requires sstate, siggen and also bitbake patches. > > > > Can this change be optional? with something like: > > > > TMPDIRAPPEND ?= "-k3r5" > > TMPDIR:append = "${TMPDIRAPPEND}" > > > > This adds the possibility of more easily reverting this change > > to being able to fix what is pending in oe-core master. > > The point of this fix was to force separate TMPDIRs to avoid potential > breakage - I was not planning on making it optional... > I've been using a separate TMPDIR directory for some time now on kirkstone and it was quite difficult to do so because there is no way to change multiconfig/k3r5.conf in place. I was told several times here on this list that such a change was not necessary. > The request seems to be very specific to layer/framework debugging > purposes > at which point this can be modified directly in the conf file. Allowing it > to > be disabled cleanly for a build seems dubious. But if there's a strong > desire, > I can send a v2... > Having something like the above gives more freedom to change it back to a single TMPDIR. I prefer a v2 where the user can choose what suits your build best. Jose > > -- > Denys >
diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf b/meta-ti-bsp/conf/multiconfig/k3r5.conf index 04b9a746..e36c87ed 100644 --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf @@ -4,4 +4,6 @@ DEPLOY_DIR_IMAGE:k3r5 = "${TI_COMMON_DEPLOY}/images/${MAINMACHINE}" MACHINE:append = "-k3r5" TCLIBC = "baremetal" +TMPDIR:append = "-k3r5" + SDKPKGSUFFIX = "k3r5-nativesdk"