| Message ID | 20220530143810.4064720-1-jose.quaresma@foundries.io |
|---|---|
| State | Rejected |
| Delegated to: | Ryan Eatmon |
| Headers | show |
| Series | multiconfig: use diferent TMPDIR for each machine | expand |
On Mon, May 30, 2022 at 03:38:10PM +0100, Jose Quaresma wrote: > Bitbake manual [1] recomends to use one TMPDIR for each machine: > > "Minimally, each configuration file must define the machine and the > temporary directory BitBake uses for the build. > Suggested practice dictates that you do not overlap the temporary > directories used during the builds." What specific issue are you trying to fix with this? I'm not aware of any immediate issues (because here multiconfig is used for completely separate machines and those are well isolated inside the temp dir), but I see few problems this will cause down the road, so I'm against this change. > [1] https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build > > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> > --- > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf b/meta-ti-bsp/conf/multiconfig/k3r5.conf > index deb07210..0ba163ae 100644 > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > @@ -3,3 +3,5 @@ MAINMACHINE := "${MACHINE}" > DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MAINMACHINE}" > > MACHINE:append = "-k3r5" > + > +TMPDIR = "${TOPDIR}/tmp-${MACHINE}" > -- > 2.36.1
Denys Dmytriyenko <denis@denix.org> escreveu no dia terça, 31/05/2022 à(s) 20:33: > On Mon, May 30, 2022 at 03:38:10PM +0100, Jose Quaresma wrote: > > Bitbake manual [1] recomends to use one TMPDIR for each machine: > > > > "Minimally, each configuration file must define the machine and the > > temporary directory BitBake uses for the build. > > Suggested practice dictates that you do not overlap the temporary > > directories used during the builds." > > What specific issue are you trying to fix with this? > The issue that I have is in the OE-core archiver.bbclasse that doesn't support multiconfig when using the same TMPDIR, this because the shared recipes like gcc, kernel It can be replicates in kirkstone using the am64xx-evm with the following: # archive sources for target recipes in conf/local.conf INHERIT += "archiver" COPYLEFT_RECIPE_TYPES = "target" ARCHIVER_MODE[src] = "original" ARCHIVER_MODE[diff] = "1" # and run bitbake with: bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch -f more info in following patch: https://lists.openembedded.org/g/openembedded-core/message/166203 > I'm not aware of any immediate issues (because here multiconfig is used > for > completely separate machines and those are well isolated inside the temp > dir), > but I see few problems this will cause down the road, so I'm against this > change. > What you are saying is that this change will introduce new problems or regressions? Jose > > > > [1] > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build > > > > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> > > --- > > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf > b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > index deb07210..0ba163ae 100644 > > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > @@ -3,3 +3,5 @@ MAINMACHINE := "${MACHINE}" > > DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MAINMACHINE}" > > > > MACHINE:append = "-k3r5" > > + > > +TMPDIR = "${TOPDIR}/tmp-${MACHINE}" > > -- > > 2.36.1 >
Multi-machine builds in a single tmpdir has been supported by oe since before multiconfig existed, and certainly works fine with multiconfig, it's only other configuration changes, such as distro, that result in conflicts and failures when using multiconfig in a single tmpdir. I don't think it's a good idea to force the matter due to flaws in archiver, personally. On Wed, Jun 1, 2022 at 2:17 AM Jose Quaresma <quaresma.jose@gmail.com> wrote: > > > Denys Dmytriyenko <denis@denix.org> escreveu no dia terça, 31/05/2022 > à(s) 20:33: > >> On Mon, May 30, 2022 at 03:38:10PM +0100, Jose Quaresma wrote: >> > Bitbake manual [1] recomends to use one TMPDIR for each machine: >> > >> > "Minimally, each configuration file must define the machine and the >> > temporary directory BitBake uses for the build. >> > Suggested practice dictates that you do not overlap the temporary >> > directories used during the builds." >> >> What specific issue are you trying to fix with this? >> > > The issue that I have is in the OE-core archiver.bbclasse that doesn't > support multiconfig when > using the same TMPDIR, this because the shared recipes like gcc, kernel > > It can be replicates in kirkstone using the am64xx-evm with the following: > > # archive sources for target recipes in conf/local.conf > INHERIT += "archiver" > COPYLEFT_RECIPE_TYPES = "target" > ARCHIVER_MODE[src] = "original" > ARCHIVER_MODE[diff] = "1" > > # and run bitbake with: > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch > -f > > more info in following patch: > https://lists.openembedded.org/g/openembedded-core/message/166203 > > >> I'm not aware of any immediate issues (because here multiconfig is used >> for >> completely separate machines and those are well isolated inside the temp >> dir), >> but I see few problems this will cause down the road, so I'm against this >> change. >> > > What you are saying is that this change will introduce new problems or > regressions? > > Jose > > >> >> >> > [1] >> https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build >> > >> > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> >> > --- >> > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ >> > 1 file changed, 2 insertions(+) >> > >> > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf >> b/meta-ti-bsp/conf/multiconfig/k3r5.conf >> > index deb07210..0ba163ae 100644 >> > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf >> > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf >> > @@ -3,3 +3,5 @@ MAINMACHINE := "${MACHINE}" >> > DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MAINMACHINE}" >> > >> > MACHINE:append = "-k3r5" >> > + >> > +TMPDIR = "${TOPDIR}/tmp-${MACHINE}" >> > -- >> > 2.36.1 >> > > > -- > Best regards, > > José Quaresma > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14769): > https://lists.yoctoproject.org/g/meta-ti/message/14769 > Mute This Topic: https://lists.yoctoproject.org/mt/91431962/3617123 > Group Owner: meta-ti+owner@lists.yoctoproject.org > Unsubscribe: https://lists.yoctoproject.org/g/meta-ti/unsub [ > kergoth@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Wed, Jun 01, 2022 at 09:32:58AM -0700, Christopher Larson wrote: > Multi-machine builds in a single tmpdir has been supported by oe since > before multiconfig existed, and certainly works fine with multiconfig, it's > only other configuration changes, such as distro, that result in conflicts > and failures when using multiconfig in a single tmpdir. I don't think it's > a good idea to force the matter due to flaws in archiver, personally. Yes, I agree that in this case it is best to fix the archiver class. The reason everything else just works is because there's already per-machine separation in many places - deploy, images, work, cache, stamps, etc. There's also a great re-use of shared components between similar machines built in the same tmpdir. As Chris says, this always worked even before multiconfig - you just change MACHINE and re-run bitbake. And multiconfig simply allows doing it in one invocation of bitbake, that's all. Denys > On Wed, Jun 1, 2022 at 2:17 AM Jose Quaresma <quaresma.jose@gmail.com> > wrote: > > > > > > > Denys Dmytriyenko <denis@denix.org> escreveu no dia terça, 31/05/2022 > > à(s) 20:33: > > > >> On Mon, May 30, 2022 at 03:38:10PM +0100, Jose Quaresma wrote: > >> > Bitbake manual [1] recomends to use one TMPDIR for each machine: > >> > > >> > "Minimally, each configuration file must define the machine and the > >> > temporary directory BitBake uses for the build. > >> > Suggested practice dictates that you do not overlap the temporary > >> > directories used during the builds." > >> > >> What specific issue are you trying to fix with this? > >> > > > > The issue that I have is in the OE-core archiver.bbclasse that doesn't > > support multiconfig when > > using the same TMPDIR, this because the shared recipes like gcc, kernel > > > > It can be replicates in kirkstone using the am64xx-evm with the following: > > > > # archive sources for target recipes in conf/local.conf > > INHERIT += "archiver" > > COPYLEFT_RECIPE_TYPES = "target" > > ARCHIVER_MODE[src] = "original" > > ARCHIVER_MODE[diff] = "1" > > > > # and run bitbake with: > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch > > -f > > > > more info in following patch: > > https://lists.openembedded.org/g/openembedded-core/message/166203 > > > > > >> I'm not aware of any immediate issues (because here multiconfig is used > >> for > >> completely separate machines and those are well isolated inside the temp > >> dir), > >> but I see few problems this will cause down the road, so I'm against this > >> change. > >> > > > > What you are saying is that this change will introduce new problems or > > regressions? > > > > Jose > > > > > >> > >> > >> > [1] > >> https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build > >> > > >> > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> > >> > --- > >> > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > >> > 1 file changed, 2 insertions(+) > >> > > >> > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf > >> b/meta-ti-bsp/conf/multiconfig/k3r5.conf > >> > index deb07210..0ba163ae 100644 > >> > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > >> > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > >> > @@ -3,3 +3,5 @@ MAINMACHINE := "${MACHINE}" > >> > DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MAINMACHINE}" > >> > > >> > MACHINE:append = "-k3r5" > >> > + > >> > +TMPDIR = "${TOPDIR}/tmp-${MACHINE}" > >> > -- > >> > 2.36.1 > >> > > > > > > -- > > Best regards, > > > > José Quaresma > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#14769): > > https://lists.yoctoproject.org/g/meta-ti/message/14769 > > Mute This Topic: https://lists.yoctoproject.org/mt/91431962/3617123 > > Group Owner: meta-ti+owner@lists.yoctoproject.org > > Unsubscribe: https://lists.yoctoproject.org/g/meta-ti/unsub [ > > kergoth@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > -- > Christopher Larson > chris_larson@mentor.com, chris.larson@siemens.com, kergoth@gmail.com > Principal Software Engineer, Embedded Linux Solutions, Siemens Digital > Industries Software
Denys Dmytriyenko <denis@denix.org> escreveu no dia quarta, 1/06/2022 à(s) 17:49: > On Wed, Jun 01, 2022 at 09:32:58AM -0700, Christopher Larson wrote: > > Multi-machine builds in a single tmpdir has been supported by oe since > > before multiconfig existed, and certainly works fine with multiconfig, > it's > > only other configuration changes, such as distro, that result in > conflicts > > and failures when using multiconfig in a single tmpdir. I don't think > it's > > a good idea to force the matter due to flaws in archiver, personally. > I will investigate a bit more but for now the only issue I see is on the archiver. > > Yes, I agree that in this case it is best to fix the archiver class. > I will try this approach instead. > > The reason everything else just works is because there's already > per-machine > separation in many places - deploy, images, work, cache, stamps, etc. > yes, everything seems to work except for the archiver class so for sure the problem is here. > > There's also a great re-use of shared components between similar machines > built in the same tmpdir. As Chris says, this always worked even before > multiconfig - you just change MACHINE and re-run bitbake. And multiconfig > simply allows doing it in one invocation of bitbake, that's all. > Right and I totally agree. Thanks for your input which always helps to better understand how this beast called bitbake works ;) Jose > > Denys > > > > On Wed, Jun 1, 2022 at 2:17 AM Jose Quaresma <quaresma.jose@gmail.com> > > wrote: > > > > > > > > > > > Denys Dmytriyenko <denis@denix.org> escreveu no dia terça, 31/05/2022 > > > à(s) 20:33: > > > > > >> On Mon, May 30, 2022 at 03:38:10PM +0100, Jose Quaresma wrote: > > >> > Bitbake manual [1] recomends to use one TMPDIR for each machine: > > >> > > > >> > "Minimally, each configuration file must define the machine and the > > >> > temporary directory BitBake uses for the build. > > >> > Suggested practice dictates that you do not overlap the temporary > > >> > directories used during the builds." > > >> > > >> What specific issue are you trying to fix with this? > > >> > > > > > > The issue that I have is in the OE-core archiver.bbclasse that doesn't > > > support multiconfig when > > > using the same TMPDIR, this because the shared recipes like gcc, kernel > > > > > > It can be replicates in kirkstone using the am64xx-evm with the > following: > > > > > > # archive sources for target recipes in conf/local.conf > > > INHERIT += "archiver" > > > COPYLEFT_RECIPE_TYPES = "target" > > > ARCHIVER_MODE[src] = "original" > > > ARCHIVER_MODE[diff] = "1" > > > > > > # and run bitbake with: > > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c > do_unpack_and_patch > > > -f > > > > > > more info in following patch: > > > https://lists.openembedded.org/g/openembedded-core/message/166203 > > > > > > > > >> I'm not aware of any immediate issues (because here multiconfig is > used > > >> for > > >> completely separate machines and those are well isolated inside the > temp > > >> dir), > > >> but I see few problems this will cause down the road, so I'm against > this > > >> change. > > >> > > > > > > What you are saying is that this change will introduce new problems or > > > regressions? > > > > > > Jose > > > > > > > > >> > > >> > > >> > [1] > > >> > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build > > >> > > > >> > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> > > >> > --- > > >> > meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ > > >> > 1 file changed, 2 insertions(+) > > >> > > > >> > diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > >> b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > >> > index deb07210..0ba163ae 100644 > > >> > --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf > > >> > +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf > > >> > @@ -3,3 +3,5 @@ MAINMACHINE := "${MACHINE}" > > >> > DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MAINMACHINE}" > > >> > > > >> > MACHINE:append = "-k3r5" > > >> > + > > >> > +TMPDIR = "${TOPDIR}/tmp-${MACHINE}" > > >> > -- > > >> > 2.36.1 > > >> > > > > > > > > > -- > > > Best regards, > > > > > > José Quaresma > > > > > > > > > > > > > > > > -- > > Christopher Larson > > chris_larson@mentor.com, chris.larson@siemens.com, kergoth@gmail.com > > Principal Software Engineer, Embedded Linux Solutions, Siemens Digital > > Industries Software > > -- > Regards, > Denys Dmytriyenko <denis@denix.org> > PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964 > Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964 > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14771): > https://lists.yoctoproject.org/g/meta-ti/message/14771 > Mute This Topic: https://lists.yoctoproject.org/mt/91431962/5052612 > Group Owner: meta-ti+owner@lists.yoctoproject.org > Unsubscribe: https://lists.yoctoproject.org/g/meta-ti/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
diff --git a/meta-ti-bsp/conf/multiconfig/k3r5.conf b/meta-ti-bsp/conf/multiconfig/k3r5.conf index deb07210..0ba163ae 100644 --- a/meta-ti-bsp/conf/multiconfig/k3r5.conf +++ b/meta-ti-bsp/conf/multiconfig/k3r5.conf @@ -3,3 +3,5 @@ MAINMACHINE := "${MACHINE}" DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MAINMACHINE}" MACHINE:append = "-k3r5" + +TMPDIR = "${TOPDIR}/tmp-${MACHINE}"
Bitbake manual [1] recomends to use one TMPDIR for each machine: "Minimally, each configuration file must define the machine and the temporary directory BitBake uses for the build. Suggested practice dictates that you do not overlap the temporary directories used during the builds." [1] https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> --- meta-ti-bsp/conf/multiconfig/k3r5.conf | 2 ++ 1 file changed, 2 insertions(+)