| Message ID | 20220222035234.463162-8-andrew@aj.id.au |
|---|---|
| State | Accepted, archived |
| Commit | ab0c2de443278625c5db54d5c51e193791f5087c |
| Headers | show |
| Series | rust: Fix powerpc64le support | expand |
I have to note that YP does not support ppc systems as build hosts; can this be kept in a bbappend? Alex On Tue, 22 Feb 2022 at 04:52, Andrew Jeffery <andrew@aj.id.au> wrote: > > Signed-off-by: Andrew Jeffery <andrew@aj.id.au> > --- > meta/recipes-devtools/rust/rust-snapshot.inc | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/meta/recipes-devtools/rust/rust-snapshot.inc b/meta/recipes-devtools/rust/rust-snapshot.inc > index 74b558262f62..d6ffe92d07ee 100644 > --- a/meta/recipes-devtools/rust/rust-snapshot.inc > +++ b/meta/recipes-devtools/rust/rust-snapshot.inc > @@ -14,6 +14,10 @@ SRC_URI[rust-std-snapshot-aarch64.sha256sum] = "3618da916a0f92f241cf1d67d04bb578 > SRC_URI[rustc-snapshot-aarch64.sha256sum] = "f26811e48d03c56c125de03d389e1ae7c6df36990953c1670c6a5676bc12d4cb" > SRC_URI[cargo-snapshot-aarch64.sha256sum] = "6d11cd94618d80cda273eeeae7285980445f61a49ebacc616777b482a41cbf3f" > > +SRC_URI[rust-std-snapshot-powerpc64le.sha256sum] = "fc07eb3e9f3d227428cc5b53ca868e3de375bc198ce4dce7b87a9246e6fec81a" > +SRC_URI[rustc-snapshot-powerpc64le.sha256sum] = "f43cb99109c3438c77c7079cdce4673df3320e310158e0b4d949c1babc4300fc" > +SRC_URI[cargo-snapshot-powerpc64le.sha256sum] = "599cf1b5a8cdbf76d591621bc9222aefa60e2f5fd378ae71c4dcf4514c47122e" > + > SRC_URI += " \ > https://static.rust-lang.org/dist/${RUST_STD_SNAPSHOT}.tar.xz;name=rust-std-snapshot-${BUILD_ARCH};subdir=rust-snapshot-components \ > https://static.rust-lang.org/dist/${RUSTC_SNAPSHOT}.tar.xz;name=rustc-snapshot-${BUILD_ARCH};subdir=rust-snapshot-components \ > -- > 2.32.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#162121): https://lists.openembedded.org/g/openembedded-core/message/162121 > Mute This Topic: https://lists.openembedded.org/mt/89310363/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: > I have to note that YP does not support ppc systems as build hosts; > can this be kept in a bbappend? It isn't official but there are obviously people using it. I think we should try and do something in core but there is a question of how we maintain this given our infrastructure/tests don't cover it :( Cheers, Richard
On Tue, 22 Feb 2022 at 13:28, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: > > I have to note that YP does not support ppc systems as build hosts; > > can this be kept in a bbappend? > > It isn't official but there are obviously people using it. I think we should try > and do something in core but there is a question of how we maintain this given > our infrastructure/tests don't cover it :( Worse yet, there is no way to test this locally either. If the ppc64 binary tarball checksums are added to the core rust recipe, presumably I'd be expected to update them together with x86 and arm64 binary checksums on version updates, but I'd have to do this completely blindly with no testing of any kind :( That's why I am asking to keep them in an external bbappend. Andrew, how hard is it to obtain a shell on ppc64 machines nowadays? Is it something specific to your employer? Alex
On Tue, Feb 22, 2022 at 6:24 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Tue, 22 Feb 2022 at 13:28, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: > > > I have to note that YP does not support ppc systems as build hosts; > > > can this be kept in a bbappend? > > > > It isn't official but there are obviously people using it. I think we should try > > and do something in core but there is a question of how we maintain this given > > our infrastructure/tests don't cover it :( > > Worse yet, there is no way to test this locally either. If the ppc64 > binary tarball checksums are added to the core rust recipe, presumably > I'd be expected to update them together with x86 and arm64 binary > checksums on version updates, but I'd have to do this completely > blindly with no testing of any kind :( as long as it does not impede main testing I would think thats ok and we can expect the OE community members like Andrew to test this and report issues or better fix them That's why I am asking to keep > them in an external bbappend. > > Andrew, how hard is it to obtain a shell on ppc64 machines nowadays? > Is it something specific to your employer? > > Alex > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#162134): https://lists.openembedded.org/g/openembedded-core/message/162134 > Mute This Topic: https://lists.openembedded.org/mt/89310363/1997914 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, 23 Feb 2022, at 05:04, Khem Raj wrote: > On Tue, Feb 22, 2022 at 6:24 AM Alexander Kanavin > <alex.kanavin@gmail.com> wrote: >> >> On Tue, 22 Feb 2022 at 13:28, Richard Purdie >> <richard.purdie@linuxfoundation.org> wrote: >> > >> > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: >> > > I have to note that YP does not support ppc systems as build hosts; >> > > can this be kept in a bbappend? >> > >> > It isn't official but there are obviously people using it. I think we should try >> > and do something in core but there is a question of how we maintain this given >> > our infrastructure/tests don't cover it :( >> >> Worse yet, there is no way to test this locally either. If the ppc64 >> binary tarball checksums are added to the core rust recipe, presumably >> I'd be expected to update them together with x86 and arm64 binary >> checksums on version updates, but I'd have to do this completely >> blindly with no testing of any kind :( > > as long as it does not impede main testing I would think thats ok and > we can expect the OE community > members like Andrew to test this and report issues or better fix them Well, hopefully I've shown I'm at least willing to try fix things :D And yeah, my preference would be that the ppc64le checksums are just bumped along with the rest, but without any explicit testing beyond that. I understand that I'm not the maintainer here so will go along with whatever approach is chosen, but keeping them here feels like we're at least solving the problem in a consistent spot. As mentioned in the cover letter we found this issue via the ppc64le CI machines used in OpenBMC. OpenBMC tries to integrate upstream YP changes every fortnight or so. That's obviously not as ideal as ppc64le resources dedicated to YP, but I think this provides enough feedback that things won't completely rot. If a bump does cause issues I'll try to resolve them as soon as possible. Andrew
On Tue, 22 Feb 2022, at 22:58, Richard Purdie wrote: > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: >> I have to note that YP does not support ppc systems as build hosts; >> can this be kept in a bbappend? > > It isn't official but there are obviously people using it. I think we should try > and do something in core but there is a question of how we maintain this given > our infrastructure/tests don't cover it :( That's fair. Is it acceptable that it's tested by proxy in OpenBMC if we can't work out anything else for YP CI? I don't expect anyone maintaining the rust support to test ppc64le explicitly if they don't have access. I'd be happy if the ppc64le checksums were just updated along with the rest whenever the snapshots are bumped. Andrew
On Wed, 2022-02-23 at 10:19 +1030, Andrew Jeffery wrote: > > On Tue, 22 Feb 2022, at 22:58, Richard Purdie wrote: > > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: > > > I have to note that YP does not support ppc systems as build hosts; > > > can this be kept in a bbappend? > > > > It isn't official but there are obviously people using it. I think we should try > > and do something in core but there is a question of how we maintain this given > > our infrastructure/tests don't cover it :( > > That's fair. Is it acceptable that it's tested by proxy in OpenBMC if > we can't work out anything else for YP CI? I don't expect anyone > maintaining the rust support to test ppc64le explicitly if they don't > have access. I'd be happy if the ppc64le checksums were just updated > along with the rest whenever the snapshots are bumped. The challenge is we have no tooling or support for "bumping" those values. We spot issues with the other two build architectures since the autobuilder tests them. There are probably some tricks we could play to handle this. The challenge is then that someone needs to take the time to write something to handle this and communicate it to the maintainers so they know to use it. We're struggling to do all the things like this we'd like to and that leads back to Alex's concern. Cheers, Richard
On Wed, 23 Feb 2022, at 00:53, Alexander Kanavin wrote: > On Tue, 22 Feb 2022 at 13:28, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: >> >> On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: >> > I have to note that YP does not support ppc systems as build hosts; >> > can this be kept in a bbappend? >> >> It isn't official but there are obviously people using it. I think we should try >> and do something in core but there is a question of how we maintain this given >> our infrastructure/tests don't cover it :( > > Worse yet, there is no way to test this locally either. If the ppc64 > binary tarball checksums are added to the core rust recipe, presumably > I'd be expected to update them together with x86 and arm64 binary > checksums on version updates, but I'd have to do this completely > blindly with no testing of any kind :( That's why I am asking to keep > them in an external bbappend. > > Andrew, how hard is it to obtain a shell on ppc64 machines nowadays? (I work for IBM Power Systems) IBM have partnered with Oregon State University to provide access to ppc64le machines: * https://osuosl.org/services/powerdev/ * https://power-developer.mybluemix.net/#hardware You can request access to the OSU setup: * For a shell: https://osuosl.org/services/powerdev/request_hosting/ * For CI: https://osuosl.org/services/powerdev/request_powerci/ So there are some forms. I'm not sure where the bar sits for you with respect to how hard it is to go this route, but feel free to reply to me privately or via andrewrj@au1.ibm.com if you have further questions about setting something up on the OSU OSL infrastructure. Andrew
On Wed, 23 Feb 2022, at 10:33, Richard Purdie wrote: > On Wed, 2022-02-23 at 10:19 +1030, Andrew Jeffery wrote: >> >> On Tue, 22 Feb 2022, at 22:58, Richard Purdie wrote: >> > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: >> > > I have to note that YP does not support ppc systems as build hosts; >> > > can this be kept in a bbappend? >> > >> > It isn't official but there are obviously people using it. I think we should try >> > and do something in core but there is a question of how we maintain this given >> > our infrastructure/tests don't cover it :( >> >> That's fair. Is it acceptable that it's tested by proxy in OpenBMC if >> we can't work out anything else for YP CI? I don't expect anyone >> maintaining the rust support to test ppc64le explicitly if they don't >> have access. I'd be happy if the ppc64le checksums were just updated >> along with the rest whenever the snapshots are bumped. > > The challenge is we have no tooling or support for "bumping" those values. Yeah, that's familiar, when I sent the initial patch fixing the URLs I hacked some stuff to force the BUILD_ARCH value to ppc64le on my x86-64 laptop. That's not a scalable approach. Not having the tooling seems reasonable, as having it does kinda suggest you'll try to maintain things you can't test. Even though I'm trying to argue for that, again, it's not scalable and I understand the push-back :) > We > spot issues with the other two build architectures since the autobuilder tests > them. > > There are probably some tricks we could play to handle this. The challenge is > then that someone needs to take the time to write something to handle this and > communicate it to the maintainers so they know to use it. We're struggling to do > all the things like this we'd like to and that leads back to Alex's concern. I understand. I have pointed Alex at some resources to access ppc64le systems: https://lore.kernel.org/openembedded-core/ee6810ac-3c22-45b0-ac49-e35b38e0f69c@www.fastmail.com/ But if that's too much fuss then we can maintain the checksums in a bbappend in OpenBMC like has been suggested. Andrew
On Wed, 2022-02-23 at 10:57 +1030, Andrew Jeffery wrote: > > On Wed, 23 Feb 2022, at 10:33, Richard Purdie wrote: > > On Wed, 2022-02-23 at 10:19 +1030, Andrew Jeffery wrote: > > > > > > On Tue, 22 Feb 2022, at 22:58, Richard Purdie wrote: > > > > On Tue, 2022-02-22 at 10:59 +0100, Alexander Kanavin wrote: > > > > > I have to note that YP does not support ppc systems as build hosts; > > > > > can this be kept in a bbappend? > > > > > > > > It isn't official but there are obviously people using it. I think we should try > > > > and do something in core but there is a question of how we maintain this given > > > > our infrastructure/tests don't cover it :( > > > > > > That's fair. Is it acceptable that it's tested by proxy in OpenBMC if > > > we can't work out anything else for YP CI? I don't expect anyone > > > maintaining the rust support to test ppc64le explicitly if they don't > > > have access. I'd be happy if the ppc64le checksums were just updated > > > along with the rest whenever the snapshots are bumped. > > > > The challenge is we have no tooling or support for "bumping" those values. > > Yeah, that's familiar, when I sent the initial patch fixing the URLs I > hacked some stuff to force the BUILD_ARCH value to ppc64le on my x86-64 > laptop. That's not a scalable approach. Funnily enough, tweaking BUILD_ARCH was something I was wondering about in some of our automated tests since that might reduce the changes of some things breaking. > Not having the tooling seems reasonable, as having it does kinda > suggest you'll try to maintain things you can't test. Even though I'm > trying to argue for that, again, it's not scalable and I understand the > push-back :) That is the big worry as we can't really test it. Cheers, Richard
diff --git a/meta/recipes-devtools/rust/rust-snapshot.inc b/meta/recipes-devtools/rust/rust-snapshot.inc index 74b558262f62..d6ffe92d07ee 100644 --- a/meta/recipes-devtools/rust/rust-snapshot.inc +++ b/meta/recipes-devtools/rust/rust-snapshot.inc @@ -14,6 +14,10 @@ SRC_URI[rust-std-snapshot-aarch64.sha256sum] = "3618da916a0f92f241cf1d67d04bb578 SRC_URI[rustc-snapshot-aarch64.sha256sum] = "f26811e48d03c56c125de03d389e1ae7c6df36990953c1670c6a5676bc12d4cb" SRC_URI[cargo-snapshot-aarch64.sha256sum] = "6d11cd94618d80cda273eeeae7285980445f61a49ebacc616777b482a41cbf3f" +SRC_URI[rust-std-snapshot-powerpc64le.sha256sum] = "fc07eb3e9f3d227428cc5b53ca868e3de375bc198ce4dce7b87a9246e6fec81a" +SRC_URI[rustc-snapshot-powerpc64le.sha256sum] = "f43cb99109c3438c77c7079cdce4673df3320e310158e0b4d949c1babc4300fc" +SRC_URI[cargo-snapshot-powerpc64le.sha256sum] = "599cf1b5a8cdbf76d591621bc9222aefa60e2f5fd378ae71c4dcf4514c47122e" + SRC_URI += " \ https://static.rust-lang.org/dist/${RUST_STD_SNAPSHOT}.tar.xz;name=rust-std-snapshot-${BUILD_ARCH};subdir=rust-snapshot-components \ https://static.rust-lang.org/dist/${RUSTC_SNAPSHOT}.tar.xz;name=rustc-snapshot-${BUILD_ARCH};subdir=rust-snapshot-components \
Signed-off-by: Andrew Jeffery <andrew@aj.id.au> --- meta/recipes-devtools/rust/rust-snapshot.inc | 4 ++++ 1 file changed, 4 insertions(+)