| Message ID | 20211217153630.1746217-1-matt@madison.systems |
|---|---|
| Headers | show |
| Series | Move crate fetcher to OE_IMPORTS | expand |
On 17/12/2021 15:36, Matt Madison wrote: > This patch series switches to using the OE_IMPORTS mechanism > for installing the crate fetcher. This simplifies the addition > and clears up issues with interactions between SRCPV and > the shared-state fetching logic. > > Matt Madison (4): > lib/crate.py: install crate fetcher on import > base.bbclass: add crate to OE_IMPORTS > cargo_common.bbclass: remove inherit of crate-fetch > classes: remove crate-fetch.bbclass > > meta/classes/base.bbclass | 2 +- > meta/classes/cargo_common.bbclass | 2 -- > meta/classes/crate-fetch.bbclass | 28 ---------------------------- > meta/lib/crate.py | 2 ++ > 4 files changed, 3 insertions(+), 31 deletions(-) > delete mode 100644 meta/classes/crate-fetch.bbclass > Would a better solution here be to move the crate fetcher into lib/bb/fetch2/ in bitbake? -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.barker@sancloud.com w: https://sancloud.co.uk/
On Mon, 2021-12-20 at 17:01 +0000, Paul Barker wrote: > On 17/12/2021 15:36, Matt Madison wrote: > > This patch series switches to using the OE_IMPORTS mechanism > > for installing the crate fetcher. This simplifies the addition > > and clears up issues with interactions between SRCPV and > > the shared-state fetching logic. > > > > Matt Madison (4): > > lib/crate.py: install crate fetcher on import > > base.bbclass: add crate to OE_IMPORTS > > cargo_common.bbclass: remove inherit of crate-fetch > > classes: remove crate-fetch.bbclass > > > > meta/classes/base.bbclass | 2 +- > > meta/classes/cargo_common.bbclass | 2 -- > > meta/classes/crate-fetch.bbclass | 28 ---------------------------- > > meta/lib/crate.py | 2 ++ > > 4 files changed, 3 insertions(+), 31 deletions(-) > > delete mode 100644 meta/classes/crate-fetch.bbclass > > > > Would a better solution here be to move the crate fetcher into > lib/bb/fetch2/ in bitbake? I've been wondering about this too. Does anyone remember how the fetcher lines up against the general fetcher requirements we have and is there a list of anything that remained to be done before it could be added to bitbake? Cheers, Richard
On 20/12/2021 22:34, Richard Purdie wrote: > On Mon, 2021-12-20 at 17:01 +0000, Paul Barker wrote: >> On 17/12/2021 15:36, Matt Madison wrote: >>> This patch series switches to using the OE_IMPORTS mechanism >>> for installing the crate fetcher. This simplifies the addition >>> and clears up issues with interactions between SRCPV and >>> the shared-state fetching logic. >>> >>> Matt Madison (4): >>> lib/crate.py: install crate fetcher on import >>> base.bbclass: add crate to OE_IMPORTS >>> cargo_common.bbclass: remove inherit of crate-fetch >>> classes: remove crate-fetch.bbclass >>> >>> meta/classes/base.bbclass | 2 +- >>> meta/classes/cargo_common.bbclass | 2 -- >>> meta/classes/crate-fetch.bbclass | 28 ---------------------------- >>> meta/lib/crate.py | 2 ++ >>> 4 files changed, 3 insertions(+), 31 deletions(-) >>> delete mode 100644 meta/classes/crate-fetch.bbclass >>> >> >> Would a better solution here be to move the crate fetcher into >> lib/bb/fetch2/ in bitbake? > > I've been wondering about this too. > > Does anyone remember how the fetcher lines up against the general fetcher > requirements we have and is there a list of anything that remained to be done > before it could be added to bitbake? If I remember correctly, it just needed some test cases writing. The crate fetcher is a subclass of the wget fetcher with changes to how urldata is set up and how unpack is handled, so it should integrate well into bitbake. Thanks,
On Tue, Dec 21, 2021 at 6:07 AM Paul Barker <paul.barker@sancloud.com> wrote: > > On 20/12/2021 22:34, Richard Purdie wrote: > > On Mon, 2021-12-20 at 17:01 +0000, Paul Barker wrote: > >> On 17/12/2021 15:36, Matt Madison wrote: > >>> This patch series switches to using the OE_IMPORTS mechanism > >>> for installing the crate fetcher. This simplifies the addition > >>> and clears up issues with interactions between SRCPV and > >>> the shared-state fetching logic. > >>> > >>> Matt Madison (4): > >>> lib/crate.py: install crate fetcher on import > >>> base.bbclass: add crate to OE_IMPORTS > >>> cargo_common.bbclass: remove inherit of crate-fetch > >>> classes: remove crate-fetch.bbclass > >>> > >>> meta/classes/base.bbclass | 2 +- > >>> meta/classes/cargo_common.bbclass | 2 -- > >>> meta/classes/crate-fetch.bbclass | 28 ---------------------------- > >>> meta/lib/crate.py | 2 ++ > >>> 4 files changed, 3 insertions(+), 31 deletions(-) > >>> delete mode 100644 meta/classes/crate-fetch.bbclass > >>> > >> > >> Would a better solution here be to move the crate fetcher into > >> lib/bb/fetch2/ in bitbake? > > > > I've been wondering about this too. > > > > Does anyone remember how the fetcher lines up against the general fetcher > > requirements we have and is there a list of anything that remained to be done > > before it could be added to bitbake? > > If I remember correctly, it just needed some test cases writing. The > crate fetcher is a subclass of the wget fetcher with changes to how > urldata is set up and how unpack is handled, so it should integrate well > into bitbake. One thing I see that's a bit worrisome is that it disables recommends_checksum for the downloaded files, and populates metadata for use (I assume) by cargo with the calculated SHA256 on the download. -Matt > > Thanks, > > -- > Paul Barker > Principal Software Engineer > SanCloud Ltd > > e: paul.barker@sancloud.com > w: https://sancloud.co.uk/
On Tue, Dec 21, 2021 at 10:15 AM Matt Madison via lists.openembedded.org <matt=madison.systems@lists.openembedded.org> wrote: > > On Tue, Dec 21, 2021 at 6:07 AM Paul Barker <paul.barker@sancloud.com> wrote: > > > > On 20/12/2021 22:34, Richard Purdie wrote: > > > On Mon, 2021-12-20 at 17:01 +0000, Paul Barker wrote: > > >> On 17/12/2021 15:36, Matt Madison wrote: > > >>> This patch series switches to using the OE_IMPORTS mechanism > > >>> for installing the crate fetcher. This simplifies the addition > > >>> and clears up issues with interactions between SRCPV and > > >>> the shared-state fetching logic. > > >>> > > >>> Matt Madison (4): > > >>> lib/crate.py: install crate fetcher on import > > >>> base.bbclass: add crate to OE_IMPORTS > > >>> cargo_common.bbclass: remove inherit of crate-fetch > > >>> classes: remove crate-fetch.bbclass > > >>> > > >>> meta/classes/base.bbclass | 2 +- > > >>> meta/classes/cargo_common.bbclass | 2 -- > > >>> meta/classes/crate-fetch.bbclass | 28 ---------------------------- > > >>> meta/lib/crate.py | 2 ++ > > >>> 4 files changed, 3 insertions(+), 31 deletions(-) > > >>> delete mode 100644 meta/classes/crate-fetch.bbclass > > >>> > > >> > > >> Would a better solution here be to move the crate fetcher into > > >> lib/bb/fetch2/ in bitbake? > > > > > > I've been wondering about this too. > > > > > > Does anyone remember how the fetcher lines up against the general fetcher > > > requirements we have and is there a list of anything that remained to be done > > > before it could be added to bitbake? > > > > If I remember correctly, it just needed some test cases writing. The > > crate fetcher is a subclass of the wget fetcher with changes to how > > urldata is set up and how unpack is handled, so it should integrate well > > into bitbake. > > One thing I see that's a bit worrisome is that it disables > recommends_checksum for the downloaded files, > and populates metadata for use (I assume) by cargo with the calculated > SHA256 on the download. Any other thoughts on this? I've modified my distro to work around the problem that led me to propose these patches, but something needs to be fixed here. crate-fetch.bbclass should not be setting SRCPV. Thanks, -Matt > > -Matt > > > > > Thanks, > > > > -- > > Paul Barker > > Principal Software Engineer > > SanCloud Ltd > > > > e: paul.barker@sancloud.com > > w: https://sancloud.co.uk/ > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#159928): https://lists.openembedded.org/g/openembedded-core/message/159928 > Mute This Topic: https://lists.openembedded.org/mt/87791267/3618418 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [matt@madison.systems] > -=-=-=-=-=-=-=-=-=-=-=- >
On Fri, 2022-01-14 at 02:31 -0800, Matt Madison wrote: > On Tue, Dec 21, 2021 at 10:15 AM Matt Madison via > lists.openembedded.org <matt=madison.systems@lists.openembedded.org> > wrote: > > > > On Tue, Dec 21, 2021 at 6:07 AM Paul Barker <paul.barker@sancloud.com> wrote: > > > > > > On 20/12/2021 22:34, Richard Purdie wrote: > > > > On Mon, 2021-12-20 at 17:01 +0000, Paul Barker wrote: > > > > > On 17/12/2021 15:36, Matt Madison wrote: > > > > > > This patch series switches to using the OE_IMPORTS mechanism > > > > > > for installing the crate fetcher. This simplifies the addition > > > > > > and clears up issues with interactions between SRCPV and > > > > > > the shared-state fetching logic. > > > > > > > > > > > > Matt Madison (4): > > > > > > lib/crate.py: install crate fetcher on import > > > > > > base.bbclass: add crate to OE_IMPORTS > > > > > > cargo_common.bbclass: remove inherit of crate-fetch > > > > > > classes: remove crate-fetch.bbclass > > > > > > > > > > > > meta/classes/base.bbclass | 2 +- > > > > > > meta/classes/cargo_common.bbclass | 2 -- > > > > > > meta/classes/crate-fetch.bbclass | 28 ---------------------------- > > > > > > meta/lib/crate.py | 2 ++ > > > > > > 4 files changed, 3 insertions(+), 31 deletions(-) > > > > > > delete mode 100644 meta/classes/crate-fetch.bbclass > > > > > > > > > > > > > > > > Would a better solution here be to move the crate fetcher into > > > > > lib/bb/fetch2/ in bitbake? > > > > > > > > I've been wondering about this too. > > > > > > > > Does anyone remember how the fetcher lines up against the general fetcher > > > > requirements we have and is there a list of anything that remained to be done > > > > before it could be added to bitbake? > > > > > > If I remember correctly, it just needed some test cases writing. The > > > crate fetcher is a subclass of the wget fetcher with changes to how > > > urldata is set up and how unpack is handled, so it should integrate well > > > into bitbake. > > > > One thing I see that's a bit worrisome is that it disables > > recommends_checksum for the downloaded files, > > and populates metadata for use (I assume) by cargo with the calculated > > SHA256 on the download. > > Any other thoughts on this? I've modified my distro to work around > the problem that led me to propose these patches, but something needs > to be fixed here. crate-fetch.bbclass should not be setting SRCPV. My thoughts are that we should look to getting this merged into bitbake. To do that we need some tests and then we can probably make it happen. I am worried about your comments on the checksum and that may be something we need to fix before it can be merged. Cheers, Richard