[01/13] cargo-update-recipe-crates.bbclass: add a class to generate SRC_URI crate lists from Cargo.lock

For better or worse, more and more rust components are appearing that do
not include their dependencies in tarballs (or git trees), and rely on cargo
to fetch them. On the other hand, bitbake does not use cargo (and quite possible
won't ever be able to), and relies on having each item explicitly listed in SRC_URI
with a crate:// prefix. This however creates a problem of both making such lists in
the first place and updating them when a recipe is updated to a newer version.

So this class can be used to perform such updates by implementing a task that does it;
the next commit shows the outcome for python3-bcrypt (which has been tested to work
and produce a successful build).

Note: the python script relies on tomllib library, which appears in Python 3.11 and
does not exist in earlier versions - I've tested this by first updating python to 3.11-rc2
in oe-core.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../cargo-update-recipe-crates.bbclass        | 41 +++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 meta/classes-recipe/cargo-update-recipe-crates.bbclass

On 2022-10-31 07:47, Alexander Kanavin wrote:
> For better or worse, more and more rust components are appearing that do
> not include their dependencies in tarballs (or git trees), and rely on cargo
> to fetch them. On the other hand, bitbake does not use cargo (and quite possible
> won't ever be able to), and relies on having each item explicitly listed in SRC_URI
> with a crate:// prefix. This however creates a problem of both making such lists in
> the first place and updating them when a recipe is updated to a newer version.
>
> So this class can be used to perform such updates by implementing a task that does it;
> the next commit shows the outcome for python3-bcrypt (which has been tested to work
> and produce a successful build).
>
> Note: the python script relies on tomllib library, which appears in Python 3.11 and
> does not exist in earlier versions - I've tested this by first updating python to 3.11-rc2
> in oe-core.


Thanks Alex, that's a nice approach.

I'm reply to the thread since Jin had some trouble getting started
with this class. This pushed me to finally try it and I have some minor
comments and suggestions.

First, it works for me for adding ripgrep (1) and for Jin for something 
he was
working on. It wasn't a perfect experience for me since at first, I
had omitted:

S = "${WORKDIR}/git"

and
$ bitbake -c update_crates  ripgrep

would fail silently with nothing useful in the log.do_update_crates file.

I'll likely send a patch to make it more verbose on error.


It also wasn't clear how to get started but in hindsight, you need 
'obviously'
need a recipe that has SRC_URI and SRCREV and S and maybe a bit more to 
get started.


Michael,

Do you think we should a document this workflow in addition to the cargo 
bitbake approach?

../Randy


1) https://github.com/BurntSushi/ripgrep

Recipe will go to meta-oe.

>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>   .../cargo-update-recipe-crates.bbclass        | 41 +++++++++++++++++++
>   1 file changed, 41 insertions(+)
>   create mode 100644 meta/classes-recipe/cargo-update-recipe-crates.bbclass
>
> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> new file mode 100644
> index 0000000000..f90938c734
> --- /dev/null
> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> @@ -0,0 +1,41 @@
> +#
> +# Copyright OpenEmbedded Contributors
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +
> +##
> +## Purpose:
> +## This class is used to update the list of crates in SRC_URI
> +## by reading Cargo.lock in the source tree.
> +##
> +## See meta/recipes-devtools/python/python3-bcrypt_*.bb for an example
> +##
> +## To perform the update: bitbake -c update_crates recipe-name
> +
> +addtask do_update_crates after do_patch
> +do_update_crates[depends] = "python3-native:do_populate_sysroot"
> +
> +do_update_crates() {
> +    nativepython3 - <<EOF
> +
> +def get_crates(f):
> +    import tomllib
> +    c_list = 'SRC_URI += " \\ \n'
> +    crates = tomllib.load(open(f, 'rb'))
> +    for c in crates['package']:
> +        if 'source' in c and 'crates.io' in c['source']:
> +            c_list += "        crate://crates.io/{}/{} \\ \n".format(c['name'], c['version'])
> +    c_list += '"\n'
> +    return c_list
> +
> +import os
> +crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
> +for root, dirs, files in os.walk('${S}'):
> +    for file in files:
> +        if file == 'Cargo.lock':
> +            crates += get_crates(os.path.join(root, file))
> +open(os.path.join('${THISDIR}', '${PN}'+"-crates.inc"), 'w').write(crates)
> +
> +EOF
> +}
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#172303): https://lists.openembedded.org/g/openembedded-core/message/172303
> Mute This Topic: https://lists.openembedded.org/mt/94683148/3616765
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Hi everyone,

I promised Randy to post my experience with Yocto and Rust, which was a
first time attempt for me. Note, that when starting I had no knowledge about 
Rust whatsoever, so this is "clean start" observation.

On Tue, Dec 06, 2022 at 06:04:18PM -0500, Randy MacLeod wrote:
> On 2022-10-31 07:47, Alexander Kanavin wrote:
> >For better or worse, more and more rust components are appearing that do
> >not include their dependencies in tarballs (or git trees), and rely on cargo
> >to fetch them. On the other hand, bitbake does not use cargo (and quite possible
> >won't ever be able to), and relies on having each item explicitly listed in SRC_URI
> >with a crate:// prefix. This however creates a problem of both making such lists in
> >the first place and updating them when a recipe is updated to a newer version.
> >
> >So this class can be used to perform such updates by implementing a task that does it;
> >the next commit shows the outcome for python3-bcrypt (which has been tested to work
> >and produce a successful build).
> >
> >Note: the python script relies on tomllib library, which appears in Python 3.11 and
> >does not exist in earlier versions - I've tested this by first updating python to 3.11-rc2
> >in oe-core.
> 
> 
> Thanks Alex, that's a nice approach.
> 
> I'm reply to the thread since Jin had some trouble getting started
> with this class. This pushed me to finally try it and I have some minor
> comments and suggestions.

my first issue was to actually find out how to approach this, it seems
most google hits point to meta-rust which has been around for a longer period
and I was lucky to get some hints in #oe on IRC, where Randy pointed me to 
cargo-update-recipe-crates.bbclass which is already in Poky.

The comment in the header about running bitbake -c update_crates recipe-name
allowed me to generate the .inc file, but it was not immediately clear to me
what I should do next.

The pointer to python3-bcrypt_*.bb was a bit confusing in the sense, that
from the rust perspective it only inherits cargo-update-recipe-crates,
but trying to build my package after having genrated the SRC_URI entries did
not produce any build output, I ended up with an empty -dev and an empty -dbg 
ipk.

By looking at the available classes I figured that I should probably also
inherit cargo, which indeed started to compile the application I needed to
package.

Next, I ran into some build issues for packages which I guess the application
was depending upon. I got an error:

error: failed to run custom build command for `libudev-sys v0.1.4`

which luckily had enough google hits to understand that it expects
a DEPENDS = "eudev"

It however still failed after that and it took me some time to realize
that I should also inherit pkgconfig, which finally did the trick.

I copied the working recipe back to Kirkstone and was happy to see, that it
builds there as well.

So overall, given that I had zero Rust knowledge, it wasn't a bad
experience, Yocto took care of most things for me.

> First, it works for me for adding ripgrep (1) and for Jin for
> something he was
> working on. It wasn't a perfect experience for me since at first, I
> had omitted:
> 
> S = "${WORKDIR}/git"
> 
> and
> $ bitbake -c update_crates  ripgrep
> 
> would fail silently with nothing useful in the log.do_update_crates file.
> 
> I'll likely send a patch to make it more verbose on error.
> 
> 
> It also wasn't clear how to get started but in hindsight, you need
> 'obviously'
> need a recipe that has SRC_URI and SRCREV and S and maybe a bit more
> to get started.
> 
> 
> Michael,
> 
> Do you think we should a document this workflow in addition to the
> cargo bitbake approach?

I think this should be documented more prominently, as missing information
on how to get going was the biggest obstacle, at least for me. The benefit of
this workflow is, that no additional layer is needed and that everything seems
to more or less work out of the box within the usual checkout.

The downside is a two step approach and I wonder if it would be possible to
append the SRC_URI crate entries on the fly, but in a way that still works
with sstate so that regeneration happens only when something has changed.

Anyway, it seems to be a good start, thanks to everyone who made this possible!

Kind regards,
Jin

 
 
> 1) https://github.com/BurntSushi/ripgrep
> 
> Recipe will go to meta-oe.
> 
> >
> >Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> >---
> >  .../cargo-update-recipe-crates.bbclass        | 41 +++++++++++++++++++
> >  1 file changed, 41 insertions(+)
> >  create mode 100644 meta/classes-recipe/cargo-update-recipe-crates.bbclass
> >
> >diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> >new file mode 100644
> >index 0000000000..f90938c734
> >--- /dev/null
> >+++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> >@@ -0,0 +1,41 @@
> >+#
> >+# Copyright OpenEmbedded Contributors
> >+#
> >+# SPDX-License-Identifier: MIT
> >+#
> >+
> >+##
> >+## Purpose:
> >+## This class is used to update the list of crates in SRC_URI
> >+## by reading Cargo.lock in the source tree.
> >+##
> >+## See meta/recipes-devtools/python/python3-bcrypt_*.bb for an example
> >+##
> >+## To perform the update: bitbake -c update_crates recipe-name
> >+
> >+addtask do_update_crates after do_patch
> >+do_update_crates[depends] = "python3-native:do_populate_sysroot"
> >+
> >+do_update_crates() {
> >+    nativepython3 - <<EOF
> >+
> >+def get_crates(f):
> >+    import tomllib
> >+    c_list = 'SRC_URI += " \\ \n'
> >+    crates = tomllib.load(open(f, 'rb'))
> >+    for c in crates['package']:
> >+        if 'source' in c and 'crates.io' in c['source']:
> >+            c_list += "        crate://crates.io/{}/{} \\ \n".format(c['name'], c['version'])
> >+    c_list += '"\n'
> >+    return c_list
> >+
> >+import os
> >+crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
> >+for root, dirs, files in os.walk('${S}'):
> >+    for file in files:
> >+        if file == 'Cargo.lock':
> >+            crates += get_crates(os.path.join(root, file))
> >+open(os.path.join('${THISDIR}', '${PN}'+"-crates.inc"), 'w').write(crates)
> >+
> >+EOF
> >+}
> >
> >-=-=-=-=-=-=-=-=-=-=-=-
> >Links: You receive all messages sent to this group.
> >View/Reply Online (#172303): https://lists.openembedded.org/g/openembedded-core/message/172303
> >Mute This Topic: https://lists.openembedded.org/mt/94683148/3616765
> >Group Owner: openembedded-core+owner@lists.openembedded.org
> >Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
> >-=-=-=-=-=-=-=-=-=-=-=-
> >
> 
> -- 
> # Randy MacLeod
> # Wind River Linux

On Wed, 7 Dec 2022 at 01:21, Sergey Bostandzhyan <jin@mediatomb.cc> wrote:
> I think this should be documented more prominently, as missing information
> on how to get going was the biggest obstacle, at least for me. The benefit of
> this workflow is, that no additional layer is needed and that everything seems
> to more or less work out of the box within the usual checkout.

Thanks for the report, I totally agree that the rust recipe workflow
should be documented somewhere in the official manuals, I'm just not
sure where. Perhaps the 'common tasks' in
https://docs.yoctoproject.org/dev-manual/index.html ?

Can you write and propose a patch to the documentation repository?

> The downside is a two step approach and I wonder if it would be possible to
> append the SRC_URI crate entries on the fly, but in a way that still works
> with sstate so that regeneration happens only when something has changed.

You can't modify anything in the layers during bitbake tasks, only if
the task is completely standalone and isolated. You also can't modify
SRC_URI internally after fetching as SRC_URI needs to be fully formed
before the fetch task runs. What we can do is provide better hints in
case cargo fails because the list of crates mismatches the lockdown
file in the source, and needs to be updated (or created). Can this be
reliably detected?

Alex

Hi Alex,

Am 07.12.2022 um 09:53 schrieb Alexander Kanavin via lists.openembedded.org:
> On Wed, 7 Dec 2022 at 01:21, Sergey Bostandzhyan <jin@mediatomb.cc> wrote:
>> I think this should be documented more prominently, as missing information
>> on how to get going was the biggest obstacle, at least for me. The benefit of
>> this workflow is, that no additional layer is needed and that everything seems
>> to more or less work out of the box within the usual checkout.
> Thanks for the report, I totally agree that the rust recipe workflow
> should be documented somewhere in the official manuals, I'm just not
> sure where. Perhaps the 'common tasks' in
> https://docs.yoctoproject.org/dev-manual/index.html ?
>
> Can you write and propose a patch to the documentation repository?

Is the recipetool obsolete or why we have yust an other tool to create a 
recipe in oe-core?

Regards
   Stefan

On Wed 7. Dec 2022 at 16.28, Stefan Herbrechtsmeier <
stefan.herbrechtsmeier-oss@weidmueller.com> wrote:

> Hi Alex,
>
> Am 07.12.2022 um 09:53 schrieb Alexander Kanavin via
> lists.openembedded.org:
> > On Wed, 7 Dec 2022 at 01:21, Sergey Bostandzhyan <jin@mediatomb.cc>
> wrote:
> >> I think this should be documented more prominently, as missing
> information
> >> on how to get going was the biggest obstacle, at least for me. The
> benefit of
> >> this workflow is, that no additional layer is needed and that
> everything seems
> >> to more or less work out of the box within the usual checkout.
> > Thanks for the report, I totally agree that the rust recipe workflow
> > should be documented somewhere in the official manuals, I'm just not
> > sure where. Perhaps the 'common tasks' in
> > https://docs.yoctoproject.org/dev-manual/index.html ?
> >
> > Can you write and propose a patch to the documentation repository?
>
> Is the recipetool obsolete or why we have yust an other tool to create a
> recipe in oe-core?


Which another tool are you referring to? Cargo bitbake is a 3rd party
project, one that I am not recommending to anyone in any way. And having a
class to update or create a list of crates does not preclude its use in
recipetool. Patches welcome.

A bit of politeness would be welcome too, Stefan, seriously. Watch your
tone.

Alex

Hi Alex,

Am 07.12.2022 um 17:21 schrieb Alexander Kanavin:
> On Wed 7. Dec 2022 at 16.28, Stefan Herbrechtsmeier 
> <stefan.herbrechtsmeier-oss@weidmueller.com> wrote:
>
>     Hi Alex,
>
>     Am 07.12.2022 um 09:53 schrieb Alexander Kanavin via
>     lists.openembedded.org <http://lists.openembedded.org>:
>     > On Wed, 7 Dec 2022 at 01:21, Sergey Bostandzhyan
>     <jin@mediatomb.cc> wrote:
>     >> I think this should be documented more prominently, as missing
>     information
>     >> on how to get going was the biggest obstacle, at least for me.
>     The benefit of
>     >> this workflow is, that no additional layer is needed and that
>     everything seems
>     >> to more or less work out of the box within the usual checkout.
>     > Thanks for the report, I totally agree that the rust recipe workflow
>     > should be documented somewhere in the official manuals, I'm just not
>     > sure where. Perhaps the 'common tasks' in
>     > https://docs.yoctoproject.org/dev-manual/index.html ?
>     >
>     > Can you write and propose a patch to the documentation repository?
>
>     Is the recipetool obsolete or why we have yust an other tool to
>     create a
>     recipe in oe-core?
>
>
> Which another tool are you referring to? Cargo bitbake is a 3rd party 
> project, one that I am not recommending to anyone in any way. And 
> having a class to update or create a list of crates does not preclude 
> its use in recipetool. Patches welcome.
>
> A bit of politeness would be welcome too, Stefan, seriously. Watch 
> your tone.

Sorry for the harsh tone but you ignore existing tools and add just 
another tool to oe-core without mention any reason or document it. Do 
you really expect that somebody will patch existing tools if main 
developers ignore existing tools and provide new tool for their use 
case? Why should somebody improve an existing tool, extend the 
documentation, add tests or even upstream its work if these same 
requirements don’t exist for the main developers. It looks like the 
requirements for foreign and main contributors are different and this 
doesn't encourage people to participant. Maybe this is only my personal 
feeling and I apologize my harsh tone, but the acceptance of patches 
should be comprehensible, and expectations should be the same for 
everyone. Should others answer to your comments that their solution 
doesn’t preclude your suggestion and that they welcome patches? For sure 
it takes more time to add rust support to recipetool but I think a 
second tool without a clear reason in oe-core hurts more in long term 
because its now unclear if new features (like checksums or licenses 
support) should be added to this tool or if this tool is only a 
temporary solution and should be replaced by recipetool in long term. 
Furthermore, this class is marked as a class for a recipe but shouldn’t 
be inherit by a recipe and manipulates a file inside the meta data.

Regards
   Stefan

On Fri, 2022-12-09 at 11:39 +0100, Stefan Herbrechtsmeier wrote:
> Sorry for the harsh tone but you ignore existing tools and add just
> another tool to oe-core without mention any reason or document it. Do
> you really expect that somebody will patch existing tools if main
> developers ignore existing tools and provide new tool for their use
> case?

I'm not sure that is entirely fair criticism. We have some challenges
with some of the new languages. Rust support was in a separate layer.
We took some of those recipes into core, then ended up making
significant changes to them. We never took the external tool.

The hope in doing that was that we'd find a better way to make things
work. I think what Alex has done is an improvement over that external
tool and it experiments with a different way of handling things. I've
had generally quite positive feedback on the approach itself. Yes,
there are some issues with documentation and some people using it have
struggled with some usability issues. None of those look like they're
unfixable.

>  Why should somebody improve an existing tool, extend the
> documentation, add tests or even upstream its work if these same
> requirements don’t exist for the main developers. It looks like the
> requirements for foreign and main contributors are different and this
> doesn't encourage people to participant.

I don't see us treating developers differently and I am concerned you
think we/I do. Any given solution that is proposed is evaluated on it's
pros and cons. In this case the solution is quite self contained and
allowed the approach to be experimented with whilst solving a real
world issue. If it doesn't work out we can easily drop it. The risk
from taking it is therefore low. Yes, I should probably insist on
documentation. In this case it is relatively simple code which is
relatively easily understood so I've been less worried about that up
front.

If it worked out well, we can integrate it further and document it. If
it doesn't it can be removed. FWIW I have heard people saying they like
the approach and that we should use it for some of the other languages
with challenges like this.

>  Maybe this is only my personal feeling and I apologize my harsh
> tone, but the acceptance of patches should be comprehensible, and
> expectations should be the same for everyone.

I do try to ensure that. My "algorithm" for accepting patches is
probably not easily documented but I think the factors here which are 
the standalone nature of the change and the easy with which we could
drop it if needed. As such it is in my "low risk" category of patches.

I'd note Alex has another patch which has been sitting for months
unmerged as it is in my "high risk" to the project category. I suspect
that one will not actually merge but I need to find the time to explain
why, right now it is more based on a feeling it is the wrong direction.

>  Should others answer to your comments that their solution doesn’t
> preclude your suggestion and that they welcome patches? For sure it
> takes more time to add rust support to recipetool but I think a
> second tool without a clear reason in oe-core hurts more in long term
> because its now unclear if new features (like checksums or licenses
> support) should be added to this tool or if this tool is only a
> temporary solution and should be replaced by recipetool in long term.
> Furthermore, this class is marked as a class for a recipe but
> shouldn’t be inherit by a recipe and manipulates a file inside the
> meta data.

We do have precedence for classes that help updates like this. Both
python and perl have code that adds tasks that function a bit like
this, in those cases for the core recipe.

Personally, I would ultimately like to see these operations handled by
recipetool and I suspect natural evolution of the code may head that
way. Both devtool and recipetool have used classes as a way to help
them perform operations so in that sense, this is actually a logical
development path for those tools.

Cheers,

Richard

On Fri, 9 Dec 2022 at 11:39, Stefan Herbrechtsmeier
<stefan.herbrechtsmeier-oss@weidmueller.com> wrote:
> Sorry for the harsh tone but you ignore existing tools and add just another tool to oe-core without mention any reason or document it.

The reason to add the class was to be able to update existing
rust-based recipes without having to manually update long lists of
crates in those recipes, which is a rather urgent issue, more urgent
than having rust support in recipetool. The class does not replace
recipetool, in fact it can, and should be used by devtool/recipetool
once someone finds time and motivation to sit down and add (currently
missing) rust support to it (class functionality can be executed from
those tools via tinfoil.build_targets()). I do not have that time, and
I am not getting paid for doing that either. Oh, and we do not have a
maintainer for those tools. Would you like to volunteer for that?

Documentation is handled in a separate repository, and is typically
decoupled from actual code. I'll get to it, but you should know that
some things are actually higher priority. Such as keeping core
updated, so that you can have a secure stack in the products you sell,
and doing paid customer work, so that I can pay my bills. I'd
appreciate if you reflect on the fact that none of my upstream work is
recognized, or supported by your company, even though without it you
would not have a product to sell - which in turn pays your bills.

Alex

On Fri, 9 Dec 2022 at 13:01, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> I'd note Alex has another patch which has been sitting for months
> unmerged as it is in my "high risk" to the project category. I suspect
> that one will not actually merge but I need to find the time to explain
> why, right now it is more based on a feeling it is the wrong direction.

I'd be happy to be 'course corrected' on oe-setup-build, I personally
feel it does more 'guessing' than I'm comfortable with but I can't
figure out a better way to handle things. The problems it's aiming to
address are real, so it shouldn't be formally rejected without a plan.

Alex

On Fri, 2022-12-09 at 15:43 +0100, Alexander Kanavin wrote:
> On Fri, 9 Dec 2022 at 13:01, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > I'd note Alex has another patch which has been sitting for months
> > unmerged as it is in my "high risk" to the project category. I suspect
> > that one will not actually merge but I need to find the time to explain
> > why, right now it is more based on a feeling it is the wrong direction.
> 
> I'd be happy to be 'course corrected' on oe-setup-build, I personally
> feel it does more 'guessing' than I'm comfortable with but I can't
> figure out a better way to handle things. The problems it's aiming to
> address are real, so it shouldn't be formally rejected without a plan.

Right, I do need to explain why I'm reluctant about it and what the
concerns are. Sadly doing that isn't straightforward, particularly if
the bar is set to the height of "show how a better approach would work"
since I then have to think through a new design before I can reply. The
challenge with that patch is that it uses a very predominant namespace
and can't easily change once we take that direction so that makes
merging much harder.

My point here was that patches from everyone are treated fairly, some
merge quickly, some don't, it really does depend.

There are also patches from some guy called Richard on the bitbake
mailing list which Richard is refusing to merge due to performance
concerns :)

Cheers,

Richard

Hi Richard,

Am 09.12.2022 um 13:01 schrieb Richard Purdie:
> On Fri, 2022-12-09 at 11:39 +0100, Stefan Herbrechtsmeier wrote:
>> Sorry for the harsh tone but you ignore existing tools and add just
>> another tool to oe-core without mention any reason or document it. Do
>> you really expect that somebody will patch existing tools if main
>> developers ignore existing tools and provide new tool for their use
>> case?
> I'm not sure that is entirely fair criticism. We have some challenges
> with some of the new languages. Rust support was in a separate layer.
> We took some of those recipes into core, then ended up making
> significant changes to them. We never took the external tool.
>
> The hope in doing that was that we'd find a better way to make things
> work. I think what Alex has done is an improvement over that external
> tool and it experiments with a different way of handling things. I've
> had generally quite positive feedback on the approach itself. Yes,
> there are some issues with documentation and some people using it have
> struggled with some usability issues. None of those look like they're
> unfixable.
>
>>   Why should somebody improve an existing tool, extend the
>> documentation, add tests or even upstream its work if these same
>> requirements don’t exist for the main developers. It looks like the
>> requirements for foreign and main contributors are different and this
>> doesn't encourage people to participant.
> I don't see us treating developers differently and I am concerned you
> think we/I do. Any given solution that is proposed is evaluated on it's
> pros and cons. In this case the solution is quite self contained and
> allowed the approach to be experimented with whilst solving a real
> world issue. If it doesn't work out we can easily drop it. The risk
> from taking it is therefore low. Yes, I should probably insist on
> documentation. In this case it is relatively simple code which is
> relatively easily understood so I've been less worried about that up
> front.
>
> If it worked out well, we can integrate it further and document it. If
> it doesn't it can be removed. FWIW I have heard people saying they like
> the approach and that we should use it for some of the other languages
> with challenges like this.
>
>>   Maybe this is only my personal feeling and I apologize my harsh
>> tone, but the acceptance of patches should be comprehensible, and
>> expectations should be the same for everyone.
> I do try to ensure that. My "algorithm" for accepting patches is
> probably not easily documented but I think the factors here which are
> the standalone nature of the change and the easy with which we could
> drop it if needed. As such it is in my "low risk" category of patches.
>
> I'd note Alex has another patch which has been sitting for months
> unmerged as it is in my "high risk" to the project category. I suspect
> that one will not actually merge but I need to find the time to explain
> why, right now it is more based on a feeling it is the wrong direction.
>
>>   Should others answer to your comments that their solution doesn’t
>> preclude your suggestion and that they welcome patches? For sure it
>> takes more time to add rust support to recipetool but I think a
>> second tool without a clear reason in oe-core hurts more in long term
>> because its now unclear if new features (like checksums or licenses
>> support) should be added to this tool or if this tool is only a
>> temporary solution and should be replaced by recipetool in long term.
>> Furthermore, this class is marked as a class for a recipe but
>> shouldn’t be inherit by a recipe and manipulates a file inside the
>> meta data.
> We do have precedence for classes that help updates like this. Both
> python and perl have code that adds tasks that function a bit like
> this, in those cases for the core recipe.
>
> Personally, I would ultimately like to see these operations handled by
> recipetool and I suspect natural evolution of the code may head that
> way. Both devtool and recipetool have used classes as a way to help
> them perform operations so in that sense, this is actually a logical
> development path for those tools.

Thanks for you feedback and the clarification. This helps me a lot.

Stefan

Hi Alex,

Am 09.12.2022 um 13:09 schrieb Alexander Kanavin:
> On Fri, 9 Dec 2022 at 11:39, Stefan Herbrechtsmeier
> <stefan.herbrechtsmeier-oss@weidmueller.com> wrote:
>> Sorry for the harsh tone but you ignore existing tools and add just another tool to oe-core without mention any reason or document it.
> The reason to add the class was to be able to update existing
> rust-based recipes without having to manually update long lists of
> crates in those recipes, which is a rather urgent issue, more urgent
> than having rust support in recipetool. The class does not replace
> recipetool, in fact it can, and should be used by devtool/recipetool
> once someone finds time and motivation to sit down and add (currently
> missing) rust support to it (class functionality can be executed from
> those tools via tinfoil.build_targets()). I do not have that time, and
> I am not getting paid for doing that either. Oh, and we do not have a
> maintainer for those tools. Would you like to volunteer for that?

Yes, I'm happy to maintain recipetool. We need to go on with our npm und 
go support anyway.

Regards
   Stefan

On Mon, 12 Dec 2022 at 15:18, Stefan Herbrechtsmeier
<stefan.herbrechtsmeier-oss@weidmueller.com> wrote:
> >> Sorry for the harsh tone but you ignore existing tools and add just another tool to oe-core without mention any reason or document it.
> > The reason to add the class was to be able to update existing
> > rust-based recipes without having to manually update long lists of
> > crates in those recipes, which is a rather urgent issue, more urgent
> > than having rust support in recipetool. The class does not replace
> > recipetool, in fact it can, and should be used by devtool/recipetool
> > once someone finds time and motivation to sit down and add (currently
> > missing) rust support to it (class functionality can be executed from
> > those tools via tinfoil.build_targets()). I do not have that time, and
> > I am not getting paid for doing that either. Oh, and we do not have a
> > maintainer for those tools. Would you like to volunteer for that?
>
> Yes, I'm happy to maintain recipetool. We need to go on with our npm und
> go support anyway.

Thank you, that would be very much appreciated.

We're all stretched thin, and need to continuously advocate the
importance of OSS maintenance as a job, as it's still not seen as
such. The problem is endemic across the software industry.

Alex