diff mbox series

[V5,6/6] cargo-update-recipe-crates: don't walk on the whole dir

Message ID 6279ccad3163dca527872a9144d63fb8f07c6f0e.1680190966.git.frederic.martinsons@gmail.com
State New
Headers show
Series [V5,1/6] cargo_common.bbclass: Support local github repos | expand

Commit Message

Frédéric Martinsons March 30, 2023, 3:44 p.m. UTC 
From: Frederic Martinsons <frederic.martinsons@gmail.com>

There is no need to do such things, Cargo.lock file
has to be at the root of CARGO_LOCK_SRC_DIR.
This avoid finding other possible Cargo.lock that
would be in subdir (for example if a patch is applied
on the recipe, we can have .pc subdir in S and a Cargo.lock
can be there)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
 .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

Comments

Alex Kiernan March 30, 2023, 4:34 p.m. UTC | #1 
On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>
> From: Frederic Martinsons <frederic.martinsons@gmail.com>
>
> There is no need to do such things, Cargo.lock file
> has to be at the root of CARGO_LOCK_SRC_DIR.
> This avoid finding other possible Cargo.lock that
> would be in subdir (for example if a patch is applied
> on the recipe, we can have .pc subdir in S and a Cargo.lock
> can be there)
>
> Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
> ---
>  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> index daa363b0dd..549cfe627e 100644
> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> @@ -68,10 +68,14 @@ def get_crates(f):
>  import os
>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>  found = False
> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
> -    for file in files:
> -        if file == 'Cargo.lock':
> -            crates += get_crates(os.path.join(root, file))
> +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
> +    if file == 'Cargo.lock':
> +        try:
> +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}', file)
> +            crates += get_crates(cargo_lock_path)
> +        except Exception as e:
> +            raise ValueError("Cannot parse '%s'" % cargo_lock_path) from e
> +        else:
>              found = True
>  if not found:
>      raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")

Isn't this just a long-winded version of something like this
(completely untested):

diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
index daa363b0dd65..22ddcfa5c1e3 100644
--- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
+++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
@@ -67,14 +67,7 @@ def get_crates(f):

 import os
 crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
-found = False
-for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
-    for file in files:
-        if file == 'Cargo.lock':
-            crates += get_crates(os.path.join(root, file))
-            found = True
-if not found:
-    raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")
+crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}", "Cargo.lock"))
 open("${TARGET_FILE}", 'w').write(crates)
 EOF
Frédéric Martinsons March 30, 2023, 5:09 p.m. UTC | #2 
Pretty much indeed, except for the exception catching part (which allows a
better error message)
Do I need to submit another whole series (PATCH V6) or can I submit just
this change ?

On Thu, 30 Mar 2023 at 18:34, Alex Kiernan <alex.kiernan@gmail.com> wrote:

> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
> >
> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
> >
> > There is no need to do such things, Cargo.lock file
> > has to be at the root of CARGO_LOCK_SRC_DIR.
> > This avoid finding other possible Cargo.lock that
> > would be in subdir (for example if a patch is applied
> > on the recipe, we can have .pc subdir in S and a Cargo.lock
> > can be there)
> >
> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
> > ---
> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
> >  1 file changed, 8 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> > index daa363b0dd..549cfe627e 100644
> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> > @@ -68,10 +68,14 @@ def get_crates(f):
> >  import os
> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
> >  found = False
> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
> > -    for file in files:
> > -        if file == 'Cargo.lock':
> > -            crates += get_crates(os.path.join(root, file))
> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
> > +    if file == 'Cargo.lock':
> > +        try:
> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
> file)
> > +            crates += get_crates(cargo_lock_path)
> > +        except Exception as e:
> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
> from e
> > +        else:
> >              found = True
> >  if not found:
> >      raise ValueError("Unable to find Cargo.lock in
> ${CARGO_LOCK_SRC_DIR}")
>
> Isn't this just a long-winded version of something like this
> (completely untested):
>
> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> index daa363b0dd65..22ddcfa5c1e3 100644
> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> @@ -67,14 +67,7 @@ def get_crates(f):
>
>  import os
>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
> -found = False
> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
> -    for file in files:
> -        if file == 'Cargo.lock':
> -            crates += get_crates(os.path.join(root, file))
> -            found = True
> -if not found:
> -    raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")
> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}", "Cargo.lock"))
>  open("${TARGET_FILE}", 'w').write(crates)
>  EOF
>
> --
> Alex Kiernan
>
Frédéric Martinsons March 30, 2023, 6:53 p.m. UTC | #3 
Nevermind, I'll submit a new v6 series tomorrow.

Your comment is totally accurate and follow the PEP20 which I should follow
too ("readability counts" and "explicit is better than implicit" at most )

Le jeu. 30 mars 2023, 19:10, Frederic Martinsons via lists.openembedded.org
<frederic.martinsons=gmail.com@lists.openembedded.org> a écrit :

> Pretty much indeed, except for the exception catching part (which allows a
> better error message)
> Do I need to submit another whole series (PATCH V6) or can I submit just
> this change ?
>
> On Thu, 30 Mar 2023 at 18:34, Alex Kiernan <alex.kiernan@gmail.com> wrote:
>
>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>> >
>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>> >
>> > There is no need to do such things, Cargo.lock file
>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>> > This avoid finding other possible Cargo.lock that
>> > would be in subdir (for example if a patch is applied
>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>> > can be there)
>> >
>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>> > ---
>> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > index daa363b0dd..549cfe627e 100644
>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > @@ -68,10 +68,14 @@ def get_crates(f):
>> >  import os
>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>> >  found = False
>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>> > -    for file in files:
>> > -        if file == 'Cargo.lock':
>> > -            crates += get_crates(os.path.join(root, file))
>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>> > +    if file == 'Cargo.lock':
>> > +        try:
>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>> file)
>> > +            crates += get_crates(cargo_lock_path)
>> > +        except Exception as e:
>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>> from e
>> > +        else:
>> >              found = True
>> >  if not found:
>> >      raise ValueError("Unable to find Cargo.lock in
>> ${CARGO_LOCK_SRC_DIR}")
>>
>> Isn't this just a long-winded version of something like this
>> (completely untested):
>>
>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> index daa363b0dd65..22ddcfa5c1e3 100644
>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> @@ -67,14 +67,7 @@ def get_crates(f):
>>
>>  import os
>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>> -found = False
>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>> -    for file in files:
>> -        if file == 'Cargo.lock':
>> -            crates += get_crates(os.path.join(root, file))
>> -            found = True
>> -if not found:
>> -    raise ValueError("Unable to find Cargo.lock in
>> ${CARGO_LOCK_SRC_DIR}")
>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}", "Cargo.lock"))
>>  open("${TARGET_FILE}", 'w').write(crates)
>>  EOF
>>
>> --
>> Alex Kiernan
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#179346):
> https://lists.openembedded.org/g/openembedded-core/message/179346
> Mute This Topic: https://lists.openembedded.org/mt/97953740/6213388
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> frederic.martinsons@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Martin Jansa March 30, 2023, 7:23 p.m. UTC | #4 
I don't remember the exact details now, but when I was working on updating
solana recipes to use this
https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4

the S was pointing to just some subdirectory and multiple Cargo.locks files
were parsed in other directories as well, that's why I was adding
https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58

in the end I've changed to use the root directory in S and just
set CARGO_SRC_DIR to right subdirectory to build.

I don't have much experience with crates other than these solana recipes,
but isn't there some valid use-case for multiple Cargo.locks?
I assume Alex in original implementation didn't use the os.walk just to
make it more complicated :).

And FWIW when trying to regenerate these .inc files with current master and
with:
$ bitbake -c update_crates solana-keygen
it fails with:
ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
'/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
add at least one to the recipe:
SRC_URI[Inflector.sha256sum] =
"fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
NoChecksumError('Missing SRC_URI checksum', '
https://crates.io/api/v1/crates/Inflector/0.11.4/download')

Isn't it an chicken&egg issue now when do_fetch enforces checksums to be
used? Or am I just missing some of the pending changes or just using it
incorrectly?


On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com> wrote:

> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
> >
> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
> >
> > There is no need to do such things, Cargo.lock file
> > has to be at the root of CARGO_LOCK_SRC_DIR.
> > This avoid finding other possible Cargo.lock that
> > would be in subdir (for example if a patch is applied
> > on the recipe, we can have .pc subdir in S and a Cargo.lock
> > can be there)
> >
> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
> > ---
> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
> >  1 file changed, 8 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> > index daa363b0dd..549cfe627e 100644
> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> > @@ -68,10 +68,14 @@ def get_crates(f):
> >  import os
> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
> >  found = False
> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
> > -    for file in files:
> > -        if file == 'Cargo.lock':
> > -            crates += get_crates(os.path.join(root, file))
> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
> > +    if file == 'Cargo.lock':
> > +        try:
> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
> file)
> > +            crates += get_crates(cargo_lock_path)
> > +        except Exception as e:
> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
> from e
> > +        else:
> >              found = True
> >  if not found:
> >      raise ValueError("Unable to find Cargo.lock in
> ${CARGO_LOCK_SRC_DIR}")
>
> Isn't this just a long-winded version of something like this
> (completely untested):
>
> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> index daa363b0dd65..22ddcfa5c1e3 100644
> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
> @@ -67,14 +67,7 @@ def get_crates(f):
>
>  import os
>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
> -found = False
> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
> -    for file in files:
> -        if file == 'Cargo.lock':
> -            crates += get_crates(os.path.join(root, file))
> -            found = True
> -if not found:
> -    raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")
> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}", "Cargo.lock"))
>  open("${TARGET_FILE}", 'w').write(crates)
>  EOF
>
> --
> Alex Kiernan
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#179344):
> https://lists.openembedded.org/g/openembedded-core/message/179344
> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Frédéric Martinsons March 30, 2023, 7:50 p.m. UTC | #5 
Well I see what you mean, I'll take a look at your example to try to find
out if multiple Cargo.lock could be expected.

And for your second remark, yes, there is a chicken and eggs issue for
updating crates checksum from scratch.
You didn't miss anything, I came across this when updating crates checksum
that was "pristine"
 and I only manage to execute update_crates by locally patching
bitbake/lib/bb/fetch2/crate.py for
recommends_checksum method to return False .
I'm aware that is not ideal but I don't know how to make this better (maybe
make do_update_crates
after do_fetch instead of after do_patch ? )


Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a écrit :

> I don't remember the exact details now, but when I was working on updating
> solana recipes to use this
>
> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>
> the S was pointing to just some subdirectory and multiple Cargo.locks
> files were parsed in other directories as well, that's why I was adding
>
> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>
> in the end I've changed to use the root directory in S and just
> set CARGO_SRC_DIR to right subdirectory to build.
>
> I don't have much experience with crates other than these solana recipes,
> but isn't there some valid use-case for multiple Cargo.locks?
> I assume Alex in original implementation didn't use the os.walk just to
> make it more complicated :).
>
> And FWIW when trying to regenerate these .inc files with current master
> and with:
> $ bitbake -c update_crates solana-keygen
> it fails with:
> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
> add at least one to the recipe:
> SRC_URI[Inflector.sha256sum] =
> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
> NoChecksumError('Missing SRC_URI checksum', '
> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>
> Isn't it an chicken&egg issue now when do_fetch enforces checksums to be
> used? Or am I just missing some of the pending changes or just using it
> incorrectly?
>
>
> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
> wrote:
>
>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>> >
>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>> >
>> > There is no need to do such things, Cargo.lock file
>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>> > This avoid finding other possible Cargo.lock that
>> > would be in subdir (for example if a patch is applied
>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>> > can be there)
>> >
>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>> > ---
>> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > index daa363b0dd..549cfe627e 100644
>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > @@ -68,10 +68,14 @@ def get_crates(f):
>> >  import os
>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>> >  found = False
>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>> > -    for file in files:
>> > -        if file == 'Cargo.lock':
>> > -            crates += get_crates(os.path.join(root, file))
>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>> > +    if file == 'Cargo.lock':
>> > +        try:
>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>> file)
>> > +            crates += get_crates(cargo_lock_path)
>> > +        except Exception as e:
>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>> from e
>> > +        else:
>> >              found = True
>> >  if not found:
>> >      raise ValueError("Unable to find Cargo.lock in
>> ${CARGO_LOCK_SRC_DIR}")
>>
>> Isn't this just a long-winded version of something like this
>> (completely untested):
>>
>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> index daa363b0dd65..22ddcfa5c1e3 100644
>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> @@ -67,14 +67,7 @@ def get_crates(f):
>>
>>  import os
>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>> -found = False
>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>> -    for file in files:
>> -        if file == 'Cargo.lock':
>> -            crates += get_crates(os.path.join(root, file))
>> -            found = True
>> -if not found:
>> -    raise ValueError("Unable to find Cargo.lock in
>> ${CARGO_LOCK_SRC_DIR}")
>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}", "Cargo.lock"))
>>  open("${TARGET_FILE}", 'w').write(crates)
>>  EOF
>>
>> --
>> Alex Kiernan
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#179344):
>> https://lists.openembedded.org/g/openembedded-core/message/179344
>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>> Martin.Jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>
Frédéric Martinsons March 30, 2023, 8:07 p.m. UTC | #6 
By the way, about this "chickens and eggs" problem, isn't it the same for a
regular recipe you just upgraded?

Personnaly, when I update a recipe, I let bitbake tell me what is the new
checksum expected and put it the recipe (as the error message says).

But I'm aware this is not exactly the same, since a cargo based recipe
could contain a ton of crate:// uri and if you apply this method, you have
to copy the new checksum one by one, bitbake error after another...

Don't know how to make this better and I plead guilty for not mentioning
that in a dedicated commit message.

Of course, if someone come up with a smoothier way of doing this, I'll make
a new patch.

Le jeu. 30 mars 2023, 21:50, Frédéric Martinsons <
frederic.martinsons@gmail.com> a écrit :

> Well I see what you mean, I'll take a look at your example to try to find
> out if multiple Cargo.lock could be expected.
>
> And for your second remark, yes, there is a chicken and eggs issue for
> updating crates checksum from scratch.
> You didn't miss anything, I came across this when updating crates checksum
> that was "pristine"
>  and I only manage to execute update_crates by locally patching
> bitbake/lib/bb/fetch2/crate.py for
> recommends_checksum method to return False .
> I'm aware that is not ideal but I don't know how to make this better
> (maybe make do_update_crates
> after do_fetch instead of after do_patch ? )
>
>
> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
> écrit :
>
>> I don't remember the exact details now, but when I was working on
>> updating solana recipes to use this
>>
>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>
>> the S was pointing to just some subdirectory and multiple Cargo.locks
>> files were parsed in other directories as well, that's why I was adding
>>
>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>
>> in the end I've changed to use the root directory in S and just
>> set CARGO_SRC_DIR to right subdirectory to build.
>>
>> I don't have much experience with crates other than these solana recipes,
>> but isn't there some valid use-case for multiple Cargo.locks?
>> I assume Alex in original implementation didn't use the os.walk just to
>> make it more complicated :).
>>
>> And FWIW when trying to regenerate these .inc files with current master
>> and with:
>> $ bitbake -c update_crates solana-keygen
>> it fails with:
>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>> add at least one to the recipe:
>> SRC_URI[Inflector.sha256sum] =
>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>> NoChecksumError('Missing SRC_URI checksum', '
>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>
>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to be
>> used? Or am I just missing some of the pending changes or just using it
>> incorrectly?
>>
>>
>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>> wrote:
>>
>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>>> >
>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>> >
>>> > There is no need to do such things, Cargo.lock file
>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>> > This avoid finding other possible Cargo.lock that
>>> > would be in subdir (for example if a patch is applied
>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>> > can be there)
>>> >
>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>> > ---
>>> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>> >
>>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > index daa363b0dd..549cfe627e 100644
>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>> >  import os
>>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>> >  found = False
>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>> > -    for file in files:
>>> > -        if file == 'Cargo.lock':
>>> > -            crates += get_crates(os.path.join(root, file))
>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>> > +    if file == 'Cargo.lock':
>>> > +        try:
>>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>>> file)
>>> > +            crates += get_crates(cargo_lock_path)
>>> > +        except Exception as e:
>>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>>> from e
>>> > +        else:
>>> >              found = True
>>> >  if not found:
>>> >      raise ValueError("Unable to find Cargo.lock in
>>> ${CARGO_LOCK_SRC_DIR}")
>>>
>>> Isn't this just a long-winded version of something like this
>>> (completely untested):
>>>
>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>
>>>  import os
>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>> -found = False
>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>> -    for file in files:
>>> -        if file == 'Cargo.lock':
>>> -            crates += get_crates(os.path.join(root, file))
>>> -            found = True
>>> -if not found:
>>> -    raise ValueError("Unable to find Cargo.lock in
>>> ${CARGO_LOCK_SRC_DIR}")
>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>> "Cargo.lock"))
>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>  EOF
>>>
>>> --
>>> Alex Kiernan
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#179344):
>>> https://lists.openembedded.org/g/openembedded-core/message/179344
>>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>>> Martin.Jansa@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>>
Martin Jansa March 30, 2023, 8:08 p.m. UTC | #7 
echo > recipes-upstreamable/sugar/sugar-crates.inc

before running "bitbake -c update_crates"

seems like easiest solution to let it regenerate the .inc file (or remove
"require" and then add it back after refresh).

But there is still something strange in this solana, after refreshing the
.inc files in sugar as well as solana-*, it's failing with:

NOTE: recipe sugar-1.1.0-r0: task do_unpack: Started
ERROR: sugar-1.1.0-r0 do_unpack: Error executing a python function in
exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:base_do_unpack(d)
     0003:
File:
'/OE/lge/build/webos/mickledore/oe-core/meta/classes-global/base.bbclass',
lineno: 160, function: base_do_unpack
     0156:        return
     0157:
     0158:    try:
     0159:        fetcher = bb.fetch2.Fetch(src_uri, d)
 *** 0160:        fetcher.unpack(d.getVar('WORKDIR'))
     0161:    except bb.fetch2.BBFetchException as e:
     0162:        bb.fatal("Bitbake Fetcher Error: " + repr(e))
     0163:}
     0164:
File: '/OE/lge/build/webos/mickledore/bitbake/lib/bb/fetch2/__init__.py',
lineno: 1882, function: unpack
     1878:
     1879:            if ud.lockfile:
     1880:                lf = bb.utils.lockfile(ud.lockfile)
     1881:
 *** 1882:            ud.method.unpack(ud, root, self.d)
     1883:
     1884:            if ud.lockfile:
     1885:                bb.utils.unlockfile(lf)
     1886:
File: '/OE/lge/build/webos/mickledore/bitbake/lib/bb/fetch2/crate.py',
lineno: 84, function: unpack
     0080:        """
     0081:        Uses the crate to build the necessary paths for cargo to
utilize it
     0082:        """
     0083:        if ud.type == 'crate':
 *** 0084:            return self._crate_unpack(ud, rootdir, d)
     0085:        else:
     0086:            super(Crate, self).unpack(ud, rootdir, d)
     0087:
     0088:    def _crate_unpack(self, ud, rootdir, d):
File: '/OE/lge/build/webos/mickledore/bitbake/lib/bb/fetch2/crate.py',
lineno: 113, function: _crate_unpack
     0109:            # ensure we've got these paths made
     0110:            bb.utils.mkdirhier(cargo_bitbake)
     0111:
     0112:            # generate metadata necessary
 *** 0113:            with open(thefile, 'rb') as f:
     0114:                # get the SHA256 of the original tarball
     0115:                tarhash = hashlib.sha256(f.read()).hexdigest()
     0116:
     0117:            metadata['files'] = {}
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/OE/lge/build/webos/mickledore/downloads/arrayvec-0.7.2.crate'

ERROR: Logfile of failure stored in:
/OE/lge/build/webos/mickledore/BUILD/work/qemux86_64-webos-linux/sugar/1.1.0-r0/temp/log.do_unpack.2600442
NOTE: recipe sugar-1.1.0-r0: task do_unpack: Failed

...

NOTE: recipe solana-cli-1.14.5-r0: task do_fetch: Started
WARNING: solana-cli-1.14.5-r0 do_fetch: Checksum failure encountered with
download of crate://crates.io/arrayvec/0.7.2 - will attempt other sources
if available
ERROR: solana-cli-1.14.5-r0 do_fetch: Fetcher failure for URL: '
https://crates.io/api/v1/crates/arrayvec/0.7.2/download'. Checksum mismatch!
File: '/OE/lge/build/webos/mickledore/downloads/arrayvec-0.7.2.crate.tmp'
has sha256 checksum
'8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6' when
'be4dc07131ffa69b8072d35f5007352af944213cde02545e2103680baed38fcd' was
expected
If this change is expected (e.g. you have upgraded to a new version without
updating the checksums) then you can use these lines within the recipe:
SRC_URI[arrayvec.sha256sum] =
"8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
Otherwise you should retry the download and/or check with upstream to
determine if the file has become corrupted or otherwise unexpectedly
modified.

This is just FYI in case you can see what's going on right away, I'll
continue to debug it from my end as well.

On Thu, Mar 30, 2023 at 9:50 PM Frédéric Martinsons <
frederic.martinsons@gmail.com> wrote:

> Well I see what you mean, I'll take a look at your example to try to find
> out if multiple Cargo.lock could be expected.
>
> And for your second remark, yes, there is a chicken and eggs issue for
> updating crates checksum from scratch.
> You didn't miss anything, I came across this when updating crates checksum
> that was "pristine"
>  and I only manage to execute update_crates by locally patching
> bitbake/lib/bb/fetch2/crate.py for
> recommends_checksum method to return False .
> I'm aware that is not ideal but I don't know how to make this better
> (maybe make do_update_crates
> after do_fetch instead of after do_patch ? )
>
>
> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
> écrit :
>
>> I don't remember the exact details now, but when I was working on
>> updating solana recipes to use this
>>
>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>
>> the S was pointing to just some subdirectory and multiple Cargo.locks
>> files were parsed in other directories as well, that's why I was adding
>>
>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>
>> in the end I've changed to use the root directory in S and just
>> set CARGO_SRC_DIR to right subdirectory to build.
>>
>> I don't have much experience with crates other than these solana recipes,
>> but isn't there some valid use-case for multiple Cargo.locks?
>> I assume Alex in original implementation didn't use the os.walk just to
>> make it more complicated :).
>>
>> And FWIW when trying to regenerate these .inc files with current master
>> and with:
>> $ bitbake -c update_crates solana-keygen
>> it fails with:
>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>> add at least one to the recipe:
>> SRC_URI[Inflector.sha256sum] =
>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>> NoChecksumError('Missing SRC_URI checksum', '
>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>
>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to be
>> used? Or am I just missing some of the pending changes or just using it
>> incorrectly?
>>
>>
>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>> wrote:
>>
>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>>> >
>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>> >
>>> > There is no need to do such things, Cargo.lock file
>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>> > This avoid finding other possible Cargo.lock that
>>> > would be in subdir (for example if a patch is applied
>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>> > can be there)
>>> >
>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>> > ---
>>> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>> >
>>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > index daa363b0dd..549cfe627e 100644
>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>> >  import os
>>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>> >  found = False
>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>> > -    for file in files:
>>> > -        if file == 'Cargo.lock':
>>> > -            crates += get_crates(os.path.join(root, file))
>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>> > +    if file == 'Cargo.lock':
>>> > +        try:
>>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>>> file)
>>> > +            crates += get_crates(cargo_lock_path)
>>> > +        except Exception as e:
>>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>>> from e
>>> > +        else:
>>> >              found = True
>>> >  if not found:
>>> >      raise ValueError("Unable to find Cargo.lock in
>>> ${CARGO_LOCK_SRC_DIR}")
>>>
>>> Isn't this just a long-winded version of something like this
>>> (completely untested):
>>>
>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>
>>>  import os
>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>> -found = False
>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>> -    for file in files:
>>> -        if file == 'Cargo.lock':
>>> -            crates += get_crates(os.path.join(root, file))
>>> -            found = True
>>> -if not found:
>>> -    raise ValueError("Unable to find Cargo.lock in
>>> ${CARGO_LOCK_SRC_DIR}")
>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>> "Cargo.lock"))
>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>  EOF
>>>
>>> --
>>> Alex Kiernan
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#179344):
>>> https://lists.openembedded.org/g/openembedded-core/message/179344
>>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>>> Martin.Jansa@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>>
Alex Kiernan March 30, 2023, 8:20 p.m. UTC | #8 
Oh my word... I wonder if we need to introspect what cargo does (a la
cargo bitbake) to get this kind of thing right?

On Thu, Mar 30, 2023 at 8:23 PM Martin Jansa <martin.jansa@gmail.com> wrote:
>
> I don't remember the exact details now, but when I was working on updating solana recipes to use this
> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>
> the S was pointing to just some subdirectory and multiple Cargo.locks files were parsed in other directories as well, that's why I was adding
> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>
> in the end I've changed to use the root directory in S and just set CARGO_SRC_DIR to right subdirectory to build.
>
> I don't have much experience with crates other than these solana recipes, but isn't there some valid use-case for multiple Cargo.locks?
> I assume Alex in original implementation didn't use the os.walk just to make it more complicated :).
>
> And FWIW when trying to regenerate these .inc files with current master and with:
> $ bitbake -c update_crates solana-keygen
> it fails with:
> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please add at least one to the recipe:
> SRC_URI[Inflector.sha256sum] = "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error: NoChecksumError('Missing SRC_URI checksum', 'https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>
> Isn't it an chicken&egg issue now when do_fetch enforces checksums to be used? Or am I just missing some of the pending changes or just using it incorrectly?
>
>
> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com> wrote:
>>
>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>> >
>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>> >
>> > There is no need to do such things, Cargo.lock file
>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>> > This avoid finding other possible Cargo.lock that
>> > would be in subdir (for example if a patch is applied
>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>> > can be there)
>> >
>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>> > ---
>> >  .../cargo-update-recipe-crates.bbclass               | 12 ++++++++----
>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > index daa363b0dd..549cfe627e 100644
>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> > @@ -68,10 +68,14 @@ def get_crates(f):
>> >  import os
>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>> >  found = False
>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>> > -    for file in files:
>> > -        if file == 'Cargo.lock':
>> > -            crates += get_crates(os.path.join(root, file))
>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>> > +    if file == 'Cargo.lock':
>> > +        try:
>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}', file)
>> > +            crates += get_crates(cargo_lock_path)
>> > +        except Exception as e:
>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path) from e
>> > +        else:
>> >              found = True
>> >  if not found:
>> >      raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")
>>
>> Isn't this just a long-winded version of something like this
>> (completely untested):
>>
>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> index daa363b0dd65..22ddcfa5c1e3 100644
>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>> @@ -67,14 +67,7 @@ def get_crates(f):
>>
>>  import os
>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>> -found = False
>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>> -    for file in files:
>> -        if file == 'Cargo.lock':
>> -            crates += get_crates(os.path.join(root, file))
>> -            found = True
>> -if not found:
>> -    raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")
>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}", "Cargo.lock"))
>>  open("${TARGET_FILE}", 'w').write(crates)
>>  EOF
>>
>> --
>> Alex Kiernan
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#179344): https://lists.openembedded.org/g/openembedded-core/message/179344
>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Martin.Jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
Martin Jansa March 30, 2023, 8:22 p.m. UTC | #9 
On Thu, Mar 30, 2023 at 10:07 PM Frédéric Martinsons <
frederic.martinsons@gmail.com> wrote:

> By the way, about this "chickens and eggs" problem, isn't it the same for
> a regular recipe you just upgraded?
>

I don't know, I haven't upgraded a recipe with crates yet :)

It's just this time I was aware that the .inc file will need to be
refreshed and it wasn't obvious at first that easiest way to avoid do_fetch
issue blocking .inc refresh is to remove all crates first by emptying .inc
file.

Personnaly, when I update a recipe, I let bitbake tell me what is the new
> checksum expected and put it the recipe (as the error message says).
>

For regular recipes with a few checksum I do the same. But it doesn't scale
here as you mention bellow.
as it reported only the first wrong checksum from MANY.

But I'm aware this is not exactly the same, since a cargo based recipe
> could contain a ton of crate:// uri and if you apply this method, you have
> to copy the new checksum one by one, bitbake error after another...
>
> Don't know how to make this better and I plead guilty for not mentioning
> that in a dedicated commit message.
>

Better late than never, you can include it in v6 of this patch (I have read
the git log of the .bbclass before going to refresh the .inc files, so
others might notice it there as well).

Of course, if someone come up with a smoothier way of doing this, I'll make
> a new patch.
>

I think I found what the other issue with arrayvec create was:

meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
crates.io/arrayvec/0.7.2 \
meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
= "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
crates.io/arrayvec/0.7.1 \
meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
= "be4dc07131ffa69b8072d35f5007352af944213cde02545e2103680baed38fcd"
meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:    crate://
crates.io/arrayvec/0.7.2 \
meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:SRC_URI[arrayvec.sha256sum]
= "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"

So different Cargo.lock files included in the same ${S} have different
version, but in this case it didn't set different name parameter, probably
because different Cargo.lock files are processed separately?

Thanks

Le jeu. 30 mars 2023, 21:50, Frédéric Martinsons <
> frederic.martinsons@gmail.com> a écrit :
>
>> Well I see what you mean, I'll take a look at your example to try to find
>> out if multiple Cargo.lock could be expected.
>>
>> And for your second remark, yes, there is a chicken and eggs issue for
>> updating crates checksum from scratch.
>> You didn't miss anything, I came across this when updating crates
>> checksum that was "pristine"
>>  and I only manage to execute update_crates by locally patching
>> bitbake/lib/bb/fetch2/crate.py for
>> recommends_checksum method to return False .
>> I'm aware that is not ideal but I don't know how to make this better
>> (maybe make do_update_crates
>> after do_fetch instead of after do_patch ? )
>>
>>
>> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
>> écrit :
>>
>>> I don't remember the exact details now, but when I was working on
>>> updating solana recipes to use this
>>>
>>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>>
>>> the S was pointing to just some subdirectory and multiple Cargo.locks
>>> files were parsed in other directories as well, that's why I was adding
>>>
>>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>>
>>> in the end I've changed to use the root directory in S and just
>>> set CARGO_SRC_DIR to right subdirectory to build.
>>>
>>> I don't have much experience with crates other than these solana
>>> recipes, but isn't there some valid use-case for multiple Cargo.locks?
>>> I assume Alex in original implementation didn't use the os.walk just to
>>> make it more complicated :).
>>>
>>> And FWIW when trying to regenerate these .inc files with current master
>>> and with:
>>> $ bitbake -c update_crates solana-keygen
>>> it fails with:
>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>>> add at least one to the recipe:
>>> SRC_URI[Inflector.sha256sum] =
>>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>>> NoChecksumError('Missing SRC_URI checksum', '
>>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>>
>>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to be
>>> used? Or am I just missing some of the pending changes or just using it
>>> incorrectly?
>>>
>>>
>>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>>> wrote:
>>>
>>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>>>> >
>>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>> >
>>>> > There is no need to do such things, Cargo.lock file
>>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>>> > This avoid finding other possible Cargo.lock that
>>>> > would be in subdir (for example if a patch is applied
>>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>>> > can be there)
>>>> >
>>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>> > ---
>>>> >  .../cargo-update-recipe-crates.bbclass               | 12
>>>> ++++++++----
>>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>>> >
>>>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> > index daa363b0dd..549cfe627e 100644
>>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>>> >  import os
>>>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>>> >  found = False
>>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>> > -    for file in files:
>>>> > -        if file == 'Cargo.lock':
>>>> > -            crates += get_crates(os.path.join(root, file))
>>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>>> > +    if file == 'Cargo.lock':
>>>> > +        try:
>>>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>>>> file)
>>>> > +            crates += get_crates(cargo_lock_path)
>>>> > +        except Exception as e:
>>>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>>>> from e
>>>> > +        else:
>>>> >              found = True
>>>> >  if not found:
>>>> >      raise ValueError("Unable to find Cargo.lock in
>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>
>>>> Isn't this just a long-winded version of something like this
>>>> (completely untested):
>>>>
>>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>>
>>>>  import os
>>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>>> -found = False
>>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>> -    for file in files:
>>>> -        if file == 'Cargo.lock':
>>>> -            crates += get_crates(os.path.join(root, file))
>>>> -            found = True
>>>> -if not found:
>>>> -    raise ValueError("Unable to find Cargo.lock in
>>>> ${CARGO_LOCK_SRC_DIR}")
>>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>>> "Cargo.lock"))
>>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>>  EOF
>>>>
>>>> --
>>>> Alex Kiernan
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#179344):
>>>> https://lists.openembedded.org/g/openembedded-core/message/179344
>>>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>>>> Martin.Jansa@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>
>>>>
Martin Jansa March 30, 2023, 9:15 p.m. UTC | #10 
I've added the updated .inc files to meta-webosose fork if it's easier to
see there:

https://github.com/shr-project/meta-webosose/commits/master-2023-03-30 (top
2 commits)

this checksum mismatch is also the reason for that ugly exception as
explained in top commit, it's race-condition between sugar.do_unpack and
solana-cli.do_fetch where sugar has right arrayvec 0.7.2 with right
checksum in do_unpack, while solana-cli renames it to _bad-checksum_
suffix, because solana-cli looks for checksum of arrayvec 0.7.1.

I've tried to manually add name= param to verify this, but there are too
many :), so I'll update the bbclass to always add name with version even if
it doesn't see duplicated crates (and then this can be improved if multiple
Cargo.lock files really need to be supported).

I honestly don't know, I've was only working on these solana recipes for
short time and wanted to offer them more as strange example.

Cheers,

On Thu, Mar 30, 2023 at 10:22 PM Martin Jansa via lists.openembedded.org
<Martin.Jansa=gmail.com@lists.openembedded.org> wrote:

> On Thu, Mar 30, 2023 at 10:07 PM Frédéric Martinsons <
> frederic.martinsons@gmail.com> wrote:
>
>> By the way, about this "chickens and eggs" problem, isn't it the same for
>> a regular recipe you just upgraded?
>>
>
> I don't know, I haven't upgraded a recipe with crates yet :)
>
> It's just this time I was aware that the .inc file will need to be
> refreshed and it wasn't obvious at first that easiest way to avoid do_fetch
> issue blocking .inc refresh is to remove all crates first by emptying .inc
> file.
>
> Personnaly, when I update a recipe, I let bitbake tell me what is the new
>> checksum expected and put it the recipe (as the error message says).
>>
>
> For regular recipes with a few checksum I do the same. But it doesn't
> scale here as you mention bellow.
> as it reported only the first wrong checksum from MANY.
>
> But I'm aware this is not exactly the same, since a cargo based recipe
>> could contain a ton of crate:// uri and if you apply this method, you have
>> to copy the new checksum one by one, bitbake error after another...
>>
>> Don't know how to make this better and I plead guilty for not mentioning
>> that in a dedicated commit message.
>>
>
> Better late than never, you can include it in v6 of this patch (I have
> read the git log of the .bbclass before going to refresh the .inc files, so
> others might notice it there as well).
>
> Of course, if someone come up with a smoothier way of doing this, I'll
>> make a new patch.
>>
>
> I think I found what the other issue with arrayvec create was:
>
> meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
> crates.io/arrayvec/0.7.2 \
> meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
> = "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
> meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
> crates.io/arrayvec/0.7.1 \
> meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
> = "be4dc07131ffa69b8072d35f5007352af944213cde02545e2103680baed38fcd"
> meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:    crate://
> crates.io/arrayvec/0.7.2 \
> meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:SRC_URI[arrayvec.sha256sum]
> = "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
>
> So different Cargo.lock files included in the same ${S} have different
> version, but in this case it didn't set different name parameter, probably
> because different Cargo.lock files are processed separately?
>
> Thanks
>
> Le jeu. 30 mars 2023, 21:50, Frédéric Martinsons <
>> frederic.martinsons@gmail.com> a écrit :
>>
>>> Well I see what you mean, I'll take a look at your example to try to
>>> find out if multiple Cargo.lock could be expected.
>>>
>>> And for your second remark, yes, there is a chicken and eggs issue for
>>> updating crates checksum from scratch.
>>> You didn't miss anything, I came across this when updating crates
>>> checksum that was "pristine"
>>>  and I only manage to execute update_crates by locally patching
>>> bitbake/lib/bb/fetch2/crate.py for
>>> recommends_checksum method to return False .
>>> I'm aware that is not ideal but I don't know how to make this better
>>> (maybe make do_update_crates
>>> after do_fetch instead of after do_patch ? )
>>>
>>>
>>> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
>>> écrit :
>>>
>>>> I don't remember the exact details now, but when I was working on
>>>> updating solana recipes to use this
>>>>
>>>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>>>
>>>> the S was pointing to just some subdirectory and multiple Cargo.locks
>>>> files were parsed in other directories as well, that's why I was adding
>>>>
>>>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>>>
>>>> in the end I've changed to use the root directory in S and just
>>>> set CARGO_SRC_DIR to right subdirectory to build.
>>>>
>>>> I don't have much experience with crates other than these solana
>>>> recipes, but isn't there some valid use-case for multiple Cargo.locks?
>>>> I assume Alex in original implementation didn't use the os.walk just to
>>>> make it more complicated :).
>>>>
>>>> And FWIW when trying to regenerate these .inc files with current master
>>>> and with:
>>>> $ bitbake -c update_crates solana-keygen
>>>> it fails with:
>>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>>>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>>>> add at least one to the recipe:
>>>> SRC_URI[Inflector.sha256sum] =
>>>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>>>> NoChecksumError('Missing SRC_URI checksum', '
>>>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>>>
>>>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to
>>>> be used? Or am I just missing some of the pending changes or just using it
>>>> incorrectly?
>>>>
>>>>
>>>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>>>> wrote:
>>>>
>>>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com> wrote:
>>>>> >
>>>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>>> >
>>>>> > There is no need to do such things, Cargo.lock file
>>>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>>>> > This avoid finding other possible Cargo.lock that
>>>>> > would be in subdir (for example if a patch is applied
>>>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>>>> > can be there)
>>>>> >
>>>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>>> > ---
>>>>> >  .../cargo-update-recipe-crates.bbclass               | 12
>>>>> ++++++++----
>>>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>>>> >
>>>>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> > index daa363b0dd..549cfe627e 100644
>>>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>>>> >  import os
>>>>> >  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>>>> >  found = False
>>>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>>> > -    for file in files:
>>>>> > -        if file == 'Cargo.lock':
>>>>> > -            crates += get_crates(os.path.join(root, file))
>>>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>>>> > +    if file == 'Cargo.lock':
>>>>> > +        try:
>>>>> > +            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}',
>>>>> file)
>>>>> > +            crates += get_crates(cargo_lock_path)
>>>>> > +        except Exception as e:
>>>>> > +            raise ValueError("Cannot parse '%s'" % cargo_lock_path)
>>>>> from e
>>>>> > +        else:
>>>>> >              found = True
>>>>> >  if not found:
>>>>> >      raise ValueError("Unable to find Cargo.lock in
>>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>>
>>>>> Isn't this just a long-winded version of something like this
>>>>> (completely untested):
>>>>>
>>>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>>>
>>>>>  import os
>>>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>>>> -found = False
>>>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>>> -    for file in files:
>>>>> -        if file == 'Cargo.lock':
>>>>> -            crates += get_crates(os.path.join(root, file))
>>>>> -            found = True
>>>>> -if not found:
>>>>> -    raise ValueError("Unable to find Cargo.lock in
>>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>>>> "Cargo.lock"))
>>>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>>>  EOF
>>>>>
>>>>> --
>>>>> Alex Kiernan
>>>>>
>>>>>
>>>>>
>>>>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#179358):
> https://lists.openembedded.org/g/openembedded-core/message/179358
> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Frédéric Martinsons March 31, 2023, 5:32 a.m. UTC | #11 
Hello,

The multiple Cargo.lock seems totally valid, a project can have multiple
binaries provided (which is the case for Solana cli suite) so I'll submit a
new v6 of my series to reflect that, I just have to ignored specify dirs
like .git (no use to walk inside it) or .pc (can contains a patched
Cargo.lock that would be redundant and syntaxically invalid )

I'll also override the crate names with their version for all the crates
found.

For the chicken and eggs issue, I need to think more of it (maybe make
bitbake print the list of all missing checksum  would be enough) and, in
order to not delay this series, I'll submit a separate bitbake patch if I
manage to find something suitable.

Many thanks for the comments Martin.

Le jeu. 30 mars 2023, 23:15, Martin Jansa <martin.jansa@gmail.com> a écrit :

> I've added the updated .inc files to meta-webosose fork if it's easier to
> see there:
>
> https://github.com/shr-project/meta-webosose/commits/master-2023-03-30
> (top 2 commits)
>
> this checksum mismatch is also the reason for that ugly exception as
> explained in top commit, it's race-condition between sugar.do_unpack and
> solana-cli.do_fetch where sugar has right arrayvec 0.7.2 with right
> checksum in do_unpack, while solana-cli renames it to _bad-checksum_
> suffix, because solana-cli looks for checksum of arrayvec 0.7.1.
>
> I've tried to manually add name= param to verify this, but there are too
> many :), so I'll update the bbclass to always add name with version even if
> it doesn't see duplicated crates (and then this can be improved if multiple
> Cargo.lock files really need to be supported).
>
> I honestly don't know, I've was only working on these solana recipes for
> short time and wanted to offer them more as strange example.
>
> Cheers,
>
> On Thu, Mar 30, 2023 at 10:22 PM Martin Jansa via lists.openembedded.org
> <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
>
>> On Thu, Mar 30, 2023 at 10:07 PM Frédéric Martinsons <
>> frederic.martinsons@gmail.com> wrote:
>>
>>> By the way, about this "chickens and eggs" problem, isn't it the same
>>> for a regular recipe you just upgraded?
>>>
>>
>> I don't know, I haven't upgraded a recipe with crates yet :)
>>
>> It's just this time I was aware that the .inc file will need to be
>> refreshed and it wasn't obvious at first that easiest way to avoid do_fetch
>> issue blocking .inc refresh is to remove all crates first by emptying .inc
>> file.
>>
>> Personnaly, when I update a recipe, I let bitbake tell me what is the new
>>> checksum expected and put it the recipe (as the error message says).
>>>
>>
>> For regular recipes with a few checksum I do the same. But it doesn't
>> scale here as you mention bellow.
>> as it reported only the first wrong checksum from MANY.
>>
>> But I'm aware this is not exactly the same, since a cargo based recipe
>>> could contain a ton of crate:// uri and if you apply this method, you have
>>> to copy the new checksum one by one, bitbake error after another...
>>>
>>> Don't know how to make this better and I plead guilty for not mentioning
>>> that in a dedicated commit message.
>>>
>>
>> Better late than never, you can include it in v6 of this patch (I have
>> read the git log of the .bbclass before going to refresh the .inc files, so
>> others might notice it there as well).
>>
>> Of course, if someone come up with a smoothier way of doing this, I'll
>>> make a new patch.
>>>
>>
>> I think I found what the other issue with arrayvec create was:
>>
>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
>> crates.io/arrayvec/0.7.2 \
>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
>> = "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
>> crates.io/arrayvec/0.7.1 \
>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
>> = "be4dc07131ffa69b8072d35f5007352af944213cde02545e2103680baed38fcd"
>> meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:    crate://
>> crates.io/arrayvec/0.7.2 \
>> meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:SRC_URI[arrayvec.sha256sum]
>> = "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
>>
>> So different Cargo.lock files included in the same ${S} have different
>> version, but in this case it didn't set different name parameter, probably
>> because different Cargo.lock files are processed separately?
>>
>> Thanks
>>
>> Le jeu. 30 mars 2023, 21:50, Frédéric Martinsons <
>>> frederic.martinsons@gmail.com> a écrit :
>>>
>>>> Well I see what you mean, I'll take a look at your example to try to
>>>> find out if multiple Cargo.lock could be expected.
>>>>
>>>> And for your second remark, yes, there is a chicken and eggs issue for
>>>> updating crates checksum from scratch.
>>>> You didn't miss anything, I came across this when updating crates
>>>> checksum that was "pristine"
>>>>  and I only manage to execute update_crates by locally patching
>>>> bitbake/lib/bb/fetch2/crate.py for
>>>> recommends_checksum method to return False .
>>>> I'm aware that is not ideal but I don't know how to make this better
>>>> (maybe make do_update_crates
>>>> after do_fetch instead of after do_patch ? )
>>>>
>>>>
>>>> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
>>>> écrit :
>>>>
>>>>> I don't remember the exact details now, but when I was working on
>>>>> updating solana recipes to use this
>>>>>
>>>>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>>>>
>>>>> the S was pointing to just some subdirectory and multiple Cargo.locks
>>>>> files were parsed in other directories as well, that's why I was adding
>>>>>
>>>>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>>>>
>>>>> in the end I've changed to use the root directory in S and just
>>>>> set CARGO_SRC_DIR to right subdirectory to build.
>>>>>
>>>>> I don't have much experience with crates other than these solana
>>>>> recipes, but isn't there some valid use-case for multiple Cargo.locks?
>>>>> I assume Alex in original implementation didn't use the os.walk just
>>>>> to make it more complicated :).
>>>>>
>>>>> And FWIW when trying to regenerate these .inc files with current
>>>>> master and with:
>>>>> $ bitbake -c update_crates solana-keygen
>>>>> it fails with:
>>>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>>>>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>>>>> add at least one to the recipe:
>>>>> SRC_URI[Inflector.sha256sum] =
>>>>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>>>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>>>>> NoChecksumError('Missing SRC_URI checksum', '
>>>>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>>>>
>>>>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to
>>>>> be used? Or am I just missing some of the pending changes or just using it
>>>>> incorrectly?
>>>>>
>>>>>
>>>>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>>>> >
>>>>>> > There is no need to do such things, Cargo.lock file
>>>>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>>>>> > This avoid finding other possible Cargo.lock that
>>>>>> > would be in subdir (for example if a patch is applied
>>>>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>>>>> > can be there)
>>>>>> >
>>>>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>>>> > ---
>>>>>> >  .../cargo-update-recipe-crates.bbclass               | 12
>>>>>> ++++++++----
>>>>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>>>>> >
>>>>>> > diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> > index daa363b0dd..549cfe627e 100644
>>>>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>>>>> >  import os
>>>>>> >  crates = "# Autogenerated with 'bitbake -c update_crates
>>>>>> ${PN}'\n\n"
>>>>>> >  found = False
>>>>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>>>> > -    for file in files:
>>>>>> > -        if file == 'Cargo.lock':
>>>>>> > -            crates += get_crates(os.path.join(root, file))
>>>>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>>>>> > +    if file == 'Cargo.lock':
>>>>>> > +        try:
>>>>>> > +            cargo_lock_path =
>>>>>> os.path.join('${CARGO_LOCK_SRC_DIR}', file)
>>>>>> > +            crates += get_crates(cargo_lock_path)
>>>>>> > +        except Exception as e:
>>>>>> > +            raise ValueError("Cannot parse '%s'" %
>>>>>> cargo_lock_path) from e
>>>>>> > +        else:
>>>>>> >              found = True
>>>>>> >  if not found:
>>>>>> >      raise ValueError("Unable to find Cargo.lock in
>>>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>>>
>>>>>> Isn't this just a long-winded version of something like this
>>>>>> (completely untested):
>>>>>>
>>>>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>>>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>>>>
>>>>>>  import os
>>>>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>>>>> -found = False
>>>>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>>>> -    for file in files:
>>>>>> -        if file == 'Cargo.lock':
>>>>>> -            crates += get_crates(os.path.join(root, file))
>>>>>> -            found = True
>>>>>> -if not found:
>>>>>> -    raise ValueError("Unable to find Cargo.lock in
>>>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>>>>> "Cargo.lock"))
>>>>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>>>>  EOF
>>>>>>
>>>>>> --
>>>>>> Alex Kiernan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#179358):
>> https://lists.openembedded.org/g/openembedded-core/message/179358
>> Mute This Topic: https://lists.openembedded.org/mt/97953740/3617156
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>> Martin.Jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>
Frédéric Martinsons April 1, 2023, 3:21 p.m. UTC | #12 
Hello, I just submitted a patch inside the bitbake fetcher to display all
the missing
checksum at once instead of only one.

For the moment, I have no better solution, let's see what bitbake folks
tell us about that.

On Fri, 31 Mar 2023 at 07:32, Frederic Martinsons via lists.openembedded.org
<frederic.martinsons=gmail.com@lists.openembedded.org> wrote:

> Hello,
>
> The multiple Cargo.lock seems totally valid, a project can have multiple
> binaries provided (which is the case for Solana cli suite) so I'll submit a
> new v6 of my series to reflect that, I just have to ignored specify dirs
> like .git (no use to walk inside it) or .pc (can contains a patched
> Cargo.lock that would be redundant and syntaxically invalid )
>
> I'll also override the crate names with their version for all the crates
> found.
>
> For the chicken and eggs issue, I need to think more of it (maybe make
> bitbake print the list of all missing checksum  would be enough) and, in
> order to not delay this series, I'll submit a separate bitbake patch if I
> manage to find something suitable.
>
> Many thanks for the comments Martin.
>
> Le jeu. 30 mars 2023, 23:15, Martin Jansa <martin.jansa@gmail.com> a
> écrit :
>
>> I've added the updated .inc files to meta-webosose fork if it's easier to
>> see there:
>>
>> https://github.com/shr-project/meta-webosose/commits/master-2023-03-30
>> (top 2 commits)
>>
>> this checksum mismatch is also the reason for that ugly exception as
>> explained in top commit, it's race-condition between sugar.do_unpack and
>> solana-cli.do_fetch where sugar has right arrayvec 0.7.2 with right
>> checksum in do_unpack, while solana-cli renames it to _bad-checksum_
>> suffix, because solana-cli looks for checksum of arrayvec 0.7.1.
>>
>> I've tried to manually add name= param to verify this, but there are too
>> many :), so I'll update the bbclass to always add name with version even if
>> it doesn't see duplicated crates (and then this can be improved if multiple
>> Cargo.lock files really need to be supported).
>>
>> I honestly don't know, I've was only working on these solana recipes for
>> short time and wanted to offer them more as strange example.
>>
>> Cheers,
>>
>> On Thu, Mar 30, 2023 at 10:22 PM Martin Jansa via lists.openembedded.org
>> <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
>>
>>> On Thu, Mar 30, 2023 at 10:07 PM Frédéric Martinsons <
>>> frederic.martinsons@gmail.com> wrote:
>>>
>>>> By the way, about this "chickens and eggs" problem, isn't it the same
>>>> for a regular recipe you just upgraded?
>>>>
>>>
>>> I don't know, I haven't upgraded a recipe with crates yet :)
>>>
>>> It's just this time I was aware that the .inc file will need to be
>>> refreshed and it wasn't obvious at first that easiest way to avoid do_fetch
>>> issue blocking .inc refresh is to remove all crates first by emptying .inc
>>> file.
>>>
>>> Personnaly, when I update a recipe, I let bitbake tell me what is the
>>>> new checksum expected and put it the recipe (as the error message says).
>>>>
>>>
>>> For regular recipes with a few checksum I do the same. But it doesn't
>>> scale here as you mention bellow.
>>> as it reported only the first wrong checksum from MANY.
>>>
>>> But I'm aware this is not exactly the same, since a cargo based recipe
>>>> could contain a ton of crate:// uri and if you apply this method, you have
>>>> to copy the new checksum one by one, bitbake error after another...
>>>>
>>>> Don't know how to make this better and I plead guilty for not
>>>> mentioning that in a dedicated commit message.
>>>>
>>>
>>> Better late than never, you can include it in v6 of this patch (I have
>>> read the git log of the .bbclass before going to refresh the .inc files, so
>>> others might notice it there as well).
>>>
>>> Of course, if someone come up with a smoothier way of doing this, I'll
>>>> make a new patch.
>>>>
>>>
>>> I think I found what the other issue with arrayvec create was:
>>>
>>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
>>> crates.io/arrayvec/0.7.2 \
>>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
>>> = "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
>>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:    crate://
>>> crates.io/arrayvec/0.7.1 \
>>> meta-webos/recipes-upstreamable/solana/solana-crates.inc:SRC_URI[arrayvec.sha256sum]
>>> = "be4dc07131ffa69b8072d35f5007352af944213cde02545e2103680baed38fcd"
>>> meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:    crate://
>>> crates.io/arrayvec/0.7.2 \
>>> meta-webos/recipes-upstreamable/sugar/sugar-crates.inc:SRC_URI[arrayvec.sha256sum]
>>> = "8da52d66c7071e2e3fa2a1e5c6d088fec47b593032b254f5e980de8ea54454d6"
>>>
>>> So different Cargo.lock files included in the same ${S} have different
>>> version, but in this case it didn't set different name parameter, probably
>>> because different Cargo.lock files are processed separately?
>>>
>>> Thanks
>>>
>>> Le jeu. 30 mars 2023, 21:50, Frédéric Martinsons <
>>>> frederic.martinsons@gmail.com> a écrit :
>>>>
>>>>> Well I see what you mean, I'll take a look at your example to try to
>>>>> find out if multiple Cargo.lock could be expected.
>>>>>
>>>>> And for your second remark, yes, there is a chicken and eggs issue for
>>>>> updating crates checksum from scratch.
>>>>> You didn't miss anything, I came across this when updating crates
>>>>> checksum that was "pristine"
>>>>>  and I only manage to execute update_crates by locally patching
>>>>> bitbake/lib/bb/fetch2/crate.py for
>>>>> recommends_checksum method to return False .
>>>>> I'm aware that is not ideal but I don't know how to make this better
>>>>> (maybe make do_update_crates
>>>>> after do_fetch instead of after do_patch ? )
>>>>>
>>>>>
>>>>> Le jeu. 30 mars 2023, 21:23, Martin Jansa <martin.jansa@gmail.com> a
>>>>> écrit :
>>>>>
>>>>>> I don't remember the exact details now, but when I was working on
>>>>>> updating solana recipes to use this
>>>>>>
>>>>>> https://github.com/webosose/meta-webosose/commit/9bdfae7988077d0211eeee79cc339d0770cd86b4
>>>>>>
>>>>>> the S was pointing to just some subdirectory and multiple Cargo.locks
>>>>>> files were parsed in other directories as well, that's why I was adding
>>>>>>
>>>>>> https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=7636a2b8080521ed2ad54b0edce47a8742a12d58
>>>>>>
>>>>>> in the end I've changed to use the root directory in S and just
>>>>>> set CARGO_SRC_DIR to right subdirectory to build.
>>>>>>
>>>>>> I don't have much experience with crates other than these solana
>>>>>> recipes, but isn't there some valid use-case for multiple Cargo.locks?
>>>>>> I assume Alex in original implementation didn't use the os.walk just
>>>>>> to make it more complicated :).
>>>>>>
>>>>>> And FWIW when trying to regenerate these .inc files with current
>>>>>> master and with:
>>>>>> $ bitbake -c update_crates solana-keygen
>>>>>> it fails with:
>>>>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: No checksum specified for
>>>>>> '/OE/lge/build/webos/mickledore/downloads/Inflector-0.11.4.crate', please
>>>>>> add at least one to the recipe:
>>>>>> SRC_URI[Inflector.sha256sum] =
>>>>>> "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
>>>>>> ERROR: solana-keygen-1.14.5-r0 do_fetch: Bitbake Fetcher Error:
>>>>>> NoChecksumError('Missing SRC_URI checksum', '
>>>>>> https://crates.io/api/v1/crates/Inflector/0.11.4/download')
>>>>>>
>>>>>> Isn't it an chicken&egg issue now when do_fetch enforces checksums to
>>>>>> be used? Or am I just missing some of the pending changes or just using it
>>>>>> incorrectly?
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 30, 2023 at 6:34 PM Alex Kiernan <alex.kiernan@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On Thu, Mar 30, 2023 at 4:45 PM <frederic.martinsons@gmail.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>>>>> >
>>>>>>> > There is no need to do such things, Cargo.lock file
>>>>>>> > has to be at the root of CARGO_LOCK_SRC_DIR.
>>>>>>> > This avoid finding other possible Cargo.lock that
>>>>>>> > would be in subdir (for example if a patch is applied
>>>>>>> > on the recipe, we can have .pc subdir in S and a Cargo.lock
>>>>>>> > can be there)
>>>>>>> >
>>>>>>> > Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
>>>>>>> > ---
>>>>>>> >  .../cargo-update-recipe-crates.bbclass               | 12
>>>>>>> ++++++++----
>>>>>>> >  1 file changed, 8 insertions(+), 4 deletions(-)
>>>>>>> >
>>>>>>> > diff --git
>>>>>>> a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> > index daa363b0dd..549cfe627e 100644
>>>>>>> > --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> > +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> > @@ -68,10 +68,14 @@ def get_crates(f):
>>>>>>> >  import os
>>>>>>> >  crates = "# Autogenerated with 'bitbake -c update_crates
>>>>>>> ${PN}'\n\n"
>>>>>>> >  found = False
>>>>>>> > -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>>>>> > -    for file in files:
>>>>>>> > -        if file == 'Cargo.lock':
>>>>>>> > -            crates += get_crates(os.path.join(root, file))
>>>>>>> > +for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
>>>>>>> > +    if file == 'Cargo.lock':
>>>>>>> > +        try:
>>>>>>> > +            cargo_lock_path =
>>>>>>> os.path.join('${CARGO_LOCK_SRC_DIR}', file)
>>>>>>> > +            crates += get_crates(cargo_lock_path)
>>>>>>> > +        except Exception as e:
>>>>>>> > +            raise ValueError("Cannot parse '%s'" %
>>>>>>> cargo_lock_path) from e
>>>>>>> > +        else:
>>>>>>> >              found = True
>>>>>>> >  if not found:
>>>>>>> >      raise ValueError("Unable to find Cargo.lock in
>>>>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>>>>
>>>>>>> Isn't this just a long-winded version of something like this
>>>>>>> (completely untested):
>>>>>>>
>>>>>>> diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> index daa363b0dd65..22ddcfa5c1e3 100644
>>>>>>> --- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> +++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
>>>>>>> @@ -67,14 +67,7 @@ def get_crates(f):
>>>>>>>
>>>>>>>  import os
>>>>>>>  crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
>>>>>>> -found = False
>>>>>>> -for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
>>>>>>> -    for file in files:
>>>>>>> -        if file == 'Cargo.lock':
>>>>>>> -            crates += get_crates(os.path.join(root, file))
>>>>>>> -            found = True
>>>>>>> -if not found:
>>>>>>> -    raise ValueError("Unable to find Cargo.lock in
>>>>>>> ${CARGO_LOCK_SRC_DIR}")
>>>>>>> +crates += get_crates(os.path.join("${CARGO_LOCK_SRC_DIR}",
>>>>>>> "Cargo.lock"))
>>>>>>>  open("${TARGET_FILE}", 'w').write(crates)
>>>>>>>  EOF
>>>>>>>
>>>>>>> --
>>>>>>> Alex Kiernan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>
>>>
>>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#179391):
> https://lists.openembedded.org/g/openembedded-core/message/179391
> Mute This Topic: https://lists.openembedded.org/mt/97953740/6213388
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> frederic.martinsons@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Richard Purdie April 3, 2023, 1:35 p.m. UTC | #13 
On Sat, 2023-04-01 at 17:21 +0200, Frederic Martinsons wrote:
> Hello, I just submitted a patch inside the bitbake fetcher to display
> all the missing checksum at once instead of only one.
> 
> For the moment, I have no better solution, let's see what bitbake
> folks tell us about that.

I'm fine with that FWIW...

Cheers,

Richard
Frédéric Martinsons April 3, 2023, 2:25 p.m. UTC | #14 
Nice, glad to hear that.

Le lun. 3 avr. 2023, 15:35, Richard Purdie <
richard.purdie@linuxfoundation.org> a écrit :

> On Sat, 2023-04-01 at 17:21 +0200, Frederic Martinsons wrote:
> > Hello, I just submitted a patch inside the bitbake fetcher to display
> > all the missing checksum at once instead of only one.
> >
> > For the moment, I have no better solution, let's see what bitbake
> > folks tell us about that.
>
> I'm fine with that FWIW...
>
> Cheers,
>
> Richard
>
diff mbox series

Patch

diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
index daa363b0dd..549cfe627e 100644
--- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
+++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
@@ -68,10 +68,14 @@  def get_crates(f):
 import os
 crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
 found = False
-for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
-    for file in files:
-        if file == 'Cargo.lock':
-            crates += get_crates(os.path.join(root, file))
+for file in os.listdir('${CARGO_LOCK_SRC_DIR}'):
+    if file == 'Cargo.lock':
+        try:
+            cargo_lock_path = os.path.join('${CARGO_LOCK_SRC_DIR}', file)
+            crates += get_crates(cargo_lock_path)
+        except Exception as e:
+            raise ValueError("Cannot parse '%s'" % cargo_lock_path) from e
+        else:
             found = True
 if not found:
     raise ValueError("Unable to find Cargo.lock in ${CARGO_LOCK_SRC_DIR}")