diff mbox series

[bitbake-devel] fetch2/git: fix shallow clone for tag containing slash

Message ID 20250514050900.2096691-1-Qi.Chen@windriver.com
State New
Headers show
Series [bitbake-devel] fetch2/git: fix shallow clone for tag containing slash | expand

Commit Message

ChenQi May 14, 2025, 5:09 a.m. UTC 
From: Chen Qi <Qi.Chen@windriver.com>

If a tag contains slash, e.g., debian/5.22, then shallow clone
fails because it's using a wrong ref.

To reproduce the issue, add the following lines in local.conf:

  BB_GIT_SHALLOW = "1"
  BB_GENERATE_SHALLOW_TARBALLS = "1"

And then run 'bitbake debianutils -c fetch'.

What the original os.path.basename(ref) wanted to do is to remove
the strings such as refs/heads/. So we do it explitly to fix this
issue.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 bitbake/lib/bb/fetch2/git.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Richard Purdie May 14, 2025, 6:28 a.m. UTC | #1 
On Wed, 2025-05-14 at 13:09 +0800, Chen Qi via lists.openembedded.org wrote:
> From: Chen Qi <Qi.Chen@windriver.com>
> 
> If a tag contains slash, e.g., debian/5.22, then shallow clone
> fails because it's using a wrong ref.
> 
> To reproduce the issue, add the following lines in local.conf:
> 
>   BB_GIT_SHALLOW = "1"
>   BB_GENERATE_SHALLOW_TARBALLS = "1"
> 
> And then run 'bitbake debianutils -c fetch'.
> 
> What the original os.path.basename(ref) wanted to do is to remove
> the strings such as refs/heads/. So we do it explitly to fix this
> issue.
> 
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  bitbake/lib/bb/fetch2/git.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
> index 11cda2007d..784a45bda2 100644
> --- a/bitbake/lib/bb/fetch2/git.py
> +++ b/bitbake/lib/bb/fetch2/git.py
> @@ -639,7 +639,7 @@ class Git(FetchMethod):
>                  extra_refs.append(r)
>  
>          for ref in extra_refs:
> -            ref_fetch = os.path.basename(ref)
> +            ref_fetch = ref.replace('refs/heads/', '').replace('refs/remotes/origin/', '').replace('refs/tags/', '')
>              runfetchcmd("%s fetch origin --depth 1 %s" % (ud.basecmd, ref_fetch), d, workdir=dest)
>              revision = runfetchcmd("%s rev-parse FETCH_HEAD" % ud.basecmd, d, workdir=dest)
>              runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest)
> 

Thanks, I just saw a few people reporting this.

Would it be possible to add a test case to bitbake-selftest so we can
esure this doesn't regress in future?

Cheers,

Richard
ChenQi May 14, 2025, 6:33 a.m. UTC | #2 
On 5/14/25 14:28, Richard Purdie via lists.openembedded.org wrote:
> On Wed, 2025-05-14 at 13:09 +0800, Chen Qi via lists.openembedded.org wrote:
>> From: Chen Qi <Qi.Chen@windriver.com>
>>
>> If a tag contains slash, e.g., debian/5.22, then shallow clone
>> fails because it's using a wrong ref.
>>
>> To reproduce the issue, add the following lines in local.conf:
>>
>>    BB_GIT_SHALLOW = "1"
>>    BB_GENERATE_SHALLOW_TARBALLS = "1"
>>
>> And then run 'bitbake debianutils -c fetch'.
>>
>> What the original os.path.basename(ref) wanted to do is to remove
>> the strings such as refs/heads/. So we do it explitly to fix this
>> issue.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> ---
>>   bitbake/lib/bb/fetch2/git.py | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
>> index 11cda2007d..784a45bda2 100644
>> --- a/bitbake/lib/bb/fetch2/git.py
>> +++ b/bitbake/lib/bb/fetch2/git.py
>> @@ -639,7 +639,7 @@ class Git(FetchMethod):
>>                   extra_refs.append(r)
>>   
>>           for ref in extra_refs:
>> -            ref_fetch = os.path.basename(ref)
>> +            ref_fetch = ref.replace('refs/heads/', '').replace('refs/remotes/origin/', '').replace('refs/tags/', '')
>>               runfetchcmd("%s fetch origin --depth 1 %s" % (ud.basecmd, ref_fetch), d, workdir=dest)
>>               revision = runfetchcmd("%s rev-parse FETCH_HEAD" % ud.basecmd, d, workdir=dest)
>>               runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest)
>>
> Thanks, I just saw a few people reporting this.
>
> Would it be possible to add a test case to bitbake-selftest so we can
> esure this doesn't regress in future?

Got it. I'll add a testcase and send out a new patch.

Regards,
Qi

>
> Cheers,
>
> Richard
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#17606): https://lists.openembedded.org/g/bitbake-devel/message/17606
> Mute This Topic: https://lists.openembedded.org/mt/113103962/7304865
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Qi.Chen@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Khem Raj May 14, 2025, 4:55 p.m. UTC | #3 
there is a bug as well - this should help with
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15862
so referring to that would be good

On Tue, May 13, 2025 at 11:33 PM Chen Qi via lists.openembedded.org
<Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>
> On 5/14/25 14:28, Richard Purdie via lists.openembedded.org wrote:
> > On Wed, 2025-05-14 at 13:09 +0800, Chen Qi via lists.openembedded.org wrote:
> >> From: Chen Qi <Qi.Chen@windriver.com>
> >>
> >> If a tag contains slash, e.g., debian/5.22, then shallow clone
> >> fails because it's using a wrong ref.
> >>
> >> To reproduce the issue, add the following lines in local.conf:
> >>
> >>    BB_GIT_SHALLOW = "1"
> >>    BB_GENERATE_SHALLOW_TARBALLS = "1"
> >>
> >> And then run 'bitbake debianutils -c fetch'.
> >>
> >> What the original os.path.basename(ref) wanted to do is to remove
> >> the strings such as refs/heads/. So we do it explitly to fix this
> >> issue.
> >>
> >> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> >> ---
> >>   bitbake/lib/bb/fetch2/git.py | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
> >> index 11cda2007d..784a45bda2 100644
> >> --- a/bitbake/lib/bb/fetch2/git.py
> >> +++ b/bitbake/lib/bb/fetch2/git.py
> >> @@ -639,7 +639,7 @@ class Git(FetchMethod):
> >>                   extra_refs.append(r)
> >>
> >>           for ref in extra_refs:
> >> -            ref_fetch = os.path.basename(ref)
> >> +            ref_fetch = ref.replace('refs/heads/', '').replace('refs/remotes/origin/', '').replace('refs/tags/', '')
> >>               runfetchcmd("%s fetch origin --depth 1 %s" % (ud.basecmd, ref_fetch), d, workdir=dest)
> >>               revision = runfetchcmd("%s rev-parse FETCH_HEAD" % ud.basecmd, d, workdir=dest)
> >>               runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest)
> >>
> > Thanks, I just saw a few people reporting this.
> >
> > Would it be possible to add a test case to bitbake-selftest so we can
> > esure this doesn't regress in future?
>
> Got it. I'll add a testcase and send out a new patch.
>
> Regards,
> Qi
>
> >
> > Cheers,
> >
> > Richard
> >
> >
> >
> >
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#17607): https://lists.openembedded.org/g/bitbake-devel/message/17607
> Mute This Topic: https://lists.openembedded.org/mt/113103962/1997914
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
diff mbox series

Patch

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index 11cda2007d..784a45bda2 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -639,7 +639,7 @@  class Git(FetchMethod):
                 extra_refs.append(r)
 
         for ref in extra_refs:
-            ref_fetch = os.path.basename(ref)
+            ref_fetch = ref.replace('refs/heads/', '').replace('refs/remotes/origin/', '').replace('refs/tags/', '')
             runfetchcmd("%s fetch origin --depth 1 %s" % (ud.basecmd, ref_fetch), d, workdir=dest)
             revision = runfetchcmd("%s rev-parse FETCH_HEAD" % ud.basecmd, d, workdir=dest)
             runfetchcmd("%s update-ref %s %s" % (ud.basecmd, ref, revision), d, workdir=dest)