mbox

[0/2] fetch2/git: Use git shallow fetch to implement clone_shallow_local()

Message ID cover.1718886475.git.liezhi.yang@windriver.com
State Not Applicable, archived
Headers show

Pull-request

https://github.com/robertlinux/yocto rbt/shallow

Message

Robert Yang June 20, 2024, 12:44 p.m. UTC
From: Robert Yang <liezhi.yang@windriver.com>

* Test info
  bitbake-selftest works well
  bitbake world works well with the shallow tarballs

// Robert

The following changes since commit 5d88faa0f35f0205c1475893d8589d1e6533dcc0:

  bitbake: siggen: catch FileNotFoundError everywhere and ConnectionError also in get_unihashes (2024-06-18 08:45:22 +0100)

are available in the Git repository at:

  https://github.com/robertlinux/yocto rbt/shallow
  https://github.com/robertlinux/yocto/tree/rbt/shallow

Robert Yang (2):
  fetch2/git: Use git shallow fetch to implement clone_shallow_local()
  bitbake: tests/fetch: Update GitShallowTest for clone_shallow_local()

 bitbake/lib/bb/fetch2/git.py  | 71 ++++++++++++++++++++++-------------
 bitbake/lib/bb/tests/fetch.py | 13 ++++---
 2 files changed, 53 insertions(+), 31 deletions(-)

Comments

Robert Yang June 27, 2024, 5:46 a.m. UTC | #1
Hi RP and Christopher,

On 6/20/24 20:44, Robert Yang via lists.openembedded.org wrote:
> From: Robert Yang <liezhi.yang@windriver.com>
> 
> * Test info
>    bitbake-selftest works well
>    bitbake world works well with the shallow tarballs

Do you have any comments on the patches, please?

// Robert

> 
> // Robert
> 
> The following changes since commit 5d88faa0f35f0205c1475893d8589d1e6533dcc0:
> 
>    bitbake: siggen: catch FileNotFoundError everywhere and ConnectionError also in get_unihashes (2024-06-18 08:45:22 +0100)
> 
> are available in the Git repository at:
> 
>    https://github.com/robertlinux/yocto rbt/shallow
>    https://github.com/robertlinux/yocto/tree/rbt/shallow
> 
> Robert Yang (2):
>    fetch2/git: Use git shallow fetch to implement clone_shallow_local()
>    bitbake: tests/fetch: Update GitShallowTest for clone_shallow_local()
> 
>   bitbake/lib/bb/fetch2/git.py  | 71 ++++++++++++++++++++++-------------
>   bitbake/lib/bb/tests/fetch.py | 13 ++++---
>   2 files changed, 53 insertions(+), 31 deletions(-)
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#16368): https://lists.openembedded.org/g/bitbake-devel/message/16368
> Mute This Topic: https://lists.openembedded.org/mt/106779045/7304958
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [liezhi.yang@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Alexandre Belloni July 1, 2024, 10:13 p.m. UTC | #2
Hello,

This fails on our ubuntu 20.04 workers:

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6923/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/3542/steps/11/logs/stdio

On 20/06/2024 05:44:39-0700, Robert Yang via lists.openembedded.org wrote:
> From: Robert Yang <liezhi.yang@windriver.com>
> 
> * Test info
>   bitbake-selftest works well
>   bitbake world works well with the shallow tarballs
> 
> // Robert
> 
> The following changes since commit 5d88faa0f35f0205c1475893d8589d1e6533dcc0:
> 
>   bitbake: siggen: catch FileNotFoundError everywhere and ConnectionError also in get_unihashes (2024-06-18 08:45:22 +0100)
> 
> are available in the Git repository at:
> 
>   https://github.com/robertlinux/yocto rbt/shallow
>   https://github.com/robertlinux/yocto/tree/rbt/shallow
> 
> Robert Yang (2):
>   fetch2/git: Use git shallow fetch to implement clone_shallow_local()
>   bitbake: tests/fetch: Update GitShallowTest for clone_shallow_local()
> 
>  bitbake/lib/bb/fetch2/git.py  | 71 ++++++++++++++++++++++-------------
>  bitbake/lib/bb/tests/fetch.py | 13 ++++---
>  2 files changed, 53 insertions(+), 31 deletions(-)
> 
> -- 
> 2.45.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#16368): https://lists.openembedded.org/g/bitbake-devel/message/16368
> Mute This Topic: https://lists.openembedded.org/mt/106779045/3617179
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Robert Yang July 2, 2024, 7:31 a.m. UTC | #3
On 7/2/24 06:13, Alexandre Belloni via lists.openembedded.org wrote:
> Hello,
> 
> This fails on our ubuntu 20.04 workers:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6923/steps/11/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/3542/steps/11/logs/stdio

Sorry, it might because the git version on ubuntu 20.04 is old, let me check how 
to fix it.

// Robert

> 
> On 20/06/2024 05:44:39-0700, Robert Yang via lists.openembedded.org wrote:
>> From: Robert Yang <liezhi.yang@windriver.com>
>>
>> * Test info
>>    bitbake-selftest works well
>>    bitbake world works well with the shallow tarballs
>>
>> // Robert
>>
>> The following changes since commit 5d88faa0f35f0205c1475893d8589d1e6533dcc0:
>>
>>    bitbake: siggen: catch FileNotFoundError everywhere and ConnectionError also in get_unihashes (2024-06-18 08:45:22 +0100)
>>
>> are available in the Git repository at:
>>
>>    https://github.com/robertlinux/yocto rbt/shallow
>>    https://github.com/robertlinux/yocto/tree/rbt/shallow
>>
>> Robert Yang (2):
>>    fetch2/git: Use git shallow fetch to implement clone_shallow_local()
>>    bitbake: tests/fetch: Update GitShallowTest for clone_shallow_local()
>>
>>   bitbake/lib/bb/fetch2/git.py  | 71 ++++++++++++++++++++++-------------
>>   bitbake/lib/bb/tests/fetch.py | 13 ++++---
>>   2 files changed, 53 insertions(+), 31 deletions(-)
>>
>> -- 
>> 2.45.1
>>
> 
>>
>>
>>
> 
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#16383): https://lists.openembedded.org/g/bitbake-devel/message/16383
> Mute This Topic: https://lists.openembedded.org/mt/106779045/3616940
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [liezhi.yang@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Robert Yang July 7, 2024, 7:11 a.m. UTC | #4
On 7/2/24 15:31, Robert Yang wrote:
> 
> 
> On 7/2/24 06:13, Alexandre Belloni via lists.openembedded.org wrote:
>> Hello,
>>
>> This fails on our ubuntu 20.04 workers:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6923/steps/11/logs/stdio
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/3542/steps/11/logs/stdio
> 
> Sorry, it might because the git version on ubuntu 20.04 is old, let me check how  > to fix it.

I had a ppa on my ubuntu host 20.04 which installed a higher version git, that
made me didn't meet the error. I've tried pure ubuntu 20.04 from docker hub, and
then reproduced the error, the fix is:

+            # Advertise the revision for lower version git such as 2.25.1:
+            # error: Server does not allow request for unadvertised object.
+            # The ud.clonedir is a local temporary dir, will be removed when
+            # fetch is done, so we can do anything on it.
+            adv_cmd = 'git branch -f advertise-%s %s' % (revision, revision)
+            runfetchcmd(adv_cmd, d, workdir=ud.clonedir)


I will send a V2 for it.

// Robert

> 
> // Robert
> 
>>
>> On 20/06/2024 05:44:39-0700, Robert Yang via lists.openembedded.org wrote:
>>> From: Robert Yang <liezhi.yang@windriver.com>
>>>
>>> * Test info
>>>    bitbake-selftest works well
>>>    bitbake world works well with the shallow tarballs
>>>
>>> // Robert
>>>
>>> The following changes since commit 5d88faa0f35f0205c1475893d8589d1e6533dcc0:
>>>
>>>    bitbake: siggen: catch FileNotFoundError everywhere and ConnectionError 
>>> also in get_unihashes (2024-06-18 08:45:22 +0100)
>>>
>>> are available in the Git repository at:
>>>
>>>    https://github.com/robertlinux/yocto rbt/shallow
>>>    https://github.com/robertlinux/yocto/tree/rbt/shallow
>>>
>>> Robert Yang (2):
>>>    fetch2/git: Use git shallow fetch to implement clone_shallow_local()
>>>    bitbake: tests/fetch: Update GitShallowTest for clone_shallow_local()
>>>
>>>   bitbake/lib/bb/fetch2/git.py  | 71 ++++++++++++++++++++++-------------
>>>   bitbake/lib/bb/tests/fetch.py | 13 ++++---
>>>   2 files changed, 53 insertions(+), 31 deletions(-)
>>>
>>> -- 
>>> 2.45.1
>>>
>>
>>>
>>>
>>>
>>
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#16383): 
>> https://lists.openembedded.org/g/bitbake-devel/message/16383
>> Mute This Topic: https://lists.openembedded.org/mt/106779045/3616940
>> Group Owner: bitbake-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub 
>> [liezhi.yang@windriver.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>