diff mbox series

[bitbake-devel,v2] Change git lfs support to allow for destsuffix being set.

Message ID ad8a8eb0-48b7-3d06-f8d3-ddab5747c4f9@braneaudio.com
State New
Headers show
Series [bitbake-devel,v2] Change git lfs support to allow for destsuffix being set. | expand

Commit Message

Richard Greer Oct. 4, 2022, 10:19 p.m. UTC
From: Richard Greer<rg@braneaudio.com>

When getting large files with git lfs need to try and recreate the
path where the repo is actually located after the unpack.

If destsuffix or subdir (where subdir is not an absolute path) is
set then it will not be in tmpdir/git.

It looks like the more correct place would be in ud.destdir.

This does not address the issue where it will fail in the unpack
if subdir is an absolute path since it cannot possibly be in the
same path as the tmpdir.  I have no idea as to how to deal with
that.

Signed-off-by: Richard Greer<rg@braneaudio.com>

---
v1->v2
  * Try to get rid of improper line breaks.
  * Tried git send-email but could not get it to work with our mail
    server for any settings.
  * Tried routing it through gmail using personal account as a login,
    but it failed due to personal account not being subscribed to
    mailing list.  Do not really want to have my personal email
    subscribed either.
  * I do not see line breaks in patch from previous email in what I
    received from the mailing list, nor do I see it if I look at the
    post in Chrome on lists.openembedded.org.  So, I hope attempt two
    works since git send-email does not look like an option for me.
---
  bitbake/lib/bb/fetch2/git.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

-- 2.34.1

Comments

Luca Ceresoli Oct. 5, 2022, 8:45 p.m. UTC | #1
Hello Richard,

On Tue, 4 Oct 2022 17:19:47 -0500
"Richard Greer" <rg@braneaudio.com> wrote:

> From: Richard Greer<rg@braneaudio.com>
> 
> When getting large files with git lfs need to try and recreate the
> path where the repo is actually located after the unpack.
> 
> If destsuffix or subdir (where subdir is not an absolute path) is
> set then it will not be in tmpdir/git.
> 
> It looks like the more correct place would be in ud.destdir.
> 
> This does not address the issue where it will fail in the unpack
> if subdir is an absolute path since it cannot possibly be in the
> same path as the tmpdir.  I have no idea as to how to deal with
> that.
> 
> Signed-off-by: Richard Greer<rg@braneaudio.com>
> 
> ---
> v1->v2
>   * Try to get rid of improper line breaks.
>   * Tried git send-email but could not get it to work with our mail
>     server for any settings.
>   * Tried routing it through gmail using personal account as a login,
>     but it failed due to personal account not being subscribed to
>     mailing list.  Do not really want to have my personal email
>     subscribed either.
>   * I do not see line breaks in patch from previous email in what I
>     received from the mailing list, nor do I see it if I look at the
>     post in Chrome on lists.openembedded.org.  So, I hope attempt two
>     works since git send-email does not look like an option for me.

This time the lines appear correct, no line wrapping. Thanks for your
perseverance!

However I'm afraid there's still something incorrect.

> ---
>   bitbake/lib/bb/fetch2/git.py | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py

As you are modifying bitbake, you should do so a clone of the bitbake
git repository. From the path above, which have a "bitbake" directory
component, I suspect you did your work in a clone if the poky
repository, which is not meant for development. Have a look at the
README.md in the poky repository for more info, and should it still be
not clear we can help you here.

I also tried to apply your patch manually to circumvent this issue but
it didn't apply. I'm not sure about why this is happening, but it might
due to some other reason related to your mailer. I'd suggest to give
'git send-email' another try: it is meant exactly to send patches and
any other tool is likely to create problems. The README.md I mentioned
above has a link to info about setting up your workstation to send
patches in a proper way.

Finally, you might want to subscribe to the mailing list and disable
email delivery, so you won't get emails from the list but you would be
allowed to send emails.

I hope this helps.
Ross Burton Oct. 6, 2022, 11:07 a.m. UTC | #2
On 5 Oct 2022, at 21:45, Luca Ceresoli via lists.openembedded.org <luca.ceresoli=bootlin.com@lists.openembedded.org> wrote:
> As you are modifying bitbake, you should do so a clone of the bitbake
> git repository. From the path above, which have a "bitbake" directory
> component, I suspect you did your work in a clone if the poky
> repository, which is not meant for development. Have a look at the
> README.md in the poky repository for more info, and should it still be
> not clear we can help you here.

For what it’s worth, git usually handles this absolutely fine.  I’ve never worked in a separate clone of bitbake, just always from poky.

> I also tried to apply your patch manually to circumvent this issue but
> it didn't apply. I'm not sure about why this is happening, but it might
> due to some other reason related to your mailer. I'd suggest to give
> 'git send-email' another try: it is meant exactly to send patches and
> any other tool is likely to create problems. The README.md I mentioned
> above has a link to info about setting up your workstation to send
> patches in a proper way.

I also tried and can confirm: whitespace is being mangled by your mailer (Thunderbird).  Please use git-send-email as mailers do like to get in the way.

Ross
diff mbox series

Patch

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index f0df6fb69b..fa454a1c07 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -421,8 +421,8 @@  class Git(FetchMethod):
                  # Only do this if the unpack resulted in a .git/lfs directory being
                  # created; this only happens if at least one blob needed to be
                  # downloaded.
-                if os.path.exists(os.path.join(tmpdir, "git", ".git", "lfs")):
-                    runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.clonedir, d, workdir="%s/git/.git" % tmpdir)
+                if os.path.exists(os.path.join(ud.destdir, ".git", "lfs")):
+                    runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.clonedir, d, workdir="%s/.git" % ud.destdir)
              finally:
                  bb.utils.remove(tmpdir, recurse=True)