| Message ID | 20260703110612.294043-1-sivakumar.bs@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [bitbake-devel] fetch2: give every FetchData a default unpack_tracer | expand |
On Fri, 2026-07-03 at 16:36 +0530, Siva Balasubramanian via lists.openembedded.org wrote: > Fetch() sets an unpack_tracer attribute on the FetchData objects it > manages, but the per-mirror FetchData objects created in > build_mirroruris() are constructed directly and never get one. When a > mirror is used for a git recipe that needs Git LFS, git.py's download() > performs a checkout (via Git.unpack()) on the mirror's FetchData to > materialise the LFS objects, and that unpack path dereferences > ud.unpack_tracer, failing with: > > AttributeError: 'FetchData' object has no attribute 'unpack_tracer' > > so PREMIRRORS/MIRRORS fetching is broken for git-lfs sources. > > Fix this at the source by initialising unpack_tracer to a > DummyUnpackTracer in FetchData.__init__(), so the attribute always > exists. Fetch() still overrides it with the real (possibly > user-configured via BB_UNPACK_TRACER_CLASS) tracer for the URLs it > manages; the mirror FetchData objects only perform an internal, > throwaway checkout that should not be traced anyway. > > Add a MirrorUriTest regression test asserting the mirror FetchData > objects carry an unpack_tracer. > > Reported-by: Oliver Feilner <oliver.feilner@yaskawa.eu> > > [YOCTO #15948] > > Signed-off-by: Siva Balasubramanian <sivakumar.bs@gmail.com> Wasn't this already fixed by: https://git.openembedded.org/bitbake/commit/?id=f0c9cf8d3885c5b1c2ba448f064421dae476fcd0 ? We don't want to add a dummy one everywhere, only add one when needed as the patch avove does... Cheers, Richard
Thanks Richard, you're right, this is already fixed in master by
f0c9cf8d ("fetch2: make unpack_tracer available when fetching mirror
URLs"). My checkout was behind and I missed it. Please drop this patch
On Fri, Jul 3, 2026 at 6:00 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2026-07-03 at 16:36 +0530, Siva Balasubramanian via
> lists.openembedded.org wrote:
> > Fetch() sets an unpack_tracer attribute on the FetchData objects it
> > manages, but the per-mirror FetchData objects created in
> > build_mirroruris() are constructed directly and never get one. When a
> > mirror is used for a git recipe that needs Git LFS, git.py's download()
> > performs a checkout (via Git.unpack()) on the mirror's FetchData to
> > materialise the LFS objects, and that unpack path dereferences
> > ud.unpack_tracer, failing with:
> >
> > AttributeError: 'FetchData' object has no attribute 'unpack_tracer'
> >
> > so PREMIRRORS/MIRRORS fetching is broken for git-lfs sources.
> >
> > Fix this at the source by initialising unpack_tracer to a
> > DummyUnpackTracer in FetchData.__init__(), so the attribute always
> > exists. Fetch() still overrides it with the real (possibly
> > user-configured via BB_UNPACK_TRACER_CLASS) tracer for the URLs it
> > manages; the mirror FetchData objects only perform an internal,
> > throwaway checkout that should not be traced anyway.
> >
> > Add a MirrorUriTest regression test asserting the mirror FetchData
> > objects carry an unpack_tracer.
> >
> > Reported-by: Oliver Feilner <oliver.feilner@yaskawa.eu>
> >
> > [YOCTO #15948]
> >
> > Signed-off-by: Siva Balasubramanian <sivakumar.bs@gmail.com>
>
> Wasn't this already fixed by:
>
>
> https://git.openembedded.org/bitbake/commit/?id=f0c9cf8d3885c5b1c2ba448f064421dae476fcd0
>
> ?
>
> We don't want to add a dummy one everywhere, only add one when needed
> as the patch avove does...
>
> Cheers,
>
> Richard
>
diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index ce7456b60..28d3d50d4 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -1325,6 +1325,12 @@ class FetchData(object): self.mirrortarballs = [] self.basename = None self.basepath = None + # Default to a no-op tracer. Fetch() replaces this with the real + # (possibly user-configured) unpack tracer for the URLs it manages, + # but FetchData objects created elsewhere (e.g. the per-mirror ones + # built in build_mirroruris()) must still have the attribute so that + # fetcher unpack methods invoked on them do not raise AttributeError. + self.unpack_tracer = DummyUnpackTracer() (self.type, self.host, self.path, self.user, self.pswd, self.parm) = decodeurl(d.expand(url)) self.date = self.getSRCDate(d) self.url = url diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index ebc80aa8c..f9c7e3116 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -530,6 +530,17 @@ class MirrorUriTest(FetcherTest): uris, uds = bb.fetch2.build_mirroruris(fetcher, mirrors, self.d) self.assertEqual(uris, ['file:///somepath/downloads/bitbake-1.0.tar.gz', 'file:///someotherpath/downloads/bitbake-1.0.tar.gz']) + def test_mirror_uds_have_unpack_tracer(self): + # The per-mirror FetchData objects must carry an unpack_tracer, otherwise + # fetcher unpack methods (e.g. the git-lfs checkout done for a mirror) + # raise AttributeError. See YOCTO #15948. + fetcher = bb.fetch.FetchData("http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) + mirrors = bb.fetch2.mirror_from_string(self.mirrorvar) + uris, uds = bb.fetch2.build_mirroruris(fetcher, mirrors, self.d) + self.assertTrue(uds) + for ud in uds: + self.assertTrue(hasattr(ud, "unpack_tracer")) + def test_urilist2(self): # Catch https:// -> files:// bug fetcher = bb.fetch.FetchData("https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d)
Fetch() sets an unpack_tracer attribute on the FetchData objects it manages, but the per-mirror FetchData objects created in build_mirroruris() are constructed directly and never get one. When a mirror is used for a git recipe that needs Git LFS, git.py's download() performs a checkout (via Git.unpack()) on the mirror's FetchData to materialise the LFS objects, and that unpack path dereferences ud.unpack_tracer, failing with: AttributeError: 'FetchData' object has no attribute 'unpack_tracer' so PREMIRRORS/MIRRORS fetching is broken for git-lfs sources. Fix this at the source by initialising unpack_tracer to a DummyUnpackTracer in FetchData.__init__(), so the attribute always exists. Fetch() still overrides it with the real (possibly user-configured via BB_UNPACK_TRACER_CLASS) tracer for the URLs it manages; the mirror FetchData objects only perform an internal, throwaway checkout that should not be traced anyway. Add a MirrorUriTest regression test asserting the mirror FetchData objects carry an unpack_tracer. Reported-by: Oliver Feilner <oliver.feilner@yaskawa.eu> [YOCTO #15948] Signed-off-by: Siva Balasubramanian <sivakumar.bs@gmail.com> --- lib/bb/fetch2/__init__.py | 6 ++++++ lib/bb/tests/fetch.py | 11 +++++++++++ 2 files changed, 17 insertions(+)