mbox series

[RFC,v2,0/7] fetch2: add support for implicit urls

Message ID 20250905064419.2589111-1-stefan.herbrechtsmeier-oss@weidmueller.com
Headers show
Series fetch2: add support for implicit urls | expand

Message

Stefan Herbrechtsmeier Sept. 5, 2025, 6:44 a.m. UTC
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>

The patch series add support for implicit URLs inside the fetcher. The
implicit URLs could be defined inside a source like a version control
system (git submodule) or a lock file (package-lock.json, cargo.lock or
go.sum). The integration of implicit URLs beside explicit URLs
simplifies the fetcher classes and avoid bugs because of iterations
between the Fetch and FetchMethod classes.

The series remove most methods inside the gitsm fetcher and only leaves
the parsing of the git submodules and the unpack functionality. It
allows the gitsm fetcher to use the premirror only feature. The current
implementation leads to problems because the download of the git
submodules is triggered via the download method which is called deeply
inside the fetcher code.

Changes in v2:
- Move test case for gitsm local paths into FetcherLocalTest class
- Remove implicit urls from localpaths
- Add commit to test the expanded_urldata function
- Fix UnboundLocalError of urldata in expand_urldata function
- Remove implicit URLs from localpaths for backward compatibility

Stefan Herbrechtsmeier (7):
  tests: fetch: add test case for gitsm implicit local paths
  tests: fetch: add test cases for expanded_urldata
  fetch2: rename u to url in Fetch class
  fetch2: call functions within loops of Fetch class
  fetch2: add helper to get urldata in Fetch class
  fetch2: add support for implicit urls
  fetch2: gitsm: use implicit urls feature

 lib/bb/fetch2/__init__.py | 127 +++++++++++++++++++++++++++-----------
 lib/bb/fetch2/gitsm.py    |  46 ++------------
 lib/bb/tests/fetch.py     |  59 ++++++++++++++++++
 3 files changed, 154 insertions(+), 78 deletions(-)

Comments

Mathieu Dubois-Briand Sept. 7, 2025, 6:22 p.m. UTC | #1
On Fri Sep 5, 2025 at 8:44 AM CEST, Stefan Herbrechtsmeier via lists.openembedded.org wrote:
> From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>
> The patch series add support for implicit URLs inside the fetcher. The
> implicit URLs could be defined inside a source like a version control
> system (git submodule) or a lock file (package-lock.json, cargo.lock or
> go.sum). The integration of implicit URLs beside explicit URLs
> simplifies the fetcher classes and avoid bugs because of iterations
> between the Fetch and FetchMethod classes.
>
> The series remove most methods inside the gitsm fetcher and only leaves
> the parsing of the git submodules and the unpack functionality. It
> allows the gitsm fetcher to use the premirror only feature. The current
> implementation leads to problems because the download of the git
> submodules is triggered via the download method which is called deeply
> inside the fetcher code.
>
> Changes in v2:
> - Move test case for gitsm local paths into FetcherLocalTest class
> - Remove implicit urls from localpaths
> - Add commit to test the expanded_urldata function
> - Fix UnboundLocalError of urldata in expand_urldata function
> - Remove implicit URLs from localpaths for backward compatibility
>
> Stefan Herbrechtsmeier (7):
>   tests: fetch: add test case for gitsm implicit local paths
>   tests: fetch: add test cases for expanded_urldata
>   fetch2: rename u to url in Fetch class
>   fetch2: call functions within loops of Fetch class
>   fetch2: add helper to get urldata in Fetch class
>   fetch2: add support for implicit urls
>   fetch2: gitsm: use implicit urls feature
>
>  lib/bb/fetch2/__init__.py | 127 +++++++++++++++++++++++++++-----------
>  lib/bb/fetch2/gitsm.py    |  46 ++------------
>  lib/bb/tests/fetch.py     |  59 ++++++++++++++++++
>  3 files changed, 154 insertions(+), 78 deletions(-)

Thanks,

I took a bit of time to test this new version, no issue on the
autobuilder.

Mathieu