| Message ID | 20250717210247.181580-1-pamathieu@poum.ca |
|---|---|
| State | New |
| Headers | show |
| Series | fetch2/wget: Keep query parameters in URL during checkstatus | expand |
On Thu Jul 17, 2025 at 11:02 PM CEST, Philippe-Alexandre Mathieu via lists.openembedded.org wrote: > When recreating the uri in wget's checkstatus method, we only use the > scheme, netloc and path. This completely strips the query parameters > from the final URI and potentially breaks the checking functionality > from certain fetchers. > > This is the case for the Azure storage fetcher, as it requires a SAS > token that is formatted as a series of query parameters. The error > manifests itself when using a private storage account as a PREMIRROR or > SSTATE_MIRROR (since regular SRC_URI won't run the checkstatus). > > This problem is present in scarthgap, but wasn't in kirkstone. > > Signed-off-by: Philippe-Alexandre Mathieu <pamathieu@poum.ca> > --- Hi Philippe-Alexandre, Thanks for your patch, It looks like this is breaking bitbake selftests: FAIL: test_wget_checkstatus (bb.tests.fetch.FetchCheckStatusTest.test_wget_checkstatus) (url='ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz') ... AssertionError: False is not true : URI ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz, can't check status ... FAIL: test_wget_checkstatus_connection_cache (bb.tests.fetch.FetchCheckStatusTest.test_wget_checkstatus_connection_cache) (url='ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz') https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/1984 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1920 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2139 Can you have a look at these failures please?
diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index 7e43d3bc97..c8870b18ad 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -372,7 +372,7 @@ class Wget(FetchMethod): try: parts = urllib.parse.urlparse(ud.url.split(";")[0]) - uri = "{}://{}{}".format(parts.scheme, parts.netloc, parts.path) + uri = "{}://{}{}?{}".format(parts.scheme, parts.netloc, parts.path, parts.query) r = urllib.request.Request(uri) r.get_method = lambda: "HEAD" # Some servers (FusionForge, as used on Alioth) require that the
When recreating the uri in wget's checkstatus method, we only use the scheme, netloc and path. This completely strips the query parameters from the final URI and potentially breaks the checking functionality from certain fetchers. This is the case for the Azure storage fetcher, as it requires a SAS token that is formatted as a series of query parameters. The error manifests itself when using a private storage account as a PREMIRROR or SSTATE_MIRROR (since regular SRC_URI won't run the checkstatus). This problem is present in scarthgap, but wasn't in kirkstone. Signed-off-by: Philippe-Alexandre Mathieu <pamathieu@poum.ca> --- bitbake/lib/bb/fetch2/wget.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)