From patchwork Mon Dec 19 08:55:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: alp.oezmert@mercedes-benz.com X-Patchwork-Id: 16914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC36BC4332F for ; Mon, 19 Dec 2022 08:55:24 +0000 (UTC) Received: from mail-out.emea.daimler.com (mail-out.emea.daimler.com [141.113.1.134]) by mx.groups.io with SMTP id smtpd.web11.16874.1671440116583375068 for ; Mon, 19 Dec 2022 00:55:18 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@mercedes-benz.com header.s=s1-20210420-dai header.b=iTNoUyKn; spf=pass (domain: mercedes-benz.com, ip: 141.113.1.134, mailfrom: alp.oezmert@mercedes-benz.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mercedes-benz.com; i=@mercedes-benz.com; q=dns/txt; s=s1-20210420-DAI; t=1671440116; h=from:to:cc:subject:date:message-id:content-type: content-id:content-transfer-encoding:mime-version; bh=a63iMzWp5kbqdZwYDJ4fpzIsZKlbnMkNwL5AeyJqaRo=; b=iTNoUyKnZQZ49SsTZb1tToPelOZwrPVH43YI5M9g22MOCpAzXAvQeRvI C6vH02yA1dm74w3I0nPeidb68ipfoGqW7sk3MeBI1OwVkLF2rz9nzN5wf RWcAiEsXI3T5gmqOm2BU8nE4yn5ZOpzBu8tKeUdRwrjTrliW9r3coYA/e 6LACcdkd6RGE3gpr2HP+3xijD8o/ugDu2WK3a611Sgvuse2ZIgWecTo4K P6LE2jx5ToaCSsXW/5XFyTQqCJ1L1q1V1U6If4I1xQXBPTP90tq3oqvTr 4WnPiNLu5qsNQv1p9/YjBrssMwI5g54d1sNJXjIljfIXQmhGDxD3EBonC Q==; From: To: CC: Subject: [PATCH] wget: Add CHECKMD_wget option Thread-Topic: [PATCH] wget: Add CHECKMD_wget option Thread-Index: AQHZE4ebhWCgVNbSp0WGf0mp421pIg== Date: Mon, 19 Dec 2022 08:55:13 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-ID: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 19 Dec 2022 08:55:24 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14197 `CHECKCMD_wget` defines a command that the wget fetcher module will execute instead of the built-in HTTP client when checking if URLs exist. This allows to completely match the behavior of the wget fetcher including proxy and authentication. By default, this option is empty, and wget fetcher behavior is not changed. Signed-off-by: Alp Özmert --- lib/bb/fetch2/wget.py | 7 +++++++ lib/bb/tests/fetch.py | 14 ++++++++++++++ 2 files changed, 21 insertions(+) -- 2.38.1 If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for your support. diff --git a/lib/bb/fetch2/wget.py b/lib/bb/fetch2/wget.py index 821afa5b..56afd24e 100644 --- a/lib/bb/fetch2/wget.py +++ b/lib/bb/fetch2/wget.py @@ -153,6 +153,13 @@ class Wget(FetchMethod): return True def checkstatus(self, fetch, ud, d, try_again=True): + checkcmd = d.getVar("CHECKCMD_wget") + if checkcmd: + if checkcmd == "auto": + checkcmd = "%s --spider --quiet" % self.basecmd + uri = ud.url.split(";")[0] + return os.system("%s %s" % (checkcmd, uri)) == os.EX_OK + class HTTPConnectionCache(http.client.HTTPConnection): if fetch.connection_cache: def connect(self): diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 0af06e46..6eb77e5c 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -1498,6 +1498,20 @@ class FetchCheckStatusTest(FetcherTest): ret = m.checkstatus(fetch, ud, self.d) self.assertTrue(ret, msg="URI %s, can't check status" % (u)) + @skipIfNoNetwork() + def test_wget_checkstatus_external_wget(self): + # exclude test case ftp:// as wget has been observed to wrongly return '8' + self.test_wget_uris = [uri for uri in self.test_wget_uris if "ftp://" not in uri] + self.d.setVar("CHECKCMD_wget", "wget --spider --quiet") + self.test_wget_checkstatus() + + @skipIfNoNetwork() + def test_wget_checkstatus_external_auto(self): + # exclude test case ftp:// as wget has been observed to wrongly return '8' + self.test_wget_uris = [uri for uri in self.test_wget_uris if "ftp://" not in uri] + self.d.setVar("CHECKCMD_wget", "auto") + self.test_wget_checkstatus() + @skipIfNoNetwork() def test_wget_checkstatus_connection_cache(self): from bb.fetch2 import FetchConnectionCache