diff mbox series

[scarthgap,17/21] recipetool: Handle unclean response in go resolver

Message ID d61934122cdc9f2d68f99e5e3363c3f1808e7782.1716211838.git.steve@sakoman.com
State Accepted
Delegated to: Steve Sakoman
Headers show
Series [scarthgap,01/21] ncurses: Fix CVE-2023-50495 | expand

Commit Message

Steve Sakoman May 20, 2024, 1:33 p.m. UTC 
From: Sven Schwermer <sven.schwermer@disruptive-technologies.com>

It appears that some go modules repond with a 404 error when trying to
resolve them dynamically. The response body may still contain the
go-import meta tag. An example for such behaviour is gonum.org/v1/gonum.

Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 8f2e14ab6562a9a68819a960c66a258ea9dbe246)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 scripts/lib/recipetool/create_go.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
diff mbox series

Patch

diff --git a/scripts/lib/recipetool/create_go.py b/scripts/lib/recipetool/create_go.py
index c560831442..0fb7115e26 100644
--- a/scripts/lib/recipetool/create_go.py
+++ b/scripts/lib/recipetool/create_go.py
@@ -16,7 +16,7 @@  from html.parser import HTMLParser
 from recipetool.create import RecipeHandler, handle_license_vars
 from recipetool.create import guess_license, tidy_licenses, fixup_license
 from recipetool.create import determine_from_url
-from urllib.error import URLError
+from urllib.error import URLError, HTTPError
 
 import bb.utils
 import json
@@ -251,15 +251,18 @@  class GoRecipeHandler(RecipeHandler):
         req = urllib.request.Request(url)
 
         try:
-            resp = urllib.request.urlopen(req)
-
+            body = urllib.request.urlopen(req).read()
+        except HTTPError as http_err:
+            logger.warning(
+                "Unclean status when fetching page from [%s]: %s", url, str(http_err))
+            body = http_err.fp.read()
         except URLError as url_err:
             logger.warning(
                 "Failed to fetch page from [%s]: %s", url, str(url_err))
             return None
 
         parser = GoImportHTMLParser()
-        parser.feed(resp.read().decode('utf-8'))
+        parser.feed(body.decode('utf-8'))
         parser.close()
 
         return GoImport(parser.import_prefix, parser.vcs, parser.repourl, None)