From patchwork Fri Sep 4 13:38:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 97318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AF17C79F88 for ; Fri, 4 Sep 2026 13:38:28 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.msgproc01-g2.14076.1788529101185685236 for ; Fri, 04 Sep 2026 06:38:21 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@arm.com header.s=foss header.b=TV3QmtEZ; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0BA72152B for ; Fri, 4 Sep 2026 06:38:17 -0700 (PDT) Received: from cesw-amp-gbt-1s-m12830-04.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5CED23F7D8 for ; Fri, 4 Sep 2026 06:38:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1788529100; bh=PuDrsPA72b2M+Fe/H4N5AvOk015S+LfOHJN0yeUxZrw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=TV3QmtEZSED9hjRaGAG9/1mPvKGEHZl9M1JzANMXypx14TfcLwU7xiIqziPD6pfeV nU21ZWj7ClMNPBT3kDjGcimlAoyjYv7noqMq5mGw7xqGXiYwCVSkSET4MGn0yXE9t4 RitCp938DkjIyCO1ydgEbo4kNN8g0J5lWPuaNWyk= From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH v2 4/4] pypi: improve upstream check regex Date: Fri, 4 Sep 2026 14:38:15 +0100 Message-ID: <20260904133815.778303-4-ross.burton@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904133815.778303-1-ross.burton@arm.com> References: <20260904133815.778303-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from 45-33-107-173.ip.linodeusercontent.com [45.33.107.173] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 04 Sep 2026 13:38:28 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/245113 Currently the upstream release checker fetches the Simple API release index which lists all of the uploaded files. However the naming rules have evolved over time[1] so matching the value of PYPI_PACKAGE fails if the sdist filename was not normalised but then is normalised. Remove UPSTREAM_CHECK_PYPI_PACKAGE which is realistically unused[2] and instead transform the PYPI_PACKAGE into a regular expression that handles both - and _ being used in the filename. Change the version number match so that it handles .postN suffixes, as defined in the Version Specifiers documentation[3]. We don't want to match any alphanumeric characters as this would also match development snapshots and release candidates. Finally escape the . between the version and the extension, so this matches a literal . and not any character. Across oe-core and meta-oe this improves the upstream detection quite significantly: - 93 recipes move from a broken check to up-to-date - 21 recipes move from a broken check to needing an update - 30 recipes move from up-to-date to needing an update [1] Specifically PEP-0427 introduced normalisation of special characters to underscores and lowercasing. [2] No users in oe-core, three in meta-oe which are no longer needed after this change. [3] https://packaging.python.org/en/latest/specifications/version-specifiers/ Signed-off-by: Ross Burton --- meta/classes-recipe/pypi.bbclass | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/meta/classes-recipe/pypi.bbclass b/meta/classes-recipe/pypi.bbclass index eb13ffe7e71..08788fcb9a9 100644 --- a/meta/classes-recipe/pypi.bbclass +++ b/meta/classes-recipe/pypi.bbclass @@ -39,8 +39,7 @@ def pypi_src_uri(d): def pypi_normalize(d): """" - Normalize the package names to match PEP625 (https://peps.python.org/pep-0625/). - For non-compliant packages, maintainers can set UPSTREAM_CHECK_PYPI_PACKAGE to override the normalization + Normalize the package names to match PEP625 (https://peps.python.org/pep-0625/). """ import re return re.sub(r"[-_.]+", "-", d.getVar('PYPI_PACKAGE')).lower() @@ -52,7 +51,10 @@ SECTION = "devel/python" SRC_URI:prepend = "${PYPI_SRC_URI} " S = "${UNPACKDIR}/${PYPI_PACKAGE}-${PV}" -UPSTREAM_CHECK_PYPI_PACKAGE ?= "${PYPI_PACKAGE}" +def pypi_normalize_regex(d): + # Use a regex wildcard instead of hyphen as the filenames + # may or may not have been normalised properly. + return pypi_normalize(d).replace("-", "[_-]") # Use the simple repository API rather than the potentially unstable project URL # More information on the pypi API specification is avaialble here: @@ -60,7 +62,7 @@ UPSTREAM_CHECK_PYPI_PACKAGE ?= "${PYPI_PACKAGE}" # # NOTE: All URLs for the simple API MUST request canonical normalized URLs per the spec UPSTREAM_CHECK_URI ?= "https://pypi.org/simple/${@pypi_normalize(d)}/" -UPSTREAM_CHECK_REGEX ?= "(?i)${UPSTREAM_CHECK_PYPI_PACKAGE}-(?P(\d+[\.\-_]*)+).(tar\.gz|tgz|zip|tar\.bz2)" +UPSTREAM_CHECK_REGEX ?= "(?i)${@pypi_normalize_regex(d)}-(?P(\d+(\.[\d\-]+)*(\.post\d+)?))\.(tar\.gz|tgz|zip|tar\.bz2)" CVE_PRODUCT ?= "python:${PYPI_PACKAGE}"