| Message ID | 20260127091829.3500551-1-leon.anavi@konsulko.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [[PATCHv2] python3-pyzstd: Upgrade 0.17.0 -> 0.19.1 | expand |
On Tue Jan 27, 2026 at 10:18 AM CET, Leon Anavi via lists.openembedded.org wrote: > Upgrade to release 0.19.1: > > - Fix SeekableZstdFile write table entries on 32-bits architectures > when there is a huge number of entries > > From release 0.19.0: > > - The project has been completely refactored to use the Zstandard > implementation from the standard library (PEP-784) > - The refactor has some minor impact on public APIs, such as > changing the exception raised on invalid input > - Add backports.zstd dependency for Python before 3.14 > - Changes in build dependency: remove setuptools and C build > toolchain, add hatchling and hatch-vcs > - Remove git submodule usage > - Drop support for Python 3.9 and below > - Use ruff as formatter and linter > - Embed type hints in Python code, and check with mypy > > All ptests pass following python3 upgrade from 3.13.11 to 3.14.0. > > Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> > --- Hi Leon, Thanks for the new version. It looks like this is breaking some ptest: Failed ptests: {'python3-pyzstd': 'START: ptest-runner\n' '2026-01-28T16:27\n' '\n' ... "E ImportError: cannot import name 'zstd' from " "'compression' (unknown location)\n" https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2996 https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/1005 Test results can be found here: https://valkyrie.yocto.io/pub/non-release/20260128-86/testresults/qemux86-64-ptest/core-image-ptest-python3-pyzstd/ Test results for riscv64 and arm64 are not published at this time, as some tests are still running. They will all end-up in https://valkyrie.yocto.io/pub/non-release/20260128-86/testresults/ Can you have a look at these failures? Thanks, Mathieu
Hi Mathieu, On Wed, Jan 28, 2026 at 7:03 PM Mathieu Dubois-Briand < mathieu.dubois-briand@bootlin.com> wrote: > On Tue Jan 27, 2026 at 10:18 AM CET, Leon Anavi via lists.openembedded.org > wrote: > > Upgrade to release 0.19.1: > > > > - Fix SeekableZstdFile write table entries on 32-bits architectures > > when there is a huge number of entries > > > > From release 0.19.0: > > > > - The project has been completely refactored to use the Zstandard > > implementation from the standard library (PEP-784) > > - The refactor has some minor impact on public APIs, such as > > changing the exception raised on invalid input > > - Add backports.zstd dependency for Python before 3.14 > > - Changes in build dependency: remove setuptools and C build > > toolchain, add hatchling and hatch-vcs > > - Remove git submodule usage > > - Drop support for Python 3.9 and below > > - Use ruff as formatter and linter > > - Embed type hints in Python code, and check with mypy > > > > All ptests pass following python3 upgrade from 3.13.11 to 3.14.0. > > > > Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> > > --- > > Hi Leon, > > Thanks for the new version. > > It looks like this is breaking some ptest: > > Failed ptests: > {'python3-pyzstd': 'START: ptest-runner\n' > '2026-01-28T16:27\n' > '\n' > ... > "E ImportError: cannot import name 'zstd' from " > "'compression' (unknown location)\n" > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2996 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/1005 > > Test results can be found here: > > > https://valkyrie.yocto.io/pub/non-release/20260128-86/testresults/qemux86-64-ptest/core-image-ptest-python3-pyzstd/ > Test results for riscv64 and arm64 are not published at this time, as > some tests are still running. They will all end-up in > https://valkyrie.yocto.io/pub/non-release/20260128-86/testresults/ > > Can you have a look at these failures? > Hm, I am not exactly sure what's going on. On my Raspberry Pi 5 test setup ptest-runner python3-pyzstd doesn't report any issues: Testsuite summary # TOTAL: 209 # PASS: 209 # SKIP: 0 # XFAIL: 0 # FAIL: 0 # XPASS: 0 # ERROR: 0 DURATION: 4 END: /usr/lib/python3-pyzstd/ptest 2026-01-29T11:17 STOP: ptest-runner TOTAL: 1 FAIL: 0 I will take another look after FOSDEM. I will likely need to reproduce your exact test environment. In the meantime, do you have any hints about what might be causing the test failures on your side? Best regards, Leon > Thanks, > Mathieu > > -- > Mathieu Dubois-Briand, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com > >
On Thu Jan 29, 2026 at 12:22 PM CET, Leon Anavi wrote: > Hi Mathieu, > > On Wed, Jan 28, 2026 at 7:03 PM Mathieu Dubois-Briand < > mathieu.dubois-briand@bootlin.com> wrote: > >> On Tue Jan 27, 2026 at 10:18 AM CET, Leon Anavi via lists.openembedded.org >> wrote: >> > Upgrade to release 0.19.1: >> > >> > - Fix SeekableZstdFile write table entries on 32-bits architectures >> > when there is a huge number of entries >> > >> > From release 0.19.0: >> > >> > - The project has been completely refactored to use the Zstandard >> > implementation from the standard library (PEP-784) >> > - The refactor has some minor impact on public APIs, such as >> > changing the exception raised on invalid input >> > - Add backports.zstd dependency for Python before 3.14 >> > - Changes in build dependency: remove setuptools and C build >> > toolchain, add hatchling and hatch-vcs >> > - Remove git submodule usage >> > - Drop support for Python 3.9 and below >> > - Use ruff as formatter and linter >> > - Embed type hints in Python code, and check with mypy >> > >> > All ptests pass following python3 upgrade from 3.13.11 to 3.14.0. >> > >> > Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> >> > --- >> >> Hi Leon, >> >> Thanks for the new version. >> >> It looks like this is breaking some ptest: >> >> Failed ptests: >> {'python3-pyzstd': 'START: ptest-runner\n' >> '2026-01-28T16:27\n' >> '\n' >> ... >> "E ImportError: cannot import name 'zstd' from " >> "'compression' (unknown location)\n" >> >> https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2996 >> https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/1005 >> >> Test results can be found here: >> >> >> https://valkyrie.yocto.io/pub/non-release/20260128-86/testresults/qemux86-64-ptest/core-image-ptest-python3-pyzstd/ >> Test results for riscv64 and arm64 are not published at this time, as >> some tests are still running. They will all end-up in >> https://valkyrie.yocto.io/pub/non-release/20260128-86/testresults/ >> >> Can you have a look at these failures? >> > > Hm, I am not exactly sure what's going on. On my Raspberry Pi 5 test > setup ptest-runner python3-pyzstd doesn't report any issues: > > Testsuite summary > # TOTAL: 209 > # PASS: 209 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 0 > # XPASS: 0 > # ERROR: 0 > DURATION: 4 > END: /usr/lib/python3-pyzstd/ptest > 2026-01-29T11:17 > STOP: ptest-runner > TOTAL: 1 FAIL: 0 > > I will take another look after FOSDEM. I will likely need to reproduce your > exact test environment. In the meantime, do you have any hints about what > might be causing the test failures on your side? > > Best regards, Leon > > Hi Leon, I'm not really sure about what is the issue here, but maybe a different configuration or a missing dependency. If you need any data about how to reproduce it in qemu, all data should be in the build, particularly in the "Write config" step: https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/5139973/raw_inline So basically: SDKMACHINE = "x86_64" TEST_SUITES = 'ping ssh ptest' SANITY_TESTED_DISTROS = '' EXTRA_IMAGE_FEATURES ?= 'allow-empty-password empty-root-password allow-root-login' bitbake-config-build enable-fragment machine/qemux86-64 bitbake-config-build enable-fragment distro/poky bitbake core-image-ptest-python3-pyzstd bitbake -c do_testimage core-image-ptest-python3-pyzstd Please tell me if you have any issue reproducing the failure. Thanks, Mathieu
diff --git a/meta/recipes-devtools/python/python3-pyzstd/0001-Remove-setuptools-version-limit-of-74.patch b/meta/recipes-devtools/python/python3-pyzstd/0001-Remove-setuptools-version-limit-of-74.patch deleted file mode 100644 index f2910322ff..0000000000 --- a/meta/recipes-devtools/python/python3-pyzstd/0001-Remove-setuptools-version-limit-of-74.patch +++ /dev/null @@ -1,27 +0,0 @@ -From bc9b975c9f41e43481a2eb0623b4180926baecec Mon Sep 17 00:00:00 2001 -From: Khem Raj <raj.khem@gmail.com> -Date: Wed, 9 Apr 2025 14:38:31 -0700 -Subject: [PATCH] Remove setuptools version limit of '74' - -Upstream sticks to 0.74 since setuptools have dropped msvc9compiler -support beyond that which is needed for python 3.9 on windows. We -do not have this problem on Linux - -Upstream-Status: Inappropriate [OE-specific] -Signed-off-by: Khem Raj <raj.khem@gmail.com> ---- - pyproject.toml | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/pyproject.toml b/pyproject.toml -index dd885e4..5d4d511 100644 ---- a/pyproject.toml -+++ b/pyproject.toml -@@ -1,6 +1,6 @@ - [build-system] - # setuptools 64+ support --build-option - # setuptools 74+ drops distutils.msvc9compiler required for Python 3.9 under Windows --requires = ["setuptools>=64,<74"] -+requires = ["setuptools>=64"] - backend-path = ["build_script"] - build-backend = "pyzstd_pep517" diff --git a/meta/recipes-devtools/python/python3-pyzstd_0.17.0.bb b/meta/recipes-devtools/python/python3-pyzstd_0.19.1.bb similarity index 53% rename from meta/recipes-devtools/python/python3-pyzstd_0.17.0.bb rename to meta/recipes-devtools/python/python3-pyzstd_0.19.1.bb index af7069b6ed..df1e4d49d2 100644 --- a/meta/recipes-devtools/python/python3-pyzstd_0.17.0.bb +++ b/meta/recipes-devtools/python/python3-pyzstd_0.19.1.bb @@ -9,12 +9,8 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=aedb5a2679cd1552fb61c181ef974b9e" PYPI_PACKAGE = "pyzstd" -SRC_URI += "file://0001-Remove-setuptools-version-limit-of-74.patch" -SRC_URI[sha256sum] = "d84271f8baa66c419204c1dd115a4dec8b266f8a2921da21b81764fa208c1db6" +SRC_URI[sha256sum] = "36723d3c915b3981de9198d0a2c82b2f5fe3eaa36e4d8d586937830a8afc7d72" -inherit pypi python_setuptools_build_meta ptest-python-pytest +inherit pypi python_hatchling ptest-python-pytest -# clang-16 with -flto segfaults on arm, therefore ignore flto for now -do_configure:append:arm:toolchain-clang() { - sed -i -e "s|'-flto'|''|" ${S}/setup.py -} +DEPENDS += "python3-hatch-vcs-native"
Upgrade to release 0.19.1: - Fix SeekableZstdFile write table entries on 32-bits architectures when there is a huge number of entries From release 0.19.0: - The project has been completely refactored to use the Zstandard implementation from the standard library (PEP-784) - The refactor has some minor impact on public APIs, such as changing the exception raised on invalid input - Add backports.zstd dependency for Python before 3.14 - Changes in build dependency: remove setuptools and C build toolchain, add hatchling and hatch-vcs - Remove git submodule usage - Drop support for Python 3.9 and below - Use ruff as formatter and linter - Embed type hints in Python code, and check with mypy All ptests pass following python3 upgrade from 3.13.11 to 3.14.0. Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> --- ...emove-setuptools-version-limit-of-74.patch | 27 ------------------- ...std_0.17.0.bb => python3-pyzstd_0.19.1.bb} | 10 +++---- 2 files changed, 3 insertions(+), 34 deletions(-) delete mode 100644 meta/recipes-devtools/python/python3-pyzstd/0001-Remove-setuptools-version-limit-of-74.patch rename meta/recipes-devtools/python/{python3-pyzstd_0.17.0.bb => python3-pyzstd_0.19.1.bb} (53%)