| Message ID | 20251219134734.842422-1-tgamblin@baylibre.com |
|---|---|
| Headers | show |
| Series | python3: upgrade 3.13.9 -> 3.14.2 | expand |
On Fri Dec 19, 2025 at 2:47 PM CET, Trevor Gamblin wrote: > Compared to v1 > (https://lists.openembedded.org/g/openembedded-core/message/227955): > > - Drop python3-bcrypt upgrade, as someone else has submitted that; > - Drop rpm patch to manually override Python3_SITEARCH with a patch, as that was > masking the real issue (see below); > - Add a new patch to fix python3targetconfig by exporting _PYTHON_PROJECT_BASE, > which Python's sysconfig looks for (as of 3.14.0) when doing > cross-compilation. The absence of this was resulting in packages like rpm > finding paths for the native interpreter when making use of sysconfig, when > they needed the host values. > > Note that I've dropped my own version of the python3-bcrypt upgrade, but I > tested this patch series on top of the other one that was submitted, as > otherwise the ptest images would break during build (version mismatch errors > around PyO3, which is fixed with the bcrypt upgrade). > > v1 details below. > > This supersedes a previous series sent, which only did the 3.14.0 upgrade. > > Compared to the first series > (https://lists.openembedded.org/g/openembedded-core/message/227310), I've > done the following: > > - Added a commit for including the '_py_warnings' and 'annotationlib' modules in > python3-core; > - Adds the python 3.14.2 upgrade; > - Removed commits for adding python3-misc to RDEPENDS for python3-pytest and > python3-jsonpointer, as these are now solved by a new patch to add new modules > to python3-core in the manifest. Doing it this way was fragile in the sense > that there were many other recipes needing this same fix that testing didn't > easily catch, and it doesn't make sense to add python3-misc to recipe RDEPENDS > just because (for example) python3-logging (another package split) is trying > to make use of warnings, but can't find the right module; > - Modified the rpm commit to apply a patch (submitted upstream) to fix > CMakeLists.txt, rather than a do_configure prepend; > - Fixed the Python 3.14.0 upgrade commit so that one of the patches we carry to > skip tests doesn't include duplicate calls to the relevant skip functions. > > I've kept the upgrades from 3.13.9 to 3.14.0 and 3.14.0 to 3.14.2 separate to > reflect separate testing and let people review those changelogs individually if > desired. > > Trevor Gamblin (5): > python3: upgrade 3.13.9 -> 3.14.0 > python3targetconfig: export _PYTHON_PROJECT_BASE > python_pep517: add --prefix to nativepython3 call > python3: upgrade 3.14.0 -> 3.14.2 > python3: add _py_warnings, annotationlib to core > Hi Trevor, Thanks for the new version. It looks like we are having issues with maturin and click. Testing 'maturin develop' requires Rust cross-canadian in the SDK Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/qemux86/build/layers/openembedded-core/meta/lib/oeqa/sdk/cases/maturin.py", line 30, in test_maturin_list_python self.assertIn(expected, output) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ AssertionError: '3.14 at /srv/pokybuild/yocto-worker/qemux86/build/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3' not found in '
On 2025-12-22 03:11, Mathieu Dubois-Briand wrote: > On Fri Dec 19, 2025 at 2:47 PM CET, Trevor Gamblin wrote: >> Compared to v1 >> (https://lists.openembedded.org/g/openembedded-core/message/227955): >> >> - Drop python3-bcrypt upgrade, as someone else has submitted that; >> - Drop rpm patch to manually override Python3_SITEARCH with a patch, as that was >> masking the real issue (see below); >> - Add a new patch to fix python3targetconfig by exporting _PYTHON_PROJECT_BASE, >> which Python's sysconfig looks for (as of 3.14.0) when doing >> cross-compilation. The absence of this was resulting in packages like rpm >> finding paths for the native interpreter when making use of sysconfig, when >> they needed the host values. >> >> Note that I've dropped my own version of the python3-bcrypt upgrade, but I >> tested this patch series on top of the other one that was submitted, as >> otherwise the ptest images would break during build (version mismatch errors >> around PyO3, which is fixed with the bcrypt upgrade). >> >> v1 details below. >> >> This supersedes a previous series sent, which only did the 3.14.0 upgrade. >> >> Compared to the first series >> (https://lists.openembedded.org/g/openembedded-core/message/227310), I've >> done the following: >> >> - Added a commit for including the '_py_warnings' and 'annotationlib' modules in >> python3-core; >> - Adds the python 3.14.2 upgrade; >> - Removed commits for adding python3-misc to RDEPENDS for python3-pytest and >> python3-jsonpointer, as these are now solved by a new patch to add new modules >> to python3-core in the manifest. Doing it this way was fragile in the sense >> that there were many other recipes needing this same fix that testing didn't >> easily catch, and it doesn't make sense to add python3-misc to recipe RDEPENDS >> just because (for example) python3-logging (another package split) is trying >> to make use of warnings, but can't find the right module; >> - Modified the rpm commit to apply a patch (submitted upstream) to fix >> CMakeLists.txt, rather than a do_configure prepend; >> - Fixed the Python 3.14.0 upgrade commit so that one of the patches we carry to >> skip tests doesn't include duplicate calls to the relevant skip functions. >> >> I've kept the upgrades from 3.13.9 to 3.14.0 and 3.14.0 to 3.14.2 separate to >> reflect separate testing and let people review those changelogs individually if >> desired. >> >> Trevor Gamblin (5): >> python3: upgrade 3.13.9 -> 3.14.0 >> python3targetconfig: export _PYTHON_PROJECT_BASE >> python_pep517: add --prefix to nativepython3 call >> python3: upgrade 3.14.0 -> 3.14.2 >> python3: add _py_warnings, annotationlib to core >> > Hi Trevor, > > Thanks for the new version. It looks like we are having issues with > maturin and click. > > Testing 'maturin develop' requires Rust cross-canadian in the SDK > Traceback (most recent call last): > File "/srv/pokybuild/yocto-worker/qemux86/build/layers/openembedded-core/meta/lib/oeqa/sdk/cases/maturin.py", line 30, in test_maturin_list_python > self.assertIn(expected, output) > ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ > AssertionError: '3.14 at /srv/pokybuild/yocto-worker/qemux86/build/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3' not found in '
On Mon, 22 Dec 2025 at 19:36, Trevor Gamblin via lists.openembedded.org <tgamblin=baylibre.com@lists.openembedded.org> wrote: > > Testing 'maturin develop' requires Rust cross-canadian in the SDK > > Traceback (most recent call last): > > File "/srv/pokybuild/yocto-worker/qemux86/build/layers/openembedded-core/meta/lib/oeqa/sdk/cases/maturin.py", line 30, in test_maturin_list_python > > self.assertIn(expected, output) > > ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ > > AssertionError: '3.14 at /srv/pokybuild/yocto-worker/qemux86/build/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3' not found in '
On 2025-12-22 14:33, Alexander Kanavin wrote: > On Mon, 22 Dec 2025 at 19:36, Trevor Gamblin via > lists.openembedded.org <tgamblin=baylibre.com@lists.openembedded.org> > wrote: >>> Testing 'maturin develop' requires Rust cross-canadian in the SDK >>> Traceback (most recent call last): >>> File "/srv/pokybuild/yocto-worker/qemux86/build/layers/openembedded-core/meta/lib/oeqa/sdk/cases/maturin.py", line 30, in test_maturin_list_python >>> self.assertIn(expected, output) >>> ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ >>> AssertionError: '3.14 at /srv/pokybuild/yocto-worker/qemux86/build/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3' not found in '
On Mon, 22 Dec 2025 at 20:52, Trevor Gamblin <tgamblin@baylibre.com> wrote: > Maybe. I tried all of this locally (on a F43 system, which includes > 3.14.2 as of writing), and after I sourced the pokysdk script, the > interpreter on my path changed to version 3.13.4. Seems that the > install-buildtools script is (at least for me) resulting in a 3.13.4 > version is getting pulled in. I'd think maybe something got contaminated > somehow from my cache, but there'd be newer 3.13.x versions for it to > grab if that were the case. install-buildtools brings in python 3.13 into PATH, as that's what is inside the buildtools tarball. That is expected and won't change until after 3.14 lands in core, and new buildtools is produced with it. The mystery is why the test first gets 3.14 from running python3 (that comes from the sdk installation), then right after that 3.13 from running maturin (that comes from buildtools). Alex
On 2025-12-22 15:00, Alexander Kanavin wrote: > On Mon, 22 Dec 2025 at 20:52, Trevor Gamblin <tgamblin@baylibre.com> wrote: > >> Maybe. I tried all of this locally (on a F43 system, which includes >> 3.14.2 as of writing), and after I sourced the pokysdk script, the >> interpreter on my path changed to version 3.13.4. Seems that the >> install-buildtools script is (at least for me) resulting in a 3.13.4 >> version is getting pulled in. I'd think maybe something got contaminated >> somehow from my cache, but there'd be newer 3.13.x versions for it to >> grab if that were the case. > install-buildtools brings in python 3.13 into PATH, as that's what is > inside the buildtools tarball. That is expected and won't change until > after 3.14 lands in core, and new buildtools is produced with it. > > The mystery is why the test first gets 3.14 from running python3 (that > comes from the sdk installation), then right after that 3.13 from > running maturin (that comes from buildtools). Ross and/or Tim, any thoughts? > > Alex
On Mon, 5 Jan 2026 at 15:18, Trevor Gamblin <tgamblin@baylibre.com> wrote: > >> Maybe. I tried all of this locally (on a F43 system, which includes > >> 3.14.2 as of writing), and after I sourced the pokysdk script, the > >> interpreter on my path changed to version 3.13.4. Seems that the > >> install-buildtools script is (at least for me) resulting in a 3.13.4 > >> version is getting pulled in. I'd think maybe something got contaminated > >> somehow from my cache, but there'd be newer 3.13.x versions for it to > >> grab if that were the case. > > install-buildtools brings in python 3.13 into PATH, as that's what is > > inside the buildtools tarball. That is expected and won't change until > > after 3.14 lands in core, and new buildtools is produced with it. > > > > The mystery is why the test first gets 3.14 from running python3 (that > > comes from the sdk installation), then right after that 3.13 from > > running maturin (that comes from buildtools). > Ross and/or Tim, any thoughts? I looked into it a bit. I believe the issue was outdated maturin, with the 1.9.4 -> 1.10.2 update having landed in master it goes away. This patchset can be re-taken into testing without modifications. Alex
On 2026-01-06 04:39, Alexander Kanavin wrote: > On Mon, 5 Jan 2026 at 15:18, Trevor Gamblin <tgamblin@baylibre.com> wrote: >>>> Maybe. I tried all of this locally (on a F43 system, which includes >>>> 3.14.2 as of writing), and after I sourced the pokysdk script, the >>>> interpreter on my path changed to version 3.13.4. Seems that the >>>> install-buildtools script is (at least for me) resulting in a 3.13.4 >>>> version is getting pulled in. I'd think maybe something got contaminated >>>> somehow from my cache, but there'd be newer 3.13.x versions for it to >>>> grab if that were the case. >>> install-buildtools brings in python 3.13 into PATH, as that's what is >>> inside the buildtools tarball. That is expected and won't change until >>> after 3.14 lands in core, and new buildtools is produced with it. >>> >>> The mystery is why the test first gets 3.14 from running python3 (that >>> comes from the sdk installation), then right after that 3.13 from >>> running maturin (that comes from buildtools). >> Ross and/or Tim, any thoughts? > I looked into it a bit. I believe the issue was outdated maturin, with > the 1.9.4 -> 1.10.2 update having landed in master it goes away. This > patchset can be re-taken into testing without modifications. Thanks for looking at this again. I did see that there was an upgrade submitted to move python from 3.13.9 -> 3.13.11, so if that goes in first then I'll still need to rebase this. > > Alex
On 2026-01-06 14:22, Trevor Gamblin via lists.openembedded.org wrote: > > On 2026-01-06 04:39, Alexander Kanavin wrote: >> On Mon, 5 Jan 2026 at 15:18, Trevor Gamblin <tgamblin@baylibre.com> >> wrote: >>>>> Maybe. I tried all of this locally (on a F43 system, which includes >>>>> 3.14.2 as of writing), and after I sourced the pokysdk script, the >>>>> interpreter on my path changed to version 3.13.4. Seems that the >>>>> install-buildtools script is (at least for me) resulting in a 3.13.4 >>>>> version is getting pulled in. I'd think maybe something got >>>>> contaminated >>>>> somehow from my cache, but there'd be newer 3.13.x versions for it to >>>>> grab if that were the case. >>>> install-buildtools brings in python 3.13 into PATH, as that's what is >>>> inside the buildtools tarball. That is expected and won't change until >>>> after 3.14 lands in core, and new buildtools is produced with it. >>>> >>>> The mystery is why the test first gets 3.14 from running python3 (that >>>> comes from the sdk installation), then right after that 3.13 from >>>> running maturin (that comes from buildtools). >>> Ross and/or Tim, any thoughts? >> I looked into it a bit. I believe the issue was outdated maturin, with >> the 1.9.4 -> 1.10.2 update having landed in master it goes away. This >> patchset can be re-taken into testing without modifications. > Thanks for looking at this again. > > I did see that there was an upgrade submitted to move python from > 3.13.9 -> 3.13.11, so if that goes in first then I'll still need to > rebase this. That upgrade is now on master-next, so I will rebase and re-send this series. - Trevor >> >> Alex > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#228922): https://lists.openembedded.org/g/openembedded-core/message/228922 > Mute This Topic: https://lists.openembedded.org/mt/116859136/7611679 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [tgamblin@baylibre.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Compared to v1 (https://lists.openembedded.org/g/openembedded-core/message/227955): - Drop python3-bcrypt upgrade, as someone else has submitted that; - Drop rpm patch to manually override Python3_SITEARCH with a patch, as that was masking the real issue (see below); - Add a new patch to fix python3targetconfig by exporting _PYTHON_PROJECT_BASE, which Python's sysconfig looks for (as of 3.14.0) when doing cross-compilation. The absence of this was resulting in packages like rpm finding paths for the native interpreter when making use of sysconfig, when they needed the host values. Note that I've dropped my own version of the python3-bcrypt upgrade, but I tested this patch series on top of the other one that was submitted, as otherwise the ptest images would break during build (version mismatch errors around PyO3, which is fixed with the bcrypt upgrade). v1 details below. This supersedes a previous series sent, which only did the 3.14.0 upgrade. Compared to the first series (https://lists.openembedded.org/g/openembedded-core/message/227310), I've done the following: - Added a commit for including the '_py_warnings' and 'annotationlib' modules in python3-core; - Adds the python 3.14.2 upgrade; - Removed commits for adding python3-misc to RDEPENDS for python3-pytest and python3-jsonpointer, as these are now solved by a new patch to add new modules to python3-core in the manifest. Doing it this way was fragile in the sense that there were many other recipes needing this same fix that testing didn't easily catch, and it doesn't make sense to add python3-misc to recipe RDEPENDS just because (for example) python3-logging (another package split) is trying to make use of warnings, but can't find the right module; - Modified the rpm commit to apply a patch (submitted upstream) to fix CMakeLists.txt, rather than a do_configure prepend; - Fixed the Python 3.14.0 upgrade commit so that one of the patches we carry to skip tests doesn't include duplicate calls to the relevant skip functions. I've kept the upgrades from 3.13.9 to 3.14.0 and 3.14.0 to 3.14.2 separate to reflect separate testing and let people review those changelogs individually if desired. Trevor Gamblin (5): python3: upgrade 3.13.9 -> 3.14.0 python3targetconfig: export _PYTHON_PROJECT_BASE python_pep517: add --prefix to nativepython3 call python3: upgrade 3.14.0 -> 3.14.2 python3: add _py_warnings, annotationlib to core meta/classes-recipe/python3-dir.bbclass | 2 +- .../python3targetconfig.bbclass | 1 + meta/classes-recipe/python_pep517.bbclass | 2 +- ...r-OpenSSL-3.4-and-add-it-to-multissl.patch | 1452 ----------------- ...-use-prefix-value-from-build-configu.patch | 35 +- ...sts-due-to-load-variability-on-YP-AB.patch | 31 +- ...ctive_children-skip-problematic-test.patch | 17 +- ...1-test_cmd-skip-bang-completion-test.patch | 31 + ...-test_unix_console.test_cursor_back_.patch | 46 + ...kip-test_sysconfig.test_sysconfigdat.patch | 32 + .../python/python3/python3-manifest.json | 5 +- .../{python3_3.13.9.bb => python3_3.14.2.bb} | 20 +- 12 files changed, 187 insertions(+), 1487 deletions(-) delete mode 100644 meta/recipes-devtools/python/python3/0001-Generate-data-for-OpenSSL-3.4-and-add-it-to-multissl.patch create mode 100644 meta/recipes-devtools/python/python3/0001-test_cmd-skip-bang-completion-test.patch create mode 100644 meta/recipes-devtools/python/python3/0001-test_pyrepl-skip-test_unix_console.test_cursor_back_.patch create mode 100644 meta/recipes-devtools/python/python3/0001-test_sysconfig-skip-test_sysconfig.test_sysconfigdat.patch rename meta/recipes-devtools/python/{python3_3.13.9.bb => python3_3.14.2.bb} (95%)