| Message ID | 20241224221625.4092917-1-javier.tia@linaro.org |
|---|---|
| State | Rejected |
| Headers | show |
| Series | Set max supported Python version | expand |
On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via lists.yoctoproject.org wrote: > When Bitbake is used with Python 3.13, it generates numerous build > issues that do not occur with Python 3.12. The documentation is clear > for the supported Linux distributions, but not for the supported Python > versions. > > Signed-off-by: Javier Tia <javier.tia@linaro.org> Can you be specific about which version of bitbake you saw the issues with? I was hoping we had resolved most of the issues with 3.13, at least in master. We do aim to work with new versions of python but it obviously takes time to catch up and for fixes to make older versions. Cheers, Richard
Hi Richard, On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via lists.yoctoproject.org wrote: >> When Bitbake is used with Python 3.13, it generates numerous build >> issues that do not occur with Python 3.12. The documentation is clear >> for the supported Linux distributions, but not for the supported Python >> versions. >> >> Signed-off-by: Javier Tia <javier.tia@linaro.org> > > Can you be specific about which version of bitbake you saw the issues > with? I was hoping we had resolved most of the issues with 3.13, at > least in master. We do aim to work with new versions of python but it > obviously takes time to catch up and for fixes to make older versions. With Bitbake from poky master (284a08b6e40761f) and BitBake from the latest master (c9906bb289dcfd9). Python 3.13.1. Issues: - "ERROR: ParseError in None: Not all recipes parsed, parser thread killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, but I reopened the bug because I believe it is still not fixed. - The problem mentioned on https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes from other layers. Please let me know if more information is needed. > Cheers, > > Richard
On Tue, 2024-12-24 at 17:13 -0600, Javier Tia wrote: > On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: > > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via > > lists.yoctoproject.org wrote: > > > When Bitbake is used with Python 3.13, it generates numerous > > > build > > > issues that do not occur with Python 3.12. The documentation is > > > clear > > > for the supported Linux distributions, but not for the supported > > > Python > > > versions. > > > > > > Signed-off-by: Javier Tia <javier.tia@linaro.org> > > > > Can you be specific about which version of bitbake you saw the > > issues > > with? I was hoping we had resolved most of the issues with 3.13, at > > least in master. We do aim to work with new versions of python but > > it > > obviously takes time to catch up and for fixes to make older > > versions. > > With Bitbake from poky master (284a08b6e40761f) and BitBake from the > latest master (c9906bb289dcfd9). Python 3.13.1. > > Issues: > > - "ERROR: ParseError in None: Not all recipes parsed, parser thread > killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, > but I reopened the bug because I believe it is still not fixed. > > - The problem mentioned on > https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky > recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes > from other layers. > > Please let me know if more information is needed. We have a fedora41 worker in the autobuilder cluster which has: fedora41-vk-1:~$ python3 --version Python 3.13.0 As you can see here: https://valkyrie.yoctoproject.org/#/workers/54 is it running builds successfully. There is therefore some difference in your environment and I'm not sure it is python 3.13 that is the cause of the problems you're seeing. Are there any tracebacks in bitbake-cookerdaemon.log? Which distro is this? The pseudo aborts are likely a different problem with a missing libc intercept or similar issue and secondary to the other problem. Cheers, Richard
FWIW: I'm using gentoo with 3.13.1 and I'm not seeing these issues since pseudo was updated. On Wed, Dec 25, 2024 at 12:22 AM Richard Purdie via lists.yoctoproject.org <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote: > > On Tue, 2024-12-24 at 17:13 -0600, Javier Tia wrote: > > On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: > > > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via > > > lists.yoctoproject.org wrote: > > > > When Bitbake is used with Python 3.13, it generates numerous > > > > build > > > > issues that do not occur with Python 3.12. The documentation is > > > > clear > > > > for the supported Linux distributions, but not for the supported > > > > Python > > > > versions. > > > > > > > > Signed-off-by: Javier Tia <javier.tia@linaro.org> > > > > > > Can you be specific about which version of bitbake you saw the > > > issues > > > with? I was hoping we had resolved most of the issues with 3.13, at > > > least in master. We do aim to work with new versions of python but > > > it > > > obviously takes time to catch up and for fixes to make older > > > versions. > > > > With Bitbake from poky master (284a08b6e40761f) and BitBake from the > > latest master (c9906bb289dcfd9). Python 3.13.1. > > > > Issues: > > > > - "ERROR: ParseError in None: Not all recipes parsed, parser thread > > killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, > > but I reopened the bug because I believe it is still not fixed. > > > > - The problem mentioned on > > https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky > > recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes > > from other layers. > > > > Please let me know if more information is needed. > > We have a fedora41 worker in the autobuilder cluster which has: > > fedora41-vk-1:~$ python3 --version > Python 3.13.0 > > As you can see here: > > https://valkyrie.yoctoproject.org/#/workers/54 > > is it running builds successfully. There is therefore some difference > in your environment and I'm not sure it is python 3.13 that is the > cause of the problems you're seeing. > > Are there any tracebacks in bitbake-cookerdaemon.log? Which distro is > this? > > The pseudo aborts are likely a different problem with a missing libc > intercept or similar issue and secondary to the other problem. > > Cheers, > > Richard > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#6028): https://lists.yoctoproject.org/g/docs/message/6028 > Mute This Topic: https://lists.yoctoproject.org/mt/110278405/3617156 > Group Owner: docs+owner@lists.yoctoproject.org > Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [martin.jansa@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hi Martin, On Tue, Dec 24, 2024, at 5:34 PM, Martin Jansa wrote: > FWIW: I'm using gentoo with 3.13.1 and I'm not seeing these issues > since pseudo was updated. Yes, I updated pseudo [1], but all build issues happened: [1] bitbake pseudo-native -c cleanall && bitbake pseudo-native. Thanks, > On Wed, Dec 25, 2024 at 12:22 AM Richard Purdie via > lists.yoctoproject.org > <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote: >> >> On Tue, 2024-12-24 at 17:13 -0600, Javier Tia wrote: >> > On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: >> > > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via >> > > lists.yoctoproject.org wrote: >> > > > When Bitbake is used with Python 3.13, it generates numerous >> > > > build >> > > > issues that do not occur with Python 3.12. The documentation is >> > > > clear >> > > > for the supported Linux distributions, but not for the supported >> > > > Python >> > > > versions. >> > > > >> > > > Signed-off-by: Javier Tia <javier.tia@linaro.org> >> > > >> > > Can you be specific about which version of bitbake you saw the >> > > issues >> > > with? I was hoping we had resolved most of the issues with 3.13, at >> > > least in master. We do aim to work with new versions of python but >> > > it >> > > obviously takes time to catch up and for fixes to make older >> > > versions. >> > >> > With Bitbake from poky master (284a08b6e40761f) and BitBake from the >> > latest master (c9906bb289dcfd9). Python 3.13.1. >> > >> > Issues: >> > >> > - "ERROR: ParseError in None: Not all recipes parsed, parser thread >> > killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, >> > but I reopened the bug because I believe it is still not fixed. >> > >> > - The problem mentioned on >> > https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky >> > recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes >> > from other layers. >> > >> > Please let me know if more information is needed. >> >> We have a fedora41 worker in the autobuilder cluster which has: >> >> fedora41-vk-1:~$ python3 --version >> Python 3.13.0 >> >> As you can see here: >> >> https://valkyrie.yoctoproject.org/#/workers/54 >> >> is it running builds successfully. There is therefore some difference >> in your environment and I'm not sure it is python 3.13 that is the >> cause of the problems you're seeing. >> >> Are there any tracebacks in bitbake-cookerdaemon.log? Which distro is >> this? >> >> The pseudo aborts are likely a different problem with a missing libc >> intercept or similar issue and secondary to the other problem. >> >> Cheers, >> >> Richard
Hi Javier, yeah I believe you. I just wanted to show that not all 3.13.1 pythons trigger this issue and it's important to figure out what's different on your system. e.g. in gentoo I'm disabling pgo in python since 3.13 due to some issues (detected in python test suite already). I meant this commit (which you already have unless you're using more layers and one of them changes SRCREV in pseudo recipe - which we also had before in one of our layers): https://git.openembedded.org/openembedded-core/commit/?id=d80e20d70d170397f9827c5a5fc75ad1f2e8cd94 Cheers, On Wed, Dec 25, 2024 at 12:39 AM Javier Tia <javier.tia@linaro.org> wrote: > > Hi Martin, > > On Tue, Dec 24, 2024, at 5:34 PM, Martin Jansa wrote: > > FWIW: I'm using gentoo with 3.13.1 and I'm not seeing these issues > > since pseudo was updated. > > Yes, I updated pseudo [1], but all build issues happened: > > [1] bitbake pseudo-native -c cleanall && bitbake pseudo-native. > > Thanks, > > > On Wed, Dec 25, 2024 at 12:22 AM Richard Purdie via > > lists.yoctoproject.org > > <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote: > >> > >> On Tue, 2024-12-24 at 17:13 -0600, Javier Tia wrote: > >> > On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: > >> > > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via > >> > > lists.yoctoproject.org wrote: > >> > > > When Bitbake is used with Python 3.13, it generates numerous > >> > > > build > >> > > > issues that do not occur with Python 3.12. The documentation is > >> > > > clear > >> > > > for the supported Linux distributions, but not for the supported > >> > > > Python > >> > > > versions. > >> > > > > >> > > > Signed-off-by: Javier Tia <javier.tia@linaro.org> > >> > > > >> > > Can you be specific about which version of bitbake you saw the > >> > > issues > >> > > with? I was hoping we had resolved most of the issues with 3.13, at > >> > > least in master. We do aim to work with new versions of python but > >> > > it > >> > > obviously takes time to catch up and for fixes to make older > >> > > versions. > >> > > >> > With Bitbake from poky master (284a08b6e40761f) and BitBake from the > >> > latest master (c9906bb289dcfd9). Python 3.13.1. > >> > > >> > Issues: > >> > > >> > - "ERROR: ParseError in None: Not all recipes parsed, parser thread > >> > killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, > >> > but I reopened the bug because I believe it is still not fixed. > >> > > >> > - The problem mentioned on > >> > https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky > >> > recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes > >> > from other layers. > >> > > >> > Please let me know if more information is needed. > >> > >> We have a fedora41 worker in the autobuilder cluster which has: > >> > >> fedora41-vk-1:~$ python3 --version > >> Python 3.13.0 > >> > >> As you can see here: > >> > >> https://valkyrie.yoctoproject.org/#/workers/54 > >> > >> is it running builds successfully. There is therefore some difference > >> in your environment and I'm not sure it is python 3.13 that is the > >> cause of the problems you're seeing. > >> > >> Are there any tracebacks in bitbake-cookerdaemon.log? Which distro is > >> this? > >> > >> The pseudo aborts are likely a different problem with a missing libc > >> intercept or similar issue and secondary to the other problem. > >> > >> Cheers, > >> > >> Richard > > -- > » Javier Tia ✍
Hi Martin, On Tue, Dec 24, 2024, at 5:47 PM, Martin Jansa wrote: > Hi Javier, > > yeah I believe you. I just wanted to show that not all 3.13.1 pythons > trigger this issue and it's important to figure out what's different > on your system. e.g. in gentoo I'm disabling pgo in python since 3.13 > due to some issues (detected in python test suite already). Disabling PGO made the difference. After rebuilding the python package, everything went back to normal. Thank you! > I meant this commit (which you already have unless you're using more > layers and one of them changes SRCREV in pseudo recipe - which we also > had before in one of our layers): > https://git.openembedded.org/openembedded-core/commit/?id=d80e20d70d170397f9827c5a5fc75ad1f2e8cd94 Thank you for this, too! > Cheers, > > On Wed, Dec 25, 2024 at 12:39 AM Javier Tia <javier.tia@linaro.org> wrote: >> >> Hi Martin, >> >> On Tue, Dec 24, 2024, at 5:34 PM, Martin Jansa wrote: >> > FWIW: I'm using gentoo with 3.13.1 and I'm not seeing these issues >> > since pseudo was updated. >> >> Yes, I updated pseudo [1], but all build issues happened: >> >> [1] bitbake pseudo-native -c cleanall && bitbake pseudo-native. >> >> Thanks, >> >> > On Wed, Dec 25, 2024 at 12:22 AM Richard Purdie via >> > lists.yoctoproject.org >> > <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote: >> >> >> >> On Tue, 2024-12-24 at 17:13 -0600, Javier Tia wrote: >> >> > On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: >> >> > > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via >> >> > > lists.yoctoproject.org wrote: >> >> > > > When Bitbake is used with Python 3.13, it generates numerous >> >> > > > build >> >> > > > issues that do not occur with Python 3.12. The documentation is >> >> > > > clear >> >> > > > for the supported Linux distributions, but not for the supported >> >> > > > Python >> >> > > > versions. >> >> > > > >> >> > > > Signed-off-by: Javier Tia <javier.tia@linaro.org> >> >> > > >> >> > > Can you be specific about which version of bitbake you saw the >> >> > > issues >> >> > > with? I was hoping we had resolved most of the issues with 3.13, at >> >> > > least in master. We do aim to work with new versions of python but >> >> > > it >> >> > > obviously takes time to catch up and for fixes to make older >> >> > > versions. >> >> > >> >> > With Bitbake from poky master (284a08b6e40761f) and BitBake from the >> >> > latest master (c9906bb289dcfd9). Python 3.13.1. >> >> > >> >> > Issues: >> >> > >> >> > - "ERROR: ParseError in None: Not all recipes parsed, parser thread >> >> > killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, >> >> > but I reopened the bug because I believe it is still not fixed. >> >> > >> >> > - The problem mentioned on >> >> > https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky >> >> > recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes >> >> > from other layers. >> >> > >> >> > Please let me know if more information is needed. >> >> >> >> We have a fedora41 worker in the autobuilder cluster which has: >> >> >> >> fedora41-vk-1:~$ python3 --version >> >> Python 3.13.0 >> >> >> >> As you can see here: >> >> >> >> https://valkyrie.yoctoproject.org/#/workers/54 >> >> >> >> is it running builds successfully. There is therefore some difference >> >> in your environment and I'm not sure it is python 3.13 that is the >> >> cause of the problems you're seeing. >> >> >> >> Are there any tracebacks in bitbake-cookerdaemon.log? Which distro is >> >> this? >> >> >> >> The pseudo aborts are likely a different problem with a missing libc >> >> intercept or similar issue and secondary to the other problem. >> >> >> >> Cheers, >> >> >> >> Richard >> >> -- >> » Javier Tia ✍
Hi Richard, On Tue, Dec 24, 2024, at 5:22 PM, Richard Purdie wrote: > On Tue, 2024-12-24 at 17:13 -0600, Javier Tia wrote: >> On Tue, Dec 24, 2024, at 4:54 PM, Richard Purdie wrote: >> > On Tue, 2024-12-24 at 16:16 -0600, Javier Tia via >> > lists.yoctoproject.org wrote: >> > > When Bitbake is used with Python 3.13, it generates numerous >> > > build >> > > issues that do not occur with Python 3.12. The documentation is >> > > clear >> > > for the supported Linux distributions, but not for the supported >> > > Python >> > > versions. >> > > >> > > Signed-off-by: Javier Tia <javier.tia@linaro.org> >> > >> > Can you be specific about which version of bitbake you saw the >> > issues >> > with? I was hoping we had resolved most of the issues with 3.13, at >> > least in master. We do aim to work with new versions of python but >> > it >> > obviously takes time to catch up and for fixes to make older >> > versions. >> >> With Bitbake from poky master (284a08b6e40761f) and BitBake from the >> latest master (c9906bb289dcfd9). Python 3.13.1. >> >> Issues: >> >> - "ERROR: ParseError in None: Not all recipes parsed, parser thread >> killed/died? Exciting." I noticed it was solved in Yocto Bugzilla, >> but I reopened the bug because I believe it is still not fixed. >> >> - The problem mentioned on >> https://wiki.yoctoproject.org/wiki/Pseudo_Abort with several poky >> recipes (base-files, update-rc.d, qemu, and u-boot) and other recipes >> from other layers. >> >> Please let me know if more information is needed. > > We have a fedora41 worker in the autobuilder cluster which has: > > fedora41-vk-1:~$ python3 --version > Python 3.13.0 > > As you can see here: > > https://valkyrie.yoctoproject.org/#/workers/54 > > is it running builds successfully. There is therefore some difference > in your environment and I'm not sure it is python 3.13 that is the > cause of the problems you're seeing. > > Are there any tracebacks in bitbake-cookerdaemon.log? Which distro is > this? Distro is Arch Linux. Understanding now that Python PGO was the underlying issue. And everything is fine now. This patch can be skipped. I could submit another documentation patch that warns against using PGO when building Python. > The pseudo aborts are likely a different problem with a missing libc > intercept or similar issue and secondary to the other problem. > > Cheers, > > Richard
diff --git a/documentation/brief-yoctoprojectqs/index.rst b/documentation/brief-yoctoprojectqs/index.rst index 3d249eefa..ed40f93d2 100644 --- a/documentation/brief-yoctoprojectqs/index.rst +++ b/documentation/brief-yoctoprojectqs/index.rst @@ -61,7 +61,7 @@ following requirements: - Git &MIN_GIT_VERSION; or greater - tar &MIN_TAR_VERSION; or greater - - Python &MIN_PYTHON_VERSION; or greater. + - Python &MIN_PYTHON_VERSION; - &MAX_PYTHON_VERSION; - gcc &MIN_GCC_VERSION; or greater. - GNU make &MIN_MAKE_VERSION; or greater diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst index 05827610b..877d51f1b 100644 --- a/documentation/dev-manual/start.rst +++ b/documentation/dev-manual/start.rst @@ -304,7 +304,7 @@ Project Build Host: - tar &MIN_TAR_VERSION; or greater - - Python &MIN_PYTHON_VERSION; or greater. + - Python &MIN_PYTHON_VERSION; - &MAX_PYTHON_VERSION; - gcc &MIN_GCC_VERSION; or greater. @@ -860,7 +860,7 @@ Initializing the Build Environment Before you can use Yocto you need to setup the build environment. From within the ``poky`` directory, source the :ref:`ref-manual/structure:\`\`oe-init-build-env\`\`` environment setup script to define Yocto Project's build environment on your build host:: - - $ source oe-init-build-env + + $ source oe-init-build-env Note, that this step will have to be repeated every time you open a new shell. diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in index 0933cbbd3..7532d8138 100644 --- a/documentation/poky.yaml.in +++ b/documentation/poky.yaml.in @@ -15,6 +15,7 @@ YOCTO_DL_URL : "https://downloads.yoctoproject.org" YOCTO_AB_URL : "https://autobuilder.yoctoproject.org" YOCTO_RELEASE_DL_URL : "&YOCTO_DL_URL;/releases/yocto/yocto-&DISTRO;" MIN_PYTHON_VERSION : "3.8.0" +MAX_PYTHON_VERSION : "3.12.z" MIN_TAR_VERSION : "1.28" MIN_GIT_VERSION : "1.8.3.1" MIN_GCC_VERSION : "8.0" diff --git a/documentation/ref-manual/system-requirements.rst b/documentation/ref-manual/system-requirements.rst index b087d374d..1f5918e89 100644 --- a/documentation/ref-manual/system-requirements.rst +++ b/documentation/ref-manual/system-requirements.rst @@ -290,7 +290,7 @@ the following version requirements for Git, tar, and Python: - tar &MIN_TAR_VERSION; or greater -- Python &MIN_PYTHON_VERSION; or greater +- Python &MIN_PYTHON_VERSION; - &MAX_PYTHON_VERSION; - GNU make &MIN_MAKE_VERSION; or greater
When Bitbake is used with Python 3.13, it generates numerous build issues that do not occur with Python 3.12. The documentation is clear for the supported Linux distributions, but not for the supported Python versions. Signed-off-by: Javier Tia <javier.tia@linaro.org> --- documentation/brief-yoctoprojectqs/index.rst | 2 +- documentation/dev-manual/start.rst | 6 +++--- documentation/poky.yaml.in | 1 + documentation/ref-manual/system-requirements.rst | 2 +- 4 files changed, 6 insertions(+), 5 deletions(-)