| Message ID | 20260325113112.1163632-1-zboszor@gmail.com |
|---|---|
| Headers | show |
| Series | Support Python freethreading | expand |
On Wed, 25 Mar 2026 at 12:31, Zoltán Böszörményi <zboszor@gmail.com> wrote: > > This patchset allows turning on Python freethreading by using > a new DISTRO_FEATURES setting: > > DISTRO_FEATURES += "python_freethreading" > > and optionally: > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > The change to python3-dir.bbclass is crucial, as all python module > build helper classes rely on it. This is fine as an RFC, but otherwise, given the large amount of tweaks required, and having to choose between one or the other (which complicates testing), I would wait and see how other distros provide a free-threading python. Ideally it should be co-installable with regular python. Alex
On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote: > This patchset allows turning on Python freethreading by using > a new DISTRO_FEATURES setting: > > DISTRO_FEATURES += "python_freethreading" > > and optionally: > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > The change to python3-dir.bbclass is crucial, as all python module > build helper classes rely on it. > > I also added native variants of some settings because another > series against meta-openembedded is needed to complete this one. > In that, the change to the fontforge recipe relies on the native > setting for setting BUILD_LDFLAGS. > Looking at those other patches in the series, they don't seem to be conditional on the distro feature. Are those changes valid for a build with and without freethreading ? That's the type of information, along with the benefits of enabling this would be information expected in a 0/N introductory patch. Cheers, Bruce > > All of the changes included here were successfully build tested, > except for the gdb-cross-canadian change. That one failed, but > it's included for completeness. > > Quite a few of python modules were build tested for native and > target builds, including python3-pytorch from my meta-python-ai > with reduced RDEPENDS because e.g. python3-onnx there failed > to build with freethreading enabled. > > All in all, this series at least allows to get things going. > Though maybe not for Yocto 6.0, as it's quite late. > > Best regards, > Zoltán Böszörményi > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#233874): > https://lists.openembedded.org/g/openembedded-core/message/233874 > Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
2026. 03. 25. 13:58 keltezéssel, Alexander Kanavin írta: > On Wed, 25 Mar 2026 at 12:31, Zoltán Böszörményi <zboszor@gmail.com> wrote: >> This patchset allows turning on Python freethreading by using >> a new DISTRO_FEATURES setting: >> >> DISTRO_FEATURES += "python_freethreading" >> >> and optionally: >> >> DISTRO_FEATURES_NATIVE += "python_freethreading" >> >> The change to python3-dir.bbclass is crucial, as all python module >> build helper classes rely on it. > This is fine as an RFC, but otherwise, given the large amount of > tweaks required, and having to choose between one or the other (which > complicates testing), I would wait and see how other distros provide a > free-threading python. Ideally it should be co-installable with > regular python. As I said, this would double the amount of python module recipe builds. See the Red Hat bugzilla ticket I created for a different reason (old bitbake needs old python, still a parallel installation use case): https://bugzilla.redhat.com/show_bug.cgi?id=2418374 "We don't have the human power to maintain several separate Python stacks packaged as RPM packages in Fedora." Python 3.14.x is present in Fedora 43, only without freethreading. I expect they won't do a parallel installation with freethreading, either, due to the same reason. When they switch, it will be all or nothing.
2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta: > > > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via lists.openembedded.org > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org> wrote: > > This patchset allows turning on Python freethreading by using > a new DISTRO_FEATURES setting: > > DISTRO_FEATURES += "python_freethreading" > > and optionally: > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > The change to python3-dir.bbclass is crucial, as all python module > build helper classes rely on it. > > I also added native variants of some settings because another > series against meta-openembedded is needed to complete this one. > In that, the change to the fontforge recipe relies on the native > setting for setting BUILD_LDFLAGS. > > > Looking at those other patches in the series, they don't seem to > be conditional on the distro feature. > > Are those changes valid for a build with and without freethreading ? Yes! The main change is in python3-dir.bbclass, and the recipe tweaks are just adaptations to the fact that only ${PYTHON_DIR} is needed, not ${PYTHON_DIR}${PYTHON_ABI} from that point. I was thinking that the introductory mail stressed this enough. FWIW, less than 30 recipes was spelling the latter out in openembedded-core and meta-openembedded combined. > That's the type of information, along with the benefits of enabling > this would be information expected in a 0/N introductory patch. > > Cheers, > > Bruce > > > All of the changes included here were successfully build tested, > except for the gdb-cross-canadian change. That one failed, but > it's included for completeness. > > Quite a few of python modules were build tested for native and > target builds, including python3-pytorch from my meta-python-ai > with reduced RDEPENDS because e.g. python3-onnx there failed > to build with freethreading enabled. > > All in all, this series at least allows to get things going. > Though maybe not for Yocto 6.0, as it's quite late. > > Best regards, > Zoltán Böszörményi > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#233874): > https://lists.openembedded.org/g/openembedded-core/message/233874 > Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub > [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: > 2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta: > > > > > > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via > lists.openembedded.org > > <http://lists.openembedded.org> <zboszor= > gmail.com@lists.openembedded.org> wrote: > > > > This patchset allows turning on Python freethreading by using > > a new DISTRO_FEATURES setting: > > > > DISTRO_FEATURES += "python_freethreading" > > > > and optionally: > > > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > > > The change to python3-dir.bbclass is crucial, as all python module > > build helper classes rely on it. > > > > I also added native variants of some settings because another > > series against meta-openembedded is needed to complete this one. > > In that, the change to the fontforge recipe relies on the native > > setting for setting BUILD_LDFLAGS. > > > > > > Looking at those other patches in the series, they don't seem to > > be conditional on the distro feature. > > > > Are those changes valid for a build with and without freethreading ? > > Yes! > > The main change is in python3-dir.bbclass, and the recipe tweaks > are just adaptations to the fact that only ${PYTHON_DIR} is needed, > not ${PYTHON_DIR}${PYTHON_ABI} from that point. > > I was thinking that the introductory mail stressed this enough. Obviously not :) I didn't get that impression at all from reading the introductory message or looking at the patches themselves. You have the overview knowledge of the feature, why it is useful, how it impacts existing recipes, how the migration would work, what is the compatibility, etc. The patch reader doesn't, so that all needs to be clearly supplied. Bruce > > > FWIW, less than 30 recipes was spelling the latter out in > openembedded-core and meta-openembedded combined. > > > That's the type of information, along with the benefits of enabling > > this would be information expected in a 0/N introductory patch. > > > > Cheers, > > > > Bruce > > > > > > All of the changes included here were successfully build tested, > > except for the gdb-cross-canadian change. That one failed, but > > it's included for completeness. > > > > Quite a few of python modules were build tested for native and > > target builds, including python3-pytorch from my meta-python-ai > > with reduced RDEPENDS because e.g. python3-onnx there failed > > to build with freethreading enabled. > > > > All in all, this series at least allows to get things going. > > Though maybe not for Yocto 6.0, as it's quite late. > > > > Best regards, > > Zoltán Böszörményi > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#233874): > > https://lists.openembedded.org/g/openembedded-core/message/233874 > > Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub > > [bruce.ashfield@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > >
2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta: > > > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: > > 2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta: > > > > > > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via lists.openembedded.org > <http://lists.openembedded.org> > > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org> wrote: > > > > This patchset allows turning on Python freethreading by using > > a new DISTRO_FEATURES setting: > > > > DISTRO_FEATURES += "python_freethreading" > > > > and optionally: > > > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > > > The change to python3-dir.bbclass is crucial, as all python module > > build helper classes rely on it. > > > > I also added native variants of some settings because another > > series against meta-openembedded is needed to complete this one. > > In that, the change to the fontforge recipe relies on the native > > setting for setting BUILD_LDFLAGS. > > > > > > Looking at those other patches in the series, they don't seem to > > be conditional on the distro feature. > > > > Are those changes valid for a build with and without freethreading ? > > Yes! > > The main change is in python3-dir.bbclass, and the recipe tweaks > are just adaptations to the fact that only ${PYTHON_DIR} is needed, > not ${PYTHON_DIR}${PYTHON_ABI} from that point. > > I was thinking that the introductory mail stressed this enough. > > > Obviously not :) Obviously. :) > I didn't get that impression at all from reading the introductory > message or looking at the patches themselves. > > You have the overview knowledge of the feature, why it is > useful, how it impacts existing recipes, how the migration would > work, what is the compatibility, etc. I think the Python 3.14 release notes has enough details. See also https://www.phoronix.com/news/Python-3.14 With the two series I sent, only a single recipe is left (cmpi-bindings) in openembedded-core and meta-openembedded that uses the PYTHON_ABI setting explicitly. Migration (of python scripts) should not be a problem. If it worked with 3.14 as is, it works with freethreading enabled. Compatibility is poor for a lot of modules, that clearly spell this out in their pyproject.toml or PKG-INFO: "Programming Language :: Python :: Free Threading :: 1 - Unstable" or something not quite production level support. It's still experimental for 3rd party modules, despite being officially supported for the interpreter and built-in modules in 3.14. This is the reason this was an RFC series. Anyway, the reason I was experimenting with this was that with heavily threaded python code, an import storm can cause a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in importlib._bootstrap according to the backtrace. I would like to see if Python 3.14 (with and without freethreading) fixes such crashes. > The patch reader doesn't, so that all needs to be clearly > supplied. Maybe next time for v2 after I have a complete image successfully built with freethreading enabled. > > Bruce > > > > FWIW, less than 30 recipes was spelling the latter out in > openembedded-core and meta-openembedded combined. > > > That's the type of information, along with the benefits of enabling > > this would be information expected in a 0/N introductory patch. > > > > Cheers, > > > > Bruce > > > > > > All of the changes included here were successfully build tested, > > except for the gdb-cross-canadian change. That one failed, but > > it's included for completeness. > > > > Quite a few of python modules were build tested for native and > > target builds, including python3-pytorch from my meta-python-ai > > with reduced RDEPENDS because e.g. python3-onnx there failed > > to build with freethreading enabled. > > > > All in all, this series at least allows to get things going. > > Though maybe not for Yocto 6.0, as it's quite late. > > > > Best regards, > > Zoltán Böszörményi > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#233874): > > https://lists.openembedded.org/g/openembedded-core/message/233874 > > Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810 > > Group Owner: openembedded-core+owner@lists.openembedded.org > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > > <mailto:openembedded-core%2Bowner@lists.openembedded.org > <mailto:openembedded-core%252Bowner@lists.openembedded.org>> > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub > > [bruce.ashfield@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
On Wed, Mar 25, 2026 at 9:51 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: > 2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta: > > > > > > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> > wrote: > > > > 2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta: > > > > > > > > > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via > lists.openembedded.org > > <http://lists.openembedded.org> > > > <http://lists.openembedded.org> <zboszor= > gmail.com@lists.openembedded.org> wrote: > > > > > > This patchset allows turning on Python freethreading by using > > > a new DISTRO_FEATURES setting: > > > > > > DISTRO_FEATURES += "python_freethreading" > > > > > > and optionally: > > > > > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > > > > > The change to python3-dir.bbclass is crucial, as all python > module > > > build helper classes rely on it. > > > > > > I also added native variants of some settings because another > > > series against meta-openembedded is needed to complete this > one. > > > In that, the change to the fontforge recipe relies on the > native > > > setting for setting BUILD_LDFLAGS. > > > > > > > > > Looking at those other patches in the series, they don't seem to > > > be conditional on the distro feature. > > > > > > Are those changes valid for a build with and without freethreading > ? > > > > Yes! > > > > The main change is in python3-dir.bbclass, and the recipe tweaks > > are just adaptations to the fact that only ${PYTHON_DIR} is needed, > > not ${PYTHON_DIR}${PYTHON_ABI} from that point. > > > > I was thinking that the introductory mail stressed this enough. > > > > > > Obviously not :) > > Obviously. :) > > > I didn't get that impression at all from reading the introductory > > message or looking at the patches themselves. > > > > You have the overview knowledge of the feature, why it is > > useful, how it impacts existing recipes, how the migration would > > work, what is the compatibility, etc. > > I think the Python 3.14 release notes has enough details. > See also https://www.phoronix.com/news/Python-3.14 The information needs to be distilled into your commit, not left to the patch reader to go off and read release notes. That's the point I'm trying to make. > > > With the two series I sent, only a single recipe is left (cmpi-bindings) > in openembedded-core and meta-openembedded that uses > the PYTHON_ABI setting explicitly. > > Migration (of python scripts) should not be a problem. > If it worked with 3.14 as is, it works with freethreading enabled. > > Compatibility is poor for a lot of modules, that clearly spell this out > in their pyproject.toml or PKG-INFO: > > "Programming Language :: Python :: Free Threading :: 1 - Unstable" > > or something not quite production level support. > It's still experimental for 3rd party modules, despite being officially > supported for the interpreter and built-in modules in 3.14. > > This is the reason this was an RFC series. > > Anyway, the reason I was experimenting with this was that > with heavily threaded python code, an import storm can cause > a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in > importlib._bootstrap according to the backtrace. > > I would like to see if Python 3.14 (with and without freethreading) > fixes such crashes. > Definitely worth putting in the RFC as well, the motivation for the change matters! > > > The patch reader doesn't, so that all needs to be clearly > > supplied. > > Maybe next time for v2 after I have a complete image > successfully built with freethreading enabled. > Ack'd Bruce > > > > > Bruce > > > > > > > > FWIW, less than 30 recipes was spelling the latter out in > > openembedded-core and meta-openembedded combined. > > > > > That's the type of information, along with the benefits of enabling > > > this would be information expected in a 0/N introductory patch. > > > > > > Cheers, > > > > > > Bruce > > > > > > > > > All of the changes included here were successfully build > tested, > > > except for the gdb-cross-canadian change. That one failed, but > > > it's included for completeness. > > > > > > Quite a few of python modules were build tested for native and > > > target builds, including python3-pytorch from my meta-python-ai > > > with reduced RDEPENDS because e.g. python3-onnx there failed > > > to build with freethreading enabled. > > > > > > All in all, this series at least allows to get things going. > > > Though maybe not for Yocto 6.0, as it's quite late. > > > > > > Best regards, > > > Zoltán Böszörményi > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > Links: You receive all messages sent to this group. > > > View/Reply Online (#233874): > > > https://lists.openembedded.org/g/openembedded-core/message/233874 > > > Mute This Topic: > https://lists.openembedded.org/mt/118499141/1050810 > > > Group Owner: openembedded-core+owner@lists.openembedded.org > > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > > > <mailto:openembedded-core%2Bowner@lists.openembedded.org > > <mailto:openembedded-core%252Bowner@lists.openembedded.org>> > > > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub > > > [bruce.ashfield@gmail.com] > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > -- > > > - Thou shalt not follow the NULL pointer, for chaos and madness > await thee at its end > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > >
2026. 03. 25. 15:17 keltezéssel, Bruce Ashfield írta: > > > On Wed, Mar 25, 2026 at 9:51 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: > > 2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta: > > > > > > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: > > > > 2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta: > > > > > > > > > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via > lists.openembedded.org <http://lists.openembedded.org> > > <http://lists.openembedded.org> > > > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org> > wrote: > > > > > > This patchset allows turning on Python freethreading by using > > > a new DISTRO_FEATURES setting: > > > > > > DISTRO_FEATURES += "python_freethreading" > > > > > > and optionally: > > > > > > DISTRO_FEATURES_NATIVE += "python_freethreading" > > > > > > The change to python3-dir.bbclass is crucial, as all python module > > > build helper classes rely on it. > > > > > > I also added native variants of some settings because another > > > series against meta-openembedded is needed to complete this one. > > > In that, the change to the fontforge recipe relies on the native > > > setting for setting BUILD_LDFLAGS. > > > > > > > > > Looking at those other patches in the series, they don't seem to > > > be conditional on the distro feature. > > > > > > Are those changes valid for a build with and without freethreading ? > > > > Yes! > > > > The main change is in python3-dir.bbclass, and the recipe tweaks > > are just adaptations to the fact that only ${PYTHON_DIR} is needed, > > not ${PYTHON_DIR}${PYTHON_ABI} from that point. > > > > I was thinking that the introductory mail stressed this enough. > > > > > > Obviously not :) > > Obviously. :) > > > I didn't get that impression at all from reading the introductory > > message or looking at the patches themselves. > > > > You have the overview knowledge of the feature, why it is > > useful, how it impacts existing recipes, how the migration would > > work, what is the compatibility, etc. > > I think the Python 3.14 release notes has enough details. > See also https://www.phoronix.com/news/Python-3.14 > > > The information needs to be distilled into your commit, not left > to the patch reader to go off and read release notes. That's the > point I'm trying to make. > > > > With the two series I sent, only a single recipe is left (cmpi-bindings) > in openembedded-core and meta-openembedded that uses > the PYTHON_ABI setting explicitly. > > Migration (of python scripts) should not be a problem. > If it worked with 3.14 as is, it works with freethreading enabled. > > Compatibility is poor for a lot of modules, that clearly spell this out > in their pyproject.toml or PKG-INFO: > > "Programming Language :: Python :: Free Threading :: 1 - Unstable" > > or something not quite production level support. > It's still experimental for 3rd party modules, despite being officially > supported for the interpreter and built-in modules in 3.14. > > This is the reason this was an RFC series. > > Anyway, the reason I was experimenting with this was that > with heavily threaded python code, an import storm can cause > a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in > importlib._bootstrap according to the backtrace. > > I would like to see if Python 3.14 (with and without freethreading) > fixes such crashes. > > > Definitely worth putting in the RFC as well, the motivation for > the change matters! > > > > The patch reader doesn't, so that all needs to be clearly > > supplied. > > Maybe next time for v2 after I have a complete image > successfully built with freethreading enabled. > No such luck. lldb in clang won't build because its swig source wants to play games with PyObject->ob_refcnt directly instead of using the proper macros. That struct member does not exist when freethreading is enabled. In fact, the whole struct _object (the PyObject implementation) is different when freethreading is enabled. The llvm sources, even in the newly released 22.x and main do not have any reference to the Py_GIL_DISABLED macro, so as a whole, it does not support Python freethreading. I can revisit this series a few year later when llvm adds the necessary support, which can be backported to older clang versions in my meta-clang-revival. This concludes my little experiment. That being said, I think the cleanup in the recipes is a good direction. PYTHON_DIR should include PYTHON_ABI in python3-dir.bbclass because build helper classes rely on it. Then PYTHON_DIR must be used consistently in recipes that need it, even if PYTHON_ABI stays empty. > > Ack'd > > Bruce > > > > > > Bruce > > > > > > > > FWIW, less than 30 recipes was spelling the latter out in > > openembedded-core and meta-openembedded combined. > > > > > That's the type of information, along with the benefits of enabling > > > this would be information expected in a 0/N introductory patch. > > > > > > Cheers, > > > > > > Bruce > > > > > > > > > All of the changes included here were successfully build tested, > > > except for the gdb-cross-canadian change. That one failed, but > > > it's included for completeness. > > > > > > Quite a few of python modules were build tested for native and > > > target builds, including python3-pytorch from my meta-python-ai > > > with reduced RDEPENDS because e.g. python3-onnx there failed > > > to build with freethreading enabled. > > > > > > All in all, this series at least allows to get things going. > > > Though maybe not for Yocto 6.0, as it's quite late. > > > > > > Best regards, > > > Zoltán Böszörményi > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > Links: You receive all messages sent to this group. > > > View/Reply Online (#233874): > > > https://lists.openembedded.org/g/openembedded-core/message/233874 > > > Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810 > > > Group Owner: openembedded-core+owner@lists.openembedded.org > <mailto:openembedded-core%2Bowner@lists.openembedded.org> > > <mailto:openembedded-core%2Bowner@lists.openembedded.org > <mailto:openembedded-core%252Bowner@lists.openembedded.org>> > > > <mailto:openembedded-core%2Bowner@lists.openembedded.org > <mailto:openembedded-core%252Bowner@lists.openembedded.org> > > <mailto:openembedded-core%252Bowner@lists.openembedded.org > <mailto:openembedded-core%25252Bowner@lists.openembedded.org>>> > > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub > > > [bruce.ashfield@gmail.com] > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > -- > > > - Thou shalt not follow the NULL pointer, for chaos and madness await thee > at its end > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
2026. 03. 26. 6:29 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta: > 2026. 03. 25. 15:17 keltezéssel, Bruce Ashfield írta: >> >> >> On Wed, Mar 25, 2026 at 9:51 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: >> >> 2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta: >> > >> > >> > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote: >> > >> > 2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta: >> > > >> > > >> > > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via >> lists.openembedded.org <http://lists.openembedded.org> >> > <http://lists.openembedded.org> >> > > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org> >> wrote: >> > > >> > > This patchset allows turning on Python freethreading by using >> > > a new DISTRO_FEATURES setting: >> > > >> > > DISTRO_FEATURES += "python_freethreading" >> > > >> > > and optionally: >> > > >> > > DISTRO_FEATURES_NATIVE += "python_freethreading" >> > > >> > > The change to python3-dir.bbclass is crucial, as all python module >> > > build helper classes rely on it. >> > > >> > > I also added native variants of some settings because another >> > > series against meta-openembedded is needed to complete this one. >> > > In that, the change to the fontforge recipe relies on the native >> > > setting for setting BUILD_LDFLAGS. >> > > >> > > >> > > Looking at those other patches in the series, they don't seem to >> > > be conditional on the distro feature. >> > > >> > > Are those changes valid for a build with and without freethreading ? >> > >> > Yes! >> > >> > The main change is in python3-dir.bbclass, and the recipe tweaks >> > are just adaptations to the fact that only ${PYTHON_DIR} is needed, >> > not ${PYTHON_DIR}${PYTHON_ABI} from that point. >> > >> > I was thinking that the introductory mail stressed this enough. >> > >> > >> > Obviously not :) >> >> Obviously. :) >> >> > I didn't get that impression at all from reading the introductory >> > message or looking at the patches themselves. >> > >> > You have the overview knowledge of the feature, why it is >> > useful, how it impacts existing recipes, how the migration would >> > work, what is the compatibility, etc. >> >> I think the Python 3.14 release notes has enough details. >> See also https://www.phoronix.com/news/Python-3.14 >> >> >> The information needs to be distilled into your commit, not left >> to the patch reader to go off and read release notes. That's the >> point I'm trying to make. >> >> >> >> With the two series I sent, only a single recipe is left (cmpi-bindings) >> in openembedded-core and meta-openembedded that uses >> the PYTHON_ABI setting explicitly. >> >> Migration (of python scripts) should not be a problem. >> If it worked with 3.14 as is, it works with freethreading enabled. >> >> Compatibility is poor for a lot of modules, that clearly spell this out >> in their pyproject.toml or PKG-INFO: >> >> "Programming Language :: Python :: Free Threading :: 1 - Unstable" >> >> or something not quite production level support. >> It's still experimental for 3rd party modules, despite being officially >> supported for the interpreter and built-in modules in 3.14. >> >> This is the reason this was an RFC series. >> >> Anyway, the reason I was experimenting with this was that >> with heavily threaded python code, an import storm can cause >> a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in >> importlib._bootstrap according to the backtrace. >> >> I would like to see if Python 3.14 (with and without freethreading) >> fixes such crashes. >> >> >> Definitely worth putting in the RFC as well, the motivation for >> the change matters! >> >> >> > The patch reader doesn't, so that all needs to be clearly >> > supplied. >> >> Maybe next time for v2 after I have a complete image >> successfully built with freethreading enabled. >> > > No such luck. > > lldb in clang won't build because its swig source wants to play games > with PyObject->ob_refcnt directly instead of using the proper macros. > That struct member does not exist when freethreading is enabled. > In fact, the whole struct _object (the PyObject implementation) > is different when freethreading is enabled. > > The llvm sources, even in the newly released 22.x and main > do not have any reference to the Py_GIL_DISABLED macro, > so as a whole, it does not support Python freethreading. > > I can revisit this series a few year later when llvm adds the > necessary support, which can be backported to older clang > versions in my meta-clang-revival. > > This concludes my little experiment. > > That being said, I think the cleanup in the recipes is a good direction. > PYTHON_DIR should include PYTHON_ABI in python3-dir.bbclass > because build helper classes rely on it. Then PYTHON_DIR must be > used consistently in recipes that need it, even if PYTHON_ABI stays empty. Which means I can re-send the whole series as a cleanup or preparation series without the DISTRO_FEATURES enablement part in python3-dir.bbclass. That one can stay RFC as a standalone patch, separate from the cleanup series. I will re-test my build without enabling freethreading before I would do that. > >> >> Ack'd >> >> Bruce >> >> >> > >> > Bruce >> > >> > >> > >> > FWIW, less than 30 recipes was spelling the latter out in >> > openembedded-core and meta-openembedded combined. >> > >> > > That's the type of information, along with the benefits of enabling >> > > this would be information expected in a 0/N introductory patch. >> > > >> > > Cheers, >> > > >> > > Bruce >> > > >> > > >> > > All of the changes included here were successfully build tested, >> > > except for the gdb-cross-canadian change. That one failed, but >> > > it's included for completeness. >> > > >> > > Quite a few of python modules were build tested for native and >> > > target builds, including python3-pytorch from my meta-python-ai >> > > with reduced RDEPENDS because e.g. python3-onnx there failed >> > > to build with freethreading enabled. >> > > >> > > All in all, this series at least allows to get things going. >> > > Though maybe not for Yocto 6.0, as it's quite late. >> > > >> > > Best regards, >> > > Zoltán Böszörményi >> > > >> > > >> > > >> > > >> > > >> > > -- >> > > - Thou shalt not follow the NULL pointer, for chaos and madness await thee >> at its end >> > > - "Use the force Harry" - Gandalf, Star Trek II >> > > >> > >> > >> > >> > -- >> > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its >> end >> > - "Use the force Harry" - Gandalf, Star Trek II >> > >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II >> > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#233951): https://lists.openembedded.org/g/openembedded-core/message/233951 > Mute This Topic: https://lists.openembedded.org/mt/118499141/3617728 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [zboszor@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >