| Message ID | 20221120175237.194899-1-f_l_k@t-online.de |
|---|---|
| State | New |
| Headers | show |
| Series | python3: add ${bindir}/python symlink | expand |
I have to ask you to resend and drop the DISTRO_FEATURE idea. We do not want to make the impression that python2 is supported in any way, and as explained earlier this implies that all scripts that ask for python would work with both 2 and 3. No, this is a clean break, and anyone who still has 2.x stuff has to adjust it to ask for #!/usr/bin/python2. Alex On Sun, 20 Nov 2022 at 18:53, Markus Volk <f_l_k@t-online.de> wrote: > > Currently /usr/bin/python is provided by the python2 package. If python2 > is not installed, the built image lacks a provider for it. > This results in failed scripts when using '/usr/bin/python' shebang. > > This patch adds a /usr/bin/python symlink for python3 to fix this issue. > For images containing python2 and python3, the do_rootfs task will fail > because now both of them provide the same file. > To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which, > if set, will prevent python3 from creating the symlink. > > The link is created with relative path because using an absolute path would fail > for native and nativesdk. > > Signed-off-by: Markus Volk <f_l_k@t-online.de> > --- > meta/recipes-devtools/python/python3/python3-manifest.json | 1 + > meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++ > 2 files changed, 5 insertions(+) > > diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json > index 64203cf0fc..7b6f509a45 100644 > --- a/meta/recipes-devtools/python/python3/python3-manifest.json > +++ b/meta/recipes-devtools/python/python3/python3-manifest.json > @@ -203,6 +203,7 @@ > "files": [ > "${bindir}/python${PYTHON_MAJMIN}", > "${bindir}/python${PYTHON_MAJMIN}.real", > + "${bindir}/python", > "${bindir}/python3", > "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h", > "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", > diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb > index 92a1f69320..18b631f79d 100644 > --- a/meta/recipes-devtools/python/python3_3.11.0.bb > +++ b/meta/recipes-devtools/python/python3_3.11.0.bb > @@ -144,6 +144,7 @@ do_install:prepend() { > > do_install:append:class-target() { > oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h > + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)} > } > > do_install:append:class-native() { > @@ -156,6 +157,7 @@ do_install:append:class-native() { > # (these often end up too long for the #! parser in the kernel as the > # buffer is 128 bytes long). > ln -s python3-native/python3 ${D}${bindir}/nativepython3 > + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)} > > # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them > # and the overhead in each recipe-sysroot-native isn't worth it, particularly > @@ -213,6 +215,7 @@ do_install:append() { > } > > do_install:append:class-nativesdk () { > + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)} > # Make sure we use /usr/bin/env python > for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do > sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT > @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX > > # For historical reasons PN is empty and provided by python3-modules > FILES:${PN} = "" > +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}" > RPROVIDES:${PN}-modules = "${PN}" > > FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173637): https://lists.openembedded.org/g/openembedded-core/message/173637 > Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Am So, 20. Nov 2022 um 19:00:39 +0100 schrieb Alexander Kanavin <alex.kanavin@gmail.com>: > I have to ask you to resend and drop the DISTRO_FEATURE idea. We do > not want to make the impression that python2 is supported in any way, > and as explained earlier this implies that all scripts that ask for > python would work with both 2 and 3. No, this is a clean break, and > anyone who still has 2.x stuff has to adjust it to ask for > #!/usr/bin/python2. I fully agree
On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@t-online.de> wrote: > Currently /usr/bin/python is provided by the python2 package. If python2 > is not installed, the built image lacks a provider for it. > This results in failed scripts when using '/usr/bin/python' shebang. > > This patch adds a /usr/bin/python symlink for python3 to fix this issue. > For images containing python2 and python3, the do_rootfs task will fail > because now both of them provide the same file. > To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced > which, > if set, will prevent python3 from creating the symlink. > > The link is created with relative path because using an absolute path > would fail > for native and nativesdk. What happens if and when python4 comes ? This might get us into same mess we had migrating python2 to 3 > > Signed-off-by: Markus Volk <f_l_k@t-online.de> > --- > meta/recipes-devtools/python/python3/python3-manifest.json | 1 + > meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++ > 2 files changed, 5 insertions(+) > > diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json > b/meta/recipes-devtools/python/python3/python3-manifest.json > index 64203cf0fc..7b6f509a45 100644 > --- a/meta/recipes-devtools/python/python3/python3-manifest.json > +++ b/meta/recipes-devtools/python/python3/python3-manifest.json > @@ -203,6 +203,7 @@ > "files": [ > "${bindir}/python${PYTHON_MAJMIN}", > "${bindir}/python${PYTHON_MAJMIN}.real", > + "${bindir}/python", > "${bindir}/python3", > "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h", > "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", > diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb > b/meta/recipes-devtools/python/python3_3.11.0.bb > index 92a1f69320..18b631f79d 100644 > --- a/meta/recipes-devtools/python/python3_3.11.0.bb > +++ b/meta/recipes-devtools/python/python3_3.11.0.bb > @@ -144,6 +144,7 @@ do_install:prepend() { > > do_install:append:class-target() { > oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h > + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf > ./python3 ${D}${bindir}/python', d)} > } > > do_install:append:class-native() { > @@ -156,6 +157,7 @@ do_install:append:class-native() { > # (these often end up too long for the #! parser in the kernel as > the > # buffer is 128 bytes long). > ln -s python3-native/python3 ${D}${bindir}/nativepython3 > + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf > ./python3-native/python3 ${D}${bindir}/nativepython', d)} > > # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 > of them > # and the overhead in each recipe-sysroot-native isn't worth it, > particularly > @@ -213,6 +215,7 @@ do_install:append() { > } > > do_install:append:class-nativesdk () { > + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf > ./python3 ${D}${bindir}/python', d)} > # Make sure we use /usr/bin/env python > for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do > sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT > @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " > ${MLPREFIX}openssl ${MLPREFIX > > # For historical reasons PN is empty and provided by python3-modules > FILES:${PN} = "" > +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', > '${bindir}/python', d)}" > RPROVIDES:${PN}-modules = "${PN}" > > FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173637): > https://lists.openembedded.org/g/openembedded-core/message/173637 > Mute This Topic: https://lists.openembedded.org/mt/95156228/1997914 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received. https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/ Alex On Mon 21. Nov 2022 at 0.17, Khem Raj <raj.khem@gmail.com> wrote: > > > On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@t-online.de> wrote: > >> Currently /usr/bin/python is provided by the python2 package. If python2 >> is not installed, the built image lacks a provider for it. >> This results in failed scripts when using '/usr/bin/python' shebang. >> >> This patch adds a /usr/bin/python symlink for python3 to fix this issue. >> For images containing python2 and python3, the do_rootfs task will fail >> because now both of them provide the same file. >> To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced >> which, >> if set, will prevent python3 from creating the symlink. >> >> The link is created with relative path because using an absolute path >> would fail >> for native and nativesdk. > > > What happens if and when python4 comes ? > This might get us into same mess we had migrating python2 to 3 > > >> >> Signed-off-by: Markus Volk <f_l_k@t-online.de> >> --- >> meta/recipes-devtools/python/python3/python3-manifest.json | 1 + >> meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++ >> 2 files changed, 5 insertions(+) >> >> diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json >> b/meta/recipes-devtools/python/python3/python3-manifest.json >> index 64203cf0fc..7b6f509a45 100644 >> --- a/meta/recipes-devtools/python/python3/python3-manifest.json >> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json >> @@ -203,6 +203,7 @@ >> "files": [ >> "${bindir}/python${PYTHON_MAJMIN}", >> "${bindir}/python${PYTHON_MAJMIN}.real", >> + "${bindir}/python", >> "${bindir}/python3", >> "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h", >> "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", >> diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb >> b/meta/recipes-devtools/python/python3_3.11.0.bb >> index 92a1f69320..18b631f79d 100644 >> --- a/meta/recipes-devtools/python/python3_3.11.0.bb >> +++ b/meta/recipes-devtools/python/python3_3.11.0.bb >> @@ -144,6 +144,7 @@ do_install:prepend() { >> >> do_install:append:class-target() { >> oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h >> + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf >> ./python3 ${D}${bindir}/python', d)} >> } >> >> do_install:append:class-native() { >> @@ -156,6 +157,7 @@ do_install:append:class-native() { >> # (these often end up too long for the #! parser in the kernel >> as the >> # buffer is 128 bytes long). >> ln -s python3-native/python3 ${D}${bindir}/nativepython3 >> + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf >> ./python3-native/python3 ${D}${bindir}/nativepython', d)} >> >> # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 >> of them >> # and the overhead in each recipe-sysroot-native isn't worth it, >> particularly >> @@ -213,6 +215,7 @@ do_install:append() { >> } >> >> do_install:append:class-nativesdk () { >> + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf >> ./python3 ${D}${bindir}/python', d)} >> # Make sure we use /usr/bin/env python >> for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do >> sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT >> @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " >> ${MLPREFIX}openssl ${MLPREFIX >> >> # For historical reasons PN is empty and provided by python3-modules >> FILES:${PN} = "" >> +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', >> '', '${bindir}/python', d)}" >> RPROVIDES:${PN}-modules = "${PN}" >> >> FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" >> -- >> 2.34.1 >> >> >> >> >> > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173642): > https://lists.openembedded.org/g/openembedded-core/message/173642 > Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Sun, Nov 20, 2022 at 3:56 PM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > Python community has no plans for python4 whatsoever. Anything but gentle > and steady evolution won’t be well received. > > > https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/ > Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises > > > Alex > > On Mon 21. Nov 2022 at 0.17, Khem Raj <raj.khem@gmail.com> wrote: > >> >> >> On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@t-online.de> wrote: >> >>> Currently /usr/bin/python is provided by the python2 package. If python2 >>> is not installed, the built image lacks a provider for it. >>> This results in failed scripts when using '/usr/bin/python' shebang. >>> >>> This patch adds a /usr/bin/python symlink for python3 to fix this issue. >>> For images containing python2 and python3, the do_rootfs task will fail >>> because now both of them provide the same file. >>> To allow this corner case, a DISTRO_FEATURE 'python2' has been >>> introduced which, >>> if set, will prevent python3 from creating the symlink. >>> >>> The link is created with relative path because using an absolute path >>> would fail >>> for native and nativesdk. >> >> >> What happens if and when python4 comes ? >> This might get us into same mess we had migrating python2 to 3 >> >> >>> >>> Signed-off-by: Markus Volk <f_l_k@t-online.de> >>> --- >>> meta/recipes-devtools/python/python3/python3-manifest.json | 1 + >>> meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++ >>> 2 files changed, 5 insertions(+) >>> >>> diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json >>> b/meta/recipes-devtools/python/python3/python3-manifest.json >>> index 64203cf0fc..7b6f509a45 100644 >>> --- a/meta/recipes-devtools/python/python3/python3-manifest.json >>> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json >>> @@ -203,6 +203,7 @@ >>> "files": [ >>> "${bindir}/python${PYTHON_MAJMIN}", >>> "${bindir}/python${PYTHON_MAJMIN}.real", >>> + "${bindir}/python", >>> "${bindir}/python3", >>> "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h", >>> "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", >>> diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb >>> b/meta/recipes-devtools/python/python3_3.11.0.bb >>> index 92a1f69320..18b631f79d 100644 >>> --- a/meta/recipes-devtools/python/python3_3.11.0.bb >>> +++ b/meta/recipes-devtools/python/python3_3.11.0.bb >>> @@ -144,6 +144,7 @@ do_install:prepend() { >>> >>> do_install:append:class-target() { >>> oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h >>> + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf >>> ./python3 ${D}${bindir}/python', d)} >>> } >>> >>> do_install:append:class-native() { >>> @@ -156,6 +157,7 @@ do_install:append:class-native() { >>> # (these often end up too long for the #! parser in the kernel >>> as the >>> # buffer is 128 bytes long). >>> ln -s python3-native/python3 ${D}${bindir}/nativepython3 >>> + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf >>> ./python3-native/python3 ${D}${bindir}/nativepython', d)} >>> >>> # Remove the opt-1.pyc and opt-2.pyc files. There are over >>> 3,000 of them >>> # and the overhead in each recipe-sysroot-native isn't worth >>> it, particularly >>> @@ -213,6 +215,7 @@ do_install:append() { >>> } >>> >>> do_install:append:class-nativesdk () { >>> + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf >>> ./python3 ${D}${bindir}/python', d)} >>> # Make sure we use /usr/bin/env python >>> for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do >>> sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT >>> @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " >>> ${MLPREFIX}openssl ${MLPREFIX >>> >>> # For historical reasons PN is empty and provided by python3-modules >>> FILES:${PN} = "" >>> +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', >>> '', '${bindir}/python', d)}" >>> RPROVIDES:${PN}-modules = "${PN}" >>> >>> FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3" >>> -- >>> 2.34.1 >>> >>> >>> >>> >>> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> > View/Reply Online (#173642): >> https://lists.openembedded.org/g/openembedded-core/message/173642 >> Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ >> alex.kanavin@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> >>
On Mon, 21 Nov 2022 at 16:17, Khem Raj <raj.khem@gmail.com> wrote: >> Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received. >> >> https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/ > > > Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises Fedora has done it 3 years ago: https://pagure.io/fedora-docs/release-notes/pull-request/410#request_diff Alex
And here's debian: https://wiki.debian.org/Python/FAQ#Python_2_support On my Debian systems, indeed, /usr/bin/python is absent. Alex On Mon, 21 Nov 2022 at 18:18, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > On Mon, 21 Nov 2022 at 16:17, Khem Raj <raj.khem@gmail.com> wrote: > >> Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received. > >> > >> https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/ > > > > > > Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises > > Fedora has done it 3 years ago: > https://pagure.io/fedora-docs/release-notes/pull-request/410#request_diff > > Alex > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173658): https://lists.openembedded.org/g/openembedded-core/message/173658 > Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
> On my Debian systems, indeed, /usr/bin/python is absent.
Debian has a package for this
<https://packages.debian.org/bookworm/python-is-python3>
On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k=t-online.de@lists.openembedded.org> wrote: > On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <alex.kanavin@gmail.com> wrote: >> On my Debian systems, indeed, /usr/bin/python is absent. > > Debian has a package for this > https://packages.debian.org/bookworm/python-is-python3 And that’s a perfectly good solution that I like. Personally, I think people should forget that /usr/bin/python exists: the recommendation from Python is to call python2 or python3. I can quote from PEP-0394: “”” Depending on a distribution or system configuration, python may or may not be installed. If python is installed its target interpreter may refer to python2 or python3.” … • Distributors may choose to set the behavior of the python command as follows: • python2, • python3, • not provide python command, • allow python to be configurable by an end user or a system administrator. “”” We’ve picked option 3. As per Python upstream, that’s absolutely fine. If you have a serious need that /usr/bin/python exists, and is a symlink to python3, then could you not make a simple recipe that RDEPENDS on python3 and ships just a /usr/bin/python -> python3 symlink? You can even put this in your layer to avoid having to debate it with the oe-core maintainers. Ross
The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest. Alex On Tue 22. Nov 2022 at 18.41, Ross Burton <Ross.Burton@arm.com> wrote: > On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k= > t-online.de@lists.openembedded.org> wrote: > > On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin < > alex.kanavin@gmail.com> wrote: > >> On my Debian systems, indeed, /usr/bin/python is absent. > > > > Debian has a package for this > > https://packages.debian.org/bookworm/python-is-python3 > > And that’s a perfectly good solution that I like. > > Personally, I think people should forget that /usr/bin/python exists: the > recommendation from Python is to call python2 or python3. I can quote from > PEP-0394: > > “”” > Depending on a distribution or system configuration, python may or may not > be installed. If python is installed its target interpreter may refer to > python2 or python3.” > > … > > • Distributors may choose to set the behavior of the python command as > follows: > • python2, > • python3, > • not provide python command, > • allow python to be configurable by an end user or a system > administrator. > “”” > > We’ve picked option 3. As per Python upstream, that’s absolutely fine. > > If you have a serious need that /usr/bin/python exists, and is a symlink > to python3, then could you not make a simple recipe that RDEPENDS on > python3 and ships just a /usr/bin/python -> python3 symlink? You can even > put this in your layer to avoid having to debate it with the oe-core > maintainers. > > Ross
I use a bbappend for python3 in meta-wayland for almost a year now because i had issues installing python scripts with /usr/bin/python shebang e.g. here in sway: <https://github.com/swaywm/sway/blob/master/contrib/inactive-windows-transparency.py> Issue has been that, if adding this script to do_install(), package_qa failed because of the missing /usr/bin/python provider. Ran into this quite a few times in the past and then created the symlink instead of patching shebangs I just wanted to mention, that having a symlink would be an alternative to patching shebangs and as i was asked to send a patch, i did. But I'm fine with whatever is done in oe-core. Basically I also think the best solution would be, upstream would deprecate /usr/bin/python usage. Unfortunaltey it's not, and so there will always be people using it. Am Di, 22. Nov 2022 um 17:41:22 +0000 schrieb Ross Burton <ross.burton@arm.com>: > On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org > <f_l_k=t-online.de@lists.openembedded.org > <mailto:f_l_k=t-online.de@lists.openembedded.org>> wrote: >> On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin >> <alex.kanavin@gmail.com <mailto:alex.kanavin@gmail.com>> wrote: >>> On my Debian systems, indeed, /usr/bin/python is absent. >> >> Debian has a package for this >> <https://packages.debian.org/bookworm/python-is-python3> > > And that’s a perfectly good solution that I like. > > Personally, I think people should forget that /usr/bin/python exists: > the recommendation from Python is to call python2 or python3. I can > quote from PEP-0394: > > “”” > Depending on a distribution or system configuration, python may or > may not be installed. If python is installed its target interpreter > may refer to python2 or python3.” > > … > > • Distributors may choose to set the behavior of the python > command as follows: > • python2, > • python3, > • not provide python command, > • allow python to be configurable by an end user or a > system administrator. > “”” > > We’ve picked option 3. As per Python upstream, that’s absolutely > fine. > > If you have a serious need that /usr/bin/python exists, and is a > symlink to python3, then could you not make a simple recipe that > RDEPENDS on python3 and ships just a /usr/bin/python -> python3 > symlink? You can even put this in your layer to avoid having to > debate it with the oe-core maintainers. > > Ross > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173694): > <https://lists.openembedded.org/g/openembedded-core/message/173694> > Mute This Topic: <https://lists.openembedded.org/mt/95156228/3618223> > Group Owner: openembedded-core+owner@lists.openembedded.org > <mailto:openembedded-core+owner@lists.openembedded.org> > Unsubscribe: > <https://lists.openembedded.org/g/openembedded-core/unsub> > [f_l_k@t-online.de <mailto:f_l_k@t-online.de>] > -=-=-=-=-=-=-=-=-=-=-=- >
On Tue, 2022-11-22 at 19:03 +0100, Alexander Kanavin wrote: > The serious need is that we need to patch all the scripts that ask > for python to add a 3 to it. And there will be more of these going > forward, not less. I’d rather just always have python available. Not > a problem worth deliberating over to be honest. I don't think it is that simple. a) If we take the change, we're no longer explicit in what we're using which means users can be taken by surprise when a python2 script finds python3 and could sometimes break. b) Our current approach means that people have at least looked at and decided something is python3 safe. c) It means that anyone still using python2 components is left with no way to make things work. Whilst nobody should be using python2 anymore, I suspect some still are and this will be a huge problem to them. d) If there ever is a python4, we'll have to redo a lot of the work we're done to reach this point where things are fairly clean. Rightly or wrongly, this patch will cause large amounts of pain for some portion of our userbase and I'm not sure we have enough justification to do that. That pain wouldn't likely be realised for some time either :/. Cheers, Richard
On Tue, 22 Nov 2022 at 19:14, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > Rightly or wrongly, this patch will cause large amounts of pain for > some portion of our userbase and I'm not sure we have enough > justification to do that. That pain wouldn't likely be realised for > some time either :/. I have to point out that meta-python2 hasn't even received a compatibility update for langsdale: https://git.openembedded.org/meta-python2 While this might be 'too soon' to conclude that python2 is truly dead, maybe a year (or two, or three) from now it won't be. Fedora has already made the switch, Debian will follow, and honestly, I just can't muster any sympathy for python2 users anymore. You can't push back paying off technical debt forever and expect others to accommodate you. Let me propose this: a PACKAGECONFIG for the python recipe that adds and installs the symlink in a dedicated package. We can keep it off for now, but somewhere down the line we could revisit that against established practice and what PEPs say then. Alex
On 22/11/2022 19:32:17+0100, Alexander Kanavin wrote: > On Tue, 22 Nov 2022 at 19:14, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > Rightly or wrongly, this patch will cause large amounts of pain for > > some portion of our userbase and I'm not sure we have enough > > justification to do that. That pain wouldn't likely be realised for > > some time either :/. > > I have to point out that meta-python2 hasn't even received a > compatibility update for langsdale: > https://git.openembedded.org/meta-python2 > > While this might be 'too soon' to conclude that python2 is truly dead, > maybe a year (or two, or three) from now it won't be. Fedora has > already made the switch, Debian will follow, and honestly, I just > can't muster any sympathy for python2 users anymore. You can't push > back paying off technical debt forever and expect others to > accommodate you. > > Let me propose this: a PACKAGECONFIG for the python recipe that adds > and installs the symlink in a dedicated package. We can keep it off > for now, but somewhere down the line we could revisit that against > established practice and what PEPs say then. > I actually like the idea of having a python-is-python3 package so that affected recipes could simply add it to their dependencies. That would make it explicit that python is not python2. > Alex > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173698): https://lists.openembedded.org/g/openembedded-core/message/173698 > Mute This Topic: https://lists.openembedded.org/mt/95156228/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
> -----Original Message----- > From: openembedded-core@lists.openembedded.org <openembedded- > core@lists.openembedded.org> On Behalf Of Alexandre Belloni via > lists.openembedded.org > Sent: den 22 november 2022 23:29 > To: Alexander Kanavin <alex.kanavin@gmail.com> > Cc: Richard Purdie <richard.purdie@linuxfoundation.org>; Ross Burton > <Ross.Burton@arm.com>; Khem Raj <raj.khem@gmail.com>; Markus Volk > <f_l_k@t-online.de>; openembedded-core@lists.openembedded.org > Subject: Re: [oe-core][PATCH] python3: add ${bindir}/python symlink > > On 22/11/2022 19:32:17+0100, Alexander Kanavin wrote: > > On Tue, 22 Nov 2022 at 19:14, Richard Purdie > > <richard.purdie@linuxfoundation.org> wrote: > > > Rightly or wrongly, this patch will cause large amounts of pain for > > > some portion of our userbase and I'm not sure we have enough > > > justification to do that. That pain wouldn't likely be realised for > > > some time either :/. > > > > I have to point out that meta-python2 hasn't even received a > > compatibility update for langsdale: > > https://git.openembedded.org/meta-python2 > > > > While this might be 'too soon' to conclude that python2 is truly dead, > > maybe a year (or two, or three) from now it won't be. Fedora has > > already made the switch, Debian will follow, and honestly, I just > > can't muster any sympathy for python2 users anymore. You can't push > > back paying off technical debt forever and expect others to > > accommodate you. > > > > Let me propose this: a PACKAGECONFIG for the python recipe that adds > > and installs the symlink in a dedicated package. We can keep it off > > for now, but somewhere down the line we could revisit that against > > established practice and what PEPs say then. > > > > I actually like the idea of having a python-is-python3 package so that > affected recipes could simply add it to their dependencies. That would > make it explicit that python is not python2. Please not that in the Debian case, nothing depends on python-is-python3. It is only provided so that users can manually install it if they have a Python script not provided by Debian that requires it. And note that Debian also provides a python-is-python2 package, and obviously things would eventually fail if packages actually were to depend on either as this would open up for conflicts. Thus all Python scripts in packages provided by Debian are required to explicitly specify either python2 or python3 on the shebang line. > > Alex //Peter
On 22 Nov 2022, at 18:03, Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest. But as per PEP 0384, /usr/bin/python could be python2, or not there at all. Software should use python3. Ross
On 22 Nov 2022, at 18:32, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > While this might be 'too soon' to conclude that python2 is truly dead, > maybe a year (or two, or three) from now it won't be. Fedora has > already made the switch, Debian will follow, and honestly, I just > can't muster any sympathy for python2 users anymore. You can't push > back paying off technical debt forever and expect others to > accommodate you. I absolutely agree with you here, which is why the one true binary should be /usr/bin/python3. Ross
On Wed, 23 Nov 2022 at 10:59, Ross Burton <Ross.Burton@arm.com> wrote: > > The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest. > > But as per PEP 0384, /usr/bin/python could be python2, or not there at all. Software should use python3. But would anyone pay attention to the PEP when writing new scripts? If /usr/bin/python is available by default, as it is already on Fedora, and probably will be on Debian (in testing/sid there is no choice between 2 and 3 anymore - 2 is gone, and that clears the road to have python->python3 installed by default), then it's not unreasonable to have the script use that. I wouldn't want to know or care about python's historical baggage. That's why I'm saying we're going to see more upstreams that are just having #!/usr/bin/python and don't give a damn, even when you point it out to them. Alex
On Wed, Nov 23, 2022 at 2:55 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Wed, 23 Nov 2022 at 10:59, Ross Burton <Ross.Burton@arm.com> wrote: > > > The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest. > > > > But as per PEP 0384, /usr/bin/python could be python2, or not there at all. Software should use python3. > > But would anyone pay attention to the PEP when writing new scripts? they should be pointed to PEP and I hope python tutors teach it from day 1. If > /usr/bin/python is available by default, as it is already on Fedora, There perhaps is a reason for that in Fedora, but it is in conflict with what PEP0384 is saying > and probably will be on Debian (in testing/sid there is no choice > between 2 and 3 anymore - 2 is gone, and that clears the road to have > python->python3 installed by default), then it's not unreasonable to > have the script use that. I wouldn't want to know or care about > python's historical baggage. That's why I'm saying we're going to see > more upstreams that are just having #!/usr/bin/python and don't give a > damn, even when you point it out to them. > If you want to ignore the baggage then know that there is no /usr/bin/python anymore. > Alex
> On 23 Nov 2022, at 16:34, Khem Raj via lists.openembedded.org <raj.khem=gmail.com@lists.openembedded.org> wrote: > > If >> /usr/bin/python is available by default, as it is already on Fedora, > > There perhaps is a reason for that in Fedora, but it is in conflict > with what PEP0384 is saying It’s not in conflict: the PEP codifies that /usr/bin/python can be python3. Or python2. Or not there at all. Basically, it’s up to the distribution, so people *writing Python* shouldn’t have any assumptions about it. Ross
diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json index 64203cf0fc..7b6f509a45 100644 --- a/meta/recipes-devtools/python/python3/python3-manifest.json +++ b/meta/recipes-devtools/python/python3/python3-manifest.json @@ -203,6 +203,7 @@ "files": [ "${bindir}/python${PYTHON_MAJMIN}", "${bindir}/python${PYTHON_MAJMIN}.real", + "${bindir}/python", "${bindir}/python3", "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h", "${libdir}/python${PYTHON_MAJMIN}/UserDict.py", diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb index 92a1f69320..18b631f79d 100644 --- a/meta/recipes-devtools/python/python3_3.11.0.bb +++ b/meta/recipes-devtools/python/python3_3.11.0.bb @@ -144,6 +144,7 @@ do_install:prepend() { do_install:append:class-target() { oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)} } do_install:append:class-native() { @@ -156,6 +157,7 @@ do_install:append:class-native() { # (these often end up too long for the #! parser in the kernel as the # buffer is 128 bytes long). ln -s python3-native/python3 ${D}${bindir}/nativepython3 + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)} # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them # and the overhead in each recipe-sysroot-native isn't worth it, particularly @@ -213,6 +215,7 @@ do_install:append() { } do_install:append:class-nativesdk () { + ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)} # Make sure we use /usr/bin/env python for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX # For historical reasons PN is empty and provided by python3-modules FILES:${PN} = "" +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}" RPROVIDES:${PN}-modules = "${PN}" FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
Currently /usr/bin/python is provided by the python2 package. If python2 is not installed, the built image lacks a provider for it. This results in failed scripts when using '/usr/bin/python' shebang. This patch adds a /usr/bin/python symlink for python3 to fix this issue. For images containing python2 and python3, the do_rootfs task will fail because now both of them provide the same file. To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which, if set, will prevent python3 from creating the symlink. The link is created with relative path because using an absolute path would fail for native and nativesdk. Signed-off-by: Markus Volk <f_l_k@t-online.de> --- meta/recipes-devtools/python/python3/python3-manifest.json | 1 + meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++ 2 files changed, 5 insertions(+)