| Message ID | 20241120195124.1041875-1-ross.burton@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] classes-recipe/python_pep517: remove all RECORDS files | expand |
On 2024-11-20 14:51, Ross Burton via lists.openembedded.org wrote: > Python modules install metadata into a .dist-info directory, one of which > is RECORD, which contains the files that were installed and their > checksum[1]. This is typically used by pip to validate the install, or > to know what files to remove when the module is uninstalled. > > This is slightly problematic when we need to do patching of installed > .py files in do_install(), as the RECORD file has already been written > at that point. > > However, the RECORD files only really have a use outside of a system- > managed environment, which our python packages are. We already have > commands to verify and remove modules (opkg, dpkg, rpm) and the RECORD > file existing simply allows people to 'sudo pip' and alter the package- > managed directories outside of the package manager. > > This is not a good idea, and some other distros remove the RECORD file > to stop this possibility: > - Debian[2] > - Fedora[3] > - Gentoo[4] > > We can follow for all packages which inherit python_pep517, which is the > majority of the Python packages now. > > [1] https://peps.python.org/pep-0491/#the-dist-info-directory > [2] https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/fs.py?ref_type=heads#L185 > [3] https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject#_105 > [4] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=73c49f3c00415dee99407dabba8d3b22895c9d25 > > Signed-off-by: Ross Burton <ross.burton@arm.com> As discussed in IRC, LGTM. > --- > meta/classes-recipe/python_pep517.bbclass | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta/classes-recipe/python_pep517.bbclass b/meta/classes-recipe/python_pep517.bbclass > index c30674c8ec8..e8cd1923ef2 100644 > --- a/meta/classes-recipe/python_pep517.bbclass > +++ b/meta/classes-recipe/python_pep517.bbclass > @@ -50,6 +50,8 @@ python_pep517_do_install () { > fi > > nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} --interpreter "${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} ${PEP517_WHEEL_PATH}/*.whl > + > + find ${D} -path *.dist-info/RECORD -delete > } > > # A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native. > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#207479): https://lists.openembedded.org/g/openembedded-core/message/207479 > Mute This Topic: https://lists.openembedded.org/mt/109691086/7611679 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [tgamblin@baylibre.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, Nov 20, 2024 at 1:51 PM Trevor Gamblin via lists.openembedded.org <tgamblin=baylibre.com@lists.openembedded.org> wrote: > > On 2024-11-20 14:51, Ross Burton via lists.openembedded.org wrote: > > Python modules install metadata into a .dist-info directory, one of which > > is RECORD, which contains the files that were installed and their > > checksum[1]. This is typically used by pip to validate the install, or > > to know what files to remove when the module is uninstalled. > > > > This is slightly problematic when we need to do patching of installed > > .py files in do_install(), as the RECORD file has already been written > > at that point. > > > > However, the RECORD files only really have a use outside of a system- > > managed environment, which our python packages are. We already have > > commands to verify and remove modules (opkg, dpkg, rpm) and the RECORD > > file existing simply allows people to 'sudo pip' and alter the package- > > managed directories outside of the package manager. > > > > This is not a good idea, and some other distros remove the RECORD file > > to stop this possibility: > > - Debian[2] > > - Fedora[3] > > - Gentoo[4] > > > > We can follow for all packages which inherit python_pep517, which is the > > majority of the Python packages now. > > > > [1] https://peps.python.org/pep-0491/#the-dist-info-directory > > [2] > https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/fs.py?ref_type=heads#L185 > > [3] > https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject#_105 > > [4] > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=73c49f3c00415dee99407dabba8d3b22895c9d25 > > > > Signed-off-by: Ross Burton <ross.burton@arm.com> > As discussed in IRC, LGTM. Reviewed-by: Tim Orling <tim.orling@konsulko.com> Let us keep sane by following the upstream distributions lead. Thank you Ross! > > > --- > > meta/classes-recipe/python_pep517.bbclass | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/meta/classes-recipe/python_pep517.bbclass > b/meta/classes-recipe/python_pep517.bbclass > > index c30674c8ec8..e8cd1923ef2 100644 > > --- a/meta/classes-recipe/python_pep517.bbclass > > +++ b/meta/classes-recipe/python_pep517.bbclass > > @@ -50,6 +50,8 @@ python_pep517_do_install () { > > fi > > > > nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} > --interpreter "${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} > ${PEP517_WHEEL_PATH}/*.whl > > + > > + find ${D} -path *.dist-info/RECORD -delete > > } > > > > # A manual do_install that just uses unzip for bootstrapping purposes. > Callers should DEPEND on unzip-native. > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#207488): > https://lists.openembedded.org/g/openembedded-core/message/207488 > Mute This Topic: https://lists.openembedded.org/mt/109691086/924729 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > ticotimo@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
diff --git a/meta/classes-recipe/python_pep517.bbclass b/meta/classes-recipe/python_pep517.bbclass index c30674c8ec8..e8cd1923ef2 100644 --- a/meta/classes-recipe/python_pep517.bbclass +++ b/meta/classes-recipe/python_pep517.bbclass @@ -50,6 +50,8 @@ python_pep517_do_install () { fi nativepython3 -m installer ${INSTALL_WHEEL_COMPILE_BYTECODE} --interpreter "${USRBINPATH}/env ${PEP517_INSTALL_PYTHON}" --destdir=${D} ${PEP517_WHEEL_PATH}/*.whl + + find ${D} -path *.dist-info/RECORD -delete } # A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native.
Python modules install metadata into a .dist-info directory, one of which is RECORD, which contains the files that were installed and their checksum[1]. This is typically used by pip to validate the install, or to know what files to remove when the module is uninstalled. This is slightly problematic when we need to do patching of installed .py files in do_install(), as the RECORD file has already been written at that point. However, the RECORD files only really have a use outside of a system- managed environment, which our python packages are. We already have commands to verify and remove modules (opkg, dpkg, rpm) and the RECORD file existing simply allows people to 'sudo pip' and alter the package- managed directories outside of the package manager. This is not a good idea, and some other distros remove the RECORD file to stop this possibility: - Debian[2] - Fedora[3] - Gentoo[4] We can follow for all packages which inherit python_pep517, which is the majority of the Python packages now. [1] https://peps.python.org/pep-0491/#the-dist-info-directory [2] https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/fs.py?ref_type=heads#L185 [3] https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject#_105 [4] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=73c49f3c00415dee99407dabba8d3b22895c9d25 Signed-off-by: Ross Burton <ross.burton@arm.com> --- meta/classes-recipe/python_pep517.bbclass | 2 ++ 1 file changed, 2 insertions(+)