mbox series

[00/20] python3: fix run-time dependencies

Message ID 20230517080639.9336-1-brgl@bgdev.pl
Headers show
Series python3: fix run-time dependencies | expand

Message

Bartosz Golaszewski May 17, 2023, 8:06 a.m. UTC 
From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>

I noticed that every now and then I import some python package into an
image only to find out the recipe didn't pull in all required run-time
dependencies. I would fix it whenever I noticed it but figured that I
could right a script that would go over all python packages, build a
minimal image and see that all its modules can be imported.

The patches in this series are the result of that. Most simply add
missing RDEPENDS, some are additional tweaks to recipes and the last
three add new standard library packages to the manifest. These are the
ones for which simply adding python3-misc would still be insufficient in
terms of run-time dependencies.

I plan to do the same for meta-python but this will take more time and I
need to think of some automated way of updating the recipes first.

Bartosz Golaszewski (20):
  python3-async: add missing run-time dependencies
  python3-attrs: unify RDEPENDS
  python3-attrs: don't use PYTHON_PN
  python3-attrs: add missing run-time dependencies
  python3-certifi: add missing run-time dependencies
  python3-libfdt: new package
  python3-dtschema: add missing run-time dependencies
  python3-hypothesis: fix run-time dependencies
  python3-tomli: add missing run-time dependencies
  python3-pathspec: add missing run-time dependencies
  python3-installer: add missing run-time dependencies
  python3-sphinx-rtd-theme: add missing run-time dependencies
  python3-setuptools-rust: fix RDEPENDS and allow target build
  python3-pyproject-hooks: add missing run-time dependencies
  python3-pycryptodome: don't use PYTHON_PN
  python3-pycryptodome: add missing run-time dependencies
  python3-pygobject: add missing run-time dependencies
  python3-manifest: cgitb: new package
  python3-manifest: zipapp: new package
  python3-manifest: turtle: new package

 meta/recipes-devtools/python/python-async.inc |  5 ++-
 .../python/python-pycryptodome.inc            |  8 ++--
 .../python/python3-attrs_23.1.0.bb            | 15 +++----
 .../python/python3-certifi_2022.12.7.bb       |  2 +
 .../python/python3-dtschema_2023.4.bb         |  7 +++-
 .../python/python3-hypothesis_6.71.0.bb       |  2 +-
 .../python/python3-installer_0.7.0.bb         |  6 +++
 .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++
 .../python/python3-pathspec_0.11.1.bb         |  2 +
 .../python/python3-pygobject_3.44.1.bb        |  5 ++-
 .../python/python3-pyproject-hooks_1.0.0.bb   |  5 ++-
 .../python/python3-setuptools-rust_1.5.2.bb   | 17 ++++----
 .../python/python3-sphinx-rtd-theme_1.2.0.bb  |  5 ++-
 .../python/python3-tomli_2.0.1.bb             |  5 +++
 .../python/python3/python3-manifest.json      | 42 +++++++++++++++++++
 15 files changed, 121 insertions(+), 25 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb

Comments

Alexander Kanavin May 17, 2023, 8:38 a.m. UTC | #1 
Thanks for this work. Perhaps we could catch this easier if there are
runtime tests for those items, maybe in the shape of ptest.

Sometimes recipes have no longer needed dependencies too, and trimming
those down is harder, but also appreciated.

Alex

On Wed, 17 May 2023 at 10:06, Bartosz Golaszewski <brgl@bgdev.pl> wrote:
>
> From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
>
> I noticed that every now and then I import some python package into an
> image only to find out the recipe didn't pull in all required run-time
> dependencies. I would fix it whenever I noticed it but figured that I
> could right a script that would go over all python packages, build a
> minimal image and see that all its modules can be imported.
>
> The patches in this series are the result of that. Most simply add
> missing RDEPENDS, some are additional tweaks to recipes and the last
> three add new standard library packages to the manifest. These are the
> ones for which simply adding python3-misc would still be insufficient in
> terms of run-time dependencies.
>
> I plan to do the same for meta-python but this will take more time and I
> need to think of some automated way of updating the recipes first.
>
> Bartosz Golaszewski (20):
>   python3-async: add missing run-time dependencies
>   python3-attrs: unify RDEPENDS
>   python3-attrs: don't use PYTHON_PN
>   python3-attrs: add missing run-time dependencies
>   python3-certifi: add missing run-time dependencies
>   python3-libfdt: new package
>   python3-dtschema: add missing run-time dependencies
>   python3-hypothesis: fix run-time dependencies
>   python3-tomli: add missing run-time dependencies
>   python3-pathspec: add missing run-time dependencies
>   python3-installer: add missing run-time dependencies
>   python3-sphinx-rtd-theme: add missing run-time dependencies
>   python3-setuptools-rust: fix RDEPENDS and allow target build
>   python3-pyproject-hooks: add missing run-time dependencies
>   python3-pycryptodome: don't use PYTHON_PN
>   python3-pycryptodome: add missing run-time dependencies
>   python3-pygobject: add missing run-time dependencies
>   python3-manifest: cgitb: new package
>   python3-manifest: zipapp: new package
>   python3-manifest: turtle: new package
>
>  meta/recipes-devtools/python/python-async.inc |  5 ++-
>  .../python/python-pycryptodome.inc            |  8 ++--
>  .../python/python3-attrs_23.1.0.bb            | 15 +++----
>  .../python/python3-certifi_2022.12.7.bb       |  2 +
>  .../python/python3-dtschema_2023.4.bb         |  7 +++-
>  .../python/python3-hypothesis_6.71.0.bb       |  2 +-
>  .../python/python3-installer_0.7.0.bb         |  6 +++
>  .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++
>  .../python/python3-pathspec_0.11.1.bb         |  2 +
>  .../python/python3-pygobject_3.44.1.bb        |  5 ++-
>  .../python/python3-pyproject-hooks_1.0.0.bb   |  5 ++-
>  .../python/python3-setuptools-rust_1.5.2.bb   | 17 ++++----
>  .../python/python3-sphinx-rtd-theme_1.2.0.bb  |  5 ++-
>  .../python/python3-tomli_2.0.1.bb             |  5 +++
>  .../python/python3/python3-manifest.json      | 42 +++++++++++++++++++
>  15 files changed, 121 insertions(+), 25 deletions(-)
>  create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb
>
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181449): https://lists.openembedded.org/g/openembedded-core/message/181449
> Mute This Topic: https://lists.openembedded.org/mt/98943996/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Tim Orling May 19, 2023, 12:06 a.m. UTC | #2 
Thank you, Bartosz. I didn't review in great depth, but others have done
so. This series looks like a
step in the right direction.

On Wed, May 17, 2023 at 1:39 AM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> Thanks for this work. Perhaps we could catch this easier if there are
> runtime tests for those items, maybe in the shape of ptest.
>
> Sometimes recipes have no longer needed dependencies too, and trimming
> those down is harder, but also appreciated.
>
> One challenge has been the additional dependencies that come in for
ptests. Another is that
many PyPi packages do not include their /tests/ directory, meaning we would
need to switch to
the git fetcher in many cases. My intent has always been "full" ptest
coverage, when practical.

Alex
>
> On Wed, 17 May 2023 at 10:06, Bartosz Golaszewski <brgl@bgdev.pl> wrote:
> >
> > From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> >
> > I noticed that every now and then I import some python package into an
> > image only to find out the recipe didn't pull in all required run-time
> > dependencies. I would fix it whenever I noticed it but figured that I
> > could right a script that would go over all python packages, build a
> > minimal image and see that all its modules can be imported.
> >
> > The patches in this series are the result of that. Most simply add
> > missing RDEPENDS, some are additional tweaks to recipes and the last
> > three add new standard library packages to the manifest. These are the
> > ones for which simply adding python3-misc would still be insufficient in
> > terms of run-time dependencies.
> >
> > I plan to do the same for meta-python but this will take more time and I
> > need to think of some automated way of updating the recipes first.
> >
> > Bartosz Golaszewski (20):
> >   python3-async: add missing run-time dependencies
> >   python3-attrs: unify RDEPENDS
> >   python3-attrs: don't use PYTHON_PN
> >   python3-attrs: add missing run-time dependencies
> >   python3-certifi: add missing run-time dependencies
> >   python3-libfdt: new package
> >   python3-dtschema: add missing run-time dependencies
> >   python3-hypothesis: fix run-time dependencies
> >   python3-tomli: add missing run-time dependencies
> >   python3-pathspec: add missing run-time dependencies
> >   python3-installer: add missing run-time dependencies
> >   python3-sphinx-rtd-theme: add missing run-time dependencies
> >   python3-setuptools-rust: fix RDEPENDS and allow target build
> >   python3-pyproject-hooks: add missing run-time dependencies
> >   python3-pycryptodome: don't use PYTHON_PN
> >   python3-pycryptodome: add missing run-time dependencies
> >   python3-pygobject: add missing run-time dependencies
> >   python3-manifest: cgitb: new package
> >   python3-manifest: zipapp: new package
> >   python3-manifest: turtle: new package
> >
> >  meta/recipes-devtools/python/python-async.inc |  5 ++-
> >  .../python/python-pycryptodome.inc            |  8 ++--
> >  .../python/python3-attrs_23.1.0.bb            | 15 +++----
> >  .../python/python3-certifi_2022.12.7.bb       |  2 +
> >  .../python/python3-dtschema_2023.4.bb         |  7 +++-
> >  .../python/python3-hypothesis_6.71.0.bb       |  2 +-
> >  .../python/python3-installer_0.7.0.bb         |  6 +++
> >  .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++
> >  .../python/python3-pathspec_0.11.1.bb         |  2 +
> >  .../python/python3-pygobject_3.44.1.bb        |  5 ++-
> >  .../python/python3-pyproject-hooks_1.0.0.bb   |  5 ++-
> >  .../python/python3-setuptools-rust_1.5.2.bb   | 17 ++++----
> >  .../python/python3-sphinx-rtd-theme_1.2.0.bb  |  5 ++-
> >  .../python/python3-tomli_2.0.1.bb             |  5 +++
> >  .../python/python3/python3-manifest.json      | 42 +++++++++++++++++++
> >  15 files changed, 121 insertions(+), 25 deletions(-)
> >  create mode 100644 meta/recipes-devtools/python/
> python3-libfdt_1.7.0.post1.bb
> >
> > --
> > 2.39.2
> >
> >
> >
> >
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181479):
> https://lists.openembedded.org/g/openembedded-core/message/181479
> Mute This Topic: https://lists.openembedded.org/mt/98943996/924729
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> ticotimo@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>