[2/2,v2] python3-numpy: fix ptests

Fix the numpy ptests by doing the following:

- Add gcc and meson as ptest RDEPENDS in the recipe, and clean up the RDEPENDS
  lists (sorting alphabetically) in the process;
- Convert run-ptest to a shell script that sets PYTEST_DEBUG_TEMPROOT
  to a directory inside the same path that contains the script, create
  that directory, and then invoke the tests in the same manner as before
  with 'python3 -c ...', before finally running 'rm -rf' on the directory we
  created (so that the space is freed up);
- Add a patch to skip the test_big_arrays test, which uses a lot of
  space;
- Set 'IMAGE_ROOTFS_EXTRA_SPACE = "3048576"' for python3-numpy in
  core-image-ptest.bb;
- Also set 'QB_MEM:virtclass-mcextend-python3-numpy = "-m 4096"' in
  core-image-ptest.bb;
- Move python3-numpy from the PTESTS_PROBLEMS list to the PTESTS_SLOW
  one.

Results on qemux86-64:

|==== 47301 passed, 4171 skipped, 33 xfailed, 5 xpassed in 148.64s (0:02:28) ====
|DURATION: 152
|END: /usr/lib/python3-numpy/ptest
|2025-04-24T14:02
|STOP: ptest-runner
|TOTAL: 1 FAIL: 0

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
---
v2 cleans up the RDEPENDS lists in the recipe (after we added gcc and meson),
and also tweaks run-ptest to remove the temporary directory after the tests are
done to save space.

 .../distro/include/ptest-packagelists.inc     |  2 +-
 meta/recipes-core/images/core-image-ptest.bb  |  4 +++
 .../python/python3-numpy/run-ptest            | 15 ++++++---
 .../python/python3-numpy_2.2.5.bb             | 33 +++++++++++--------
 4 files changed, 35 insertions(+), 19 deletions(-)

On Thu, 2025-04-24 at 12:27 -0400, Trevor Gamblin via lists.openembedded.org wrote:
> Fix the numpy ptests by doing the following:
> 
> - Add gcc and meson as ptest RDEPENDS in the recipe, and clean up the RDEPENDS
>   lists (sorting alphabetically) in the process;
> - Convert run-ptest to a shell script that sets PYTEST_DEBUG_TEMPROOT
>   to a directory inside the same path that contains the script, create
>   that directory, and then invoke the tests in the same manner as before
>   with 'python3 -c ...', before finally running 'rm -rf' on the directory we
>   created (so that the space is freed up);
> - Add a patch to skip the test_big_arrays test, which uses a lot of
>   space;
> - Set 'IMAGE_ROOTFS_EXTRA_SPACE = "3048576"' for python3-numpy in
>   core-image-ptest.bb;
> - Also set 'QB_MEM:virtclass-mcextend-python3-numpy = "-m 4096"' in
>   core-image-ptest.bb;
> - Move python3-numpy from the PTESTS_PROBLEMS list to the PTESTS_SLOW
>   one.
> 
> Results on qemux86-64:
> 
> > ==== 47301 passed, 4171 skipped, 33 xfailed, 5 xpassed in 148.64s (0:02:28) ====
> > DURATION: 152
> > END: /usr/lib/python3-numpy/ptest
> > 2025-04-24T14:02
> > STOP: ptest-runner
> > TOTAL: 1 FAIL: 0
> 
> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
> ---
> v2 cleans up the RDEPENDS lists in the recipe (after we added gcc and meson),
> and also tweaks run-ptest to remove the temporary directory after the tests are
> done to save space.
> 
>  .../distro/include/ptest-packagelists.inc     |  2 +-
>  meta/recipes-core/images/core-image-ptest.bb  |  4 +++
>  .../python/python3-numpy/run-ptest            | 15 ++++++---
>  .../python/python3-numpy_2.2.5.bb             | 33 +++++++++++--------
>  4 files changed, 35 insertions(+), 19 deletions(-)
> 
> diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
> index f3fdededf66..2a27c0c5d41 100644
> --- a/meta/conf/distro/include/ptest-packagelists.inc
> +++ b/meta/conf/distro/include/ptest-packagelists.inc
> @@ -120,6 +120,7 @@ PTESTS_SLOW = "\
>      python3-cffi \
>      python3-click \
>      python3-cryptography \
> +    python3-numpy \
>      python3-xmltodict \
>      strace \
>      tar \
> @@ -155,5 +156,4 @@ PTESTS_PROBLEMS = "\
>      mdadm \
>      numactl \
>      python3-license-expression \
> -    python3-numpy \
>  "
> diff --git a/meta/recipes-core/images/core-image-ptest.bb b/meta/recipes-core/images/core-image-ptest.bb
> index a0ff8d9528e..73270d10676 100644
> --- a/meta/recipes-core/images/core-image-ptest.bb
> +++ b/meta/recipes-core/images/core-image-ptest.bb
> @@ -27,11 +27,15 @@ IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-lttng-tools = "1524288"
>  # tar-ptest in particular needs more space
>  IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-tar = "1524288"
>  
> +# python3-numpy-ptest requires a lot of extra space
> +IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-python3-numpy = "3048576"
> +
>  # ptests need more memory than standard to avoid the OOM killer
>  QB_MEM = "-m 1024"
>  QB_MEM:virtclass-mcextend-lttng-tools = "-m 4096"
>  QB_MEM:virtclass-mcextend-python3 = "-m 2048"
>  QB_MEM:virtclass-mcextend-python3-cryptography = "-m 5100"
> +QB_MEM:virtclass-mcextend-python3-numpy = "-m 4096"
>  QB_MEM:virtclass-mcextend-tcl = "-m 5100"
>  
>  TEST_SUITES = "ping ssh parselogs ptest"
> diff --git a/meta/recipes-devtools/python/python3-numpy/run-ptest b/meta/recipes-devtools/python/python3-numpy/run-ptest
> index 9a1c72aeb10..10dfb1246d3 100644
> --- a/meta/recipes-devtools/python/python3-numpy/run-ptest
> +++ b/meta/recipes-devtools/python/python3-numpy/run-ptest
> @@ -1,5 +1,12 @@
> -#!/usr/bin/env python3
> -
> -import numpy
> -numpy.test(label='full', verbose=2)
> +#!/bin/sh
>  
> +# By default, numpy will use /tmp as the root path for temporary files used
> +# during tests, but if this is a tmpfs it may fill up quickly and cause many of
> +# the tests to report "no space left on device" errors. Create a custom
> +# directory for these and point pytest at it so we can take advantage of the
> +# storage provided in the rootfs.
> +TEMPROOT="/usr/lib/python3-numpy/ptest/tmp"
> +export PYTEST_DEBUG_TEMPROOT="$TEMPROOT"
> +mkdir -p "$TEMPROOT"
> +python3 -c "import numpy; numpy.test(label='full', verbose=2)"
> +rm -rf "$TEMPROOT"
> diff --git a/meta/recipes-devtools/python/python3-numpy_2.2.5.bb b/meta/recipes-devtools/python/python3-numpy_2.2.5.bb
> index c6a37c0a85f..05d2b1a1f2c 100644
> --- a/meta/recipes-devtools/python/python3-numpy_2.2.5.bb
> +++ b/meta/recipes-devtools/python/python3-numpy_2.2.5.bb
> @@ -10,6 +10,7 @@ SRCNAME = "numpy"
>  SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/${SRCNAME}-${PV}.tar.gz \
>             file://0001-Don-t-search-usr-and-so-on-for-libraries-by-default-.patch \
>             file://fix_reproducibility.patch \
> +           file://0001-lib-tests-test_io-skip-test_big_arrays.patch \
>             file://run-ptest \
>             "
>  SRC_URI[sha256sum] = "a9c0d994680cd991b1cb772e8b297340085466a6fe964bc9d4e80f5e2f43c291"

The patch file is missing from the patch you've sent!

Cheers,

Richard