| Message ID | 20250428174416.379821-3-tgamblin@baylibre.com |
|---|---|
| State | New |
| Headers | show |
| Series | python3-numpy: upgrade and ptest fix | expand |
On 28 Apr 2025, at 18:44, Trevor Gamblin via lists.openembedded.org <tgamblin=baylibre.com@lists.openembedded.org> wrote: > - Add gcc and meson as ptest RDEPENDS in the recipe; Personally I’d prefer to skip a few tests than pull in GCC and Meson… > +@@ -205,6 +205,7 @@ def roundtrip(self, *args, **kwargs): > + self.arr_reloaded.fid.close() > + os.remove(self.arr_reloaded.fid.name) > + > ++ @pytest.mark.skip(reason="Requires a large amount of space") > + @pytest.mark.skipif(IS_PYPY, reason="Hangs on PyPy") > + @pytest.mark.skipif(not IS_64BIT, reason="Needs 64bit platform") > + @pytest.mark.slow How many tests are marked as slow, and how slow is this test suite in total? Just skipping the slow tests is an option instead of carrying a patch forever. > +TEMPROOT="/usr/lib/python3-numpy/ptest/tmp" > +export PYTEST_DEBUG_TEMPROOT="$TEMPROOT" > +mkdir -p "$TEMPROOT" > +pytest --automake /usr/lib/python3.*/site-packages/numpy > +rm -rf "$TEMPROOT" Why two variables? Ross
On 2025-04-29 08:56, Ross Burton wrote: > On 28 Apr 2025, at 18:44, Trevor Gamblin via lists.openembedded.org <tgamblin=baylibre.com@lists.openembedded.org> wrote: >> - Add gcc and meson as ptest RDEPENDS in the recipe; > Personally I’d prefer to skip a few tests than pull in GCC and Meson… > >> +@@ -205,6 +205,7 @@ def roundtrip(self, *args, **kwargs): >> + self.arr_reloaded.fid.close() >> + os.remove(self.arr_reloaded.fid.name) >> + >> ++ @pytest.mark.skip(reason="Requires a large amount of space") >> + @pytest.mark.skipif(IS_PYPY, reason="Hangs on PyPy") >> + @pytest.mark.skipif(not IS_64BIT, reason="Needs 64bit platform") >> + @pytest.mark.slow > How many tests are marked as slow, and how slow is this test suite in total? Just skipping the slow tests is an option instead of carrying a patch forever. It looks like adding `-m "not slow"` to the pytest invocation results in 2815 tests being omitted. slow tests in: Testsuite summary # TOTAL: 51510 # PASS: 47301 # SKIP: 4171 # XFAIL: 33 # FAIL: 0 # XPASS: 5 # ERROR: 0 without: Testsuite summary # TOTAL: 48695 # PASS: 44851 # SKIP: 3807 # XFAIL: 32 # FAIL: 0 # XPASS: 5 # ERROR: 0 If I drop both gcc and meson from RDEPENDS while trying to skip slow tests, the ptest script fails after reporting 30 errors. If I drop just gcc, it runs but still generates 9 errors. I can remove those nine from the output by skipping the test_mem_policy test. Dropping meson looks like a different story - that "30 errors" thing that I mentioned above comes from 30 different test modules (each containing various numbers of test cases) failing to find meson when they begin setup. I'm not sure if we want to try disabling all of those... > >> +TEMPROOT="/usr/lib/python3-numpy/ptest/tmp" >> +export PYTEST_DEBUG_TEMPROOT="$TEMPROOT" >> +mkdir -p "$TEMPROOT" >> +pytest --automake /usr/lib/python3.*/site-packages/numpy >> +rm -rf "$TEMPROOT" > Why two variables? Good point - another case of me leaving stuff in from early versions of the script... > > Ross
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index f3fdededf66..2a27c0c5d41 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -120,6 +120,7 @@ PTESTS_SLOW = "\ python3-cffi \ python3-click \ python3-cryptography \ + python3-numpy \ python3-xmltodict \ strace \ tar \ @@ -155,5 +156,4 @@ PTESTS_PROBLEMS = "\ mdadm \ numactl \ python3-license-expression \ - python3-numpy \ " diff --git a/meta/recipes-core/images/core-image-ptest.bb b/meta/recipes-core/images/core-image-ptest.bb index a0ff8d9528e..73270d10676 100644 --- a/meta/recipes-core/images/core-image-ptest.bb +++ b/meta/recipes-core/images/core-image-ptest.bb @@ -27,11 +27,15 @@ IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-lttng-tools = "1524288" # tar-ptest in particular needs more space IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-tar = "1524288" +# python3-numpy-ptest requires a lot of extra space +IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-python3-numpy = "3048576" + # ptests need more memory than standard to avoid the OOM killer QB_MEM = "-m 1024" QB_MEM:virtclass-mcextend-lttng-tools = "-m 4096" QB_MEM:virtclass-mcextend-python3 = "-m 2048" QB_MEM:virtclass-mcextend-python3-cryptography = "-m 5100" +QB_MEM:virtclass-mcextend-python3-numpy = "-m 4096" QB_MEM:virtclass-mcextend-tcl = "-m 5100" TEST_SUITES = "ping ssh parselogs ptest" diff --git a/meta/recipes-devtools/python/python3-numpy/0001-lib-tests-test_io-skip-test_big_arrays.patch b/meta/recipes-devtools/python/python3-numpy/0001-lib-tests-test_io-skip-test_big_arrays.patch new file mode 100644 index 00000000000..0732f6ef6ab --- /dev/null +++ b/meta/recipes-devtools/python/python3-numpy/0001-lib-tests-test_io-skip-test_big_arrays.patch @@ -0,0 +1,34 @@ +From 9ab82e22a02a17754bcd45b6b4de6bfd16c6db51 Mon Sep 17 00:00:00 2001 +From: Trevor Gamblin <tgamblin@baylibre.com> +Date: Thu, 24 Apr 2025 09:10:34 -0400 +Subject: [PATCH] lib/tests/test_io: skip test_big_arrays + +This test consistently fails with the following error in our ptest +images: + +|OSError: [Errno 28] No space left on device + +So skip this test. + +Upstream-Status: Inappropriate [embedded-specific] + +Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> +--- + numpy/lib/tests/test_io.py | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/numpy/lib/tests/test_io.py b/numpy/lib/tests/test_io.py +index 742915e22e..97ae18d7f8 100644 +--- a/numpy/lib/tests/test_io.py ++++ b/numpy/lib/tests/test_io.py +@@ -205,6 +205,7 @@ def roundtrip(self, *args, **kwargs): + self.arr_reloaded.fid.close() + os.remove(self.arr_reloaded.fid.name) + ++ @pytest.mark.skip(reason="Requires a large amount of space") + @pytest.mark.skipif(IS_PYPY, reason="Hangs on PyPy") + @pytest.mark.skipif(not IS_64BIT, reason="Needs 64bit platform") + @pytest.mark.slow +-- +2.39.5 + diff --git a/meta/recipes-devtools/python/python3-numpy/run-ptest b/meta/recipes-devtools/python/python3-numpy/run-ptest index 9a1c72aeb10..261916d0c12 100644 --- a/meta/recipes-devtools/python/python3-numpy/run-ptest +++ b/meta/recipes-devtools/python/python3-numpy/run-ptest @@ -1,5 +1,12 @@ -#!/usr/bin/env python3 - -import numpy -numpy.test(label='full', verbose=2) +#!/bin/sh +# By default, numpy will use /tmp as the root path for temporary files used +# during tests, but if this is a tmpfs it may fill up quickly and cause many of +# the tests to report "no space left on device" errors. Create a custom +# directory for these and point pytest at it so we can take advantage of the +# storage provided in the rootfs. +TEMPROOT="/usr/lib/python3-numpy/ptest/tmp" +export PYTEST_DEBUG_TEMPROOT="$TEMPROOT" +mkdir -p "$TEMPROOT" +pytest --automake /usr/lib/python3.*/site-packages/numpy +rm -rf "$TEMPROOT" diff --git a/meta/recipes-devtools/python/python3-numpy_2.2.5.bb b/meta/recipes-devtools/python/python3-numpy_2.2.5.bb index c6a37c0a85f..273e1665e96 100644 --- a/meta/recipes-devtools/python/python3-numpy_2.2.5.bb +++ b/meta/recipes-devtools/python/python3-numpy_2.2.5.bb @@ -10,6 +10,7 @@ SRCNAME = "numpy" SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/${SRCNAME}-${PV}.tar.gz \ file://0001-Don-t-search-usr-and-so-on-for-libraries-by-default-.patch \ file://fix_reproducibility.patch \ + file://0001-lib-tests-test_io-skip-test_big_arrays.patch \ file://run-ptest \ " SRC_URI[sha256sum] = "a9c0d994680cd991b1cb772e8b297340085466a6fe964bc9d4e80f5e2f43c291" @@ -41,32 +42,37 @@ FILES:${PN}-staticdev += "${PYTHON_SITEPACKAGES_DIR}/numpy/_core/lib/*.a \ " # install what is needed for numpy.test() -RDEPENDS:${PN} = "python3-unittest \ +RDEPENDS:${PN} = "\ + python3-compression \ + python3-ctypes \ + python3-datetime \ python3-difflib \ - python3-pprint \ - python3-pickle \ - python3-shell \ python3-doctest \ - python3-datetime \ + python3-email \ + python3-json \ python3-misc \ python3-mmap \ + python3-multiprocessing \ python3-netclient \ python3-numbers \ - python3-pydoc \ + python3-pickle \ python3-pkgutil \ - python3-email \ - python3-compression \ - python3-ctypes \ + python3-pprint \ + python3-pydoc \ + python3-shell \ python3-threading \ - python3-multiprocessing \ - python3-json \ + python3-unittest \ " -RDEPENDS:${PN}-ptest += "python3-pytest \ +RDEPENDS:${PN}-ptest += "\ + gcc \ + ldd \ + meson \ python3-hypothesis \ - python3-sortedcontainers \ + python3-pytest \ python3-resource \ + python3-sortedcontainers \ python3-typing-extensions \ - ldd \ + python3-unittest-automake-output \ " BBCLASSEXTEND = "native nativesdk"
Fix the numpy ptests by doing the following: - Add gcc and meson as ptest RDEPENDS in the recipe; - Add python3-unittest-automake-output as a ptest RDEPENDS; - Convert run-ptest to a shell script that sets PYTEST_DEBUG_TEMPROOT to a directory inside the same path that contains the script, create that directory, and then invoke the tests with `pytest --automake` pointed at the numpy path in site-packages (copying the tests into the normal PTEST_DIRECTORY seems to cause module import breakages in some tests); - Add a patch to skip the test_big_arrays test, which uses a lot of space; - Set 'IMAGE_ROOTFS_EXTRA_SPACE = "3048576"' for python3-numpy in core-image-ptest.bb; - Also set 'QB_MEM:virtclass-mcextend-python3-numpy = "-m 4096"' in core-image-ptest.bb; - Move python3-numpy from the PTESTS_PROBLEMS list to the PTESTS_SLOW one. Results on qemux86-64: |============================================================================ |Testsuite summary |# TOTAL: 51510 |# PASS: 47301 |# SKIP: 4171 |# XFAIL: 33 |# FAIL: 0 |# XPASS: 5 |# ERROR: 0 |DURATION: 164 |END: /usr/lib/python3-numpy/ptest |2025-04-28T17:26 |STOP: ptest-runner |TOTAL: 1 FAIL: 0 Note that fewer tests would be skipped if python3-mypy were present, but since that's in meta-python I've opted to leave things as-is. Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> --- v4 modifies the run-ptest script to use `pytest --automake` for better test output formatting (matching what we expect from other ptests), and adds python3-unittest-automake-output to the ptest RDEPENDS list. Also add a note about mypy in the commit message. v3 includes the 0001-lib-tests-test_io-skip-test_big_arrays.patch file, which was mentioned in the recipe in v2 but not committed before sending. v2 cleans up the RDEPENDS lists in the recipe (after we added gcc and meson), and also tweaks run-ptest to remove the temporary directory after the tests are done to save space. .../distro/include/ptest-packagelists.inc | 2 +- meta/recipes-core/images/core-image-ptest.bb | 4 +++ ...b-tests-test_io-skip-test_big_arrays.patch | 34 +++++++++++++++++++ .../python/python3-numpy/run-ptest | 15 +++++--- .../python/python3-numpy_2.2.5.bb | 34 +++++++++++-------- 5 files changed, 70 insertions(+), 19 deletions(-) create mode 100644 meta/recipes-devtools/python/python3-numpy/0001-lib-tests-test_io-skip-test_big_arrays.patch