Message ID | 20241029193745.2919129-1-tgamblin@baylibre.com |
---|---|
State | Under Review |
Headers | show |
Series | python3-numpy: upgrade 1.26.4 -> 2.1.2 | expand |
On Tue, Oct 29, 2024 at 03:37:45PM -0400, Trevor Gamblin wrote: > - This release contains numerous API changes and bug fixes. > Changelog: https://github.com/numpy/numpy/releases > > - Notably, the build backend has switched to mesonpy since 1.26.4. > > - Patch 0001-numpy-core-Define-RISCV-32-support.patch had to be updated > to change the paths for the target files. It has now been merged > upstream, but isn't included in any release yet. PR for reference: > > https://github.com/numpy/numpy/pull/17780 > > - Builds OK and seems to be compatible with current piglit and pandas > (meta-python), which was not the case for the previous upgrade attempt > to numpy 2.0.1. > > - For numpy, add 'pkgconfig' to inherit to avoid errors like this at > do_compile: > > | Found Pkg-config: NO > | Run-time dependency python found: YES 3.12 > | Has header "Python.h" with dependency python: NO > | > | ../numpy-2.0.0/meson.build:44:2: ERROR: Problem encountered: Cannot compile `Python.h`. Perhaps you need to install python-dev|python-devel > | > | A full log can be found at /home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/2.0.0/build/meson-logs/meson-log.txt > > - Also modify the FILES:${PN}-staticdev line to fix a QA issue. > > License-Update: Change copyright year to 2024 > > Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Hi Trevor, It looks like this is breaking reproducible autobuilder: AssertionError: The following deb packages are different and not in exclusion list: /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/python3-numpy-dbg_2.1.2-r0_amd64.deb /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/python3-numpy-src_2.1.2-r0_amd64.deb /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/python3-numpy_2.1.2-r0_amd64.deb The following ipk packages are different and not in exclusion list: /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/python3-numpy-dbg_2.1.2-r0_core2-64.ipk /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/python3-numpy-src_2.1.2-r0_core2-64.ipk /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/python3-numpy_2.1.2-r0_core2-64.ipk The following rpm packages are different and not in exclusion list: /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/python3-numpy-2.1.2-r0.core2_64.rpm /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/python3-numpy-dbg-2.1.2-r0.core2_64.rpm /srv/pokybuild/yocto-worker/reproducible/build/build-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/python3-numpy-src-2.1.2-r0.core2_64.rpm https://valkyrie.yoctoproject.org/#/builders/37/builds/369/steps/12/logs/stdio Differences can be seen here: - https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241030-a5e77d8y/ - https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241030-a5e77d8y/packages/diff-html/ Can you have a look please ?
On Wed, 30 Oct 2024 at 15:23, Mathieu Dubois-Briand via lists.openembedded.org <mathieu.dubois-briand=bootlin.com@lists.openembedded.org> wrote: > Differences can be seen here: > - https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241030-a5e77d8y/ > - https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241030-a5e77d8y/packages/diff-html/ > > Can you have a look please ? I believe there are two separate reproducibility issues there: 1. usr/lib/python3.13/site-packages/numpy/__config__.py contains numerous references to compiler and linker flags which contains build paths. This is not uncommon, and we generally handle it by patching these out after the fact with sed in do_install:append() or similar. Plenty of examples all over oe-core. 2. The second issue is slightly trickier. There's a number of generated c files produced by cython and they contain lines like: static const char __pyx_k_srv_storage_alex_yocto_build_64[] = "/srv/storage/alex/yocto/build-64/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/numpy-2.1.2/numpy/random/bit_generator.pyx"; PyObject *__pyx_kp_s_srv_storage_alex_yocto_build_64; etc. This needs to be fixed by tracking down where in cython this stuff is generated, and replacing real paths (or parts of them) with something reproducible, like 'reproducible_build_path' or similar. Then this should be submitted or raised with upstream. 3. We should also drop # Python pyx -> c -> so build leaves absolute build paths in the code INSANE_SKIP:${PN} += "buildpaths" INSANE_SKIP:${PN}-src += "buildpaths" from meta/classes-recipe/python_mesonpy.bbclass when point two is addressed. Hope this helps, Alex
On 2024-11-01 06:08, Alexander Kanavin wrote: > On Wed, 30 Oct 2024 at 15:23, Mathieu Dubois-Briand via > lists.openembedded.org > <mathieu.dubois-briand=bootlin.com@lists.openembedded.org> wrote: > >> Differences can be seen here: >> - https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241030-a5e77d8y/ >> - https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20241030-a5e77d8y/packages/diff-html/ >> >> Can you have a look please ? > I believe there are two separate reproducibility issues there: > > 1. usr/lib/python3.13/site-packages/numpy/__config__.py contains > numerous references to compiler and linker flags which contains build > paths. This is not uncommon, and we generally handle it by patching > these out after the fact with sed in do_install:append() or similar. > Plenty of examples all over oe-core. > > 2. The second issue is slightly trickier. There's a number of > generated c files produced by cython and they contain lines like: > > static const char __pyx_k_srv_storage_alex_yocto_build_64[] = > "/srv/storage/alex/yocto/build-64/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/numpy-2.1.2/numpy/random/bit_generator.pyx"; > PyObject *__pyx_kp_s_srv_storage_alex_yocto_build_64; > etc. > > This needs to be fixed by tracking down where in cython this stuff is > generated, and replacing real paths (or parts of them) with something > reproducible, like 'reproducible_build_path' or similar. Then this > should be submitted or raised with upstream. Yeah, I've gotten this far, but I think the real issue is that these should be configurable via some (possibly-nonexistent) bitbake variable in the recipe. I haven't yet determined how to add this stuff yet, though. In the python3-pandas recipe in meta-python, there's a fix_cythonized_sources() function that seems to be of the right spirit: https://git.openembedded.org/meta-openembedded/tree/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb#n26 I don't think this is quite the right fix, though - it seems like these adjustments should be happening during do_configure if possible, so that the host paths aren't being propagated through all of the different copies of files like __config__.py... There are quite a lot of hits under the numpy workdir for "Generated by Cython": tgamblin@megalith ~/workspace/yocto/poky (master)$ grep -r "Generated by Cython" build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/temp/run.fix_cythonized_sources.581946: for f in `grep -l -r -e '\/* Generated by Cython.*/$' /home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2`; do grep: build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Utils.cpython-313-x86_64-linux-gnu.so: binary file matches build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Utils.py:GENERATED_BY_MARKER = "/* Generated by Cython %s */" % cython_version build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Utils.py: file_content.startswith(b"/* Generated by Cython ") or build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Compiler/Annotate.py: <!-- Generated by Cython {watermark} --> build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Compiler/Annotate.py: <p><span style="border-bottom: solid 1px grey;">Generated by Cython {watermark}</span>{more_info}</p> build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Compiler/Main.py: if u'<!-- Generated by Cython' in html_file.read(100): grep: build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Compiler/__pycache__/Annotate.cpython-313.pyc: binary file matches grep: build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/Compiler/__pycache__/Main.cpython-313.pyc: binary file matches grep: build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/recipe-sysroot-native/usr/lib/python3.13/site-packages/Cython/__pycache__/Utils.cpython-313.pyc: binary file matches build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_mt19937.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_common.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_common.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_generator.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_generator.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_bounded_integers.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_bounded_integers.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_pcg64.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_pcg64.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_sfc64.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_sfc64.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/bit_generator.cpython-313-x86_64-linux-gnu.so.p/numpy/random/bit_generator.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/mtrand.cpython-313-x86_64-linux-gnu.so.p/numpy/random/mtrand.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/build/numpy/random/_philox.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_philox.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_mt19937.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_common.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_common.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_generator.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_generator.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_bounded_integers.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_bounded_integers.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_pcg64.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_pcg64.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_sfc64.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_sfc64.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/bit_generator.cpython-313-x86_64-linux-gnu.so.p/numpy/random/bit_generator.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/mtrand.cpython-313-x86_64-linux-gnu.so.p/numpy/random/mtrand.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/package/usr/src/debug/python3-numpy/2.1.2/numpy/random/_philox.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_philox.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_mt19937.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_common.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_common.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_generator.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_generator.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_bounded_integers.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_bounded_integers.pyx.c:/* Generated by Cython 3.0.11/ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_pcg64.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_pcg64.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_sfc64.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_sfc64.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/bit_generator.cpython-313-x86_64-linux-gnu.so.p/numpy/random/bit_generator.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/mtrand.cpython-313-x86_64-linux-gnu.so.p/numpy/random/mtrand.pyx.c:/* Generated by Cython 3.0.11 */ build/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/packages-split/python3-numpy-src/usr/src/debug/python3-numpy/2.1.2/numpy/random/_philox.cpython-313-x86_64-linux-gnu.so.p/numpy/random/_philox.pyx.c:/* Generated by Cython 3.0.11 */ This makes me think that the Cython recipe is also missing a few cases in cython_fix_sources() (namely, ../Cython/Compiler/Annotate.py, ../Cython/Compiler/Main.py, and ../Cython/Utils.py, but locally adding those and rebuild doesn't seem to fix the numpy issue. Still looking. > > 3. We should also drop > # Python pyx -> c -> so build leaves absolute build paths in the code > INSANE_SKIP:${PN} += "buildpaths" > INSANE_SKIP:${PN}-src += "buildpaths" > > from meta/classes-recipe/python_mesonpy.bbclass when point two is addressed. Completely agree, it's on my radar. Apparently I need to learn a lot more about how all of this is actually happening underneath... > > Hope this helps, > Alex
On 1 Nov 2024, at 10:08, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > 2. The second issue is slightly trickier. There's a number of > generated c files produced by cython and they contain lines like: > > static const char __pyx_k_srv_storage_alex_yocto_build_64[] = > "/srv/storage/alex/yocto/build-64/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/numpy-2.1.2/numpy/random/bit_generator.pyx"; > PyObject *__pyx_kp_s_srv_storage_alex_yocto_build_64; > etc. > > This needs to be fixed by tracking down where in cython this stuff is > generated, and replacing real paths (or parts of them) with something > reproducible, like 'reproducible_build_path' or similar. Then this > should be submitted or raised with upstream. There’s an upstream ticket already: https://github.com/cython/cython/issues/5949. > 3. We should also drop > # Python pyx -> c -> so build leaves absolute build paths in the code > INSANE_SKIP:${PN} += "buildpaths" > INSANE_SKIP:${PN}-src += "buildpaths" > > from meta/classes-recipe/python_mesonpy.bbclass when point two is addressed. This should be removed from that class anyway, this is a cython workaround so doesn’t have any place in the mesonpy class. There’s also that .c files generated by cython embed the python include dir (thus, build path) inside a “Cython Metadata” blob. A number of recipes in meta-python strip this out so I’m wondering if we need a cython.bbclass which does something like: do_compile[postfuncs] = “strip_cython_metadata” strip_cython_metadata() { find ${B} -name \*.c -print0 | xargs -0 sed -i -e "/BEGIN: Cython Metadata/,/END: Cython Metadata/d” } Alternatively, patch cython to not generate that metadata in the first place. Apparently there are tools that read it, so leaving it in the tree but stripping it before install seems like the safest option. Ross
On 1 Nov 2024, at 21:47, Ross Burton <Ross.Burton@arm.com> wrote: > > On 1 Nov 2024, at 10:08, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: >> 2. The second issue is slightly trickier. There's a number of >> generated c files produced by cython and they contain lines like: >> >> static const char __pyx_k_srv_storage_alex_yocto_build_64[] = >> "/srv/storage/alex/yocto/build-64/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/numpy-2.1.2/numpy/random/bit_generator.pyx"; >> PyObject *__pyx_kp_s_srv_storage_alex_yocto_build_64; >> etc. >> >> This needs to be fixed by tracking down where in cython this stuff is >> generated, and replacing real paths (or parts of them) with something >> reproducible, like 'reproducible_build_path' or similar. Then this >> should be submitted or raised with upstream. I’ve got a prototype patch that fixes this in cython itself, meaning a pretty large chunk of horrible code in recipes (that silenced the buildpath warnings, but didnt actually fix reproduciblity) can be removed. I’ll hopefully get it good enough to post today. Ross
On 2024-11-04 09:20, Ross Burton wrote: > On 1 Nov 2024, at 21:47, Ross Burton <Ross.Burton@arm.com> wrote: >> On 1 Nov 2024, at 10:08, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: >>> 2. The second issue is slightly trickier. There's a number of >>> generated c files produced by cython and they contain lines like: >>> >>> static const char __pyx_k_srv_storage_alex_yocto_build_64[] = >>> "/srv/storage/alex/yocto/build-64/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/numpy-2.1.2/numpy/random/bit_generator.pyx"; >>> PyObject *__pyx_kp_s_srv_storage_alex_yocto_build_64; >>> etc. >>> >>> This needs to be fixed by tracking down where in cython this stuff is >>> generated, and replacing real paths (or parts of them) with something >>> reproducible, like 'reproducible_build_path' or similar. Then this >>> should be submitted or raised with upstream. > I’ve got a prototype patch that fixes this in cython itself, meaning a pretty large chunk of horrible code in recipes (that silenced the buildpath warnings, but didnt actually fix reproduciblity) can be removed. I’ll hopefully get it good enough to post today. Let me know if you want help testing - I started looking at the PR you mentioned over the weekend but didn't get anywhere yet. > > Ross
On 1 Nov 2024, at 10:08, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > 2. The second issue is slightly trickier. There's a number of > generated c files produced by cython and they contain lines like: > > static const char __pyx_k_srv_storage_alex_yocto_build_64[] = > "/srv/storage/alex/yocto/build-64/tmp/work/core2-64-poky-linux/python3-numpy/2.1.2/numpy-2.1.2/numpy/random/bit_generator.pyx"; > PyObject *__pyx_kp_s_srv_storage_alex_yocto_build_64; > etc. > > This needs to be fixed by tracking down where in cython this stuff is > generated, and replacing real paths (or parts of them) with something > reproducible, like 'reproducible_build_path' or similar. Then this > should be submitted or raised with upstream. I’ve just posted a patchset that fixes this. The patch isn’t yet upstreamed but I’m working on that, and I think integrating it to test it would be useful data for upstream. > 3. We should also drop > # Python pyx -> c -> so build leaves absolute build paths in the code > INSANE_SKIP:${PN} += "buildpaths" > INSANE_SKIP:${PN}-src += "buildpaths" > > from meta/classes-recipe/python_mesonpy.bbclass when point two is addressed. My patchset also drops that. Ross
diff --git a/meta/recipes-devtools/python/python3-numpy/0001-numpy-core-Define-RISCV-32-support.patch b/meta/recipes-devtools/python/python3-numpy/0001-numpy-core-Define-RISCV-32-support.patch index 676bdbb3afd..8dfb20edbb9 100644 --- a/meta/recipes-devtools/python/python3-numpy/0001-numpy-core-Define-RISCV-32-support.patch +++ b/meta/recipes-devtools/python/python3-numpy/0001-numpy-core-Define-RISCV-32-support.patch @@ -1,4 +1,4 @@ -From eb6d6579150bf4684603ce377c51e90ad3bb8109 Mon Sep 17 00:00:00 2001 +From df38843aaa7184115d262f14eae23ff38a20372a Mon Sep 17 00:00:00 2001 From: Khem Raj <raj.khem@gmail.com> Date: Sun, 15 Nov 2020 15:32:39 -0800 Subject: [PATCH] numpy/core: Define RISCV-32 support @@ -7,15 +7,19 @@ Helps compile on riscv32 Upstream-Status: Submitted [https://github.com/numpy/numpy/pull/17780] Signed-off-by: Khem Raj <raj.khem@gmail.com> + +Update the patch with the new paths for numpy release 2.0.0. + +Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> --- - numpy/core/include/numpy/npy_cpu.h | 3 +++ - numpy/core/include/numpy/npy_endian.h | 1 + + numpy/_core/include/numpy/npy_cpu.h | 3 +++ + numpy/_core/include/numpy/npy_endian.h | 1 + 2 files changed, 4 insertions(+) -diff --git a/numpy/core/include/numpy/npy_cpu.h b/numpy/core/include/numpy/npy_cpu.h -index 78d229e..04be511 100644 ---- a/numpy/core/include/numpy/npy_cpu.h -+++ b/numpy/core/include/numpy/npy_cpu.h +diff --git a/numpy/_core/include/numpy/npy_cpu.h b/numpy/_core/include/numpy/npy_cpu.h +index a19f8e6bbd..d824d4efb6 100644 +--- a/numpy/_core/include/numpy/npy_cpu.h ++++ b/numpy/_core/include/numpy/npy_cpu.h @@ -19,6 +19,7 @@ * NPY_CPU_ARCEB * NPY_CPU_RISCV64 @@ -33,10 +37,10 @@ index 78d229e..04be511 100644 #elif defined(__loongarch__) #define NPY_CPU_LOONGARCH #elif defined(__EMSCRIPTEN__) -diff --git a/numpy/core/include/numpy/npy_endian.h b/numpy/core/include/numpy/npy_endian.h -index 5e58a7f..0926212 100644 ---- a/numpy/core/include/numpy/npy_endian.h -+++ b/numpy/core/include/numpy/npy_endian.h +diff --git a/numpy/_core/include/numpy/npy_endian.h b/numpy/_core/include/numpy/npy_endian.h +index 5e58a7f52c..09262120bf 100644 +--- a/numpy/_core/include/numpy/npy_endian.h ++++ b/numpy/_core/include/numpy/npy_endian.h @@ -49,6 +49,7 @@ || defined(NPY_CPU_PPC64LE) \ || defined(NPY_CPU_ARCEL) \ @@ -46,5 +50,5 @@ index 5e58a7f..0926212 100644 || defined(NPY_CPU_WASM) #define NPY_BYTE_ORDER NPY_LITTLE_ENDIAN -- -2.20.1 +2.45.2 diff --git a/meta/recipes-devtools/python/python3-numpy_1.26.4.bb b/meta/recipes-devtools/python/python3-numpy_2.1.2.bb similarity index 84% rename from meta/recipes-devtools/python/python3-numpy_1.26.4.bb rename to meta/recipes-devtools/python/python3-numpy_2.1.2.bb index 914245507c7..83e921f753c 100644 --- a/meta/recipes-devtools/python/python3-numpy_1.26.4.bb +++ b/meta/recipes-devtools/python/python3-numpy_2.1.2.bb @@ -3,7 +3,7 @@ HOMEPAGE = "https://numpy.org/" DESCRIPTION = "NumPy is the fundamental package needed for scientific computing with Python." SECTION = "devel/python" LICENSE = "BSD-3-Clause & BSD-2-Clause & PSF-2.0 & Apache-2.0 & MIT" -LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=a752eb20459cf74a9d84ee4825e8317c" +LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=1de863c37a83e71b1e97b64d036ea78b" SRCNAME = "numpy" @@ -13,14 +13,14 @@ SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/${SRCNAME}-${PV}.tar.gz \ file://fix_reproducibility.patch \ file://run-ptest \ " -SRC_URI[sha256sum] = "2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010" +SRC_URI[sha256sum] = "13532a088217fa624c99b843eeb54640de23b3414b14aa66d023805eb731066c" GITHUB_BASE_URI = "https://github.com/numpy/numpy/releases" UPSTREAM_CHECK_REGEX = "releases/tag/v?(?P<pver>\d+(\.\d+)+)$" DEPENDS += "python3-cython-native" -inherit ptest setuptools3 github-releases +inherit ptest python_mesonpy pkgconfig github-releases S = "${WORKDIR}/numpy-${PV}" @@ -30,7 +30,9 @@ do_compile:prepend() { export NPY_DISABLE_SVML=1 } -FILES:${PN}-staticdev += "${PYTHON_SITEPACKAGES_DIR}/numpy/core/lib/*.a ${PYTHON_SITEPACKAGES_DIR}/numpy/random/lib/*.a" +FILES:${PN}-staticdev += "${PYTHON_SITEPACKAGES_DIR}/numpy/_core/lib/*.a \ + ${PYTHON_SITEPACKAGES_DIR}/numpy/random/lib/*.a \ +" # install what is needed for numpy.test() RDEPENDS:${PN} = "python3-unittest \
- This release contains numerous API changes and bug fixes. Changelog: https://github.com/numpy/numpy/releases - Notably, the build backend has switched to mesonpy since 1.26.4. - Patch 0001-numpy-core-Define-RISCV-32-support.patch had to be updated to change the paths for the target files. It has now been merged upstream, but isn't included in any release yet. PR for reference: https://github.com/numpy/numpy/pull/17780 - Builds OK and seems to be compatible with current piglit and pandas (meta-python), which was not the case for the previous upgrade attempt to numpy 2.0.1. - For numpy, add 'pkgconfig' to inherit to avoid errors like this at do_compile: | Found Pkg-config: NO | Run-time dependency python found: YES 3.12 | Has header "Python.h" with dependency python: NO | | ../numpy-2.0.0/meson.build:44:2: ERROR: Problem encountered: Cannot compile `Python.h`. Perhaps you need to install python-dev|python-devel | | A full log can be found at /home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/2.0.0/build/meson-logs/meson-log.txt - Also modify the FILES:${PN}-staticdev line to fix a QA issue. License-Update: Change copyright year to 2024 Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> --- ...1-numpy-core-Define-RISCV-32-support.patch | 28 +++++++++++-------- ...numpy_1.26.4.bb => python3-numpy_2.1.2.bb} | 10 ++++--- 2 files changed, 22 insertions(+), 16 deletions(-) rename meta/recipes-devtools/python/{python3-numpy_1.26.4.bb => python3-numpy_2.1.2.bb} (84%)