diff mbox series

[meta-python,v3] python3-pandas: compile against target version of numpy

Message ID 20250310133117.1356742-1-skandigraun@gmail.com
State Accepted
Headers show
Series [meta-python,v3] python3-pandas: compile against target version of numpy | expand

Commit Message

Gyorgy Sarvari March 10, 2025, 1:31 p.m. UTC
python3-numpy headers are required by pandas to compile successfully.
By default, this recipe used python3-numpy-native for compilation,
which usually works. However in case the bitness of the build-host
differs from the target, then problems arise.

For example when compiling for 32-bit ARM on a x86-64 machine, the following
error appears when trying to import the module:

ValueError: Buffer dtype mismatch, expected 'const int64_t' but got 'long long'

When running a diff on all numpy headers across native and target,
only one header differs, _numpyconfig.h, in a significant way.
This header defines the sizes of different datatypes used by numpy,
and these sizes strongly depend on the arch bitness.

This change switches from python3-numpy-native dependency to python3-numpy to
ensure that the correct headers are used. Beside this also patch the meson script,
so it accepts an option (numpy_inc_dir) to specify the location of these headers,
since it is not able to query them from the class-target module

The PYTHONPATH variable is extended with the target's RECIPE_SYSROOT, because
numpy is specified as a dependency in meson, and it needs to find the module
to continue successfully.

Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
---
 ...ion-to-specify-numpy-header-location.patch | 66 +++++++++++++++++++
 .../python/python3-pandas_2.2.2.bb            |  9 ++-
 2 files changed, 74 insertions(+), 1 deletion(-)
 create mode 100644 meta-python/recipes-devtools/python/python3-pandas/0001-BLD-add-option-to-specify-numpy-header-location.patch
diff mbox series

Patch

diff --git a/meta-python/recipes-devtools/python/python3-pandas/0001-BLD-add-option-to-specify-numpy-header-location.patch b/meta-python/recipes-devtools/python/python3-pandas/0001-BLD-add-option-to-specify-numpy-header-location.patch
new file mode 100644
index 0000000000..87023a24ec
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-pandas/0001-BLD-add-option-to-specify-numpy-header-location.patch
@@ -0,0 +1,66 @@ 
+From 16dd09e6c79768a24f5a50ec5985e0b6fdf17f35 Mon Sep 17 00:00:00 2001
+From: Gyorgy Sarvari <skandigraun@gmail.com>
+Date: Mon, 10 Mar 2025 13:52:11 +0100
+Subject: [PATCH] BLD: add option to specify numpy header location
+
+In some cases the numpy module might not be usable during build-time,
+especially when cross-compiling. (E.g. when compiling for arm32 on a
+x86-64 machine, the arm32 module is not usable at build time).
+
+This makes meson fail, as it isn't able to figure out the location of
+numpy headers.
+
+To allow an alternative way to find these headers, introduce a meson
+build option, where the location of the numpy headers can be specified.
+
+In case numpy module cannot be loaded for some reason to query the
+include folder location, fall back to the value of this meson option.
+
+Upstream-Status: Submitted [https://github.com/pandas-dev/pandas/pull/61095]
+
+Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
+---
+ meson.options      |  1 +
+ pandas/meson.build | 13 ++++++++++---
+ 2 files changed, 11 insertions(+), 3 deletions(-)
+ create mode 100644 meson.options
+
+diff --git a/meson.options b/meson.options
+new file mode 100644
+index 0000000000..3d3a18d145
+--- /dev/null
++++ b/meson.options
+@@ -0,0 +1 @@
++option('numpy_inc_dir', type : 'string', description : 'The absolute path to the numpy headers')
+diff --git a/pandas/meson.build b/pandas/meson.build
+index 435103a..a08c77a 100644
+--- a/pandas/meson.build
++++ b/pandas/meson.build
+@@ -3,17 +3,24 @@ incdir_numpy = run_command(py,
+     '-c',
+     '''
+ import os
+-import numpy as np
++
++try:
++    import numpy as np
++    base_incdir = np.get_include()
++except Exception:
++    base_incdir = os.getenv('NUMPY_INC_DIR')
++
+ try:
+     # Check if include directory is inside the pandas dir
+     # e.g. a venv created inside the pandas dir
+     # If so, convert it to a relative path
+-    incdir = os.path.relpath(np.get_include())
++    incdir = os.path.relpath(base_incdir)
+ except Exception:
+-    incdir = np.get_include()
++    incdir = base_incdir
+ print(incdir)
+      '''
+   ],
++  env: {'NUMPY_INC_DIR': get_option('numpy_inc_dir')},
+   check: true
+ ).stdout().strip()
+ 
diff --git a/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb b/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb
index c12a237653..ad9482e7dd 100644
--- a/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb
+++ b/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb
@@ -11,12 +11,14 @@  SRC_URI += " \
             file://0001-pyproject.toml-Downgrade-numpy-version-needs-to-1.x.patch \
             "
 
+SRC_URI:append:class-target = " file://0001-BLD-add-option-to-specify-numpy-header-location.patch "
+
 SRC_URI[sha256sum] = "9e79019aba43cb4fda9e4d983f8e88ca0373adbb697ae9c6c43093218de28b54"
 
 inherit pkgconfig pypi python_mesonpy cython
 
 DEPENDS += " \
-    python3-numpy-native \
+    python3-numpy \
     python3-versioneer-native \
 "
 
@@ -30,3 +32,8 @@  RDEPENDS:${PN} += " \
     python3-pytz \
     python3-profile \
 "
+
+PYTHONPATH:prepend:class-target = "${RECIPE_SYSROOT}${PYTHON_SITEPACKAGES_DIR}:"
+export PYTHONPATH
+
+EXTRA_OEMESON:append:class-target = " -Dnumpy_inc_dir=${RECIPE_SYSROOT}${PYTHON_SITEPACKAGES_DIR}/numpy/_core/include "