diff mbox series

[meta-python,v2] python3-pandas: compile against target version of numpy

Message ID 20250310081910.3106973-1-skandigraun@gmail.com
State New
Headers show
Series [meta-python,v2] python3-pandas: compile against target version of numpy | expand

Commit Message

Gyorgy Sarvari March 10, 2025, 8:19 a.m. UTC
python3-numpy headers are required by pandas to compile successfully.
By default, this recipe used python3-numpy-native for compilation,
which usually works. However in case the bitness of the build-host
differs from the target, then problems arise.

For example when compiling for 32-bit ARM on a x86-64 machine, the following
error appears when trying to import the module:

ValueError: Buffer dtype mismatch, expected 'const int64_t' but got 'long long'

When running a diff on all numpy headers across native and target,
only one header differs, _numpyconfig.h, in a significant way.
This header defines the sizes of different datatypes used by numpy,
and these sizes strongly depend on the arch bitness.

This change switches from python3-numpy-native dependency to python3-numpy to
ensure that the correct headers are used. Beside this also patch the meson script,
so it accepts an option (numpy_inc_dir) to specify the location of these headers,
since it is not able to query them from the class-target module

The PYTHONPATH variable is extended with the target's RECIPE_SYSROOT, because
numpy is specified as a dependency in meson, and it needs to find the module
to continue successfully.

Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
---
 ...fy-numpy-header-path-as-meson-option.patch | 52 +++++++++++++++++++
 .../python/python3-pandas_2.2.2.bb            |  9 +++-
 2 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100644 meta-python/recipes-devtools/python/python3-pandas/0001-specify-numpy-header-path-as-meson-option.patch
diff mbox series

Patch

diff --git a/meta-python/recipes-devtools/python/python3-pandas/0001-specify-numpy-header-path-as-meson-option.patch b/meta-python/recipes-devtools/python/python3-pandas/0001-specify-numpy-header-path-as-meson-option.patch
new file mode 100644
index 0000000000..039f116ff1
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-pandas/0001-specify-numpy-header-path-as-meson-option.patch
@@ -0,0 +1,52 @@ 
+From 46b3c3ee68f271acbc178260939ca8423f9244a5 Mon Sep 17 00:00:00 2001
+From: Gyorgy Sarvari <skandigraun@gmail.com>
+Date: Sun, 9 Mar 2025 19:57:26 +0100
+Subject: [PATCH] specify numpy header path as meson option
+
+In case pandas is compiled against the class-target version of numpy, it is
+not possible to import the module, due to target-compiled binary files - 
+it just fails with an error. However this also means that meson isn't
+able to query the include folder location, so the compilation fails.
+
+To avoid this, the new "numpy_inc_dir" meson option allows specifying the
+location of the required headers.
+
+Upstream-Status: Inappropriate [oe-specific]
+---
+ meson.options      | 1 +
+ pandas/meson.build | 6 +++---
+ 2 files changed, 4 insertions(+), 3 deletions(-)
+ create mode 100644 meson.options
+
+diff --git a/meson.options b/meson.options
+new file mode 100644
+index 0000000000..3d3a18d145
+--- /dev/null
++++ b/meson.options
+@@ -0,0 +1 @@
++option('numpy_inc_dir', type : 'string', description : 'The absolute path to the numpy headers')
+diff --git a/pandas/meson.build b/pandas/meson.build
+index 435103a954..32ad04bd8e 100644
+--- a/pandas/meson.build
++++ b/pandas/meson.build
+@@ -3,17 +3,17 @@ incdir_numpy = run_command(py,
+     '-c',
+     '''
+ import os
+-import numpy as np
+ try:
+     # Check if include directory is inside the pandas dir
+     # e.g. a venv created inside the pandas dir
+     # If so, convert it to a relative path
+-    incdir = os.path.relpath(np.get_include())
++    incdir = os.path.relpath(os.getenv('NUMPY_INC_DIR'))
+ except Exception:
+-    incdir = np.get_include()
++    incdir = os.getenv('NUMPY_INC_DIR')
+ print(incdir)
+      '''
+   ],
++  env: {'NUMPY_INC_DIR': get_option('numpy_inc_dir')},
+   check: true
+ ).stdout().strip()
+ 
diff --git a/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb b/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb
index c12a237653..5676d98cd0 100644
--- a/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb
+++ b/meta-python/recipes-devtools/python/python3-pandas_2.2.2.bb
@@ -11,12 +11,14 @@  SRC_URI += " \
             file://0001-pyproject.toml-Downgrade-numpy-version-needs-to-1.x.patch \
             "
 
+SRC_URI:append:class-target = " file://0001-specify-numpy-header-path-as-meson-option.patch "
+
 SRC_URI[sha256sum] = "9e79019aba43cb4fda9e4d983f8e88ca0373adbb697ae9c6c43093218de28b54"
 
 inherit pkgconfig pypi python_mesonpy cython
 
 DEPENDS += " \
-    python3-numpy-native \
+    python3-numpy \
     python3-versioneer-native \
 "
 
@@ -30,3 +32,8 @@  RDEPENDS:${PN} += " \
     python3-pytz \
     python3-profile \
 "
+
+PYTHONPATH:prepend:class-target = "${RECIPE_SYSROOT}${PYTHON_SITEPACKAGES_DIR}:"
+export PYTHONPATH
+
+EXTRA_OEMESON:append:class-target = " -Dnumpy_inc_dir=${RECIPE_SYSROOT}${PYTHON_SITEPACKAGES_DIR}/numpy/_core/include "