| Message ID | 20250917121016.15784-1-hongxu.jia@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | python3-cython: make generated source file be reproducible | expand |
On 9/17/25 14:10, hongxu via lists.openembedded.org wrote: > +diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py > +index 6672cb986..ae1625ea2 100644 > +--- a/Cython/Compiler/ModuleNode.py > ++++ b/Cython/Compiler/ModuleNode.py > +@@ -779,7 +779,9 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): > + code.put_generated_by() > + if metadata: > + code.putln("/* BEGIN: Cython Metadata") > +- code.putln(json.dumps(metadata, indent=4, sort_keys=True)) > ++ _metadata = json.dumps(metadata, indent=4, sort_keys=True) > ++ _metadata = _metadata.replace(os.getcwd()+'/', '') I don't think that this is going to work on Windows due to the different default path separator (of course for Yocto that doesn't matter much, but I think upstream cares about it) Maybe os.path.sep would be more portable?
On Wed, Sep 17, 2025 at 05:26 AM, Gyorgy Sarvari wrote: > > >> +diff --git a/Cython/Compiler/ModuleNode.py >> b/Cython/Compiler/ModuleNode.py >> +index 6672cb986..ae1625ea2 100644 >> +--- a/Cython/Compiler/ModuleNode.py >> ++++ b/Cython/Compiler/ModuleNode.py >> +@@ -779,7 +779,9 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): >> + code.put_generated_by() >> + if metadata: >> + code.putln("/* BEGIN: Cython Metadata") >> +- code.putln(json.dumps(metadata, indent=4, sort_keys=True)) >> ++ _metadata = json.dumps(metadata, indent=4, sort_keys=True) >> ++ _metadata = _metadata.replace(os.getcwd()+'/', '') > > I don't think that this is going to work on Windows due to the different > default path separator (of course for Yocto that doesn't matter much, > but I think upstream cares about it) OK, I will use hardcode string 'build_path' to replace actual build path prefix, it should be portable for both of Linux and Windows. Please drop this review, v2 incomig //Hongxu > > Maybe os.path.sep would be more portable?
diff --git a/meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch b/meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch new file mode 100644 index 0000000000..b560e9d020 --- /dev/null +++ b/meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch @@ -0,0 +1,70 @@ +From f33847ca55e508106999471c1d6a2f7371503a1a Mon Sep 17 00:00:00 2001 +From: Hongxu Jia <hongxu.jia@windriver.com> +Date: Wed, 17 Sep 2025 01:42:08 -0700 +Subject: [PATCH] Remove build path prefix from metadata in the generated output file + +The build path may contain tmp dir which is not predictable, it caused +the generated output file is not stable at each build and made +the generated library is not reproducible [1] between builds + +vim frozenlist/_frozenlist.cpp +... +/* BEGIN: Cython Metadata +{ + "distutils": { + "depends": [], + "language": "c++", + "name": "frozenlist._frozenlist", + "sources": [ + "/tmp/.tmp-frozenlist-pep517-cfdvygni/src/frozenlist/_frozenlist.pyx" + ] + }, + "module_name": "frozenlist._frozenlist" +} +END: Cython Metadata */ +... + +Remove build path prefix from metadata, after applied this commit, +vim frozenlist/_frozenlist.cpp +... +/* BEGIN: Cython Metadata +{ + "distutils": { + "depends": [], + "language": "c++", + "name": "frozenlist._frozenlist", + "sources": [ + "frozenlist/_frozenlist.pyx" + ] + }, + "module_name": "frozenlist._frozenlist" +} +END: Cython Metadata */ +... + +[1] https://reproducible-builds.org/ + +Upstream-Status: Submitted [https://github.com/cython/cython/pull/7161] +Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> +--- + Cython/Compiler/ModuleNode.py | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py +index 6672cb986..ae1625ea2 100644 +--- a/Cython/Compiler/ModuleNode.py ++++ b/Cython/Compiler/ModuleNode.py +@@ -779,7 +779,9 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): + code.put_generated_by() + if metadata: + code.putln("/* BEGIN: Cython Metadata") +- code.putln(json.dumps(metadata, indent=4, sort_keys=True)) ++ _metadata = json.dumps(metadata, indent=4, sort_keys=True) ++ _metadata = _metadata.replace(os.getcwd()+'/', '') ++ code.putln(_metadata) + code.putln("END: Cython Metadata */") + code.putln("") + +-- +2.49.0 + diff --git a/meta/recipes-devtools/python/python3-cython_3.1.3.bb b/meta/recipes-devtools/python/python3-cython_3.1.3.bb index dcb61a3634..1e0b28b12b 100644 --- a/meta/recipes-devtools/python/python3-cython_3.1.3.bb +++ b/meta/recipes-devtools/python/python3-cython_3.1.3.bb @@ -11,6 +11,10 @@ SRC_URI[sha256sum] = "10ee785e42328924b78f75a74f66a813cb956b4a9bc91c44816d089d59 inherit pypi setuptools3 cython +SRC_URI += " \ + file://0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch \ +" + # No need to depend on self DEPENDS:remove = "python3-cython-native"
While python3 module use cython to build library, the generated source file is not stable at each build and made the generated library not be reproducible This commit remove un-predictable string from generated source file, thus the generated library should be reproducible Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> --- ...h-prefix-from-metadata-in-the-genera.patch | 70 +++++++++++++++++++ .../python/python3-cython_3.1.3.bb | 4 ++ 2 files changed, 74 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch