| Message ID | 20211128094532.1145820-1-jacob.kroon@gmail.com |
|---|---|
| Headers | show |
| Series | Improve native/cross reproducibility | expand |
Thanks Jacob. When looking at this patchset I kept asking myself, why is this or that change necessary for -native but not for -target. I think it would help if you include that information in the commits, particularly, in 1/9 as it is the most invasive change of all. Maybe we can then figure out a better way. Alex On Sun, 28 Nov 2021 at 10:46, Jacob Kroon <jacob.kroon@gmail.com> wrote: > This patch series is not intended for merge. I only send it out to > highlight where the problems are and to get some discussion going on > how/if we want to improve the sitation. > > This is a patch series that tries to improve the reproducibility of the > native/cross binaries when building in different directories. This has > been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of > writing. > > The RUNTIME hack is questionable, maybe there is a better way to enforce > a fixed RUNTIME entry size during linking. It probably breaks for > recipes that do additional rpath manipulations at link-time. > > If in the end we do come up with a solution, then it should be tested on > the autobuilders, since otherwise this is going to degrade overtime. It > would then be important that the build paths are of significantly different > lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* > uncover all > rpath problems. > > The end result of this patch series is that I can build python3-native > in two different build paths, and the resuling sysroot-components/x86_64/ > directories are identical, except for the 'fixmepath.cmd' files, which > are not included in the hash equiv calculations. Even so, there remains a > lot of > other native builds that are going to need to be fixed in similar ways > as the ones in this patch series. > > For my images to build I had to avoid the rpath-hack for icedtea7-native > and openjdk-8-native. > > /Jacob > > Jacob Kroon (9): > bitbake.conf: Pad rpath and remove build ID in native binaries > libtool: Improve native reproducibility > openssl: Improve native reproducibility > perl/perlcross: Improve native reproducability > pkgconfig: Improve native reproducibility > ncurses: Improve native reproducibility > util-linux: Improve native reproducibility > python3: Improve native reproducibility > bitbake.conf: Avoid rpath hack for Java recipes > > meta/classes/chrpath.bbclass | 3 + > meta/conf/bitbake.conf | 8 +- > ...sysroot-and-debug-prefix-map-from-co.patch | 78 ------------------- > .../openssl/openssl/strip-buildinfo.patch | 13 ++++ > .../openssl/openssl_3.0.0.bb | 10 +-- > meta/recipes-core/ncurses/ncurses.inc | 4 + > .../util-linux/util-linux_2.37.2.bb | 2 +- > .../libtool/libtool-native_2.4.6.bb | 1 + > ...ism.patch => perl-cross-determinism.patch} | 0 > .../perl-cross/perlcross_1.3.6.bb | 4 +- > meta/recipes-devtools/perl/perl_5.34.0.bb | 5 ++ > .../pkgconfig/pkgconfig_git.bb | 1 + > .../python/python3/determinism.patch | 15 ++++ > .../recipes-devtools/python/python3_3.10.0.bb | 8 ++ > 14 files changed, 65 insertions(+), 87 deletions(-) > delete mode 100644 > meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch > create mode 100644 > meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch > rename meta/recipes-devtools/perl-cross/files/{determinism.patch => > perl-cross-determinism.patch} (100%) > create mode 100644 meta/recipes-devtools/python/python3/determinism.patch > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#158867): > https://lists.openembedded.org/g/openembedded-core/message/158867 > Mute This Topic: https://lists.openembedded.org/mt/87352786/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On 11/29/21 10:16, Alexander Kanavin wrote: > Thanks Jacob. When looking at this patchset I kept asking myself, why is > this or that change necessary for -native but not for -target. I think > it would help if you include that information in the commits, > particularly, in 1/9 as it is the most invasive change of all. Maybe we > can then figure out a better way. > > Alex > Yeah, I think its because there is no extra manipulation of rpath done for the target binaries at all, since they wont be running in a location other than what they were configured for. RP, Khem, please correct me if I'm wrong. The native binaries on the other hand will need to run from each recipe specific sysroot. I can update the commit message in patch 1 if that makes sense. Jacob
This patch series is not intended for merge. I only send it out to highlight where the problems are and to get some discussion going on how/if we want to improve the sitation. This is a patch series that tries to improve the reproducibility of the native/cross binaries when building in different directories. This has been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of writing. The RUNTIME hack is questionable, maybe there is a better way to enforce a fixed RUNTIME entry size during linking. It probably breaks for recipes that do additional rpath manipulations at link-time. If in the end we do come up with a solution, then it should be tested on the autobuilders, since otherwise this is going to degrade overtime. It would then be important that the build paths are of significantly different lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all rpath problems. The end result of this patch series is that I can build python3-native in two different build paths, and the resuling sysroot-components/x86_64/ directories are identical, except for the 'fixmepath.cmd' files, which are not included in the hash equiv calculations. Even so, there remains a lot of other native builds that are going to need to be fixed in similar ways as the ones in this patch series. For my images to build I had to avoid the rpath-hack for icedtea7-native and openjdk-8-native. /Jacob Jacob Kroon (9): bitbake.conf: Pad rpath and remove build ID in native binaries libtool: Improve native reproducibility openssl: Improve native reproducibility perl/perlcross: Improve native reproducability pkgconfig: Improve native reproducibility ncurses: Improve native reproducibility util-linux: Improve native reproducibility python3: Improve native reproducibility bitbake.conf: Avoid rpath hack for Java recipes meta/classes/chrpath.bbclass | 3 + meta/conf/bitbake.conf | 8 +- ...sysroot-and-debug-prefix-map-from-co.patch | 78 ------------------- .../openssl/openssl/strip-buildinfo.patch | 13 ++++ .../openssl/openssl_3.0.0.bb | 10 +-- meta/recipes-core/ncurses/ncurses.inc | 4 + .../util-linux/util-linux_2.37.2.bb | 2 +- .../libtool/libtool-native_2.4.6.bb | 1 + ...ism.patch => perl-cross-determinism.patch} | 0 .../perl-cross/perlcross_1.3.6.bb | 4 +- meta/recipes-devtools/perl/perl_5.34.0.bb | 5 ++ .../pkgconfig/pkgconfig_git.bb | 1 + .../python/python3/determinism.patch | 15 ++++ .../recipes-devtools/python/python3_3.10.0.bb | 8 ++ 14 files changed, 65 insertions(+), 87 deletions(-) delete mode 100644 meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch create mode 100644 meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%) create mode 100644 meta/recipes-devtools/python/python3/determinism.patch