| Message ID | 20251125225858.1361559-1-raj.khem@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [meta-filesystem] e2tools: Fix buildpaths in ptests | expand |
Le mar. 25 nov. 2025 à 23:59, Khem Raj via lists.openembedded.org <raj.khem= gmail.com@lists.openembedded.org> a écrit : > Currently the path checks are escaping QA check for buildpath detection > but config.status > still has paths which show up in reproduciblity failures, comparing build > in path A and > build in path B, content of config.status don't end up same. FYI, last time we tried to fixed that, some of the lines in config.status looked like this: S["LDFLAGS"]="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -ffile-prefix-map=/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-filesyste"\ "ms/build-st/reproducibleB-extended/tmp/work/x86-64-v3-poky-linux/e2tools/0.1.2/sources/e2tools-0.1.2=/usr/src/debug/e2tools/0.1.2 -ffile-prefix-map"\ "=/work/x86-64-v3-poky-linu"\ "x/e2tools/0.1.2/build=/usr/src/debug/e2tools/0.1.2 -ffile-prefix-map=/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-fil"\ "esystems/build-st/reproducibleB-extended/tmp/work/x86-64-v3-poky-linux/e2tools/0.1.2/recipe-sysroot= -ffile-prefix-map=/srv/pokybuild/yocto-worker/"\ "reproducible-meta-oe/build/build/build-st-meta-filesystems/build-st/reproducibleB-extended/tmp/work/x86-64-v3-poky-linux/e2tools/0.1.2/recipe-sysroo"\ "t-native= -Wl,-z,relro,-z,now" => The non-reproducible paths split across line breaks. This is autoconf trying to generate a portable AWK script (meaning the maximum line size for a literal is around 150). Since the "portable" here means supporting prehistoric UNIX version of awk we started working on modifying this limit in autoconf (_AC_AWK_LITERAL_LIMIT) to something more modern (1000). The goal was to avoid having non-reproducible paths split across line breaks to let sed work correctly but we never quite finished this patch. Signed-off-by: Khem Raj <raj.khem@gmail.com> > --- > .../recipes-utils/e2tools/e2tools_0.1.2.bb | 16 +++++++++++++++- > 1 file changed, 15 insertions(+), 1 deletion(-) > > diff --git a/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb > b/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb > index 219735acd1..b7b925da0e 100644 > --- a/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb > +++ b/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb > @@ -39,7 +39,21 @@ do_install_ptest() { > ${D}${PTEST_PATH}/build/autom4te.cache \ > ${D}${PTEST_PATH}/*/*/.git ${D}${PTEST_PATH}/*/*/.github \ > ${D}${PTEST_PATH}/*/*/autom4te.cache > - sed -i -e 's;${TMPDIR};;g' ${D}${PTEST_PATH}/build/config.status > + sed -e 's@[^ ]*-ffile-prefix-map=[^ "]*@@g' \ > + -e 's@[^ ]*-fdebug-prefix-map=[^ "]*@@g' \ > + -e 's@[^ ]*-fmacro-prefix-map=[^ "]*@@g' \ > + -e 's@[^ ]*--sysroot=[^ "]*@@g' \ > + -e 's@[^ ]*--with-libtool-sysroot=[^ "]*@@g' \ > + -e 's@[^ ]*--with-install-prefix=[^ "]*@@g' \ > + -e '/EXT2FS_CFLAGS/d' \ > + -e '/LDFLAGS/d' \ > + -e '/PKG_CONFIG_PATH/d' \ > + -e '/PKG_CONFIG_LIBDIR/d' \ > + -e 's@${S}@${PTEST_PATH}@g' \ > + -e 's@${B}@${PTEST_PATH}/build@g' \ > + -e 's@${HOSTTOOLS_DIR}@@g' \ > + -e 's@${RECIPE_SYSROOT}@@g' \ > + -i ${D}${PTEST_PATH}/build/config.status > } > > RDEPENDS:${PN}-ptest += "bash coreutils e2fsprogs e2tools gawk make perl" > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#122065): > https://lists.openembedded.org/g/openembedded-devel/message/122065 > Mute This Topic: https://lists.openembedded.org/mt/116477328/4316185 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ > yoann.congal@smile.fr] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Tue, Nov 25, 2025 at 3:55 PM Yoann Congal <yoann.congal@smile.fr> wrote: > Le mar. 25 nov. 2025 à 23:59, Khem Raj via lists.openembedded.org > <raj.khem=gmail.com@lists.openembedded.org> a écrit : > >> Currently the path checks are escaping QA check for buildpath detection >> but config.status >> still has paths which show up in reproduciblity failures, comparing build >> in path A and >> build in path B, content of config.status don't end up same. > > > FYI, last time we tried to fixed that, some of the lines in config.status > looked like this: > S["LDFLAGS"]="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed > -ffile-prefix-map=/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-filesyste"\ > "ms/build-st/reproducibleB-extended/tmp/work/x86-64-v3-poky-linux/e2tools/0.1.2/sources/e2tools-0.1.2=/usr/src/debug/e2tools/0.1.2 > -ffile-prefix-map"\ > "=/work/x86-64-v3-poky-linu"\ > "x/e2tools/0.1.2/build=/usr/src/debug/e2tools/0.1.2 > -ffile-prefix-map=/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-fil"\ > "esystems/build-st/reproducibleB-extended/tmp/work/x86-64-v3-poky-linux/e2tools/0.1.2/recipe-sysroot= > -ffile-prefix-map=/srv/pokybuild/yocto-worker/"\ > > "reproducible-meta-oe/build/build/build-st-meta-filesystems/build-st/reproducibleB-extended/tmp/work/x86-64-v3-poky-linux/e2tools/0.1.2/recipe-sysroo"\ > "t-native= -Wl,-z,relro,-z,now" > > => The non-reproducible paths split across line breaks. > > This is autoconf trying to generate a portable AWK script (meaning the > maximum line size for a literal is around 150). Since the "portable" here > means supporting prehistoric UNIX version of awk we started working on > modifying this limit in autoconf (_AC_AWK_LITERAL_LIMIT) to something more > modern (1000). The goal was to avoid having non-reproducible paths split > across line breaks to let sed work correctly but we never quite finished > this patch. > Thanks for adding more context here. I guess we can fix OEs autoconf like [1] These variables are useless for running/building ptests on the target. So omitting them here gets us moving as well. you can test the autoconf patch and see if it is helpful in making things better in general we might be ok to maintain it in OE upstream will not accept the raised limits [1] https://git.openembedded.org/openembedded-core-contrib/commit/?h=kraj/i-told-you&id=b7c107fc6f0142cf848393937f3d0ee09ea90228 > > Signed-off-by: Khem Raj <raj.khem@gmail.com> >> --- >> .../recipes-utils/e2tools/e2tools_0.1.2.bb | 16 +++++++++++++++- >> 1 file changed, 15 insertions(+), 1 deletion(-) >> >> diff --git a/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb >> b/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb >> index 219735acd1..b7b925da0e 100644 >> --- a/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb >> +++ b/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb >> @@ -39,7 +39,21 @@ do_install_ptest() { >> ${D}${PTEST_PATH}/build/autom4te.cache \ >> ${D}${PTEST_PATH}/*/*/.git ${D}${PTEST_PATH}/*/*/.github \ >> ${D}${PTEST_PATH}/*/*/autom4te.cache >> - sed -i -e 's;${TMPDIR};;g' ${D}${PTEST_PATH}/build/config.status >> + sed -e 's@[^ ]*-ffile-prefix-map=[^ "]*@@g' \ >> + -e 's@[^ ]*-fdebug-prefix-map=[^ "]*@@g' \ >> + -e 's@[^ ]*-fmacro-prefix-map=[^ "]*@@g' \ >> + -e 's@[^ ]*--sysroot=[^ "]*@@g' \ >> + -e 's@[^ ]*--with-libtool-sysroot=[^ "]*@@g' \ >> + -e 's@[^ ]*--with-install-prefix=[^ "]*@@g' \ >> + -e '/EXT2FS_CFLAGS/d' \ >> + -e '/LDFLAGS/d' \ >> + -e '/PKG_CONFIG_PATH/d' \ >> + -e '/PKG_CONFIG_LIBDIR/d' \ >> + -e 's@${S}@${PTEST_PATH}@g' \ >> + -e 's@${B}@${PTEST_PATH}/build@g' \ >> + -e 's@${HOSTTOOLS_DIR}@@g' \ >> + -e 's@${RECIPE_SYSROOT}@@g' \ >> + -i ${D}${PTEST_PATH}/build/config.status >> } >> >> RDEPENDS:${PN}-ptest += "bash coreutils e2fsprogs e2tools gawk make perl" >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#122065): >> https://lists.openembedded.org/g/openembedded-devel/message/122065 >> Mute This Topic: https://lists.openembedded.org/mt/116477328/4316185 >> Group Owner: openembedded-devel+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ >> yoann.congal@smile.fr] >> -=-=-=-=-=-=-=-=-=-=-=- >> >> > > -- > Yoann Congal > Smile ECS >
diff --git a/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb b/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb index 219735acd1..b7b925da0e 100644 --- a/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb +++ b/meta-filesystems/recipes-utils/e2tools/e2tools_0.1.2.bb @@ -39,7 +39,21 @@ do_install_ptest() { ${D}${PTEST_PATH}/build/autom4te.cache \ ${D}${PTEST_PATH}/*/*/.git ${D}${PTEST_PATH}/*/*/.github \ ${D}${PTEST_PATH}/*/*/autom4te.cache - sed -i -e 's;${TMPDIR};;g' ${D}${PTEST_PATH}/build/config.status + sed -e 's@[^ ]*-ffile-prefix-map=[^ "]*@@g' \ + -e 's@[^ ]*-fdebug-prefix-map=[^ "]*@@g' \ + -e 's@[^ ]*-fmacro-prefix-map=[^ "]*@@g' \ + -e 's@[^ ]*--sysroot=[^ "]*@@g' \ + -e 's@[^ ]*--with-libtool-sysroot=[^ "]*@@g' \ + -e 's@[^ ]*--with-install-prefix=[^ "]*@@g' \ + -e '/EXT2FS_CFLAGS/d' \ + -e '/LDFLAGS/d' \ + -e '/PKG_CONFIG_PATH/d' \ + -e '/PKG_CONFIG_LIBDIR/d' \ + -e 's@${S}@${PTEST_PATH}@g' \ + -e 's@${B}@${PTEST_PATH}/build@g' \ + -e 's@${HOSTTOOLS_DIR}@@g' \ + -e 's@${RECIPE_SYSROOT}@@g' \ + -i ${D}${PTEST_PATH}/build/config.status } RDEPENDS:${PN}-ptest += "bash coreutils e2fsprogs e2tools gawk make perl"
Currently the path checks are escaping QA check for buildpath detection but config.status still has paths which show up in reproduciblity failures, comparing build in path A and build in path B, content of config.status don't end up same. Signed-off-by: Khem Raj <raj.khem@gmail.com> --- .../recipes-utils/e2tools/e2tools_0.1.2.bb | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)