Message ID | 20221103075606.208982-3-nathan@nathanrossi.com |
---|---|
State | New |
Headers | show |
Series | [1/3] oeqa/selftest/minidebuginfo: Create selftest for minidebuginfo | expand |
Hello, This series causes the following failure on the autobuilders: https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4354/steps/14/logs/stdio 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_bindir: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_machine: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_alternatives: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_basic: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_binary: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_extlayer: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_func: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_glob: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_todir_glob: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_invalid: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_orig: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_patch: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_postinstall: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_renamed: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_script: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_subdir: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_todir: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_wildcard: ERROR On 03/11/2022 07:56:06+0000, Nathan Rossi wrote: > From: Nathan Rossi <nathan.rossi@digi.com> > > Newer versions of binutils (2.38+) have changed how the > "--only-keep-debug" of objcopy behaves when stripping non-debug sections > from an ELF. > > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80 > > This change causes associated sections to be correctly marked as NOBITS > with the section contents removed from the output. The side effect is > that this causes issues with objcopy's ability to perform symbol and > relocation stripping (-S/--strip-all) on the debug split ELF, such that > with some object files (e.g. kernel modules) objcopy fails to strip > symbols/relocations with an error like the following: > > .../.debug/nls_cp950.ko[.rodata]: file truncated > > Because of this it is now problematic to generate minidebuginfo for > these types of ELF objects. However it is not typically useful to inject > minidebuginfo into these types of ELFs, and other distributions (e.g. > Fedora, referring to find-debuginfo.sh of debugedit) only insert > minidebuginfo into executables and shared libraries. > > This change causes the minidebuginfo injection to only apply to EXEC/DYN > type ELFs, which limits the injection to executables and shared > libraires. > > Additionally this change fixes the parsing of the sections from the > "readelf -W -S" output which was not accounting for the section index > column having leading spaces for single digit index values e.g. "[ 1]". > > Signed-off-by: Nathan Rossi <nathan.rossi@digi.com> > --- > meta/classes-global/package.bbclass | 21 ++++++++++++++++++--- > 1 file changed, 18 insertions(+), 3 deletions(-) > > diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass > index 2d985d8aff..7a0a428b30 100644 > --- a/meta/classes-global/package.bbclass > +++ b/meta/classes-global/package.bbclass > @@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d): > bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file)) > return > > + # minidebuginfo does not make sense to apply to ELF objects other than > + # executables and shared libraries, skip applying the minidebuginfo > + # generation for objects like kernel modules. > + for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines(): > + if not line.strip().startswith("Type:"): > + continue > + elftype = line.split(":")[1].strip() > + if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]): > + bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file)) > + return > + break > + > # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo. > # We will exclude all of these from minidebuginfo to save space. > remove_section_names = [] > for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines(): > - fields = line.split() > - if len(fields) < 8: > + # strip the leading " [ 1]" section index to allow splitting on space > + if ']' not in line: > + continue > + fields = line[line.index(']') + 1:].split() > + if len(fields) < 7: > continue > name = fields[0] > type = fields[1] > - flags = fields[7] > + flags = fields[6] > # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them > if name.startswith('.debug_'): > continue > --- > 2.37.2 > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#172617): https://lists.openembedded.org/g/openembedded-core/message/172617 > Mute This Topic: https://lists.openembedded.org/mt/94752153/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Mon, 7 Nov 2022 at 01:55, Alexandre Belloni <alexandre.belloni@bootlin.com> wrote: > > Hello, > > This series causes the following failure on the autobuilders: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4354/steps/14/logs/stdio I looked into these failures, the SRC_URI error appears to be an issue with the lic_checksum test itself, and it presents without this series. I have posted a patch to resolve it https://lists.openembedded.org/g/openembedded-core/message/172945. The other errors around "test.bb" bash hash changes do not appear to be errors from the recipetools bitbake execution (note the "2 ERROR messages"). I suspect they are just captured during the fetch selftest cases in a common stderr buffer? I was unable to reproduce them locally when running fetch/recipetool. Regards, Nathan > > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_bindir: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_machine: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_alternatives: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_basic: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_binary: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_extlayer: ERROR > 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_func: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_glob: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_todir_glob: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_invalid: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_orig: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_patch: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_postinstall: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_renamed: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_script: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_subdir: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_todir: ERROR > 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_wildcard: ERROR > > On 03/11/2022 07:56:06+0000, Nathan Rossi wrote: > > From: Nathan Rossi <nathan.rossi@digi.com> > > > > Newer versions of binutils (2.38+) have changed how the > > "--only-keep-debug" of objcopy behaves when stripping non-debug sections > > from an ELF. > > > > https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80 > > > > This change causes associated sections to be correctly marked as NOBITS > > with the section contents removed from the output. The side effect is > > that this causes issues with objcopy's ability to perform symbol and > > relocation stripping (-S/--strip-all) on the debug split ELF, such that > > with some object files (e.g. kernel modules) objcopy fails to strip > > symbols/relocations with an error like the following: > > > > .../.debug/nls_cp950.ko[.rodata]: file truncated > > > > Because of this it is now problematic to generate minidebuginfo for > > these types of ELF objects. However it is not typically useful to inject > > minidebuginfo into these types of ELFs, and other distributions (e.g. > > Fedora, referring to find-debuginfo.sh of debugedit) only insert > > minidebuginfo into executables and shared libraries. > > > > This change causes the minidebuginfo injection to only apply to EXEC/DYN > > type ELFs, which limits the injection to executables and shared > > libraires. > > > > Additionally this change fixes the parsing of the sections from the > > "readelf -W -S" output which was not accounting for the section index > > column having leading spaces for single digit index values e.g. "[ 1]". > > > > Signed-off-by: Nathan Rossi <nathan.rossi@digi.com> > > --- > > meta/classes-global/package.bbclass | 21 ++++++++++++++++++--- > > 1 file changed, 18 insertions(+), 3 deletions(-) > > > > diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass > > index 2d985d8aff..7a0a428b30 100644 > > --- a/meta/classes-global/package.bbclass > > +++ b/meta/classes-global/package.bbclass > > @@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d): > > bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file)) > > return > > > > + # minidebuginfo does not make sense to apply to ELF objects other than > > + # executables and shared libraries, skip applying the minidebuginfo > > + # generation for objects like kernel modules. > > + for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines(): > > + if not line.strip().startswith("Type:"): > > + continue > > + elftype = line.split(":")[1].strip() > > + if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]): > > + bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file)) > > + return > > + break > > + > > # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo. > > # We will exclude all of these from minidebuginfo to save space. > > remove_section_names = [] > > for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines(): > > - fields = line.split() > > - if len(fields) < 8: > > + # strip the leading " [ 1]" section index to allow splitting on space > > + if ']' not in line: > > + continue > > + fields = line[line.index(']') + 1:].split() > > + if len(fields) < 7: > > continue > > name = fields[0] > > type = fields[1] > > - flags = fields[7] > > + flags = fields[6] > > # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them > > if name.startswith('.debug_'): > > continue > > --- > > 2.37.2 > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#172617): https://lists.openembedded.org/g/openembedded-core/message/172617 > > Mute This Topic: https://lists.openembedded.org/mt/94752153/3617179 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > -- > Alexandre Belloni, co-owner and COO, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com
diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass index 2d985d8aff..7a0a428b30 100644 --- a/meta/classes-global/package.bbclass +++ b/meta/classes-global/package.bbclass @@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d): bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file)) return + # minidebuginfo does not make sense to apply to ELF objects other than + # executables and shared libraries, skip applying the minidebuginfo + # generation for objects like kernel modules. + for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines(): + if not line.strip().startswith("Type:"): + continue + elftype = line.split(":")[1].strip() + if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]): + bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file)) + return + break + # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo. # We will exclude all of these from minidebuginfo to save space. remove_section_names = [] for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines(): - fields = line.split() - if len(fields) < 8: + # strip the leading " [ 1]" section index to allow splitting on space + if ']' not in line: + continue + fields = line[line.index(']') + 1:].split() + if len(fields) < 7: continue name = fields[0] type = fields[1] - flags = fields[7] + flags = fields[6] # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them if name.startswith('.debug_'): continue