| Message ID | 20250701112317.1204143-1-michalwsieron@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | bitbake.conf: Add relative path from B to S to DEBUG_PREFIX_MAP | expand |
On Tue, 2025-07-01 at 04:23 -0700, Michal Sieron via lists.openembedded.org wrote: > Basically since forever, meson uses relative paths when generating > ninja rules [1]. This breaks prefix mapping that bitbake sets up in > DEBUG_PREFIX_MAP. As a result practicaly no files are added to the -src > packages built with meson. > > We could add it with :append in meson.bbclass, but then it's not really > possible to override it (DEBUG_PREFIX_MAP is defined with `?=` probably > for a reason). > > [1]: https://github.com/mesonbuild/meson/commit/8d1641d6a42e0a5f6db03bc38d252d3712195929 > > Signed-off-by: Michal Sieron <michalwsieron@gmail.com> > --- > meta/conf/bitbake.conf | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > index b1f8ac5b11..0ecbc8c71f 100644 > --- a/meta/conf/bitbake.conf > +++ b/meta/conf/bitbake.conf > @@ -656,6 +656,7 @@ TARGET_DBGSRC_DIR ?= "/usr/src/debug/${PN}/${PV}" > DEBUG_PREFIX_MAP ?= "\ > -ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \ > -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \ > + -ffile-prefix-map=${@os.path.relpath(S, B)}=${TARGET_DBGSRC_DIR} \ > -ffile-prefix-map=${STAGING_DIR_HOST}= \ > -ffile-prefix-map=${STAGING_DIR_NATIVE}= \ > " This worries me a bit since going from memory when we originally implemented this, the use of relative paths shouldn't matter, gcc and other users of file-prefix-map should canonicalise paths before making comparisons. If it doesn't do that, there are so many combinations, it doesn't make sense. As such, I think the bug is elsewhere and we shouldn't apply this patch but root cause it... Cheers, Richard
> This worries me a bit since going from memory when we originally > implemented this, the use of relative paths shouldn't matter, gcc and > other users of file-prefix-map should canonicalise paths before making > comparisons. If it doesn't do that, there are so many combinations, it > doesn't make sense. As such, I think the bug is elsewhere and we > shouldn't apply this patch but root cause it... I did a test build outside of Yocto to make sure I am not imagining things. meson.build: project('hello', 'c') map_to = '/usr/src/debug/hello/1.0' executable('hello', 'hello.c', c_args: [ '-fmacro-prefix-map=' + meson.project_source_root() + '=' + map_to, '-fdebug-prefix-map=' + meson.project_source_root() + '=' + map_to, '-ffile-prefix-map=' + meson.project_source_root() + '=' + map_to, '-fmacro-prefix-map=' + meson.project_build_root() + '=' + map_to, '-fdebug-prefix-map=' + meson.project_build_root() + '=' + map_to, '-ffile-prefix-map=' + meson.project_build_root() + '=' + map_to, ]) hello.c: #include <stdio.h> int main() { printf("Hello, world!\n"); return 0; } Running dwarfsrcfiles on it I get: $ dwarfsrcfiles build/hello /usr/src/debug/hello/1.0/../hello.c /usr/src/debug/hello/1.0/../hello.c Best regards, Michal
On Thu, 2025-07-03 at 17:14 +0200, Michal Sieron wrote: > > This worries me a bit since going from memory when we originally > > implemented this, the use of relative paths shouldn't matter, gcc and > > other users of file-prefix-map should canonicalise paths before making > > comparisons. If it doesn't do that, there are so many combinations, it > > doesn't make sense. As such, I think the bug is elsewhere and we > > shouldn't apply this patch but root cause it... > > I did a test build outside of Yocto to make sure I am not imagining > things. > > meson.build: > project('hello', 'c') > > map_to = '/usr/src/debug/hello/1.0' > > executable('hello', 'hello.c', c_args: [ > '-fmacro-prefix-map=' + meson.project_source_root() + '=' + map_to, > '-fdebug-prefix-map=' + meson.project_source_root() + '=' + map_to, > '-ffile-prefix-map=' + meson.project_source_root() + '=' + map_to, > '-fmacro-prefix-map=' + meson.project_build_root() + '=' + map_to, > '-fdebug-prefix-map=' + meson.project_build_root() + '=' + map_to, > '-ffile-prefix-map=' + meson.project_build_root() + '=' + map_to, > ]) > > hello.c: > #include <stdio.h> > > int main() { > printf("Hello, world!\n"); > return 0; > } > > Running dwarfsrcfiles on it I get: > $ dwarfsrcfiles build/hello > /usr/src/debug/hello/1.0/../hello.c > /usr/src/debug/hello/1.0/../hello.c It gets even more strange as if you swap the build_root() to be before the source_root() parameters, you see: /usr/src/debug/hello/1.0/build/../hello.c /usr/src/debug/hello/1.0/build/../hello.c which is actually correct. You are supposed to order them from most specific to least specific. That doesn't change the fact the output from the other way around isn't correct. I still maintain this is a bug in gcc's handling of the prefix-maps. Cheers, Richard
> It gets even more strange as if you swap the build_root() to be before > the source_root() parameters, you see: > > /usr/src/debug/hello/1.0/build/../hello.c > /usr/src/debug/hello/1.0/build/../hello.c > > which is actually correct. You are supposed to order them from most > specific to least specific. That doesn't change the fact the output > from the other way around isn't correct. > > I still maintain this is a bug in gcc's handling of the prefix-maps. Welp, I used the same ordering as in bitbake.conf... Anyway, using now smaller set of flags '-ffile-prefix-map=' + meson.project_build_root() + '=' + map_to, '-ffile-prefix-map=' + meson.project_source_root() + '=' + map_to, I tried doing the same, but now with clang: build_root() first: /usr/src/debug/hello/1.0/../hello.c source_root() first: /usr/src/debug/hello/1.0/build/../hello.c So I don't think your comment about this being a bug in gcc is correct. Unless the same bug is present in clang. I can't do the check in Yocto at the moment, but if you say the [...]/build/../hello.c variant is correct and Yocto would correctly populate -src package, then I would say that's the answer? That is, to change the order of those two flags in the bitbake.conf. Best regards, Michal
On Fri, 2025-07-04 at 17:11 +0200, Michal Sieron wrote: > > It gets even more strange as if you swap the build_root() to be before > > the source_root() parameters, you see: > > > > /usr/src/debug/hello/1.0/build/../hello.c > > /usr/src/debug/hello/1.0/build/../hello.c > > > > which is actually correct. You are supposed to order them from most > > specific to least specific. That doesn't change the fact the output > > from the other way around isn't correct. > > > > I still maintain this is a bug in gcc's handling of the prefix-maps. > > Welp, I used the same ordering as in bitbake.conf... > > Anyway, using now smaller set of flags > '-ffile-prefix-map=' + meson.project_build_root() + '=' + map_to, > '-ffile-prefix-map=' + meson.project_source_root() + '=' + map_to, > I tried doing the same, but now with clang: > build_root() first: /usr/src/debug/hello/1.0/../hello.c > source_root() first: /usr/src/debug/hello/1.0/build/../hello.c > > So I don't think your comment about this being a bug in gcc is correct. > Unless the same bug is present in clang. > > I can't do the check in Yocto at the moment, but if you say the > [...]/build/../hello.c variant is correct and Yocto would correctly > populate -src package, then I would say that's the answer? > That is, to change the order of those two flags in the bitbake.conf. Experimenting a bit with the commands and gcc, it appears that the first prefix remap will apply to relative paths. My expectation was that it should add cwd and then canonicalise the path, then apply the maps. The problem is that if cwd changes, the file-prefix-map parameter is wrong, so it all depends on where the command is being run from. We can't guarantee that the command will always be run from a given directory. I went digging and this is the real problem: https://git.yoctoproject.org/poky/commit/?id=ea00dbbcf3bacf6e115fb261d9880f52be25ca05 in that we removed -fcanon-prefix-map thinking it worked and it clearly doesn't. We *need* that option and it will need to be put back, probably gcc specific. Copying Khem. Cheers, Richard
> in that we removed -fcanon-prefix-map thinking it worked and it clearly > doesn't. > > We *need* that option and it will need to be put back, probably gcc > specific. Copying Khem. But then what do we do about clang, which behaves the same way and doesn't support -fcanon-prefix-map? There is also Kirkstone, which is still on gcc 11 (that option was introduced in gcc 13). I also went and adjusted the 'reproduction' example a little bit to be more similar to Yocto setup. I put the meson.build and hello.c files in a 'git' directory. Then from parent of 'git' directory I am calling meson with absolute paths to source and build dirs (again to mirror how Yocto works). Now regardless of the order of prefix-map flags I get the same output: $ dwarfsrcfiles build/hello /usr/src/debug/hello/1.0/../git/hello.c /usr/src/debug/hello/1.0/../git/hello.c which reminded me that this is the actual reason I first saw this as a bug couple weeks ago. After all, Yocto won't create a 'git' directory in the -src package. With -fcanon-prefix-map it's a little better: $ dwarfsrcfiles build/hello /usr/src/debug/hello/1.0//usr/src/debug/hello/1.0/hello.c /usr/src/debug/hello/1.0/hello.c Best regards, Michal
On Sat, 2025-07-05 at 16:41 +0200, Michal Sieron wrote: > > in that we removed -fcanon-prefix-map thinking it worked and it > > clearly > > doesn't. > > > > We *need* that option and it will need to be put back, probably gcc > > specific. Copying Khem. > > But then what do we do about clang, which behaves the same way and > doesn't support -fcanon-prefix-map? There is also Kirkstone, which is > still on gcc 11 (that option was introduced in gcc 13). We ran into this issue and convinced gcc it needed a patch. I suspect kirkstone has that patch. The patch was merged by gcc 13 so we no longer had to carry it. I suspect someone needs to add this support to clang as I suspect there isn't another way to handl relative paths. > I also went and adjusted the 'reproduction' example a little bit to > be more similar to Yocto setup. I put the meson.build and hello.c > files in a 'git' directory. Then from parent of 'git' directory I am > calling meson with absolute paths to source and build dirs (again to > mirror how Yocto works). > > Now regardless of the order of prefix-map flags I get the same > output: > $ dwarfsrcfiles build/hello > /usr/src/debug/hello/1.0/../git/hello.c > /usr/src/debug/hello/1.0/../git/hello.c > which reminded me that this is the actual reason I first saw this as > a bug couple weeks ago. After all, Yocto won't create a 'git' > directory in the -src package. > > With -fcanon-prefix-map it's a little better: > $ dwarfsrcfiles build/hello > /usr/src/debug/hello/1.0//usr/src/debug/hello/1.0/hello.c > /usr/src/debug/hello/1.0/hello.c That does look a bit more like what I'd have expected. Cheers, Richard
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index b1f8ac5b11..0ecbc8c71f 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -656,6 +656,7 @@ TARGET_DBGSRC_DIR ?= "/usr/src/debug/${PN}/${PV}" DEBUG_PREFIX_MAP ?= "\ -ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \ + -ffile-prefix-map=${@os.path.relpath(S, B)}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${STAGING_DIR_HOST}= \ -ffile-prefix-map=${STAGING_DIR_NATIVE}= \ "
Basically since forever, meson uses relative paths when generating ninja rules [1]. This breaks prefix mapping that bitbake sets up in DEBUG_PREFIX_MAP. As a result practicaly no files are added to the -src packages built with meson. We could add it with :append in meson.bbclass, but then it's not really possible to override it (DEBUG_PREFIX_MAP is defined with `?=` probably for a reason). [1]: https://github.com/mesonbuild/meson/commit/8d1641d6a42e0a5f6db03bc38d252d3712195929 Signed-off-by: Michal Sieron <michalwsieron@gmail.com> --- meta/conf/bitbake.conf | 1 + 1 file changed, 1 insertion(+)