| Message ID | 1728037005-31667-1-git-send-email-diego.santacruz@spinetix.com |
|---|---|
| State | New |
| Headers | show |
| Series | pseudo: add man user and group to fallback | expand |
On Fri, 2024-10-04 at 12:16 +0200, Diego Santa Cruz via lists.openembedded.org wrote: > When installing packages with man pages the man user and group are > expected to exist. These are included in base-passwd and installed as > part of the rootfs. However, when populating the SDK the rootfs may not > have been built and thus the user and group cannot be resolved and the > installation fails with the following error. > > chown: invalid user: ‘man:man’ > > Adding the man user and group to the pseudo fallback passwd DB solves > the problem. > > Note that PSEUDO_PASSWD cannot be made task specific as to include the > SDK sysroot path in the populate_sdk and populate_sdk_ext tasks, as > PSEUDO_PASSWD is embedded in FAKEROOTENV, and FAKEROOTENV is cached > globally for all the tasks of a recipe. Adding the path of the SDK > sysroot to all tasks of image recipes would be a bit risky, so it is > best avoided. > > Signed-off-by: Diego Santa Cruz <diego.santacruz@spinetix.com> > --- > meta/recipes-devtools/pseudo/files/fallback-group | 1 + > meta/recipes-devtools/pseudo/files/fallback-passwd | 1 + > 2 files changed, 2 insertions(+) We don't preserve user/group ownership in the SDK so setting this ownership doesn't make sense in the first place. Which release is this with? It sounds very similar to what this was fixing: https://git.yoctoproject.org/poky/commit/?id=9ccde59c5303abb04e9bb8e2c987ff4b8f44bb43 Cheers, Richard
> -----Original Message----- > From: Richard Purdie <richard.purdie@linuxfoundation.org> > Sent: 04 October 2024 12:28 > To: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>; openembedded- > core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH] pseudo: add man user and group to fallback > > On Fri, 2024-10-04 at 12:16 +0200, Diego Santa Cruz via lists.openembedded.org > wrote: > > When installing packages with man pages the man user and group are > > expected to exist. These are included in base-passwd and installed as > > part of the rootfs. However, when populating the SDK the rootfs may not > > have been built and thus the user and group cannot be resolved and the > > installation fails with the following error. > > > > chown: invalid user: ‘man:man’ > > > > Adding the man user and group to the pseudo fallback passwd DB solves > > the problem. > > > > Note that PSEUDO_PASSWD cannot be made task specific as to include the > > SDK sysroot path in the populate_sdk and populate_sdk_ext tasks, as > > PSEUDO_PASSWD is embedded in FAKEROOTENV, and FAKEROOTENV is cached > > globally for all the tasks of a recipe. Adding the path of the SDK > > sysroot to all tasks of image recipes would be a bit risky, so it is > > best avoided. > > > > Signed-off-by: Diego Santa Cruz <diego.santacruz@spinetix.com> > > --- > > meta/recipes-devtools/pseudo/files/fallback-group | 1 + > > meta/recipes-devtools/pseudo/files/fallback-passwd | 1 + > > 2 files changed, 2 insertions(+) > > We don't preserve user/group ownership in the SDK so setting this > ownership doesn't make sense in the first place. > > Which release is this with? This is with kirkstone, not master, I was maybe too eager to assume the problem still existed in master, but after looking things up I think it does. The problem is that, before mickledore, manpages.bbclass has a postinst script which does a "chown man:man", while running under pseudo. So pseudo attempts to resolve the man user and group, but the path that it uses to look up the passwd and group files points to the target image, not the SDK image. In our builds we build both the target image and SDK in the same bitbake invocation so the do_rootfs and do_populate_sdk tasks race against each other and we have random failures, which occur when the postinst script runs before the do_rootfs has populated the passwd and group files of the target image. Granted the chown is not useful for the SDK in the end, but the postinst is common between the target image and the SDK, I do not know if it can easily be made conditional on target only. But I now see that this was changed in mickledore in https://git.yoctoproject.org/poky/commit/?id=d42904ba0cf0eac28ef79b0e6015150d376d134d This "chown man:man" is still run but in an intercept script (scripts/postinst-intercepts/update_mandb), but as I understand this still runs under pseudo, so presumable the problem is still present in master. I doubt the fix you mention affects this because that affects PATH, not the path used by pseudo to look up the passwd and group files. That is PSEUDO_PASSWD, which is still set to "${IMAGE_ROOTFS}:${STAGING_DIR_NATIVE}" in image.bbclass. If you think it necessary I can maybe do a build with master to see if I can reproduce the problem. > > It sounds very similar to what this was fixing: > > https://git.yoctopr/ > oject.org%2Fpoky%2Fcommit%2F%3Fid%3D9ccde59c5303abb04e9bb8e2c987ff4 > b8f44bb43&data=05%7C02%7Cdiego.santacruz%40spinetix.com%7C972427b255 > 4041e4505908dce45f41e0%7C5f4034faed2d4840a93facb1e9633b93%7C0%7C0 > %7C638636344994463896%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&s > data=Bw%2BPwwYiu0GoEhvHHdaQXGI8urEA6Uj%2FGGQr3%2BCgLuE%3D&rese > rved=0 > > Cheers, > > Richard Best, Diego
On Fri, 2024-10-04 at 10:59 +0000, Diego Santa Cruz wrote: > > -----Original Message----- > > From: Richard Purdie <richard.purdie@linuxfoundation.org> > > Sent: 04 October 2024 12:28 > > To: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>; openembedded- > > core@lists.openembedded.org > > Subject: Re: [OE-core] [PATCH] pseudo: add man user and group to fallback > > > > On Fri, 2024-10-04 at 12:16 +0200, Diego Santa Cruz via lists.openembedded.org > > wrote: > > > When installing packages with man pages the man user and group are > > > expected to exist. These are included in base-passwd and installed as > > > part of the rootfs. However, when populating the SDK the rootfs may not > > > have been built and thus the user and group cannot be resolved and the > > > installation fails with the following error. > > > > > > chown: invalid user: ‘man:man’ > > > > > > Adding the man user and group to the pseudo fallback passwd DB solves > > > the problem. > > > > > > Note that PSEUDO_PASSWD cannot be made task specific as to include the > > > SDK sysroot path in the populate_sdk and populate_sdk_ext tasks, as > > > PSEUDO_PASSWD is embedded in FAKEROOTENV, and FAKEROOTENV is cached > > > globally for all the tasks of a recipe. Adding the path of the SDK > > > sysroot to all tasks of image recipes would be a bit risky, so it is > > > best avoided. > > > > > > Signed-off-by: Diego Santa Cruz <diego.santacruz@spinetix.com> > > > --- > > > meta/recipes-devtools/pseudo/files/fallback-group | 1 + > > > meta/recipes-devtools/pseudo/files/fallback-passwd | 1 + > > > 2 files changed, 2 insertions(+) > > > > We don't preserve user/group ownership in the SDK so setting this > > ownership doesn't make sense in the first place. > > > > Which release is this with? > > This is with kirkstone, not master, I was maybe too eager to assume > the problem still existed in master, but after looking things up I > think it does. > > The problem is that, before mickledore, manpages.bbclass has a > postinst script which does a "chown man:man", while running under > pseudo. So pseudo attempts to resolve the man user and group, but the > path that it uses to look up the passwd and group files points to the > target image, not the SDK image. In our builds we build both the > target image and SDK in the same bitbake invocation so the do_rootfs > and do_populate_sdk tasks race against each other and we have random > failures, which occur when the postinst script runs before the > do_rootfs has populated the passwd and group files of the target > image. > > Granted the chown is not useful for the SDK in the end, but the > postinst is common between the target image and the SDK, I do not > know if it can easily be made conditional on target only. > > But I now see that this was changed in mickledore in > https://git.yoctoproject.org/poky/commit/?id=d42904ba0cf0eac28ef79b0e6015150d376d134d > This "chown man:man" is still run but in an intercept script > (scripts/postinst-intercepts/update_mandb), but as I understand this > still runs under pseudo, so presumable the problem is still present > in master. > > I doubt the fix you mention affects this because that affects PATH, > not the path used by pseudo to look up the passwd and group files. > That is PSEUDO_PASSWD, which is still set to > "${IMAGE_ROOTFS}:${STAGING_DIR_NATIVE}" in image.bbclass. > > If you think it necessary I can maybe do a build with master to see > if I can reproduce the problem. This change may make it clearer: https://git.yoctoproject.org/poky/commit/?id=b5dbdfe90497867a70fcdd69a3e347952dff7d3a We intercept chgrp/chown calls so those won't happen on master. There was at least one bug in the way they were working which that change fixed. There may be elements of this which could be backported to kirkstone. Cheers, Richard
> -----Original Message----- > From: Richard Purdie <richard.purdie@linuxfoundation.org> ... > > This change may make it clearer: > > https://git.yoctopr/ > oject.org%2Fpoky%2Fcommit%2F%3Fid%3Db5dbdfe90497867a70fcdd69a3e3479 > 52dff7d3a&data=05%7C02%7CDiego.SantaCruz%40spinetix.com%7Ce1c23609fd > d8496109bd08dce4667060%7C5f4034faed2d4840a93facb1e9633b93%7C0%7C0 > %7C638636375834304081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&s > data=rNFZhKXj%2BACXIKC5hi%2F0du4S3sGD25f6KFE3oc7oPl4%3D&reserved=0 > > We intercept chgrp/chown calls so those won't happen on master. There > was at least one bug in the way they were working which that change > fixed. Got it! Better that way then, no need for my fix
On Fri, 2024-10-04 at 12:08 +0000, Diego Santa Cruz wrote: > > -----Original Message----- > > From: Richard Purdie <richard.purdie@linuxfoundation.org> > ... > > > > This change may make it clearer: > > > > https://git.yoctopr/ > > oject.org%2Fpoky%2Fcommit%2F%3Fid%3Db5dbdfe90497867a70fcdd69a3e3479 > > 52dff7d3a&data=05%7C02%7CDiego.SantaCruz%40spinetix.com%7Ce1c23609f > > d > > d8496109bd08dce4667060%7C5f4034faed2d4840a93facb1e9633b93%7C0%7C0 > > %7C638636375834304081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&s > > data=rNFZhKXj%2BACXIKC5hi%2F0du4S3sGD25f6KFE3oc7oPl4%3D&reserved=0 > > > > We intercept chgrp/chown calls so those won't happen on master. > > There > > was at least one bug in the way they were working which that change > > fixed. > > Got it! Better that way then, no need for my fix
diff --git a/meta/recipes-devtools/pseudo/files/fallback-group b/meta/recipes-devtools/pseudo/files/fallback-group index bb3dbca..068ea7d 100644 --- a/meta/recipes-devtools/pseudo/files/fallback-group +++ b/meta/recipes-devtools/pseudo/files/fallback-group @@ -1,3 +1,4 @@ root:*:0: mail:*:8: +man:*:12: nobody:*:99: diff --git a/meta/recipes-devtools/pseudo/files/fallback-passwd b/meta/recipes-devtools/pseudo/files/fallback-passwd index 08611ba..270935a 100644 --- a/meta/recipes-devtools/pseudo/files/fallback-passwd +++ b/meta/recipes-devtools/pseudo/files/fallback-passwd @@ -1,3 +1,4 @@ root::0:0:root:/home/root:/bin/sh pseudopasswd:*:1:1:this-is-the-pseudo-passwd:/nonexistent:/bin/sh +man:*:6:12:man:/var/cache/man:/bin/sh nobody:*:65534:65534:nobody:/nonexistent:/bin/sh
When installing packages with man pages the man user and group are expected to exist. These are included in base-passwd and installed as part of the rootfs. However, when populating the SDK the rootfs may not have been built and thus the user and group cannot be resolved and the installation fails with the following error. chown: invalid user: ‘man:man’ Adding the man user and group to the pseudo fallback passwd DB solves the problem. Note that PSEUDO_PASSWD cannot be made task specific as to include the SDK sysroot path in the populate_sdk and populate_sdk_ext tasks, as PSEUDO_PASSWD is embedded in FAKEROOTENV, and FAKEROOTENV is cached globally for all the tasks of a recipe. Adding the path of the SDK sysroot to all tasks of image recipes would be a bit risky, so it is best avoided. Signed-off-by: Diego Santa Cruz <diego.santacruz@spinetix.com> --- meta/recipes-devtools/pseudo/files/fallback-group | 1 + meta/recipes-devtools/pseudo/files/fallback-passwd | 1 + 2 files changed, 2 insertions(+)