| Message ID | BYAPR12MB2710069E5D018DBB8F800AECF5569@BYAPR12MB2710.namprd12.prod.outlook.com |
|---|---|
| State | New |
| Headers | show |
| Series | bitbake: wget: fix netrc with pseudo | expand |
How can the problem be reproduced? Alex On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero> wrote: > From: Nicholas Jackson <nicholas.jackson@zii.aero> > > The netrc module will refuse to use a file that is not owned by the > current user. Psuedo can trigger this failure due to a reported uid of > 0. To fix this, change ownership of the default netrc file to 0:0. This > has no permanent effect on the actual netrc file. > > Note that while it is not possible to ask the netrc module what the > default netrc file is, the documentation explicitly mentions that > expanduser is used. > > Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero> > --- > bitbake/lib/bb/fetch2/wget.py | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py > index 821afa5b58..0ce6e25c5c 100644 > --- a/bitbake/lib/bb/fetch2/wget.py > +++ b/bitbake/lib/bb/fetch2/wget.py > @@ -360,10 +360,16 @@ class Wget(FetchMethod): > > try: > import netrc > + # Pseudo can cause ownership mismatch issues with > netrc, which is > + # required to be owned by the current user (root). > + if os.environ.get("PSEUDO_DISABLED") == "0": > + os.chown(os.path.join(os.path.expanduser("~"), > ".netrc"), 0, 0) > n = netrc.netrc() > login, unused, password = > n.authenticators(urllib.parse.urlparse(uri).hostname) > add_basic_auth("%s:%s" % (login, password), r) > - except (TypeError, ImportError, IOError, > netrc.NetrcParseError): > + except netrc.NetrcParseError as e: > + logger.debug("netrc failed: %s" % e) > + except (TypeError, ImportError, IOError) as e: > pass > > with opener.open(r, timeout=30) as response: > -- > 2.37.2 > > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <http://www.safran-passenger-innovations.com/> > > > > > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > > <http://twitter.com/SAFRAN> > > <http://www.facebook.com/GroupeSafran> > > <http://fr.linkedin.com/company/safran> > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14001): > https://lists.openembedded.org/g/bitbake-devel/message/14001 > Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ > alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Thanks for the quick response. My setup is an Artifactory replication of https://sstate.yoctoproject.org/, with the Artifactory credentials stored in a netrc file. The issue specifically occurs when attempting to build certain packages using bitbake --setscene-only. (It probably also happens without --setscene-only). A warning is produced, e.g.: WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 WARNING: Setscene task (/home/foo/bar/poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) failed with exit code '1' - real task will be run instead In certain cases, bitbake will spawn a bitbake-worker with PSEUDO_DISABLED=0. Unfortunately, I have not been able to pinpoint when exactly this happens. I'm not familiar with bitbake internals, so I don't know when pseudo is used specifically. What I can say for sure is that bitbake --setscene-only world will always reproduce the issue for me without this patch. For my environment, I am building the default core-image-minimal for qemux86-64 with the three default poky layers. I've reproduced the issue in both Ubuntu 22.04 and Docker using the image ghcr.io/siemens/kas/kas:3.1. In case it matters, I am using kas. Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/> [cid:a12642b7-396b-4452-b2d3-91ac0a516ca8] [cid:626cabaa-b02b-4a31-835a-cadcfc582c1f]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:c444f1c1-8306-4c88-85ca-2f546f771eda]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> [cid:5252f91d-e5e1-4a1f-932e-753107817fdd]<http://twitter.com/SAFRAN> [cid:f22e4c4f-d42e-4b5c-b99a-c93fabda9b97]<http://www.facebook.com/GroupeSafran> [cid:c9b34257-95c4-42b0-ae51-5801c62f858f]<http://fr.linkedin.com/company/safran> ________________________________ From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin <alex.kanavin@gmail.com> Sent: Friday, September 30, 2022 10:59 AM To: Nicholas Jackson <Nicholas.Jackson@zii.aero> Cc: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo How can the problem be reproduced? Alex On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote: From: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>> The netrc module will refuse to use a file that is not owned by the current user. Psuedo can trigger this failure due to a reported uid of 0. To fix this, change ownership of the default netrc file to 0:0. This has no permanent effect on the actual netrc file. Note that while it is not possible to ask the netrc module what the default netrc file is, the documentation explicitly mentions that expanduser is used. Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>> --- bitbake/lib/bb/fetch2/wget.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index 821afa5b58..0ce6e25c5c 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -360,10 +360,16 @@ class Wget(FetchMethod): try: import netrc + # Pseudo can cause ownership mismatch issues with netrc, which is + # required to be owned by the current user (root). + if os.environ.get("PSEUDO_DISABLED") == "0": + os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0) n = netrc.netrc() login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname) add_basic_auth("%s:%s" % (login, password), r) - except (TypeError, ImportError, IOError, netrc.NetrcParseError): + except netrc.NetrcParseError as e: + logger.debug("netrc failed: %s" % e) + except (TypeError, ImportError, IOError) as e: pass with opener.open(r, timeout=30) as response: -- 2.37.2 Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> [cid:1838f8deba81078ea171] [cid:1838f8deba9b4d74fd92]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>[cid:1838f8deba9b501410f3]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> [cid:1838f8deba964f504804]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> [cid:1838f8deba96698e6055]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> [cid:1838f8deba9d184584d6]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized. ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.
This does seem like there’s something particular about your setup triggering this. Can you try outside if docker? Alex On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero> wrote: > Thanks for the quick response. > My setup is an Artifactory replication of https://sstate.yoctoproject.org/, > with > the Artifactory credentials stored in a netrc file. The issue specifically > occurs > when attempting to build certain packages using bitbake --setscene-only. > (It probably also happens without --setscene-only). > A warning is produced, e.g.: > WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive > obtainable, will run full task instead. > WARNING: Logfile for failed setscene task is > /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 > WARNING: Setscene task (/home/foo/bar > /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) > failed with exit code '1' - real task will be run instead > > In certain cases, bitbake will spawn a bitbake-worker with > PSEUDO_DISABLED=0. > Unfortunately, I have not been able to pinpoint when exactly this happens. > I'm not > familiar with bitbake internals, so I don't know when pseudo is used > specifically. > What I can say for sure is that bitbake --setscene-only world will always > reproduce the issue for me without this patch. > > For my environment, I am building the default core-image-minimal for > qemux86-64 with > the three default poky layers. I've reproduced the issue in both Ubuntu > 22.04 and > Docker using the image ghcr.io/siemens/kas/kas:3.1. In case it matters, I > am using kas. > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <http://www.safran-passenger-innovations.com/> > > > > > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > > <http://twitter.com/SAFRAN> > > <http://www.facebook.com/GroupeSafran> > > <http://fr.linkedin.com/company/safran> > > ------------------------------ > *From:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin < > alex.kanavin@gmail.com> > *Sent:* Friday, September 30, 2022 10:59 AM > *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> > *Cc:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> > *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with > pseudo > > How can the problem be reproduced? > > Alex > > On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > > From: Nicholas Jackson <nicholas.jackson@zii.aero> > > The netrc module will refuse to use a file that is not owned by the > current user. Psuedo can trigger this failure due to a reported uid of > 0. To fix this, change ownership of the default netrc file to 0:0. This > has no permanent effect on the actual netrc file. > > Note that while it is not possible to ask the netrc module what the > default netrc file is, the documentation explicitly mentions that > expanduser is used. > > Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero> > --- > bitbake/lib/bb/fetch2/wget.py | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py > index 821afa5b58..0ce6e25c5c 100644 > --- a/bitbake/lib/bb/fetch2/wget.py > +++ b/bitbake/lib/bb/fetch2/wget.py > @@ -360,10 +360,16 @@ class Wget(FetchMethod): > > try: > import netrc > + # Pseudo can cause ownership mismatch issues with > netrc, which is > + # required to be owned by the current user (root). > + if os.environ.get("PSEUDO_DISABLED") == "0": > + os.chown(os.path.join(os.path.expanduser("~"), > ".netrc"), 0, 0) > n = netrc.netrc() > login, unused, password = > n.authenticators(urllib.parse.urlparse(uri).hostname) > add_basic_auth("%s:%s" % (login, password), r) > - except (TypeError, ImportError, IOError, > netrc.NetrcParseError): > + except netrc.NetrcParseError as e: > + logger.debug("netrc failed: %s" % e) > + except (TypeError, ImportError, IOError) as e: > pass > > with opener.open(r, timeout=30) as response: > -- > 2.37.2 > > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> > > > > > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> > > > <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> > > > <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> > > > <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. >
Yes, the same issue occurs outside docker. Are you thinking that the root issue may perhaps be that PSEUDO_DISABLED is being set to 0 in the first place? It does seem strange to me that we'd need a fakeroot-like environment for a task involving wget. Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/> [cid:5f2a4653-757e-4e47-9b2a-32f303ab1e0a] [cid:c33ce570-c829-448d-a54f-75aee8791bdf]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:4bc65a6f-d39d-4eb0-9865-e31276eec185]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> [cid:a2a2ba47-4eb4-4f80-8c27-efb22f0d23c1]<http://twitter.com/SAFRAN> [cid:f273792d-7493-452b-b299-67fb5439b5c7]<http://www.facebook.com/GroupeSafran> [cid:7ffb676a-6b4e-479e-8ee8-16159c401ee8]<http://fr.linkedin.com/company/safran> ________________________________ From: Alexander Kanavin <alex.kanavin@gmail.com> Sent: Friday, September 30, 2022 12:51 PM To: Nicholas Jackson <Nicholas.Jackson@zii.aero> Cc: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo This does seem like there’s something particular about your setup triggering this. Can you try outside if docker? Alex On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote: Thanks for the quick response. My setup is an Artifactory replication of https://sstate.yoctoproject.org/<https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, with the Artifactory credentials stored in a netrc file. The issue specifically occurs when attempting to build certain packages using bitbake --setscene-only. (It probably also happens without --setscene-only). A warning is produced, e.g.: WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 WARNING: Setscene task (/home/foo/bar/poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) failed with exit code '1' - real task will be run instead In certain cases, bitbake will spawn a bitbake-worker with PSEUDO_DISABLED=0. Unfortunately, I have not been able to pinpoint when exactly this happens. I'm not familiar with bitbake internals, so I don't know when pseudo is used specifically. What I can say for sure is that bitbake --setscene-only world will always reproduce the issue for me without this patch. For my environment, I am building the default core-image-minimal for qemux86-64 with the three default poky layers. I've reproduced the issue in both Ubuntu 22.04 and Docker using the image ghcr.io/siemens/kas/kas:3.1<https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. In case it matters, I am using kas. Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$> [cid:1838fef3a6420d4b2947] [cid:1838fef3a64a87b16978]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>[cid:1838fef3a64f7432f799]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> [cid:1838fef3a64c82a9bc9a]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$> [cid:1838fef3a64f82a6facb]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$> [cid:1838fef3a64923d3295c]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$> ________________________________ From: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> on behalf of Alexander Kanavin <alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>> Sent: Friday, September 30, 2022 10:59 AM To: Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> Cc: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo How can the problem be reproduced? Alex On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote: From: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>> The netrc module will refuse to use a file that is not owned by the current user. Psuedo can trigger this failure due to a reported uid of 0. To fix this, change ownership of the default netrc file to 0:0. This has no permanent effect on the actual netrc file. Note that while it is not possible to ask the netrc module what the default netrc file is, the documentation explicitly mentions that expanduser is used. Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>> --- bitbake/lib/bb/fetch2/wget.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index 821afa5b58..0ce6e25c5c 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -360,10 +360,16 @@ class Wget(FetchMethod): try: import netrc + # Pseudo can cause ownership mismatch issues with netrc, which is + # required to be owned by the current user (root). + if os.environ.get("PSEUDO_DISABLED") == "0": + os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0) n = netrc.netrc() login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname) add_basic_auth("%s:%s" % (login, password), r) - except (TypeError, ImportError, IOError, netrc.NetrcParseError): + except netrc.NetrcParseError as e: + logger.debug("netrc failed: %s" % e) + except (TypeError, ImportError, IOError) as e: pass with opener.open(r, timeout=30) as response: -- 2.37.2 Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> [cid:1838fef3a63bb3ee15b1] [cid:1838fef3a63acc5a1812]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>[cid:1838fef3a63ffaecb153]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> [cid:1838fef3a63a8ac7fa4]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> [cid:1838fef3a634dcaf8f15]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> [cid:1838fef3a6379eb4ca36]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized. ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized. ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.
Can you try with a plain poky checkout (git clone https://git.yoctoproject.org/poky ), and outside of docker? It is rather odd that no one else sees this. Alex On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero> wrote: > Yes, the same issue occurs outside docker. > Are you thinking that the root issue may perhaps be that > PSEUDO_DISABLED is being set to 0 in the first place? It does > seem strange to me that we'd need a fakeroot-like environment for > a task involving wget. > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <http://www.safran-passenger-innovations.com/> > > > > > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > > <http://twitter.com/SAFRAN> > > <http://www.facebook.com/GroupeSafran> > > <http://fr.linkedin.com/company/safran> > > ------------------------------ > *From:* Alexander Kanavin <alex.kanavin@gmail.com> > *Sent:* Friday, September 30, 2022 12:51 PM > *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> > *Cc:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> > *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with > pseudo > > This does seem like there’s something particular about your setup > triggering this. Can you try outside if docker? > > Alex > > On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > > Thanks for the quick response. > My setup is an Artifactory replication of https://sstate.yoctoproject.org/ > <https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, > with > the Artifactory credentials stored in a netrc file. The issue specifically > occurs > when attempting to build certain packages using bitbake --setscene-only. > (It probably also happens without --setscene-only). > A warning is produced, e.g.: > WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive > obtainable, will run full task instead. > WARNING: Logfile for failed setscene task is > /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 > WARNING: Setscene task (/home/foo/bar > /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) > failed with exit code '1' - real task will be run instead > > In certain cases, bitbake will spawn a bitbake-worker with > PSEUDO_DISABLED=0. > Unfortunately, I have not been able to pinpoint when exactly this happens. > I'm not > familiar with bitbake internals, so I don't know when pseudo is used > specifically. > What I can say for sure is that bitbake --setscene-only world will always > reproduce the issue for me without this patch. > > For my environment, I am building the default core-image-minimal for > qemux86-64 with > the three default poky layers. I've reproduced the issue in both Ubuntu > 22.04 and > Docker using the image ghcr.io/siemens/kas/kas:3.1 > <https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. > In case it matters, I am using kas. > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$> > > > > > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> > > > <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$> > > > <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$> > > > <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$> > > ------------------------------ > *From:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin < > alex.kanavin@gmail.com> > *Sent:* Friday, September 30, 2022 10:59 AM > *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> > *Cc:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> > *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with > pseudo > > How can the problem be reproduced? > > Alex > > On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > > From: Nicholas Jackson <nicholas.jackson@zii.aero> > > The netrc module will refuse to use a file that is not owned by the > current user. Psuedo can trigger this failure due to a reported uid of > 0. To fix this, change ownership of the default netrc file to 0:0. This > has no permanent effect on the actual netrc file. > > Note that while it is not possible to ask the netrc module what the > default netrc file is, the documentation explicitly mentions that > expanduser is used. > > Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero> > --- > bitbake/lib/bb/fetch2/wget.py | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py > index 821afa5b58..0ce6e25c5c 100644 > --- a/bitbake/lib/bb/fetch2/wget.py > +++ b/bitbake/lib/bb/fetch2/wget.py > @@ -360,10 +360,16 @@ class Wget(FetchMethod): > > try: > import netrc > + # Pseudo can cause ownership mismatch issues with > netrc, which is > + # required to be owned by the current user (root). > + if os.environ.get("PSEUDO_DISABLED") == "0": > + os.chown(os.path.join(os.path.expanduser("~"), > ".netrc"), 0, 0) > n = netrc.netrc() > login, unused, password = > n.authenticators(urllib.parse.urlparse(uri).hostname) > add_basic_auth("%s:%s" % (login, password), r) > - except (TypeError, ImportError, IOError, > netrc.NetrcParseError): > + except netrc.NetrcParseError as e: > + logger.debug("netrc failed: %s" % e) > + except (TypeError, ImportError, IOError) as e: > pass > > with opener.open(r, timeout=30) as response: > -- > 2.37.2 > > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> > > > > > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> > > > <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> > > > <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> > > > <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. >
Basically what would help is finding how this can be reproduced by someone else, e.g. me :) Alex On Fri, 30 Sept 2022 at 22:36, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > Can you try with a plain poky checkout (git clone > https://git.yoctoproject.org/poky ), and outside of docker? It is rather > odd that no one else sees this. > > Alex > > On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > >> Yes, the same issue occurs outside docker. >> Are you thinking that the root issue may perhaps be that >> PSEUDO_DISABLED is being set to 0 in the first place? It does >> seem strange to me that we'd need a fakeroot-like environment for >> a task involving wget. >> >> >> *Nicholas JACKSON* >> Software Engineer II >> >> Safran Passenger Innovations >> >> T +1 714 854 8600 ext 527 >> >> 3151 East Imperial Hwy >> >> Brea, Ca 92821 USA >> >> *www.safran-passenger-innovations.com* >> <http://www.safran-passenger-innovations.com/> >> >> >> >> >> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> >> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> >> >> <http://twitter.com/SAFRAN> >> >> <http://www.facebook.com/GroupeSafran> >> >> <http://fr.linkedin.com/company/safran> >> >> ------------------------------ >> *From:* Alexander Kanavin <alex.kanavin@gmail.com> >> *Sent:* Friday, September 30, 2022 12:51 PM >> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> >> *Cc:* bitbake-devel@lists.openembedded.org < >> bitbake-devel@lists.openembedded.org> >> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with >> pseudo >> >> This does seem like there’s something particular about your setup >> triggering this. Can you try outside if docker? >> >> Alex >> >> On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero> >> wrote: >> >> Thanks for the quick response. >> My setup is an Artifactory replication of >> https://sstate.yoctoproject.org/ >> <https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, >> with >> the Artifactory credentials stored in a netrc file. The issue >> specifically occurs >> when attempting to build certain packages using bitbake --setscene-only. >> (It probably also happens without --setscene-only). >> A warning is produced, e.g.: >> WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive >> obtainable, will run full task instead. >> WARNING: Logfile for failed setscene task is >> /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 >> WARNING: Setscene task (/home/foo/bar >> /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) >> failed with exit code '1' - real task will be run instead >> >> In certain cases, bitbake will spawn a bitbake-worker with >> PSEUDO_DISABLED=0. >> Unfortunately, I have not been able to pinpoint when exactly this >> happens. I'm not >> familiar with bitbake internals, so I don't know when pseudo is used >> specifically. >> What I can say for sure is that bitbake --setscene-only world will >> always >> reproduce the issue for me without this patch. >> >> For my environment, I am building the default core-image-minimal for >> qemux86-64 with >> the three default poky layers. I've reproduced the issue in both Ubuntu >> 22.04 and >> Docker using the image ghcr.io/siemens/kas/kas:3.1 >> <https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. >> In case it matters, I am using kas. >> >> >> *Nicholas JACKSON* >> Software Engineer II >> >> Safran Passenger Innovations >> >> T +1 714 854 8600 ext 527 >> >> 3151 East Imperial Hwy >> >> Brea, Ca 92821 USA >> >> *www.safran-passenger-innovations.com* >> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$> >> >> >> >> >> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> >> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> >> >> >> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$> >> >> >> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$> >> >> >> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$> >> >> ------------------------------ >> *From:* bitbake-devel@lists.openembedded.org < >> bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin < >> alex.kanavin@gmail.com> >> *Sent:* Friday, September 30, 2022 10:59 AM >> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> >> *Cc:* bitbake-devel@lists.openembedded.org < >> bitbake-devel@lists.openembedded.org> >> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with >> pseudo >> >> How can the problem be reproduced? >> >> Alex >> >> On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson < >> Nicholas.Jackson@zii.aero> wrote: >> >> From: Nicholas Jackson <nicholas.jackson@zii.aero> >> >> The netrc module will refuse to use a file that is not owned by the >> current user. Psuedo can trigger this failure due to a reported uid of >> 0. To fix this, change ownership of the default netrc file to 0:0. This >> has no permanent effect on the actual netrc file. >> >> Note that while it is not possible to ask the netrc module what the >> default netrc file is, the documentation explicitly mentions that >> expanduser is used. >> >> Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero> >> --- >> bitbake/lib/bb/fetch2/wget.py | 8 +++++++- >> 1 file changed, 7 insertions(+), 1 deletion(-) >> >> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py >> index 821afa5b58..0ce6e25c5c 100644 >> --- a/bitbake/lib/bb/fetch2/wget.py >> +++ b/bitbake/lib/bb/fetch2/wget.py >> @@ -360,10 +360,16 @@ class Wget(FetchMethod): >> >> try: >> import netrc >> + # Pseudo can cause ownership mismatch issues with >> netrc, which is >> + # required to be owned by the current user (root). >> + if os.environ.get("PSEUDO_DISABLED") == "0": >> + os.chown(os.path.join(os.path.expanduser("~"), >> ".netrc"), 0, 0) >> n = netrc.netrc() >> login, unused, password = >> n.authenticators(urllib.parse.urlparse(uri).hostname) >> add_basic_auth("%s:%s" % (login, password), r) >> - except (TypeError, ImportError, IOError, >> netrc.NetrcParseError): >> + except netrc.NetrcParseError as e: >> + logger.debug("netrc failed: %s" % e) >> + except (TypeError, ImportError, IOError) as e: >> pass >> >> with opener.open(r, timeout=30) as response: >> -- >> 2.37.2 >> >> >> >> *Nicholas JACKSON* >> Software Engineer II >> >> Safran Passenger Innovations >> >> T +1 714 854 8600 ext 527 >> >> 3151 East Imperial Hwy >> >> Brea, Ca 92821 USA >> >> *www.safran-passenger-innovations.com* >> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> >> >> >> >> >> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> >> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> >> >> >> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> >> >> >> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> >> >> >> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> >> >> >> ------------------------------ >> >> This email and any files transmitted with it are confidential & >> proprietary to Safran Passenger Innovations. This information is intended >> solely for the use of the individual or entity to which it is addressed. >> Access or transmittal of the information contained in this e-mail, in full >> or in part, to any other organization or persons is not authorized. >> >> >> >> >> ------------------------------ >> >> This email and any files transmitted with it are confidential & >> proprietary to Safran Passenger Innovations. This information is intended >> solely for the use of the individual or entity to which it is addressed. >> Access or transmittal of the information contained in this e-mail, in full >> or in part, to any other organization or persons is not authorized. >> >> >> ------------------------------ >> >> This email and any files transmitted with it are confidential & >> proprietary to Safran Passenger Innovations. This information is intended >> solely for the use of the individual or entity to which it is addressed. >> Access or transmittal of the information contained in this e-mail, in full >> or in part, to any other organization or persons is not authorized. >> > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14006): > https://lists.openembedded.org/g/bitbake-devel/message/14006 > Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ > alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Apologies! I was too vague earlier, here's the critical piece for reproduction in my local.conf: BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687" BB_SIGNATURE_HANDLER = "OEEquivHash" BB_HASHSERVE = "auto" SSTATE_MIRRORS ?= "file://.* https://mycompany.yoctosstatecache.mirror/all/PATH;downloadfilename=PATH" I just reproduced the issue by adding just the above lines to the default local.conf using a fresh poky clone outside docker. My netrc contains this line: machine mycompany.yoctosstatecache.mirror login myuser password mypass Unfortunately, this isn't the easiest environment to set up for reproducing the issue - one needs to set up an sstate cache that requires authentication. Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/> [cid:7cb0bad3-9b13-4d2f-bf70-26ab7a884e74] [cid:9dbf5e5c-4918-488e-a04f-75c07e553619]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:7f74b8d7-b722-4a07-9a9c-381759f74db9]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> [cid:0017a284-bcce-4505-abaf-6ac1856691c9]<http://twitter.com/SAFRAN> [cid:7cf51fa3-2619-49d3-8243-0eff37d9664f]<http://www.facebook.com/GroupeSafran> [cid:7554e28e-1422-49a0-94c0-1f8b6ca6c518]<http://fr.linkedin.com/company/safran> ________________________________ From: Alexander Kanavin <alex.kanavin@gmail.com> Sent: Friday, September 30, 2022 1:41 PM To: alex.kanavin@gmail.com <alex.kanavin@gmail.com> Cc: Nicholas Jackson <Nicholas.Jackson@zii.aero>; bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo Basically what would help is finding how this can be reproduced by someone else, e.g. me :) Alex On Fri, 30 Sept 2022 at 22:36, Alexander Kanavin via lists.openembedded.org<https://urldefense.com/v3/__http://lists.openembedded.org__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM3EQs7yE$> <alex.kanavin=gmail.com@lists.openembedded.org<mailto:gmail.com@lists.openembedded.org>> wrote: Can you try with a plain poky checkout (git clone https://git.yoctoproject.org/poky<https://urldefense.com/v3/__https://git.yoctoproject.org/poky__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMyIKVqTd$> ), and outside of docker? It is rather odd that no one else sees this. Alex On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote: Yes, the same issue occurs outside docker. Are you thinking that the root issue may perhaps be that PSEUDO_DISABLED is being set to 0 in the first place? It does seem strange to me that we'd need a fakeroot-like environment for a task involving wget. Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxn0glJG$> [cid:183901b7e473d1cd8fdd] [cid:183901b7e4754b4d084e]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$>[cid:183901b7e4895378eeff]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$> [cid:183901b7e48594944d610]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxmUQ5bj$> [cid:183901b7e482efa680d11]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMwmQ89Gq$> [cid:183901b7e48c2b406cd12]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM7y8vnd8$> ________________________________ From: Alexander Kanavin <alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>> Sent: Friday, September 30, 2022 12:51 PM To: Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> Cc: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo This does seem like there’s something particular about your setup triggering this. Can you try outside if docker? Alex On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote: Thanks for the quick response. My setup is an Artifactory replication of https://sstate.yoctoproject.org/<https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, with the Artifactory credentials stored in a netrc file. The issue specifically occurs when attempting to build certain packages using bitbake --setscene-only. (It probably also happens without --setscene-only). A warning is produced, e.g.: WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 WARNING: Setscene task (/home/foo/bar/poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) failed with exit code '1' - real task will be run instead In certain cases, bitbake will spawn a bitbake-worker with PSEUDO_DISABLED=0. Unfortunately, I have not been able to pinpoint when exactly this happens. I'm not familiar with bitbake internals, so I don't know when pseudo is used specifically. What I can say for sure is that bitbake --setscene-only world will always reproduce the issue for me without this patch. For my environment, I am building the default core-image-minimal for qemux86-64 with the three default poky layers. I've reproduced the issue in both Ubuntu 22.04 and Docker using the image ghcr.io/siemens/kas/kas:3.1<https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. In case it matters, I am using kas. Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$> [cid:183901b7e46d1b17b6a9] [cid:183901b7e4677411a697]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>[cid:183901b7e45d4a1a5a54]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> [cid:183901b7e4637ad9ced8]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$> [cid:183901b7e476a5a23b4a]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$> [cid:183901b7e47eb602e07c]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$> ________________________________ From: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> on behalf of Alexander Kanavin <alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>> Sent: Friday, September 30, 2022 10:59 AM To: Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> Cc: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo How can the problem be reproduced? Alex On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote: From: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>> The netrc module will refuse to use a file that is not owned by the current user. Psuedo can trigger this failure due to a reported uid of 0. To fix this, change ownership of the default netrc file to 0:0. This has no permanent effect on the actual netrc file. Note that while it is not possible to ask the netrc module what the default netrc file is, the documentation explicitly mentions that expanduser is used. Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>> --- bitbake/lib/bb/fetch2/wget.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index 821afa5b58..0ce6e25c5c 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -360,10 +360,16 @@ class Wget(FetchMethod): try: import netrc + # Pseudo can cause ownership mismatch issues with netrc, which is + # required to be owned by the current user (root). + if os.environ.get("PSEUDO_DISABLED") == "0": + os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0) n = netrc.netrc() login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname) add_basic_auth("%s:%s" % (login, password), r) - except (TypeError, ImportError, IOError, netrc.NetrcParseError): + except netrc.NetrcParseError as e: + logger.debug("netrc failed: %s" % e) + except (TypeError, ImportError, IOError) as e: pass with opener.open(r, timeout=30) as response: -- 2.37.2 Nicholas JACKSON Software Engineer II Safran Passenger Innovations T +1 714 854 8600 ext 527 3151 East Imperial Hwy Brea, Ca 92821 USA www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> [cid:183901b7e471078ea17b] [cid:183901b7e46b4d74fd96]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>[cid:183901b7e44b501410f1]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> [cid:183901b7e4564f504805]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> [cid:183901b7e456698e6052]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> [cid:183901b7e45d184584d3]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized. ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized. ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14006): https://lists.openembedded.org/g/bitbake-devel/message/14006<https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/message/14006__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-okVai8$> Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489<https://urldefense.com/v3/__https://lists.openembedded.org/mt/94022677/1686489__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-RMoNep$> Group Owner: bitbake-devel+owner@lists.openembedded.org<mailto:bitbake-devel%2Bowner@lists.openembedded.org> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub<https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/unsub__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM6Q9whAG$> [alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>] -=-=-=-=-=-=-=-=-=-=-=- ________________________________ This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.
Thanks, I've reproduced this too using plain python3 and .netrc with tweaked permissions (e.g. run python as regular user, set .netrc ownership to root). I am not sure why fetching sstate items happens under pseudo, or whether the fix is correct, but the issue is real. Alex On Fri, 30 Sept 2022 at 23:24, Nicholas Jackson <Nicholas.Jackson@zii.aero> wrote: > Apologies! I was too vague earlier, here's the critical piece for > reproduction in my local.conf: > > BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687" > BB_SIGNATURE_HANDLER = "OEEquivHash" > BB_HASHSERVE = "auto" > SSTATE_MIRRORS ?= "file://.* > https://mycompany.yoctosstatecache.mirror/all/PATH;downloadfilename=PATH" > > I just reproduced the issue by adding just the above lines to the default > local.conf using a fresh poky clone outside docker. > My netrc contains this line: > machine mycompany.yoctosstatecache.mirror login myuser password mypass > > Unfortunately, this isn't the easiest environment to set up for > reproducing the issue - one needs to > set up an sstate cache that requires authentication. > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <http://www.safran-passenger-innovations.com/> > > > > > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN> > > <http://twitter.com/SAFRAN> > > <http://www.facebook.com/GroupeSafran> > > <http://fr.linkedin.com/company/safran> > > ------------------------------ > *From:* Alexander Kanavin <alex.kanavin@gmail.com> > *Sent:* Friday, September 30, 2022 1:41 PM > *To:* alex.kanavin@gmail.com <alex.kanavin@gmail.com> > *Cc:* Nicholas Jackson <Nicholas.Jackson@zii.aero>; > bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org > > > *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with > pseudo > > Basically what would help is finding how this can be reproduced by someone > else, e.g. me :) > > Alex > > On Fri, 30 Sept 2022 at 22:36, Alexander Kanavin via > lists.openembedded.org > <https://urldefense.com/v3/__http://lists.openembedded.org__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM3EQs7yE$> > <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > Can you try with a plain poky checkout (git clone > https://git.yoctoproject.org/poky > <https://urldefense.com/v3/__https://git.yoctoproject.org/poky__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMyIKVqTd$> > ), and outside of docker? It is rather odd that no one else sees this. > > Alex > > On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > > Yes, the same issue occurs outside docker. > Are you thinking that the root issue may perhaps be that > PSEUDO_DISABLED is being set to 0 in the first place? It does > seem strange to me that we'd need a fakeroot-like environment for > a task involving wget. > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxn0glJG$> > > > > > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$> > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$> > > > <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxmUQ5bj$> > > > <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMwmQ89Gq$> > > > <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM7y8vnd8$> > > ------------------------------ > *From:* Alexander Kanavin <alex.kanavin@gmail.com> > *Sent:* Friday, September 30, 2022 12:51 PM > *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> > *Cc:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> > *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with > pseudo > > This does seem like there’s something particular about your setup > triggering this. Can you try outside if docker? > > Alex > > On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > > Thanks for the quick response. > My setup is an Artifactory replication of https://sstate.yoctoproject.org/ > <https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, > with > the Artifactory credentials stored in a netrc file. The issue specifically > occurs > when attempting to build certain packages using bitbake --setscene-only. > (It probably also happens without --setscene-only). > A warning is produced, e.g.: > WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive > obtainable, will run full task instead. > WARNING: Logfile for failed setscene task is > /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771 > WARNING: Setscene task (/home/foo/bar > /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) > failed with exit code '1' - real task will be run instead > > In certain cases, bitbake will spawn a bitbake-worker with > PSEUDO_DISABLED=0. > Unfortunately, I have not been able to pinpoint when exactly this happens. > I'm not > familiar with bitbake internals, so I don't know when pseudo is used > specifically. > What I can say for sure is that bitbake --setscene-only world will always > reproduce the issue for me without this patch. > > For my environment, I am building the default core-image-minimal for > qemux86-64 with > the three default poky layers. I've reproduced the issue in both Ubuntu > 22.04 and > Docker using the image ghcr.io/siemens/kas/kas:3.1 > <https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. > In case it matters, I am using kas. > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$> > > > > > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$> > > > <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$> > > > <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$> > > > <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$> > > ------------------------------ > *From:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin < > alex.kanavin@gmail.com> > *Sent:* Friday, September 30, 2022 10:59 AM > *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero> > *Cc:* bitbake-devel@lists.openembedded.org < > bitbake-devel@lists.openembedded.org> > *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with > pseudo > > How can the problem be reproduced? > > Alex > > On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero> > wrote: > > From: Nicholas Jackson <nicholas.jackson@zii.aero> > > The netrc module will refuse to use a file that is not owned by the > current user. Psuedo can trigger this failure due to a reported uid of > 0. To fix this, change ownership of the default netrc file to 0:0. This > has no permanent effect on the actual netrc file. > > Note that while it is not possible to ask the netrc module what the > default netrc file is, the documentation explicitly mentions that > expanduser is used. > > Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero> > --- > bitbake/lib/bb/fetch2/wget.py | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py > index 821afa5b58..0ce6e25c5c 100644 > --- a/bitbake/lib/bb/fetch2/wget.py > +++ b/bitbake/lib/bb/fetch2/wget.py > @@ -360,10 +360,16 @@ class Wget(FetchMethod): > > try: > import netrc > + # Pseudo can cause ownership mismatch issues with > netrc, which is > + # required to be owned by the current user (root). > + if os.environ.get("PSEUDO_DISABLED") == "0": > + os.chown(os.path.join(os.path.expanduser("~"), > ".netrc"), 0, 0) > n = netrc.netrc() > login, unused, password = > n.authenticators(urllib.parse.urlparse(uri).hostname) > add_basic_auth("%s:%s" % (login, password), r) > - except (TypeError, ImportError, IOError, > netrc.NetrcParseError): > + except netrc.NetrcParseError as e: > + logger.debug("netrc failed: %s" % e) > + except (TypeError, ImportError, IOError) as e: > pass > > with opener.open(r, timeout=30) as response: > -- > 2.37.2 > > > > *Nicholas JACKSON* > Software Engineer II > > Safran Passenger Innovations > > T +1 714 854 8600 ext 527 > > 3151 East Imperial Hwy > > Brea, Ca 92821 USA > > *www.safran-passenger-innovations.com* > <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$> > > > > > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> > <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$> > > > <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$> > > > <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$> > > > <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$> > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#14006): > https://lists.openembedded.org/g/bitbake-devel/message/14006 > <https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/message/14006__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-okVai8$> > Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489 > <https://urldefense.com/v3/__https://lists.openembedded.org/mt/94022677/1686489__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-RMoNep$> > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub > <https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/unsub__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM6Q9whAG$> > [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > > > ------------------------------ > > This email and any files transmitted with it are confidential & > proprietary to Safran Passenger Innovations. This information is intended > solely for the use of the individual or entity to which it is addressed. > Access or transmittal of the information contained in this e-mail, in full > or in part, to any other organization or persons is not authorized. >
On 30 Sep 2022, at 22:36, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > Thanks, I've reproduced this too using plain python3 and .netrc with tweaked permissions (e.g. run python as regular user, set .netrc ownership to root). I am not sure why fetching sstate items happens under pseudo, or whether the fix is correct, but the issue is real. My hunch response would be that setscene tasks shouldn’t be fakeroot, or at least the fetch should drop that power. Ross
On Sat, 2022-10-01 at 19:57 +0000, Ross Burton wrote: > On 30 Sep 2022, at 22:36, Alexander Kanavin via > lists.openembedded.org > <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > > > Thanks, I've reproduced this too using plain python3 and .netrc > > with tweaked permissions (e.g. run python as regular user, set > > .netrc ownership to root). I am not sure why fetching sstate items > > happens under pseudo, or whether the fix is correct, but the issue > > is real. > > My hunch response would be that setscene tasks shouldn’t be fakeroot Since the setscene tasks replicate things like do_package whch need to preserve fileownership, they very much need fakeroot. > or at least the fetch should drop that power. The fetcher does run commands with PSEUDO_DISABLED=1 set, the issue is that we're using library functions here. I'm not sure we can drop the power within a library funciton call so easily. Cheers, Richard
On 10/1/22 4:01 PM, Richard Purdie wrote: > On Sat, 2022-10-01 at 19:57 +0000, Ross Burton wrote: >> On 30 Sep 2022, at 22:36, Alexander Kanavin via >> lists.openembedded.org >> <alex.kanavin=gmail.com@lists.openembedded.org> wrote: >>> Thanks, I've reproduced this too using plain python3 and .netrc >>> with tweaked permissions (e.g. run python as regular user, set >>> .netrc ownership to root). I am not sure why fetching sstate items >>> happens under pseudo, or whether the fix is correct, but the issue >>> is real. >> My hunch response would be that setscene tasks shouldn’t be fakeroot > > Since the setscene tasks replicate things like do_package whch need to > preserve fileownership, they very much need fakeroot. So do you think that getting netrc to work with pseudo is a necessary fix after all? >> or at least the fetch should drop that power. > The fetcher does run commands with PSEUDO_DISABLED=1 set, the issue is > that we're using library functions here. I'm not sure we can drop the > power within a library funciton call so easily. > > Cheers, > > Richard > > >
Ping
On Mon, 2022-10-10 at 11:19 -0700, Nicholas Jackson wrote: > On 10/1/22 4:01 PM, Richard Purdie wrote: > > On Sat, 2022-10-01 at 19:57 +0000, Ross Burton wrote: > > > On 30 Sep 2022, at 22:36, Alexander Kanavin via > > > lists.openembedded.org > > > <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > > > Thanks, I've reproduced this too using plain python3 and .netrc > > > > with tweaked permissions (e.g. run python as regular user, set > > > > .netrc ownership to root). I am not sure why fetching sstate items > > > > happens under pseudo, or whether the fix is correct, but the issue > > > > is real. > > > My hunch response would be that setscene tasks shouldn’t be fakeroot > > > > Since the setscene tasks replicate things like do_package whch need to > > preserve fileownership, they very much need fakeroot. > > > So do you think that getting netrc to work with pseudo is a necessary > fix after all? We need to run the setscene tasks under pseudo. We don't need to fetch the objects under pseudo so we may need to look at something along those lines, dropping the pseudo environment for the fetching. Cheers, Richard
We encounter the same issue, interestingly only with the packagegroup-cross-canadian-qemux86-64-1.0-r0 package. For our case a few potential solutions come to my mind: * Excluding packagegroup-cross-canadian from sstate caching. Since this is only the packagegroup, it won't significantly impact performance. However, this approach merely circumvents the symptom without addressing the underlying cause. * Subclassing the netrc class and patching the _security_check method to become a no-op. This solution is compatible only with Python 3.11 and later versions, as the relevant code wasn't refactored into a dedicated method in earlier releases. See https://github.com/python/cpython/issues/72992 * Refactoring the checkstatus method to parse the .netrc file in a subprocess without fakeroot. I guess this would have a performance impact though. While I haven't tested this yet, here's a rough sketch (non-working example): cmd = "export PSEUDO_DISABLED=1; python3 -c 'import json,netrc,sys; print(json.dumps(netrc.netrc().authenticators(sys.argv[4]))' " + hostname (output, errors) = bb.process.run(cmd, log=log, shell=True, stderr=subprocess.PIPE, cwd=workdir) auth_data = json.loads(output)
diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index 821afa5b58..0ce6e25c5c 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -360,10 +360,16 @@ class Wget(FetchMethod): try: import netrc + # Pseudo can cause ownership mismatch issues with netrc, which is + # required to be owned by the current user (root). + if os.environ.get("PSEUDO_DISABLED") == "0": + os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0) n = netrc.netrc() login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname) add_basic_auth("%s:%s" % (login, password), r) - except (TypeError, ImportError, IOError, netrc.NetrcParseError): + except netrc.NetrcParseError as e: + logger.debug("netrc failed: %s" % e) + except (TypeError, ImportError, IOError) as e: pass with opener.open(r, timeout=30) as response: