diff mbox series

bitbake: wget: fix netrc with pseudo

Message ID BYAPR12MB2710069E5D018DBB8F800AECF5569@BYAPR12MB2710.namprd12.prod.outlook.com
State New
Headers show
Series bitbake: wget: fix netrc with pseudo | expand

Commit Message

Nicholas Jackson Sept. 30, 2022, 5:55 p.m. UTC
From: Nicholas Jackson <nicholas.jackson@zii.aero>

The netrc module will refuse to use a file that is not owned by the
current user. Psuedo can trigger this failure due to a reported uid of
0. To fix this, change ownership of the default netrc file to 0:0. This
has no permanent effect on the actual netrc file.

Note that while it is not possible to ask the netrc module what the
default netrc file is, the documentation explicitly mentions that
expanduser is used.

Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero>
---
 bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

--
2.37.2




Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/>



[cid:e940da09-3ffd-4243-a1be-90740bf07b55]

[cid:4965b635-fca1-4eb7-aac6-6354afdfa6b8]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:bda2fb5a-eb92-44b3-a349-47b31f606076]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>

[cid:8e4bfd42-7dfb-40e7-9249-5a84cbaa7d15]<http://twitter.com/SAFRAN>

[cid:1eb82668-d2af-4417-a928-1c672824ea6f]<http://www.facebook.com/GroupeSafran>

[cid:0cb4a8d9-39c9-450b-998d-eecd5a47d786]<http://fr.linkedin.com/company/safran>

Comments

Alexander Kanavin Sept. 30, 2022, 5:59 p.m. UTC | #1
How can the problem be reproduced?

Alex

On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero>
wrote:

> From: Nicholas Jackson <nicholas.jackson@zii.aero>
>
> The netrc module will refuse to use a file that is not owned by the
> current user. Psuedo can trigger this failure due to a reported uid of
> 0. To fix this, change ownership of the default netrc file to 0:0. This
> has no permanent effect on the actual netrc file.
>
> Note that while it is not possible to ask the netrc module what the
> default netrc file is, the documentation explicitly mentions that
> expanduser is used.
>
> Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero>
> ---
>  bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
> index 821afa5b58..0ce6e25c5c 100644
> --- a/bitbake/lib/bb/fetch2/wget.py
> +++ b/bitbake/lib/bb/fetch2/wget.py
> @@ -360,10 +360,16 @@ class Wget(FetchMethod):
>
>                  try:
>                      import netrc
> +                    # Pseudo can cause ownership mismatch issues with
> netrc, which is
> +                    # required to be owned by the current user (root).
> +                    if os.environ.get("PSEUDO_DISABLED") == "0":
> +                        os.chown(os.path.join(os.path.expanduser("~"),
> ".netrc"), 0, 0)
>                      n = netrc.netrc()
>                      login, unused, password =
> n.authenticators(urllib.parse.urlparse(uri).hostname)
>                      add_basic_auth("%s:%s" % (login, password), r)
> -                except (TypeError, ImportError, IOError,
> netrc.NetrcParseError):
> +                except netrc.NetrcParseError as e:
> +                    logger.debug("netrc failed: %s" % e)
> +                except (TypeError, ImportError, IOError) as e:
>                      pass
>
>                  with opener.open(r, timeout=30) as response:
> --
> 2.37.2
>
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <http://www.safran-passenger-innovations.com/>
>
>
>
>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
>
> <http://twitter.com/SAFRAN>
>
> <http://www.facebook.com/GroupeSafran>
>
> <http://fr.linkedin.com/company/safran>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#14001):
> https://lists.openembedded.org/g/bitbake-devel/message/14001
> Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Nicholas Jackson Sept. 30, 2022, 7:24 p.m. UTC | #2
Thanks for the quick response.
My setup is an Artifactory replication of https://sstate.yoctoproject.org/, with
the Artifactory credentials stored in a netrc file. The issue specifically occurs
when attempting to build certain packages using bitbake --setscene-only​.
(It probably also happens without --setscene-only).
A warning is produced, e.g.:
WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
WARNING: Setscene task (/home/foo/bar/poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) failed with exit code '1' - real task will be run instead

In certain cases, bitbake will spawn a bitbake-worker with PSEUDO_DISABLED=0.
Unfortunately, I have not been able to pinpoint when exactly this happens. I'm not
familiar with bitbake internals, so I don't know when pseudo is used specifically.
What I can say for sure is that bitbake --setscene-only world​ will always
reproduce the issue for me without this patch.

For my environment, I am building the default core-image-minimal for qemux86-64 with
the three default poky layers. I've reproduced the issue in both Ubuntu 22.04 and
Docker using the image ghcr.io/siemens/kas/kas:3.1. In case it matters, I am using kas.



Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/>



[cid:a12642b7-396b-4452-b2d3-91ac0a516ca8]

[cid:626cabaa-b02b-4a31-835a-cadcfc582c1f]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:c444f1c1-8306-4c88-85ca-2f546f771eda]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>

[cid:5252f91d-e5e1-4a1f-932e-753107817fdd]<http://twitter.com/SAFRAN>

[cid:f22e4c4f-d42e-4b5c-b99a-c93fabda9b97]<http://www.facebook.com/GroupeSafran>

[cid:c9b34257-95c4-42b0-ae51-5801c62f858f]<http://fr.linkedin.com/company/safran>

________________________________
From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Friday, September 30, 2022 10:59 AM
To: Nicholas Jackson <Nicholas.Jackson@zii.aero>
Cc: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org>
Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo

How can the problem be reproduced?

Alex

On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote:
From: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>>

The netrc module will refuse to use a file that is not owned by the
current user. Psuedo can trigger this failure due to a reported uid of
0. To fix this, change ownership of the default netrc file to 0:0. This
has no permanent effect on the actual netrc file.

Note that while it is not possible to ask the netrc module what the
default netrc file is, the documentation explicitly mentions that
expanduser is used.

Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>>
---
 bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 821afa5b58..0ce6e25c5c 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -360,10 +360,16 @@ class Wget(FetchMethod):

                 try:
                     import netrc
+                    # Pseudo can cause ownership mismatch issues with netrc, which is
+                    # required to be owned by the current user (root).
+                    if os.environ.get("PSEUDO_DISABLED") == "0":
+                        os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0)
                     n = netrc.netrc()
                     login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname)
                     add_basic_auth("%s:%s" % (login, password), r)
-                except (TypeError, ImportError, IOError, netrc.NetrcParseError):
+                except netrc.NetrcParseError as e:
+                    logger.debug("netrc failed: %s" % e)
+                except (TypeError, ImportError, IOError) as e:
                     pass

                 with opener.open(r, timeout=30) as response:
--
2.37.2




Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>



[cid:1838f8deba81078ea171]

[cid:1838f8deba9b4d74fd92]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>[cid:1838f8deba9b501410f3]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>

[cid:1838f8deba964f504804]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>

[cid:1838f8deba96698e6055]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>

[cid:1838f8deba9d184584d6]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>

________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.




________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.
Alexander Kanavin Sept. 30, 2022, 7:51 p.m. UTC | #3
This does seem like there’s something particular about your setup
triggering this. Can you try outside if docker?

Alex

On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero>
wrote:

> Thanks for the quick response.
> My setup is an Artifactory replication of https://sstate.yoctoproject.org/,
> with
> the Artifactory credentials stored in a netrc file. The issue specifically
> occurs
> when attempting to build certain packages using bitbake --setscene-only​.
> (It probably also happens without --setscene-only).
> A warning is produced, e.g.:
> WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive
> obtainable, will run full task instead.
> WARNING: Logfile for failed setscene task is
> /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
> WARNING: Setscene task (/home/foo/bar
> /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene)
> failed with exit code '1' - real task will be run instead
>
> In certain cases, bitbake will spawn a bitbake-worker with
> PSEUDO_DISABLED=0.
> Unfortunately, I have not been able to pinpoint when exactly this happens.
> I'm not
> familiar with bitbake internals, so I don't know when pseudo is used
> specifically.
> What I can say for sure is that bitbake --setscene-only world​ will always
> reproduce the issue for me without this patch.
>
> For my environment, I am building the default core-image-minimal for
> qemux86-64 with
> the three default poky layers. I've reproduced the issue in both Ubuntu
> 22.04 and
> Docker using the image ghcr.io/siemens/kas/kas:3.1. In case it matters, I
> am using kas.
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <http://www.safran-passenger-innovations.com/>
>
>
>
>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
>
> <http://twitter.com/SAFRAN>
>
> <http://www.facebook.com/GroupeSafran>
>
> <http://fr.linkedin.com/company/safran>
>
> ------------------------------
> *From:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin <
> alex.kanavin@gmail.com>
> *Sent:* Friday, September 30, 2022 10:59 AM
> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
> *Cc:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org>
> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
> pseudo
>
> How can the problem be reproduced?
>
> Alex
>
> On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
> From: Nicholas Jackson <nicholas.jackson@zii.aero>
>
> The netrc module will refuse to use a file that is not owned by the
> current user. Psuedo can trigger this failure due to a reported uid of
> 0. To fix this, change ownership of the default netrc file to 0:0. This
> has no permanent effect on the actual netrc file.
>
> Note that while it is not possible to ask the netrc module what the
> default netrc file is, the documentation explicitly mentions that
> expanduser is used.
>
> Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero>
> ---
>  bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
> index 821afa5b58..0ce6e25c5c 100644
> --- a/bitbake/lib/bb/fetch2/wget.py
> +++ b/bitbake/lib/bb/fetch2/wget.py
> @@ -360,10 +360,16 @@ class Wget(FetchMethod):
>
>                  try:
>                      import netrc
> +                    # Pseudo can cause ownership mismatch issues with
> netrc, which is
> +                    # required to be owned by the current user (root).
> +                    if os.environ.get("PSEUDO_DISABLED") == "0":
> +                        os.chown(os.path.join(os.path.expanduser("~"),
> ".netrc"), 0, 0)
>                      n = netrc.netrc()
>                      login, unused, password =
> n.authenticators(urllib.parse.urlparse(uri).hostname)
>                      add_basic_auth("%s:%s" % (login, password), r)
> -                except (TypeError, ImportError, IOError,
> netrc.NetrcParseError):
> +                except netrc.NetrcParseError as e:
> +                    logger.debug("netrc failed: %s" % e)
> +                except (TypeError, ImportError, IOError) as e:
>                      pass
>
>                  with opener.open(r, timeout=30) as response:
> --
> 2.37.2
>
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>
>
>
>
>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
>
>
> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>
>
>
> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>
>
>
> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
Nicholas Jackson Sept. 30, 2022, 8:02 p.m. UTC | #4
Yes, the same issue occurs outside docker.
Are you thinking that the root issue may perhaps be that
PSEUDO_DISABLED is being set to 0 in the first place? It does
seem strange to me that we'd need a fakeroot-like environment for
a task involving wget.



Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/>



[cid:5f2a4653-757e-4e47-9b2a-32f303ab1e0a]

[cid:c33ce570-c829-448d-a54f-75aee8791bdf]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:4bc65a6f-d39d-4eb0-9865-e31276eec185]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>

[cid:a2a2ba47-4eb4-4f80-8c27-efb22f0d23c1]<http://twitter.com/SAFRAN>

[cid:f273792d-7493-452b-b299-67fb5439b5c7]<http://www.facebook.com/GroupeSafran>

[cid:7ffb676a-6b4e-479e-8ee8-16159c401ee8]<http://fr.linkedin.com/company/safran>

________________________________
From: Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Friday, September 30, 2022 12:51 PM
To: Nicholas Jackson <Nicholas.Jackson@zii.aero>
Cc: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org>
Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo

This does seem like there’s something particular about your setup triggering this. Can you try outside if docker?

Alex

On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote:
Thanks for the quick response.
My setup is an Artifactory replication of https://sstate.yoctoproject.org/<https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, with
the Artifactory credentials stored in a netrc file. The issue specifically occurs
when attempting to build certain packages using bitbake --setscene-only​.
(It probably also happens without --setscene-only).
A warning is produced, e.g.:
WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
WARNING: Setscene task (/home/foo/bar/poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) failed with exit code '1' - real task will be run instead

In certain cases, bitbake will spawn a bitbake-worker with PSEUDO_DISABLED=0.
Unfortunately, I have not been able to pinpoint when exactly this happens. I'm not
familiar with bitbake internals, so I don't know when pseudo is used specifically.
What I can say for sure is that bitbake --setscene-only world​ will always
reproduce the issue for me without this patch.

For my environment, I am building the default core-image-minimal for qemux86-64 with
the three default poky layers. I've reproduced the issue in both Ubuntu 22.04 and
Docker using the image ghcr.io/siemens/kas/kas:3.1<https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. In case it matters, I am using kas.



Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$>



[cid:1838fef3a6420d4b2947]

[cid:1838fef3a64a87b16978]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>[cid:1838fef3a64f7432f799]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>

[cid:1838fef3a64c82a9bc9a]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$>

[cid:1838fef3a64f82a6facb]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$>

[cid:1838fef3a64923d3295c]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$>

________________________________
From: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> on behalf of Alexander Kanavin <alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>>
Sent: Friday, September 30, 2022 10:59 AM
To: Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>>
Cc: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>>
Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo

How can the problem be reproduced?

Alex

On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote:
From: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>>

The netrc module will refuse to use a file that is not owned by the
current user. Psuedo can trigger this failure due to a reported uid of
0. To fix this, change ownership of the default netrc file to 0:0. This
has no permanent effect on the actual netrc file.

Note that while it is not possible to ask the netrc module what the
default netrc file is, the documentation explicitly mentions that
expanduser is used.

Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>>
---
 bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 821afa5b58..0ce6e25c5c 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -360,10 +360,16 @@ class Wget(FetchMethod):

                 try:
                     import netrc
+                    # Pseudo can cause ownership mismatch issues with netrc, which is
+                    # required to be owned by the current user (root).
+                    if os.environ.get("PSEUDO_DISABLED") == "0":
+                        os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0)
                     n = netrc.netrc()
                     login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname)
                     add_basic_auth("%s:%s" % (login, password), r)
-                except (TypeError, ImportError, IOError, netrc.NetrcParseError):
+                except netrc.NetrcParseError as e:
+                    logger.debug("netrc failed: %s" % e)
+                except (TypeError, ImportError, IOError) as e:
                     pass

                 with opener.open(r, timeout=30) as response:
--
2.37.2




Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>



[cid:1838fef3a63bb3ee15b1]

[cid:1838fef3a63acc5a1812]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>[cid:1838fef3a63ffaecb153]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>

[cid:1838fef3a63a8ac7fa4]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>

[cid:1838fef3a634dcaf8f15]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>

[cid:1838fef3a6379eb4ca36]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>

________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.




________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.

________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.
Alexander Kanavin Sept. 30, 2022, 8:36 p.m. UTC | #5
Can you try with a plain poky checkout (git clone
https://git.yoctoproject.org/poky ), and outside of docker? It is rather
odd that no one else sees this.

Alex

On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero>
wrote:

> Yes, the same issue occurs outside docker.
> Are you thinking that the root issue may perhaps be that
> PSEUDO_DISABLED is being set to 0 in the first place? It does
> seem strange to me that we'd need a fakeroot-like environment for
> a task involving wget.
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <http://www.safran-passenger-innovations.com/>
>
>
>
>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
>
> <http://twitter.com/SAFRAN>
>
> <http://www.facebook.com/GroupeSafran>
>
> <http://fr.linkedin.com/company/safran>
>
> ------------------------------
> *From:* Alexander Kanavin <alex.kanavin@gmail.com>
> *Sent:* Friday, September 30, 2022 12:51 PM
> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
> *Cc:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org>
> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
> pseudo
>
> This does seem like there’s something particular about your setup
> triggering this. Can you try outside if docker?
>
> Alex
>
> On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
> Thanks for the quick response.
> My setup is an Artifactory replication of https://sstate.yoctoproject.org/
> <https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>,
> with
> the Artifactory credentials stored in a netrc file. The issue specifically
> occurs
> when attempting to build certain packages using bitbake --setscene-only​.
> (It probably also happens without --setscene-only).
> A warning is produced, e.g.:
> WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive
> obtainable, will run full task instead.
> WARNING: Logfile for failed setscene task is
> /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
> WARNING: Setscene task (/home/foo/bar
> /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene)
> failed with exit code '1' - real task will be run instead
>
> In certain cases, bitbake will spawn a bitbake-worker with
> PSEUDO_DISABLED=0.
> Unfortunately, I have not been able to pinpoint when exactly this happens.
> I'm not
> familiar with bitbake internals, so I don't know when pseudo is used
> specifically.
> What I can say for sure is that bitbake --setscene-only world​ will always
> reproduce the issue for me without this patch.
>
> For my environment, I am building the default core-image-minimal for
> qemux86-64 with
> the three default poky layers. I've reproduced the issue in both Ubuntu
> 22.04 and
> Docker using the image ghcr.io/siemens/kas/kas:3.1
> <https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>.
> In case it matters, I am using kas.
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$>
>
>
>
>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>
>
>
> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$>
>
>
> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$>
>
>
> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$>
>
> ------------------------------
> *From:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin <
> alex.kanavin@gmail.com>
> *Sent:* Friday, September 30, 2022 10:59 AM
> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
> *Cc:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org>
> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
> pseudo
>
> How can the problem be reproduced?
>
> Alex
>
> On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
> From: Nicholas Jackson <nicholas.jackson@zii.aero>
>
> The netrc module will refuse to use a file that is not owned by the
> current user. Psuedo can trigger this failure due to a reported uid of
> 0. To fix this, change ownership of the default netrc file to 0:0. This
> has no permanent effect on the actual netrc file.
>
> Note that while it is not possible to ask the netrc module what the
> default netrc file is, the documentation explicitly mentions that
> expanduser is used.
>
> Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero>
> ---
>  bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
> index 821afa5b58..0ce6e25c5c 100644
> --- a/bitbake/lib/bb/fetch2/wget.py
> +++ b/bitbake/lib/bb/fetch2/wget.py
> @@ -360,10 +360,16 @@ class Wget(FetchMethod):
>
>                  try:
>                      import netrc
> +                    # Pseudo can cause ownership mismatch issues with
> netrc, which is
> +                    # required to be owned by the current user (root).
> +                    if os.environ.get("PSEUDO_DISABLED") == "0":
> +                        os.chown(os.path.join(os.path.expanduser("~"),
> ".netrc"), 0, 0)
>                      n = netrc.netrc()
>                      login, unused, password =
> n.authenticators(urllib.parse.urlparse(uri).hostname)
>                      add_basic_auth("%s:%s" % (login, password), r)
> -                except (TypeError, ImportError, IOError,
> netrc.NetrcParseError):
> +                except netrc.NetrcParseError as e:
> +                    logger.debug("netrc failed: %s" % e)
> +                except (TypeError, ImportError, IOError) as e:
>                      pass
>
>                  with opener.open(r, timeout=30) as response:
> --
> 2.37.2
>
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>
>
>
>
>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
>
>
> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>
>
>
> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>
>
>
> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
Alexander Kanavin Sept. 30, 2022, 8:41 p.m. UTC | #6
Basically what would help is finding how this can be reproduced by someone
else, e.g. me :)

Alex

On Fri, 30 Sept 2022 at 22:36, Alexander Kanavin via lists.openembedded.org
<alex.kanavin=gmail.com@lists.openembedded.org> wrote:

> Can you try with a plain poky checkout (git clone
> https://git.yoctoproject.org/poky ), and outside of docker? It is rather
> odd that no one else sees this.
>
> Alex
>
> On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
>> Yes, the same issue occurs outside docker.
>> Are you thinking that the root issue may perhaps be that
>> PSEUDO_DISABLED is being set to 0 in the first place? It does
>> seem strange to me that we'd need a fakeroot-like environment for
>> a task involving wget.
>>
>>
>> *Nicholas JACKSON*
>> Software Engineer II
>>
>> Safran Passenger Innovations
>>
>> T +1 714 854 8600 ext 527
>>
>> 3151 East Imperial Hwy
>>
>> Brea, Ca 92821 USA
>>
>> *www.safran-passenger-innovations.com*
>> <http://www.safran-passenger-innovations.com/>
>>
>>
>>
>>
>> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
>> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
>>
>> <http://twitter.com/SAFRAN>
>>
>> <http://www.facebook.com/GroupeSafran>
>>
>> <http://fr.linkedin.com/company/safran>
>>
>> ------------------------------
>> *From:* Alexander Kanavin <alex.kanavin@gmail.com>
>> *Sent:* Friday, September 30, 2022 12:51 PM
>> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
>> *Cc:* bitbake-devel@lists.openembedded.org <
>> bitbake-devel@lists.openembedded.org>
>> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
>> pseudo
>>
>> This does seem like there’s something particular about your setup
>> triggering this. Can you try outside if docker?
>>
>> Alex
>>
>> On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero>
>> wrote:
>>
>> Thanks for the quick response.
>> My setup is an Artifactory replication of
>> https://sstate.yoctoproject.org/
>> <https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>,
>> with
>> the Artifactory credentials stored in a netrc file. The issue
>> specifically occurs
>> when attempting to build certain packages using bitbake --setscene-only​.
>> (It probably also happens without --setscene-only).
>> A warning is produced, e.g.:
>> WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive
>> obtainable, will run full task instead.
>> WARNING: Logfile for failed setscene task is
>> /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
>> WARNING: Setscene task (/home/foo/bar
>> /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene)
>> failed with exit code '1' - real task will be run instead
>>
>> In certain cases, bitbake will spawn a bitbake-worker with
>> PSEUDO_DISABLED=0.
>> Unfortunately, I have not been able to pinpoint when exactly this
>> happens. I'm not
>> familiar with bitbake internals, so I don't know when pseudo is used
>> specifically.
>> What I can say for sure is that bitbake --setscene-only world​ will
>> always
>> reproduce the issue for me without this patch.
>>
>> For my environment, I am building the default core-image-minimal for
>> qemux86-64 with
>> the three default poky layers. I've reproduced the issue in both Ubuntu
>> 22.04 and
>> Docker using the image ghcr.io/siemens/kas/kas:3.1
>> <https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>.
>> In case it matters, I am using kas.
>>
>>
>> *Nicholas JACKSON*
>> Software Engineer II
>>
>> Safran Passenger Innovations
>>
>> T +1 714 854 8600 ext 527
>>
>> 3151 East Imperial Hwy
>>
>> Brea, Ca 92821 USA
>>
>> *www.safran-passenger-innovations.com*
>> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$>
>>
>>
>>
>>
>> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>
>> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>
>>
>>
>> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$>
>>
>>
>> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$>
>>
>>
>> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$>
>>
>> ------------------------------
>> *From:* bitbake-devel@lists.openembedded.org <
>> bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin <
>> alex.kanavin@gmail.com>
>> *Sent:* Friday, September 30, 2022 10:59 AM
>> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
>> *Cc:* bitbake-devel@lists.openembedded.org <
>> bitbake-devel@lists.openembedded.org>
>> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
>> pseudo
>>
>> How can the problem be reproduced?
>>
>> Alex
>>
>> On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <
>> Nicholas.Jackson@zii.aero> wrote:
>>
>> From: Nicholas Jackson <nicholas.jackson@zii.aero>
>>
>> The netrc module will refuse to use a file that is not owned by the
>> current user. Psuedo can trigger this failure due to a reported uid of
>> 0. To fix this, change ownership of the default netrc file to 0:0. This
>> has no permanent effect on the actual netrc file.
>>
>> Note that while it is not possible to ask the netrc module what the
>> default netrc file is, the documentation explicitly mentions that
>> expanduser is used.
>>
>> Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero>
>> ---
>>  bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
>> index 821afa5b58..0ce6e25c5c 100644
>> --- a/bitbake/lib/bb/fetch2/wget.py
>> +++ b/bitbake/lib/bb/fetch2/wget.py
>> @@ -360,10 +360,16 @@ class Wget(FetchMethod):
>>
>>                  try:
>>                      import netrc
>> +                    # Pseudo can cause ownership mismatch issues with
>> netrc, which is
>> +                    # required to be owned by the current user (root).
>> +                    if os.environ.get("PSEUDO_DISABLED") == "0":
>> +                        os.chown(os.path.join(os.path.expanduser("~"),
>> ".netrc"), 0, 0)
>>                      n = netrc.netrc()
>>                      login, unused, password =
>> n.authenticators(urllib.parse.urlparse(uri).hostname)
>>                      add_basic_auth("%s:%s" % (login, password), r)
>> -                except (TypeError, ImportError, IOError,
>> netrc.NetrcParseError):
>> +                except netrc.NetrcParseError as e:
>> +                    logger.debug("netrc failed: %s" % e)
>> +                except (TypeError, ImportError, IOError) as e:
>>                      pass
>>
>>                  with opener.open(r, timeout=30) as response:
>> --
>> 2.37.2
>>
>>
>>
>> *Nicholas JACKSON*
>> Software Engineer II
>>
>> Safran Passenger Innovations
>>
>> T +1 714 854 8600 ext 527
>>
>> 3151 East Imperial Hwy
>>
>> Brea, Ca 92821 USA
>>
>> *www.safran-passenger-innovations.com*
>> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>
>>
>>
>>
>>
>> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
>> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
>>
>>
>> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>
>>
>>
>> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>
>>
>>
>> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>
>>
>>
>> ------------------------------
>>
>> This email and any files transmitted with it are confidential &
>> proprietary to Safran Passenger Innovations. This information is intended
>> solely for the use of the individual or entity to which it is addressed.
>> Access or transmittal of the information contained in this e-mail, in full
>> or in part, to any other organization or persons is not authorized.
>>
>>
>>
>>
>> ------------------------------
>>
>> This email and any files transmitted with it are confidential &
>> proprietary to Safran Passenger Innovations. This information is intended
>> solely for the use of the individual or entity to which it is addressed.
>> Access or transmittal of the information contained in this e-mail, in full
>> or in part, to any other organization or persons is not authorized.
>>
>>
>> ------------------------------
>>
>> This email and any files transmitted with it are confidential &
>> proprietary to Safran Passenger Innovations. This information is intended
>> solely for the use of the individual or entity to which it is addressed.
>> Access or transmittal of the information contained in this e-mail, in full
>> or in part, to any other organization or persons is not authorized.
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#14006):
> https://lists.openembedded.org/g/bitbake-devel/message/14006
> Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Nicholas Jackson Sept. 30, 2022, 9:24 p.m. UTC | #7
Apologies! I was too vague earlier, here's the critical piece for reproduction in my local.conf:

BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"
BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
SSTATE_MIRRORS ?= "file://.* https://mycompany.yoctosstatecache.mirror/all/PATH;downloadfilename=PATH"

I just reproduced the issue by adding just the above lines to the default local.conf using a fresh poky clone outside docker.
My netrc contains this line:
machine mycompany.yoctosstatecache.mirror login myuser password mypass

Unfortunately, this isn't the easiest environment to set up for reproducing the issue - one needs to
set up an sstate cache that requires authentication.


Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<http://www.safran-passenger-innovations.com/>



[cid:7cb0bad3-9b13-4d2f-bf70-26ab7a884e74]

[cid:9dbf5e5c-4918-488e-a04f-75c07e553619]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>[cid:7f74b8d7-b722-4a07-9a9c-381759f74db9]<http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>

[cid:0017a284-bcce-4505-abaf-6ac1856691c9]<http://twitter.com/SAFRAN>

[cid:7cf51fa3-2619-49d3-8243-0eff37d9664f]<http://www.facebook.com/GroupeSafran>

[cid:7554e28e-1422-49a0-94c0-1f8b6ca6c518]<http://fr.linkedin.com/company/safran>

________________________________
From: Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Friday, September 30, 2022 1:41 PM
To: alex.kanavin@gmail.com <alex.kanavin@gmail.com>
Cc: Nicholas Jackson <Nicholas.Jackson@zii.aero>; bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org>
Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo

Basically what would help is finding how this can be reproduced by someone else, e.g. me :)

Alex

On Fri, 30 Sept 2022 at 22:36, Alexander Kanavin via lists.openembedded.org<https://urldefense.com/v3/__http://lists.openembedded.org__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM3EQs7yE$> <alex.kanavin=gmail.com@lists.openembedded.org<mailto:gmail.com@lists.openembedded.org>> wrote:
Can you try with a plain poky checkout (git clone https://git.yoctoproject.org/poky<https://urldefense.com/v3/__https://git.yoctoproject.org/poky__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMyIKVqTd$> ), and outside of docker? It is rather odd that no one else sees this.

Alex

On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote:
Yes, the same issue occurs outside docker.
Are you thinking that the root issue may perhaps be that
PSEUDO_DISABLED is being set to 0 in the first place? It does
seem strange to me that we'd need a fakeroot-like environment for
a task involving wget.



Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxn0glJG$>



[cid:183901b7e473d1cd8fdd]

[cid:183901b7e4754b4d084e]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$>[cid:183901b7e4895378eeff]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$>

[cid:183901b7e48594944d610]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxmUQ5bj$>

[cid:183901b7e482efa680d11]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMwmQ89Gq$>

[cid:183901b7e48c2b406cd12]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM7y8vnd8$>

________________________________
From: Alexander Kanavin <alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>>
Sent: Friday, September 30, 2022 12:51 PM
To: Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>>
Cc: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>>
Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo

This does seem like there’s something particular about your setup triggering this. Can you try outside if docker?

Alex

On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote:
Thanks for the quick response.
My setup is an Artifactory replication of https://sstate.yoctoproject.org/<https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>, with
the Artifactory credentials stored in a netrc file. The issue specifically occurs
when attempting to build certain packages using bitbake --setscene-only​.
(It probably also happens without --setscene-only).
A warning is produced, e.g.:
WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
WARNING: Setscene task (/home/foo/bar/poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene) failed with exit code '1' - real task will be run instead

In certain cases, bitbake will spawn a bitbake-worker with PSEUDO_DISABLED=0.
Unfortunately, I have not been able to pinpoint when exactly this happens. I'm not
familiar with bitbake internals, so I don't know when pseudo is used specifically.
What I can say for sure is that bitbake --setscene-only world​ will always
reproduce the issue for me without this patch.

For my environment, I am building the default core-image-minimal for qemux86-64 with
the three default poky layers. I've reproduced the issue in both Ubuntu 22.04 and
Docker using the image ghcr.io/siemens/kas/kas:3.1<https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>. In case it matters, I am using kas.



Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$>



[cid:183901b7e46d1b17b6a9]

[cid:183901b7e4677411a697]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>[cid:183901b7e45d4a1a5a54]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>

[cid:183901b7e4637ad9ced8]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$>

[cid:183901b7e476a5a23b4a]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$>

[cid:183901b7e47eb602e07c]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$>

________________________________
From: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>> on behalf of Alexander Kanavin <alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>>
Sent: Friday, September 30, 2022 10:59 AM
To: Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>>
Cc: bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org> <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>>
Subject: Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with pseudo

How can the problem be reproduced?

Alex

On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero<mailto:Nicholas.Jackson@zii.aero>> wrote:
From: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>>

The netrc module will refuse to use a file that is not owned by the
current user. Psuedo can trigger this failure due to a reported uid of
0. To fix this, change ownership of the default netrc file to 0:0. This
has no permanent effect on the actual netrc file.

Note that while it is not possible to ask the netrc module what the
default netrc file is, the documentation explicitly mentions that
expanduser is used.

Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero<mailto:nicholas.jackson@zii.aero>>
---
 bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 821afa5b58..0ce6e25c5c 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -360,10 +360,16 @@ class Wget(FetchMethod):

                 try:
                     import netrc
+                    # Pseudo can cause ownership mismatch issues with netrc, which is
+                    # required to be owned by the current user (root).
+                    if os.environ.get("PSEUDO_DISABLED") == "0":
+                        os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0)
                     n = netrc.netrc()
                     login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname)
                     add_basic_auth("%s:%s" % (login, password), r)
-                except (TypeError, ImportError, IOError, netrc.NetrcParseError):
+                except netrc.NetrcParseError as e:
+                    logger.debug("netrc failed: %s" % e)
+                except (TypeError, ImportError, IOError) as e:
                     pass

                 with opener.open(r, timeout=30) as response:
--
2.37.2




Nicholas JACKSON
Software Engineer II

Safran Passenger Innovations

T +1 714 854 8600 ext 527

3151 East Imperial Hwy

Brea, Ca 92821 USA

www.safran-passenger-innovations.com<https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>



[cid:183901b7e471078ea17b]

[cid:183901b7e46b4d74fd96]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>[cid:183901b7e44b501410f1]<https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>

[cid:183901b7e4564f504805]<https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>

[cid:183901b7e456698e6052]<https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>

[cid:183901b7e45d184584d3]<https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>

________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.




________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.

________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14006): https://lists.openembedded.org/g/bitbake-devel/message/14006<https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/message/14006__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-okVai8$>
Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489<https://urldefense.com/v3/__https://lists.openembedded.org/mt/94022677/1686489__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-RMoNep$>
Group Owner: bitbake-devel+owner@lists.openembedded.org<mailto:bitbake-devel%2Bowner@lists.openembedded.org>
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub<https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/unsub__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM6Q9whAG$> [alex.kanavin@gmail.com<mailto:alex.kanavin@gmail.com>]
-=-=-=-=-=-=-=-=-=-=-=-


________________________________

This email and any files transmitted with it are confidential & proprietary to Safran Passenger Innovations. This information is intended solely for the use of the individual or entity to which it is addressed. Access or transmittal of the information contained in this e-mail, in full or in part, to any other organization or persons is not authorized.
Alexander Kanavin Sept. 30, 2022, 9:36 p.m. UTC | #8
Thanks, I've reproduced this too using plain python3 and .netrc with
tweaked permissions (e.g. run python as regular user, set .netrc ownership
to root). I am not sure why fetching sstate items happens under pseudo, or
whether the fix is correct, but the issue is real.

Alex

On Fri, 30 Sept 2022 at 23:24, Nicholas Jackson <Nicholas.Jackson@zii.aero>
wrote:

> Apologies! I was too vague earlier, here's the critical piece for
> reproduction in my local.conf:
>
> BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"
> BB_SIGNATURE_HANDLER = "OEEquivHash"
> BB_HASHSERVE = "auto"
> SSTATE_MIRRORS ?= "file://.*
> https://mycompany.yoctosstatecache.mirror/all/PATH;downloadfilename=PATH"
>
> I just reproduced the issue by adding just the above lines to the default
> local.conf using a fresh poky clone outside docker.
> My netrc contains this line:
> machine mycompany.yoctosstatecache.mirror login myuser password mypass
>
> Unfortunately, this isn't the easiest environment to set up for
> reproducing the issue - one needs to
> set up an sstate cache that requires authentication.
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <http://www.safran-passenger-innovations.com/>
>
>
>
>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
> <http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN>
>
> <http://twitter.com/SAFRAN>
>
> <http://www.facebook.com/GroupeSafran>
>
> <http://fr.linkedin.com/company/safran>
>
> ------------------------------
> *From:* Alexander Kanavin <alex.kanavin@gmail.com>
> *Sent:* Friday, September 30, 2022 1:41 PM
> *To:* alex.kanavin@gmail.com <alex.kanavin@gmail.com>
> *Cc:* Nicholas Jackson <Nicholas.Jackson@zii.aero>;
> bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org
> >
> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
> pseudo
>
> Basically what would help is finding how this can be reproduced by someone
> else, e.g. me :)
>
> Alex
>
> On Fri, 30 Sept 2022 at 22:36, Alexander Kanavin via
> lists.openembedded.org
> <https://urldefense.com/v3/__http://lists.openembedded.org__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM3EQs7yE$>
> <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
>
> Can you try with a plain poky checkout (git clone
> https://git.yoctoproject.org/poky
> <https://urldefense.com/v3/__https://git.yoctoproject.org/poky__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMyIKVqTd$>
> ), and outside of docker? It is rather odd that no one else sees this.
>
> Alex
>
> On Fri, 30 Sept 2022 at 22:02, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
> Yes, the same issue occurs outside docker.
> Are you thinking that the root issue may perhaps be that
> PSEUDO_DISABLED is being set to 0 in the first place? It does
> seem strange to me that we'd need a fakeroot-like environment for
> a task involving wget.
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxn0glJG$>
>
>
>
>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM9ge9Ioq$>
>
>
> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMxmUQ5bj$>
>
>
> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeMwmQ89Gq$>
>
>
> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM7y8vnd8$>
>
> ------------------------------
> *From:* Alexander Kanavin <alex.kanavin@gmail.com>
> *Sent:* Friday, September 30, 2022 12:51 PM
> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
> *Cc:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org>
> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
> pseudo
>
> This does seem like there’s something particular about your setup
> triggering this. Can you try outside if docker?
>
> Alex
>
> On Fri 30. Sep 2022 at 21.24, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
> Thanks for the quick response.
> My setup is an Artifactory replication of https://sstate.yoctoproject.org/
> <https://urldefense.com/v3/__https://sstate.yoctoproject.org/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TkJgA78$>,
> with
> the Artifactory credentials stored in a netrc file. The issue specifically
> occurs
> when attempting to build certain packages using bitbake --setscene-only​.
> (It probably also happens without --setscene-only).
> A warning is produced, e.g.:
> WARNING: efibootmgr-18-r0 do_package_setscene: No sstate archive
> obtainable, will run full task instead.
> WARNING: Logfile for failed setscene task is
> /home/foo/bar/build/tmp/work/core2-64-poky-linux/efibootmgr/18-r0/temp/log.do_package_setscene.472771
> WARNING: Setscene task (/home/foo/bar
> /poky/meta/recipes-bsp/efibootmgr/efibootmgr_18.bb:do_package_setscene)
> failed with exit code '1' - real task will be run instead
>
> In certain cases, bitbake will spawn a bitbake-worker with
> PSEUDO_DISABLED=0.
> Unfortunately, I have not been able to pinpoint when exactly this happens.
> I'm not
> familiar with bitbake internals, so I don't know when pseudo is used
> specifically.
> What I can say for sure is that bitbake --setscene-only world​ will always
> reproduce the issue for me without this patch.
>
> For my environment, I am building the default core-image-minimal for
> qemux86-64 with
> the three default poky layers. I've reproduced the issue in both Ubuntu
> 22.04 and
> Docker using the image ghcr.io/siemens/kas/kas:3.1
> <https://urldefense.com/v3/__http://ghcr.io/siemens/kas/kas:3.1__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TKPQAyS$>.
> In case it matters, I am using kas.
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_TOfrCXE$>
>
>
>
>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_eBgAh8f$>
>
>
> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_azwuY37$>
>
>
> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_T5mTjcB$>
>
>
> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!FbyY92yX6Ab2wL6z84GEifystfPM_rtQNCO7rqvq21ddVS90QOTCASJ2KP6SSGxggHMuX16yplCGYp4D16fL_UMGP7F-$>
>
> ------------------------------
> *From:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org> on behalf of Alexander Kanavin <
> alex.kanavin@gmail.com>
> *Sent:* Friday, September 30, 2022 10:59 AM
> *To:* Nicholas Jackson <Nicholas.Jackson@zii.aero>
> *Cc:* bitbake-devel@lists.openembedded.org <
> bitbake-devel@lists.openembedded.org>
> *Subject:* Re: [bitbake-devel] [PATCH] bitbake: wget: fix netrc with
> pseudo
>
> How can the problem be reproduced?
>
> Alex
>
> On Fri, 30 Sept 2022 at 19:55, Nicholas Jackson <Nicholas.Jackson@zii.aero>
> wrote:
>
> From: Nicholas Jackson <nicholas.jackson@zii.aero>
>
> The netrc module will refuse to use a file that is not owned by the
> current user. Psuedo can trigger this failure due to a reported uid of
> 0. To fix this, change ownership of the default netrc file to 0:0. This
> has no permanent effect on the actual netrc file.
>
> Note that while it is not possible to ask the netrc module what the
> default netrc file is, the documentation explicitly mentions that
> expanduser is used.
>
> Signed-off-by: Nicholas Jackson <nicholas.jackson@zii.aero>
> ---
>  bitbake/lib/bb/fetch2/wget.py | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
> index 821afa5b58..0ce6e25c5c 100644
> --- a/bitbake/lib/bb/fetch2/wget.py
> +++ b/bitbake/lib/bb/fetch2/wget.py
> @@ -360,10 +360,16 @@ class Wget(FetchMethod):
>
>                  try:
>                      import netrc
> +                    # Pseudo can cause ownership mismatch issues with
> netrc, which is
> +                    # required to be owned by the current user (root).
> +                    if os.environ.get("PSEUDO_DISABLED") == "0":
> +                        os.chown(os.path.join(os.path.expanduser("~"),
> ".netrc"), 0, 0)
>                      n = netrc.netrc()
>                      login, unused, password =
> n.authenticators(urllib.parse.urlparse(uri).hostname)
>                      add_basic_auth("%s:%s" % (login, password), r)
> -                except (TypeError, ImportError, IOError,
> netrc.NetrcParseError):
> +                except netrc.NetrcParseError as e:
> +                    logger.debug("netrc failed: %s" % e)
> +                except (TypeError, ImportError, IOError) as e:
>                      pass
>
>                  with opener.open(r, timeout=30) as response:
> --
> 2.37.2
>
>
>
> *Nicholas JACKSON*
> Software Engineer II
>
> Safran Passenger Innovations
>
> T +1 714 854 8600 ext 527
>
> 3151 East Imperial Hwy
>
> Brea, Ca 92821 USA
>
> *www.safran-passenger-innovations.com*
> <https://urldefense.com/v3/__http://www.safran-passenger-innovations.com/__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyCHL6szf$>
>
>
>
>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
> <https://urldefense.com/v3/__http://www.safran-group.com/subscription-account?utm_source=subscribemail&utm_medium=mailpro&utm_campaign=subscribemailEN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyBduGohQ$>
>
>
> <https://urldefense.com/v3/__http://twitter.com/SAFRAN__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyNnzvPqy$>
>
>
> <https://urldefense.com/v3/__http://www.facebook.com/GroupeSafran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyHmr7D_P$>
>
>
> <https://urldefense.com/v3/__http://fr.linkedin.com/company/safran__;!!BXNzmnhnlE-olPyVU9tI!As8zj7fVv20WKGLUTF4RmEeqROZCvPaimgdLyag0Ird5wj8wRp_VGu9fGuLkpGGLDedRzDSMUZ1LJCoQAS8jyGC1R16c$>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
>
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#14006):
> https://lists.openembedded.org/g/bitbake-devel/message/14006
> <https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/message/14006__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-okVai8$>
> Mute This Topic: https://lists.openembedded.org/mt/94022677/1686489
> <https://urldefense.com/v3/__https://lists.openembedded.org/mt/94022677/1686489__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM-RMoNep$>
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub
> <https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/unsub__;!!BXNzmnhnlE-olPyVU9tI!Gp-M49WeloAU78cAoBvhQ7H666CKF8pQvndx4nk4n4YKlPJeGGJT-erdamXxq_vFIZu97nOC1u2HsyhAX-SeM6Q9whAG$>
> [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
> ------------------------------
>
> This email and any files transmitted with it are confidential &
> proprietary to Safran Passenger Innovations. This information is intended
> solely for the use of the individual or entity to which it is addressed.
> Access or transmittal of the information contained in this e-mail, in full
> or in part, to any other organization or persons is not authorized.
>
Ross Burton Oct. 1, 2022, 7:57 p.m. UTC | #9
On 30 Sep 2022, at 22:36, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> 
> Thanks, I've reproduced this too using plain python3 and .netrc with tweaked permissions (e.g. run python as regular user, set .netrc ownership to root). I am not sure why fetching sstate items happens under pseudo, or whether the fix is correct, but the issue is real.

My hunch response would be that setscene tasks shouldn’t be fakeroot, or at least the fetch should drop that power.

Ross
Richard Purdie Oct. 1, 2022, 11:01 p.m. UTC | #10
On Sat, 2022-10-01 at 19:57 +0000, Ross Burton wrote:
> On 30 Sep 2022, at 22:36, Alexander Kanavin via
> lists.openembedded.org
> <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> > 
> > Thanks, I've reproduced this too using plain python3 and .netrc
> > with tweaked permissions (e.g. run python as regular user, set
> > .netrc ownership to root). I am not sure why fetching sstate items
> > happens under pseudo, or whether the fix is correct, but the issue
> > is real.
> 
> My hunch response would be that setscene tasks shouldn’t be fakeroot


Since the setscene tasks replicate things like do_package whch need to
preserve fileownership, they very much need fakeroot.

> or at least the fetch should drop that power.

The fetcher does run commands with PSEUDO_DISABLED=1 set, the issue is
that we're using library functions here. I'm not sure we can drop the
power within a library funciton call so easily.

Cheers,

Richard
Nicholas Jackson Oct. 10, 2022, 6:19 p.m. UTC | #11
On 10/1/22 4:01 PM, Richard Purdie wrote:
> On Sat, 2022-10-01 at 19:57 +0000, Ross Burton wrote:
>> On 30 Sep 2022, at 22:36, Alexander Kanavin via
>> lists.openembedded.org
>> <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
>>> Thanks, I've reproduced this too using plain python3 and .netrc
>>> with tweaked permissions (e.g. run python as regular user, set
>>> .netrc ownership to root). I am not sure why fetching sstate items
>>> happens under pseudo, or whether the fix is correct, but the issue
>>> is real.
>> My hunch response would be that setscene tasks shouldn’t be fakeroot
>
> Since the setscene tasks replicate things like do_package whch need to
> preserve fileownership, they very much need fakeroot.


So do you think that getting netrc to work with pseudo is a necessary
fix after all?


>> or at least the fetch should drop that power.
> The fetcher does run commands with PSEUDO_DISABLED=1 set, the issue is
> that we're using library functions here. I'm not sure we can drop the
> power within a library funciton call so easily.
>
> Cheers,
>
> Richard
>
>
>
diff mbox series

Patch

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 821afa5b58..0ce6e25c5c 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -360,10 +360,16 @@  class Wget(FetchMethod):

                 try:
                     import netrc
+                    # Pseudo can cause ownership mismatch issues with netrc, which is
+                    # required to be owned by the current user (root).
+                    if os.environ.get("PSEUDO_DISABLED") == "0":
+                        os.chown(os.path.join(os.path.expanduser("~"), ".netrc"), 0, 0)
                     n = netrc.netrc()
                     login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname)
                     add_basic_auth("%s:%s" % (login, password), r)
-                except (TypeError, ImportError, IOError, netrc.NetrcParseError):
+                except netrc.NetrcParseError as e:
+                    logger.debug("netrc failed: %s" % e)
+                except (TypeError, ImportError, IOError) as e:
                     pass

                 with opener.open(r, timeout=30) as response: