diff mbox series

Revert "python3-ctypes: depend on ldconfig only if distro-feature set"

Message ID 20250220035724.1460062-1-changqing.li@windriver.com
State Accepted, archived
Commit f98299ec2fa65804ceeff634fa50c8d154e1c153
Headers show
Series Revert "python3-ctypes: depend on ldconfig only if distro-feature set" | expand

Commit Message

Changqing Li Feb. 20, 2025, 3:57 a.m. UTC 
From: Changqing Li <changqing.li@windriver.com>

In oe-core, function ldconfig_postinst_fragment use exist of
/sbin/ldconfig to decide if ldconfig is runned to generate the cache,
and function _run_ldconfig will run ldconfig to generate cache during
generate rootfs.  ldconfig.service is actually not used since we have
generate ld.so.cache during do_rootfs, refer[1][2][3]. ldconfig
dependency is necessary when ldconfig not in DISTRO_FEATURES.

The reverted commit causes regression when ldconfig not in
DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
ctypes.util.find_library(name) can find the lib if it is installed, now,
since ldconfig is not installed, ctypes.util.find_library(name) cannot
find the lib even if it is installed.

Here is one usecase(gtk+3 lib is installed, ctypes.util.find_library
used to find the lib):
import wx.lib.wxcairo as wxcairo
File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/{}init{}.py", line 59, in <module>
  from .wx_cairocffi import _ContextFromDC, _FontFaceFromFont
File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 189, in <module>
  gdkLib = _findGDKLib()
File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 181, in _findGDKLib
  return _findHelper([libname], 'gdk',
                     "Unable to find the GDK shared library")
File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 170, in _findHelper
  raise RuntimeError(msg)
RuntimeError: Unable to find the GDK shared library

[1] https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass#n394
[2] https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/rootfs.py#n316
[3] https://github.com/systemd/systemd-stable/blob/v255-stable/units/ldconfig.service

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 meta/recipes-devtools/python/python3_3.13.2.bb | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

Comments

Mathieu Dubois-Briand Feb. 21, 2025, 2:34 p.m. UTC | #1 
On Thu Feb 20, 2025 at 4:57 AM CET, Changqing Li via lists.openembedded.org wrote:
> From: Changqing Li <changqing.li@windriver.com>
>
> In oe-core, function ldconfig_postinst_fragment use exist of
> /sbin/ldconfig to decide if ldconfig is runned to generate the cache,
> and function _run_ldconfig will run ldconfig to generate cache during
> generate rootfs.  ldconfig.service is actually not used since we have
> generate ld.so.cache during do_rootfs, refer[1][2][3]. ldconfig
> dependency is necessary when ldconfig not in DISTRO_FEATURES.
>
> The reverted commit causes regression when ldconfig not in
> DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
> ctypes.util.find_library(name) can find the lib if it is installed, now,
> since ldconfig is not installed, ctypes.util.find_library(name) cannot
> find the lib even if it is installed.
>
> Here is one usecase(gtk+3 lib is installed, ctypes.util.find_library
> used to find the lib):
> import wx.lib.wxcairo as wxcairo
> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/{}init{}.py", line 59, in <module>
>   from .wx_cairocffi import _ContextFromDC, _FontFaceFromFont
> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 189, in <module>
>   gdkLib = _findGDKLib()
> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 181, in _findGDKLib
>   return _findHelper([libname], 'gdk',
>                      "Unable to find the GDK shared library")
> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 170, in _findHelper
>   raise RuntimeError(msg)
> RuntimeError: Unable to find the GDK shared library
>
> [1] https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass#n394
> [2] https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/rootfs.py#n316
> [3] https://github.com/systemd/systemd-stable/blob/v255-stable/units/ldconfig.service
>
> Signed-off-by: Changqing Li <changqing.li@windriver.com>
> ---

Hi,

Thanks for your patch. However, I believe this is introducing some
regression: I've got some ptest failures on the autobuilder:

AssertionError:
Failed ptests:
{'python3': ['test_timerfd_TFD_TIMER_ABSTIME', 'python3']}

https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/992

You can get some more advanced about ptest failures in there:

https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/

Can you have a look at this issue please?
Changqing Li Feb. 24, 2025, 12:27 a.m. UTC | #2 
On 2/21/25 22:34, Mathieu Dubois-Briand wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Thu Feb 20, 2025 at 4:57 AM CET, Changqing Li via lists.openembedded.org wrote:
>> From: Changqing Li<changqing.li@windriver.com>
>>
>> In oe-core, function ldconfig_postinst_fragment use exist of
>> /sbin/ldconfig to decide if ldconfig is runned to generate the cache,
>> and function _run_ldconfig will run ldconfig to generate cache during
>> generate rootfs.  ldconfig.service is actually not used since we have
>> generate ld.so.cache during do_rootfs, refer[1][2][3]. ldconfig
>> dependency is necessary when ldconfig not in DISTRO_FEATURES.
>>
>> The reverted commit causes regression when ldconfig not in
>> DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
>> ctypes.util.find_library(name) can find the lib if it is installed, now,
>> since ldconfig is not installed, ctypes.util.find_library(name) cannot
>> find the lib even if it is installed.
>>
>> Here is one usecase(gtk+3 lib is installed, ctypes.util.find_library
>> used to find the lib):
>> import wx.lib.wxcairo as wxcairo
>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/{}init{}.py", line 59, in <module>
>>    from .wx_cairocffi import _ContextFromDC, _FontFaceFromFont
>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 189, in <module>
>>    gdkLib = _findGDKLib()
>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 181, in _findGDKLib
>>    return _findHelper([libname], 'gdk',
>>                       "Unable to find the GDK shared library")
>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 170, in _findHelper
>>    raise RuntimeError(msg)
>> RuntimeError: Unable to find the GDK shared library
>>
>> [1]https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass#n394
>> [2]https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/rootfs.py#n316
>> [3]https://github.com/systemd/systemd-stable/blob/v255-stable/units/ldconfig.service
>>
>> Signed-off-by: Changqing Li<changqing.li@windriver.com>
>> ---
> Hi,
>
> Thanks for your patch. However, I believe this is introducing some
> regression: I've got some ptest failures on the autobuilder:
>
> AssertionError:
> Failed ptests:
> {'python3': ['test_timerfd_TFD_TIMER_ABSTIME', 'python3']}
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/992
>
> You can get some more advanced about ptest failures in there:
>
> https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/
>
> Can you have a look at this issue please?

Sure. I will check this.

Regards

Changqing

>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
Changqing Li Feb. 24, 2025, 8:20 a.m. UTC | #3 
On 2/24/25 08:27, Changqing Li via lists.openembedded.org wrote:
>
>
> On 2/21/25 22:34, Mathieu Dubois-Briand wrote:
>> CAUTION: This email comes from a non Wind River email account!
>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>
>> On Thu Feb 20, 2025 at 4:57 AM CET, Changqing Li via lists.openembedded.org wrote:
>>> From: Changqing Li<changqing.li@windriver.com>
>>>
>>> In oe-core, function ldconfig_postinst_fragment use exist of
>>> /sbin/ldconfig to decide if ldconfig is runned to generate the cache,
>>> and function _run_ldconfig will run ldconfig to generate cache during
>>> generate rootfs.  ldconfig.service is actually not used since we have
>>> generate ld.so.cache during do_rootfs, refer[1][2][3]. ldconfig
>>> dependency is necessary when ldconfig not in DISTRO_FEATURES.
>>>
>>> The reverted commit causes regression when ldconfig not in
>>> DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
>>> ctypes.util.find_library(name) can find the lib if it is installed, now,
>>> since ldconfig is not installed, ctypes.util.find_library(name) cannot
>>> find the lib even if it is installed.
>>>
>>> Here is one usecase(gtk+3 lib is installed, ctypes.util.find_library
>>> used to find the lib):
>>> import wx.lib.wxcairo as wxcairo
>>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/{}init{}.py", line 59, in <module>
>>>    from .wx_cairocffi import _ContextFromDC, _FontFaceFromFont
>>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 189, in <module>
>>>    gdkLib = _findGDKLib()
>>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 181, in _findGDKLib
>>>    return _findHelper([libname], 'gdk',
>>>                       "Unable to find the GDK shared library")
>>> File "/usr/lib/python3.13/site-packages/wx/lib/wxcairo/wx_cairocffi.py", line 170, in _findHelper
>>>    raise RuntimeError(msg)
>>> RuntimeError: Unable to find the GDK shared library
>>>
>>> [1]https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass#n394
>>> [2]https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/rootfs.py#n316
>>> [3]https://github.com/systemd/systemd-stable/blob/v255-stable/units/ldconfig.service
>>>
>>> Signed-off-by: Changqing Li<changqing.li@windriver.com>
>>> ---
>> Hi,
>>
>> Thanks for your patch. However, I believe this is introducing some
>> regression: I've got some ptest failures on the autobuilder:
>>
>> AssertionError:
>> Failed ptests:
>> {'python3': ['test_timerfd_TFD_TIMER_ABSTIME', 'python3']}
>>
>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/992
>>
>> You can get some more advanced about ptest failures in there:
>>
>> https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/
>>
>> Can you have a look at this issue please?
>
> Sure. I will check this.
>
> Regards
>
> Changqing
>
Hi, Mathieu

I cannot reproduce this ptest failure on my host. No failure of 
test_timerfd_TFD_TIMER_ABSTIME in log.do_testimage. and the result is:

RESULTS - parselogs.ParseLogsTest.test_get_context: PASSED (0.00s) 
RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (2.71s) RESULTS 
- ping.PingTest.test_ping: PASSED (0.02s) RESULTS - 
ssh.SSHTest.test_ssh: PASSED (0.56s) RESULTS - 
ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) 
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: 
EXPECTEDFAIL (4455.53s) SUMMARY: core-image-ptest-python3 () - Ran 6 
tests in 4458.839s core-image-ptest-python3 - OK - All required tests 
passed (successes=4, skipped=1, failures=0, errors=0) DEBUG: Python 
function do_testimage finished

And I checked 
https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/, 


there is no AssertionError in log.do_testimage.2624139.20250221054435 
<https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/log.do_testimage.2624139.20250221054435>.

So I checked test_timerfd_TFD_TIMER_ABSTIME, refer [1]. Seems like this 
is not related to my changes.

My commit changes RDEPENDS for package pthon3-ctype, and this is 
actually no influence for poky, since poky enable distro_feature 
ldconfig by default.

There are some similar timerfd issue in upstream python, I guess maybe 
this is an intermittent failure of python13

[1] https://github.com/python/cpython/blob/main/Lib/test/test_os.py#L4402

[2] https://github.com/python/cpython/issues/126112

Regards
Changqing

>> --
>> Mathieu Dubois-Briand, Bootlin
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#211834):https://lists.openembedded.org/g/openembedded-core/message/211834
> Mute This Topic:https://lists.openembedded.org/mt/111283758/3616873
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub  [changqing.li@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Mathieu Dubois-Briand Feb. 24, 2025, 11:02 a.m. UTC | #4 
On Mon Feb 24, 2025 at 9:20 AM CET, Changqing Li wrote:
>
> On 2/24/25 08:27, Changqing Li via lists.openembedded.org wrote:
> Hi, Mathieu
>
> I cannot reproduce this ptest failure on my host. No failure of 
> test_timerfd_TFD_TIMER_ABSTIME in log.do_testimage. and the result is:
>
> RESULTS - parselogs.ParseLogsTest.test_get_context: PASSED (0.00s) 
> RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (2.71s) RESULTS 
> - ping.PingTest.test_ping: PASSED (0.02s) RESULTS - 
> ssh.SSHTest.test_ssh: PASSED (0.56s) RESULTS - 
> ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) 
> RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: 
> EXPECTEDFAIL (4455.53s) SUMMARY: core-image-ptest-python3 () - Ran 6 
> tests in 4458.839s core-image-ptest-python3 - OK - All required tests 
> passed (successes=4, skipped=1, failures=0, errors=0) DEBUG: Python 
> function do_testimage finished
>
> And I checked 
> https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/, 
>
>
> there is no AssertionError in log.do_testimage.2624139.20250221054435 
> <https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/log.do_testimage.2624139.20250221054435>.
>
> So I checked test_timerfd_TFD_TIMER_ABSTIME, refer [1]. Seems like this 
> is not related to my changes.
>
> My commit changes RDEPENDS for package pthon3-ctype, and this is 
> actually no influence for poky, since poky enable distro_feature 
> ldconfig by default.
>
> There are some similar timerfd issue in upstream python, I guess maybe 
> this is an intermittent failure of python13
>
> [1] https://github.com/python/cpython/blob/main/Lib/test/test_os.py#L4402
>
> [2] https://github.com/python/cpython/issues/126112
>
> Regards
> Changqing

Hi Changqing,

Thanks for testing. This might be a intermittent failure, but I believe
we never saw this one before.

I will add back this patch in my branch and let's see how it goes!
Jörg Sommer Feb. 24, 2025, 11:36 p.m. UTC | #5 
Changqing Li via lists.openembedded.org schrieb am Do 20. Feb, 11:57 (+0800):
> The reverted commit causes regression when ldconfig not in
> DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
> ctypes.util.find_library(name) can find the lib if it is installed,

Hello Changqing Li,

Did I understand correctly that gtk+3 was found before, indent of ldconfig
was in der DISTRO_FEATURES? But after the change, if fails.

Where is libgdk installed? In /usr/lib or somewhere else?


Bye, Jörg
Changqing Li Feb. 25, 2025, 2:02 a.m. UTC | #6 
On 2/25/25 07:36, Jörg Sommer wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> Changqing Li via lists.openembedded.org schrieb am Do 20. Feb, 11:57 (+0800):
>> The reverted commit causes regression when ldconfig not in
>> DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
>> ctypes.util.find_library(name) can find the lib if it is installed,
> Hello Changqing Li,
>
> Did I understand correctly that gtk+3 was found before, indent of ldconfig
> was in der DISTRO_FEATURES? But after the change, if fails.
>
> Where is libgdk installed? In /usr/lib or somewhere else?

Hi,

libgtk is under /usr/lib. See:

root@qemux86-64:~# ldconfig -p | grep gtk-3

libgtk-3.so.0 (libc6,x86-64) => /usr/lib/libgtk-3.so.0

root@qemux86-64:~# python3

Python 3.13.2 (main, Feb 4 2025, 14:51:09) [GCC 14.2.0] on linux Type 
"help", "copyright", "credits" or "license" for more information.

 >>> from ctypes.util import find_library

 >>> find_library('gtk-3')

  'libgtk-3.so.0'


For the case that ldconfig is NOT in DISTRO_FEATURES, gtk+3 can be 
found. After commit "python3-ctypes: depend on ldconfig only if 
distro-feature set" is merged,

ldconfig is not installed, so ctypes.util.find_library cannot found gtk+3.

//Changqing

>
>
> Bye, Jörg
>
> --
> Navimatix GmbH           T: 03641 - 327 99 0
> Tatzendpromenade 2       F: 03641 - 526 306
> 07745 Jenawww.navimatix.de
>
> Geschäftsführer: Steffen Späthe, Jan Rommeley
> Registergericht: Amtsgericht Jena, HRB 501480
Jörg Sommer Feb. 25, 2025, 5:33 a.m. UTC | #7 
Changqing Li schrieb am Di 25. Feb, 10:02 (+0800):
> On 2/25/25 07:36, Jörg Sommer wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > 
> > Changqing Li via lists.openembedded.org schrieb am Do 20. Feb, 11:57 (+0800):
> > > The reverted commit causes regression when ldconfig not in
> > > DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
> > > ctypes.util.find_library(name) can find the lib if it is installed,
> > Hello Changqing Li,
> > 
> > Did I understand correctly that gtk+3 was found before, indent of ldconfig
> > was in der DISTRO_FEATURES? But after the change, if fails.
> > 
> > Where is libgdk installed? In /usr/lib or somewhere else?

> libgtk is under /usr/lib. See:
> 
> root@qemux86-64:~# ldconfig -p | grep gtk-3
> 
> libgtk-3.so.0 (libc6,x86-64) => /usr/lib/libgtk-3.so.0

Interesting. I expected a subdirectory of lib.

> root@qemux86-64:~# python3
> 
> >>> from ctypes.util import find_library
> >>> find_library('gtk-3')
> 
>  'libgtk-3.so.0'

Thanks. That is a nice test case.

> For the case that ldconfig is NOT in DISTRO_FEATURES, gtk+3 can be found.
> After commit "python3-ctypes: depend on ldconfig only if distro-feature set"
> is merged,
> 
> ldconfig is not installed, so ctypes.util.find_library cannot found gtk+3.

Yes, this was the contradiction: Even through you removed ldconfig from
DISTRO_FEATURES, it got pulled in. I could not remove ldconfig (even through
our system with python works fine without it).


Kind regards, Jörg
Mathieu Dubois-Briand Feb. 25, 2025, 9:12 a.m. UTC | #8 
On Mon Feb 24, 2025 at 12:02 PM CET, Mathieu Dubois-Briand via lists.openembedded.org wrote:
> On Mon Feb 24, 2025 at 9:20 AM CET, Changqing Li wrote:
> >
> > On 2/24/25 08:27, Changqing Li via lists.openembedded.org wrote:
> > Hi, Mathieu
> >
> > I cannot reproduce this ptest failure on my host. No failure of 
> > test_timerfd_TFD_TIMER_ABSTIME in log.do_testimage. and the result is:
> >
> > RESULTS - parselogs.ParseLogsTest.test_get_context: PASSED (0.00s) 
> > RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (2.71s) RESULTS 
> > - ping.PingTest.test_ping: PASSED (0.02s) RESULTS - 
> > ssh.SSHTest.test_ssh: PASSED (0.56s) RESULTS - 
> > ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) 
> > RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: 
> > EXPECTEDFAIL (4455.53s) SUMMARY: core-image-ptest-python3 () - Ran 6 
> > tests in 4458.839s core-image-ptest-python3 - OK - All required tests 
> > passed (successes=4, skipped=1, failures=0, errors=0) DEBUG: Python 
> > function do_testimage finished
> >
> > And I checked 
> > https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/, 
> >
> >
> > there is no AssertionError in log.do_testimage.2624139.20250221054435 
> > <https://valkyrie.yocto.io/pub/non-release/20250221-33/testresults/qemux86-64-ptest/core-image-ptest-python3/log.do_testimage.2624139.20250221054435>.
> >
> > So I checked test_timerfd_TFD_TIMER_ABSTIME, refer [1]. Seems like this 
> > is not related to my changes.
> >
> > My commit changes RDEPENDS for package pthon3-ctype, and this is 
> > actually no influence for poky, since poky enable distro_feature 
> > ldconfig by default.
> >
> > There are some similar timerfd issue in upstream python, I guess maybe 
> > this is an intermittent failure of python13
> >
> > [1] https://github.com/python/cpython/blob/main/Lib/test/test_os.py#L4402
> >
> > [2] https://github.com/python/cpython/issues/126112
> >
> > Regards
> > Changqing
>
> Hi Changqing,
>
> Thanks for testing. This might be a intermittent failure, but I believe
> we never saw this one before.
>
> I will add back this patch in my branch and let's see how it goes!

Hi Changqing,

I confirm I added this back and now it's working. So it might be some
new AB-INT issue, nothing is wrong with your patch from the autobuilder
perspective.

Thanks for your time.
Jörg Sommer Feb. 27, 2025, 10:32 p.m. UTC | #9 
Changqing Li schrieb am Di 25. Feb, 10:02 (+0800):
> On 2/25/25 07:36, Jörg Sommer wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > 
> > Changqing Li via lists.openembedded.org schrieb am Do 20. Feb, 11:57 (+0800):
> > > The reverted commit causes regression when ldconfig not in
> > > DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
> > > ctypes.util.find_library(name) can find the lib if it is installed,
> > Hello Changqing Li,
> > 
> > Did I understand correctly that gtk+3 was found before, indent of ldconfig
> > was in der DISTRO_FEATURES? But after the change, if fails.
> > 
> > Where is libgdk installed? In /usr/lib or somewhere else?
> 
> Hi,
> 
> libgtk is under /usr/lib. See:
> 
> root@qemux86-64:~# ldconfig -p | grep gtk-3
> 
> libgtk-3.so.0 (libc6,x86-64) => /usr/lib/libgtk-3.so.0
> 
> root@qemux86-64:~# python3
> 
> Python 3.13.2 (main, Feb 4 2025, 14:51:09) [GCC 14.2.0] on linux Type
> "help", "copyright", "credits" or "license" for more information.
> 
> >>> from ctypes.util import find_library
> 
> >>> find_library('gtk-3')
> 
>  'libgtk-3.so.0'

I have reverted your patch and this works:

```
root@qemux86-64:~# python3 -c 'from ctypes.util import find_library; print(find_library("m"))'
libm.so.6
root@qemux86-64:~# python3 -c 'from ctypes.util import find_library; print(find_library("gtk-3"))'
libgtk-3.so.0

root@qemux86-64:~# ldconfig --version
ldconfig (GNU libc) 2.41
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Andreas Jaeger.
```

Here are my settings from local.conf:

```
EXTRA_IMAGE_FEATURES += "\
    allow-empty-password allow-root-login empty-root-password \
    ssh-server-openssh tools-debug \
"

INIT_MANAGER = "systemd"
DISTRO_FEATURES:append = " usrmerge"
IMAGE_INSTALL:append = " conserver-ptest"
PACKAGECONFIG:append:pn-systemd = " coredump"

IMAGE_INSTALL:append = " python3-ctypes gtk+3"

MACHINE ??= "qemux86-64"
DISTRO ??= "poky"
```

I have used poky with 40b5f61a8f and the revert and meta-openembedded with
0b83551ff2.

> For the case that ldconfig is NOT in DISTRO_FEATURES, gtk+3 can be found.
> After commit "python3-ctypes: depend on ldconfig only if distro-feature set"
> is merged,
> 
> ldconfig is not installed, so ctypes.util.find_library cannot found gtk+3.

You can also install ld or gcc. But if you prefer ldconfig, you should set
this DISTRO_FEATURES. It's like x11 support, if you want it you have to set
the feature.

Quoting https://docs.yoctoproject.org/ref-manual/features.html:

| ldconfig: Include support for ldconfig and ld.so.conf on the target.


@Mathieu: I think you should revert the commit 0df1318e5c, because it pulls
in ldconfig even it is unselected in DISTRO_FEATURES. It leaves no way to
build without ldconfig (what was the intend of my commit).


Kind regards, Jörg
Changqing Li Feb. 28, 2025, 2:56 a.m. UTC | #10 
On 2/28/25 06:32, Jörg Sommer wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> Changqing Li schrieb am Di 25. Feb, 10:02 (+0800):
>> On 2/25/25 07:36, Jörg Sommer wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> Changqing Li via lists.openembedded.org schrieb am Do 20. Feb, 11:57 (+0800):
>>>> The reverted commit causes regression when ldconfig not in
>>>> DISTRO_FEATURES, before, without ldconfig in DISTRO_FEATURES,
>>>> ctypes.util.find_library(name) can find the lib if it is installed,
>>> Hello Changqing Li,
>>>
>>> Did I understand correctly that gtk+3 was found before, indent of ldconfig
>>> was in der DISTRO_FEATURES? But after the change, if fails.
>>>
>>> Where is libgdk installed? In /usr/lib or somewhere else?
>> Hi,
>>
>> libgtk is under /usr/lib. See:
>>
>> root@qemux86-64:~# ldconfig -p | grep gtk-3
>>
>> libgtk-3.so.0 (libc6,x86-64) => /usr/lib/libgtk-3.so.0
>>
>> root@qemux86-64:~# python3
>>
>> Python 3.13.2 (main, Feb 4 2025, 14:51:09) [GCC 14.2.0] on linux Type
>> "help", "copyright", "credits" or "license" for more information.
>>
>>>>> from ctypes.util import find_library
>>>>> find_library('gtk-3')
>>   'libgtk-3.so.0'
> I have reverted your patch and this works:
>
> ```
> root@qemux86-64:~# python3 -c 'from ctypes.util import find_library; print(find_library("m"))'
> libm.so.6
> root@qemux86-64:~# python3 -c 'from ctypes.util import find_library; print(find_library("gtk-3"))'
> libgtk-3.so.0
>
> root@qemux86-64:~# ldconfig --version
> ldconfig (GNU libc) 2.41
> Copyright (C) 2024 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> Written by Andreas Jaeger.
> ```
>
> Here are my settings from local.conf:
>
> ```
> EXTRA_IMAGE_FEATURES += "\
>      allow-empty-password allow-root-login empty-root-password \
>      ssh-server-openssh tools-debug \
> "
>
> INIT_MANAGER = "systemd"
> DISTRO_FEATURES:append = " usrmerge"
> IMAGE_INSTALL:append = " conserver-ptest"
> PACKAGECONFIG:append:pn-systemd = " coredump"
>
> IMAGE_INSTALL:append = " python3-ctypes gtk+3"
>
> MACHINE ??= "qemux86-64"
> DISTRO ??= "poky"
> ```
>
> I have used poky with 40b5f61a8f and the revert and meta-openembedded with
> 0b83551ff2.

This works because poky have ldconfig in DISTRO_FEATURES by default, 
https://git.yoctoproject.org/poky/tree/meta/conf/bitbake.conf#n933

if remove ldconfig from DISTRO_FEATURES in local.conf, it will not work.

>> For the case that ldconfig is NOT in DISTRO_FEATURES, gtk+3 can be found.
>> After commit "python3-ctypes: depend on ldconfig only if distro-feature set"
>> is merged,
>>
>> ldconfig is not installed, so ctypes.util.find_library cannot found gtk+3.
> You can also install ld or gcc. But if you prefer ldconfig, you should set
> this DISTRO_FEATURES. It's like x11 support, if you want it you have to set
> the feature.

ctypes.util.find_library need ldconfig or ld or gcc to work. No matter 
what the DISTRO_FEATURES is, we should make a package at least work once 
it is installed.

python3-ctypes need rdepend at least one of ld, ldconfig, gcc, it should 
be a hard dependency.

certainly，it doesn't have to be ldconfig.  I am not sure depends on 
binutils or gcc it better, since binutils and gcc are more big packges.

>
> Quotinghttps://docs.yoctoproject.org/ref-manual/features.html:
>
> | ldconfig: Include support for ldconfig and ld.so.conf on the target.
>
>
> @Mathieu: I think you should revert the commit 0df1318e5c, because it pulls
> in ldconfig even it is unselected in DISTRO_FEATURES. It leaves no way to
> build without ldconfig (what was the intend of my commit).

when ldconfig is disabled in DISTRO_FEATURES, if we don't want ldconfig 
be pulled in, how about just not install the package that depend on 
ldconfig, eg: python3-ctypes.

if revert my commit, and ldconfig is disabled in DISTRO_FEATURES, and 
python3-ctype is installed, ctypes.util.find_library will not work.

I tried with my commit in oe-core, and 
DISTRO_FEATURES_BACKFILL_CONSIDERED += "ldconfig" in local.conf:

boot core-image-minimal, ldconfig is not pulled.

root@qemux86-64:~# ldconfig --version -sh: ldconfig: command not found 
root@qemux86-64:~#

Regards

Changqing

>
>
> Kind regards, Jörg
>
> --
> Navimatix GmbH           T: 03641 - 327 99 0
> Tatzendpromenade 2       F: 03641 - 526 306
> 07745 Jenawww.navimatix.de
>
> Geschäftsführer: Steffen Späthe, Jan Rommeley
> Registergericht: Amtsgericht Jena, HRB 501480
Jörg Sommer Feb. 28, 2025, 5:42 a.m. UTC | #11 
Changqing Li schrieb am Fr 28. Feb, 10:56 (+0800):
> On 2/28/25 06:32, Jörg Sommer wrote:
> > ```
> > root@qemux86-64:~# python3 -c 'from ctypes.util import find_library; print(find_library("m"))'
> > libm.so.6
> > root@qemux86-64:~# python3 -c 'from ctypes.util import find_library; print(find_library("gtk-3"))'
> > libgtk-3.so.0
> > 
> > root@qemux86-64:~# ldconfig --version
> > ldconfig (GNU libc) 2.41
> > Copyright (C) 2024 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions.  There is NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > Written by Andreas Jaeger.
> > ```
> > 
> > Here are my settings from local.conf:
> > 
> > ```
> > EXTRA_IMAGE_FEATURES += "\
> >      allow-empty-password allow-root-login empty-root-password \
> >      ssh-server-openssh tools-debug \
> > "
> > 
> > INIT_MANAGER = "systemd"
> > DISTRO_FEATURES:append = " usrmerge"
> > IMAGE_INSTALL:append = " conserver-ptest"
> > PACKAGECONFIG:append:pn-systemd = " coredump"
> > 
> > IMAGE_INSTALL:append = " python3-ctypes gtk+3"
> > 
> > MACHINE ??= "qemux86-64"
> > DISTRO ??= "poky"
> > ```
> > 
> > I have used poky with 40b5f61a8f and the revert and meta-openembedded with
> > 0b83551ff2.
> 
> This works because poky have ldconfig in DISTRO_FEATURES by default,
> https://git.yoctoproject.org/poky/tree/meta/conf/bitbake.conf#n933
> 
> if remove ldconfig from DISTRO_FEATURES in local.conf, it will not work.

But you can't remove ldconfig if you need it. Why do you remove it?

If you want the ldconfig feature of python3-ctype, you have to enable
ldconfig. Or at least install the package.

Or should we downgrade the dependency to a recommendation?

> ctypes.util.find_library need ldconfig or ld or gcc to work. No matter what
> the DISTRO_FEATURES is, we should make a package at least work once it is
> installed.

In our image python3-ctype works without ldconfig.

> > Quotinghttps://docs.yoctoproject.org/ref-manual/features.html:
> > 
> > | ldconfig: Include support for ldconfig and ld.so.conf on the target.
> > 
> > 
> > @Mathieu: I think you should revert the commit 0df1318e5c, because it pulls
> > in ldconfig even it is unselected in DISTRO_FEATURES. It leaves no way to
> > build without ldconfig (what was the intend of my commit).
> 
> when ldconfig is disabled in DISTRO_FEATURES, if we don't want ldconfig be
> pulled in, how about just not install the package that depend on ldconfig,
> eg: python3-ctypes.

In our case python3-ctypes is a dependency of django.

> if revert my commit, and ldconfig is disabled in DISTRO_FEATURES, and
> python3-ctype is installed, ctypes.util.find_library will not work.

But then you need ldconfig. Either install it with IMAGE_INSTALL or include
it in DISTRO_FEATURES. I see this the same way like other DISTRO_FEATURES
for example pam: if you want to use pam with e.g. systemd you have to
include it in DISTRO_FEATURES.


Regards, Jörg
Richard Purdie Feb. 28, 2025, 10:20 a.m. UTC | #12 
On Fri, 2025-02-28 at 06:42 +0100, Jörg Sommer wrote:
> Changqing Li schrieb am Fr 28. Feb, 10:56 (+0800):
> 
> But you can't remove ldconfig if you need it. Why do you remove it?
> 
> If you want the ldconfig feature of python3-ctype, you have to enable
> ldconfig. Or at least install the package.
> 
> Or should we downgrade the dependency to a recommendation?
> 
> > ctypes.util.find_library need ldconfig or ld or gcc to work. No matter what
> > the DISTRO_FEATURES is, we should make a package at least work once it is
> > installed.
> 
> In our image python3-ctype works without ldconfig.
> 
> > > Quotinghttps://docs.yoctoproject.org/ref-manual/features.html:
> > > 
> > > > ldconfig: Include support for ldconfig and ld.so.conf on the target.
> > > 
> > > 
> > > @Mathieu: I think you should revert the commit 0df1318e5c, because it pulls
> > > in ldconfig even it is unselected in DISTRO_FEATURES. It leaves no way to
> > > build without ldconfig (what was the intend of my commit).
> > 
> > when ldconfig is disabled in DISTRO_FEATURES, if we don't want ldconfig be
> > pulled in, how about just not install the package that depend on ldconfig,
> > eg: python3-ctypes.
> 
> In our case python3-ctypes is a dependency of django.
> 
> > if revert my commit, and ldconfig is disabled in DISTRO_FEATURES, and
> > python3-ctype is installed, ctypes.util.find_library will not work.
> 
> But then you need ldconfig. Either install it with IMAGE_INSTALL or include
> it in DISTRO_FEATURES. I see this the same way like other DISTRO_FEATURES
> for example pam: if you want to use pam with e.g. systemd you have to
> include it in DISTRO_FEATURES.

DISTRO_FEATURES are high level policy controls, not "this should be
included or excluded" controls. For example "pam" controls whether the
access control modules are configured/used, not whether libpam is or is
not included.

For ldconfig, as I remember, the distro config was about whether to
include "ldconfig" calls after installing libraries and whether to use
the ld cache.

Packages being installed or not is an image level decision, not distro
level.

In this case, perhaps we should drop the RDEPENDS to a RRECOMMENDS?

Cheers,

Richard
Jörg Sommer Feb. 28, 2025, 10:35 a.m. UTC | #13 
Richard Purdie schrieb am Fr 28. Feb, 10:20 (+0000):
> On Fri, 2025-02-28 at 06:42 +0100, Jörg Sommer wrote:
> > Changqing Li schrieb am Fr 28. Feb, 10:56 (+0800):
> > 
> > But you can't remove ldconfig if you need it. Why do you remove it?
> > 
> > If you want the ldconfig feature of python3-ctype, you have to enable
> > ldconfig. Or at least install the package.
> > 
> > Or should we downgrade the dependency to a recommendation?
> > 
> > > ctypes.util.find_library need ldconfig or ld or gcc to work. No matter what
> > > the DISTRO_FEATURES is, we should make a package at least work once it is
> > > installed.
> > 
> > In our image python3-ctype works without ldconfig.
> > 
> > > > Quotinghttps://docs.yoctoproject.org/ref-manual/features.html:
> > > > 
> > > > > ldconfig: Include support for ldconfig and ld.so.conf on the target.
> > > > 
> > > > 
> > > > @Mathieu: I think you should revert the commit 0df1318e5c, because it pulls
> > > > in ldconfig even it is unselected in DISTRO_FEATURES. It leaves no way to
> > > > build without ldconfig (what was the intend of my commit).
> > > 
> > > when ldconfig is disabled in DISTRO_FEATURES, if we don't want ldconfig be
> > > pulled in, how about just not install the package that depend on ldconfig,
> > > eg: python3-ctypes.
> > 
> > In our case python3-ctypes is a dependency of django.
> > 
> > > if revert my commit, and ldconfig is disabled in DISTRO_FEATURES, and
> > > python3-ctype is installed, ctypes.util.find_library will not work.
> > 
> > But then you need ldconfig. Either install it with IMAGE_INSTALL or include
> > it in DISTRO_FEATURES. I see this the same way like other DISTRO_FEATURES
> > for example pam: if you want to use pam with e.g. systemd you have to
> > include it in DISTRO_FEATURES.
> 
> DISTRO_FEATURES are high level policy controls, not "this should be
> included or excluded" controls. For example "pam" controls whether the
> access control modules are configured/used, not whether libpam is or is
> not included.

But it controls if other packages (e.g. systemd) enable pam support or not.
The same goes with ldconfig: Should ldconfig support in the packages be
enabled (which would pull-in ldconfig as dependency).

> For ldconfig, as I remember, the distro config was about whether to
> include "ldconfig" calls after installing libraries and whether to use the
> ld cache.

If the feature is only about enabling code in the postinst script, than we
should find a better description of the feature

| ldconfig: Include support for ldconfig and ld.so.conf on the target.

https://docs.yoctoproject.org/ref-manual/features.html

> Packages being installed or not is an image level decision, not distro
> level.

Yes, that's also my understanding. But enabling some features (e.g. x11
support) leads to recipes set DEPENDS on libs and this leads to the
installation of some packages. So DISTRO_FEATURES influence indirectly what
gets into the image.

> In this case, perhaps we should drop the RDEPENDS to a RRECOMMENDS?

This would be helpful to remove ldconfig, if django is installed.

But the main question is still open: Why should I want ldconfig and remove
it from DISTRO_FEATURES? What is this use case?


Regards Jörg
Richard Purdie Feb. 28, 2025, 10:50 a.m. UTC | #14 
On Fri, 2025-02-28 at 11:35 +0100, Jörg Sommer wrote:
> Richard Purdie schrieb am Fr 28. Feb, 10:20 (+0000):
> > DISTRO_FEATURES are high level policy controls, not "this should be
> > included or excluded" controls. For example "pam" controls whether the
> > access control modules are configured/used, not whether libpam is or is
> > not included.
> 
> But it controls if other packages (e.g. systemd) enable pam support or not.
> The same goes with ldconfig: Should ldconfig support in the packages be
> enabled (which would pull-in ldconfig as dependency).
> 
> > For ldconfig, as I remember, the distro config was about whether to
> > include "ldconfig" calls after installing libraries and whether to use the
> > ld cache.
> 
> If the feature is only about enabling code in the postinst script, than we
> should find a better description of the feature

I think the description needs improving and I think we'd take patches
to do that.

> > ldconfig: Include support for ldconfig and ld.so.conf on the target.
> 
> https://docs.yoctoproject.org/ref-manual/features.html
> 
> > Packages being installed or not is an image level decision, not distro
> > level.
> 
> Yes, that's also my understanding. But enabling some features (e.g. x11
> support) leads to recipes set DEPENDS on libs and this leads to the
> installation of some packages. So DISTRO_FEATURES influence indirectly what
> gets into the image.

The key word is indirectly. Where things have a hard dependency on
something, they do include that dependency and it isn't conditional on
a distro feature.

In this case, the usage is tangential to what the DISTRO_FEATURE was
designed for (ld cache). I do agree ldconfig is optional so RRECOMMENDS
seems better than a hard RDEPENDS though.

> > In this case, perhaps we should drop the RDEPENDS to a RRECOMMENDS?
> 
> This would be helpful to remove ldconfig, if django is installed.
> 
> But the main question is still open: Why should I want ldconfig and remove
> it from DISTRO_FEATURES? What is this use case?

It is being used to control whether the ld cache is being used or not.
That has to be done at a distro wide policy level. Whether ldconfig
makes it into the image or not is an indirect effect.

We can control the inclusion of ldconfig for ctypes at a package/image
level, it isn't a distro config level issue.

Cheers,

Richard
Changqing Li March 3, 2025, 3:30 a.m. UTC | #15 
On 2/28/25 18:50, Richard Purdie wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Fri, 2025-02-28 at 11:35 +0100, Jörg Sommer wrote:
>> Richard Purdie schrieb am Fr 28. Feb, 10:20 (+0000):
>>> DISTRO_FEATURES are high level policy controls, not "this should be
>>> included or excluded" controls. For example "pam" controls whether the
>>> access control modules are configured/used, not whether libpam is or is
>>> not included.
>> But it controls if other packages (e.g. systemd) enable pam support or not.
>> The same goes with ldconfig: Should ldconfig support in the packages be
>> enabled (which would pull-in ldconfig as dependency).
>>
>>> For ldconfig, as I remember, the distro config was about whether to
>>> include "ldconfig" calls after installing libraries and whether to use the
>>> ld cache.
>> If the feature is only about enabling code in the postinst script, than we
>> should find a better description of the feature
> I think the description needs improving and I think we'd take patches
> to do that.
>
>>> ldconfig: Include support for ldconfig and ld.so.conf on the target.
>> https://docs.yoctoproject.org/ref-manual/features.html
>>
>>> Packages being installed or not is an image level decision, not distro
>>> level.
>> Yes, that's also my understanding. But enabling some features (e.g. x11
>> support) leads to recipes set DEPENDS on libs and this leads to the
>> installation of some packages. So DISTRO_FEATURES influence indirectly what
>> gets into the image.
> The key word is indirectly. Where things have a hard dependency on
> something, they do include that dependency and it isn't conditional on
> a distro feature.
>
> In this case, the usage is tangential to what the DISTRO_FEATURE was
> designed for (ld cache). I do agree ldconfig is optional so RRECOMMENDS
> seems better than a hard RDEPENDS though.

I will send a patch to update RDEPENDS -> RRECOMMENDS

Thanks

Changqing

>
>>> In this case, perhaps we should drop the RDEPENDS to a RRECOMMENDS?
>> This would be helpful to remove ldconfig, if django is installed.
>>
>> But the main question is still open: Why should I want ldconfig and remove
>> it from DISTRO_FEATURES? What is this use case?
> It is being used to control whether the ld cache is being used or not.
> That has to be done at a distro wide policy level. Whether ldconfig
> makes it into the image or not is an indirect effect.
>
> We can control the inclusion of ldconfig for ctypes at a package/image
> level, it isn't a distro config level issue.
>
> Cheers,
>
> Richard
>
>
diff mbox series

Patch

diff --git a/meta/recipes-devtools/python/python3_3.13.2.bb b/meta/recipes-devtools/python/python3_3.13.2.bb
index 52fac76c00..f64d67da89 100644
--- a/meta/recipes-devtools/python/python3_3.13.2.bb
+++ b/meta/recipes-devtools/python/python3_3.13.2.bb
@@ -477,9 +477,7 @@  FILES:${PN}-man = "${datadir}/man"
 
 # See https://bugs.python.org/issue18748 and https://bugs.python.org/issue37395
 RDEPENDS:libpython3:append:libc-glibc = " libgcc"
-RDEPENDS:${PN}-ctypes:append:libc-glibc = "\
-    ${@bb.utils.contains('DISTRO_FEATURES', 'ldconfig', '${MLPREFIX}ldconfig', '', d)} \
-"
+RDEPENDS:${PN}-ctypes:append:libc-glibc = " ${MLPREFIX}ldconfig"
 RDEPENDS:${PN}-ptest = "\
     ${PN}-dev \
     ${PN}-modules \