| Message ID | 20240922073618.1359707-1-yi.zhao@windriver.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [meta-oe,1/2] protobuf: add abseil-cpp to RDEPENDS | expand |
> -----Original Message----- > From: openembedded-devel@lists.openembedded.org <openembedded-devel@lists.openembedded.org> On Behalf Of Yi Zhao > Sent: den 22 september 2024 09:36 > To: openembedded-devel@lists.openembedded.org > Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS > > Since commit[1], the libraries of abseil-cpp have been splitted into > separate packages. When protobuf is installed into the image, only 48 > abseil libraries are installed as runtime dependencies. But the output > of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > required. Add abseil-cpp to RDEPENDS to ensure that all required > abseil-cpp libraries are installed. > > [1] https://git.openembedded.org/meta-openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035ecc > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > --- > meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > index cc5204de4..354b3213a 100644 > --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}" > # CMake requires protoc binary to exist in sysroot, even if it has wrong architecture. > SYSROOT_DIRS += "${bindir}" > > +RDEPENDS:${PN} = "abseil-cpp" > +RDEPENDS:${PN}-lite = "abseil-cpp" This defeats the whole purpose of splitting up the abseil libraries. What triggered this change? Was it an actual runtime error, or were you only looking at the output from pkg-config --libs protobuf? Since the .pc files are in the protobuf-dev package, which depends on the abseil-cpp-dev package, all abseil libraries should be available if you are actually building on target or with an SDK. > RDEPENDS:${PN}-compiler = "${PN}" > RDEPENDS:${PN}-dev += "${PN}-compiler" > RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}" > -- > 2.25.1 //Peter
On 9/24/24 03:02, Peter Kjellerstedt wrote: >> -----Original Message----- >> From: openembedded-devel@lists.openembedded.org <openembedded-devel@lists.openembedded.org> On Behalf Of Yi Zhao >> Sent: den 22 september 2024 09:36 >> To: openembedded-devel@lists.openembedded.org >> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS >> >> Since commit[1], the libraries of abseil-cpp have been splitted into >> separate packages. When protobuf is installed into the image, only 48 >> abseil libraries are installed as runtime dependencies. But the output >> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are >> required. Add abseil-cpp to RDEPENDS to ensure that all required >> abseil-cpp libraries are installed. >> >> [1] https://git.openembedded.org/meta-openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035ecc >> >> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> >> --- >> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >> index cc5204de4..354b3213a 100644 >> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}" >> # CMake requires protoc binary to exist in sysroot, even if it has wrong architecture. >> SYSROOT_DIRS += "${bindir}" >> >> +RDEPENDS:${PN} = "abseil-cpp" >> +RDEPENDS:${PN}-lite = "abseil-cpp" > This defeats the whole purpose of splitting up the abseil > libraries. What triggered this change? Was it an actual > runtime error, or were you only looking at the output > from pkg-config --libs protobuf? > > Since the .pc files are in the protobuf-dev package, which > depends on the abseil-cpp-dev package, all abseil libraries > should be available if you are actually building on target > or with an SDK. Our customer reported this issue. Here is his description: The abseil libraries will be included in the SDK, but not on the target. The application is building with SDK. The yocto build is adding protobuf to the image (no explicit abseil libraries are added). If an application then are built using the SDK it can not be executed on target as libraries are missing. //Yi > >> RDEPENDS:${PN}-compiler = "${PN}" >> RDEPENDS:${PN}-dev += "${PN}-compiler" >> RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}" >> -- >> 2.25.1 > //Peter >
> -----Original Message----- > From: Yi Zhao <yi.zhao@windriver.com> > Sent: den 25 september 2024 04:17 > To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao <yi.zhao@eng.windriver.com> > Cc: openembedded-devel@lists.openembedded.org > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS > > > On 9/24/24 03:02, Peter Kjellerstedt wrote: > >> -----Original Message----- > >> From: openembedded-devel@lists.openembedded.org <openembedded-devel@lists.openembedded.org> On Behalf Of Yi Zhao > >> Sent: den 22 september 2024 09:36 > >> To: openembedded-devel@lists.openembedded.org > >> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS > >> > >> Since commit[1], the libraries of abseil-cpp have been splitted into > >> separate packages. When protobuf is installed into the image, only 48 > >> abseil libraries are installed as runtime dependencies. But the output > >> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > >> required. Add abseil-cpp to RDEPENDS to ensure that all required > >> abseil-cpp libraries are installed. > >> > >> [1] https://git.openembedded.org/meta-openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035ecc > >> > >> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > >> --- > >> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > >> 1 file changed, 2 insertions(+) > >> > >> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >> index cc5204de4..354b3213a 100644 > >> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}" > >> # CMake requires protoc binary to exist in sysroot, even if it has wrong architecture. > >> SYSROOT_DIRS += "${bindir}" > >> > >> +RDEPENDS:${PN} = "abseil-cpp" > >> +RDEPENDS:${PN}-lite = "abseil-cpp" > > > > This defeats the whole purpose of splitting up the abseil > > libraries. What triggered this change? Was it an actual > > runtime error, or were you only looking at the output > > from pkg-config --libs protobuf? > > > > Since the .pc files are in the protobuf-dev package, which > > depends on the abseil-cpp-dev package, all abseil libraries > > should be available if you are actually building on target > > or with an SDK. > > Our customer reported this issue. Here is his description: > The abseil libraries will be included in the SDK, but not on the target. > The application is building with SDK. The yocto build is adding protobuf > to the image (no explicit abseil libraries are added). If an application > then are built using the SDK it can not be executed on target as > libraries are missing. Can we at least change the dependency to a recommendation so that it can be avoided via BAD_RECOMMENDATIONS? Because as it is, it penalizes everyone that use protobuf for the case where one provides an SDK with protobuf. My suggestion would be to change it to: # Recommend abseil-cpp so that all its libraries are installed in case # protobuf is provided as part of an SDK. Everyone else should add # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. RRECOMMENDS:${PN} = "abseil-cpp" RRECOMMENDS:${PN}-lite = "abseil-cpp" > //Yi > > >> RDEPENDS:${PN}-compiler = "${PN}" > >> RDEPENDS:${PN}-dev += "${PN}-compiler" > >> RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}" > >> -- > >> 2.25.1 > > //Peter //Peter
> -----Original Message----- > From: openembedded-devel@lists.openembedded.org <openembedded- > devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via > lists.openembedded.org > Sent: Tuesday, October 1, 2024 14:00 > To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao <yi.zhao@eng.windriver.com> > Cc: openembedded-devel@lists.openembedded.org > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > RDEPENDS > > > -----Original Message----- > > From: Yi Zhao <yi.zhao@windriver.com> > > Sent: den 25 september 2024 04:17 > > To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao > <yi.zhao@eng.windriver.com> > > Cc: openembedded-devel@lists.openembedded.org > > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > RDEPENDS > > > > > > On 9/24/24 03:02, Peter Kjellerstedt wrote: > > >> -----Original Message----- > > >> From: openembedded-devel@lists.openembedded.org <openembedded- > devel@lists.openembedded.org> On Behalf Of Yi Zhao > > >> Sent: den 22 september 2024 09:36 > > >> To: openembedded-devel@lists.openembedded.org > > >> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > RDEPENDS > > >> > > >> Since commit[1], the libraries of abseil-cpp have been splitted into > > >> separate packages. When protobuf is installed into the image, only 48 > > >> abseil libraries are installed as runtime dependencies. But the output > > >> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > > >> required. Add abseil-cpp to RDEPENDS to ensure that all required > > >> abseil-cpp libraries are installed. > > >> > > >> [1] https://git.openembedded.org/meta- > openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > ecc > > >> > > >> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > >> --- > > >> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > >> 1 file changed, 2 insertions(+) > > >> > > >> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >> index cc5204de4..354b3213a 100644 > > >> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf- > lite${SOLIBS}" > > >> # CMake requires protoc binary to exist in sysroot, even if it has wrong > architecture. > > >> SYSROOT_DIRS += "${bindir}" > > >> > > >> +RDEPENDS:${PN} = "abseil-cpp" > > >> +RDEPENDS:${PN}-lite = "abseil-cpp" > > > > > > This defeats the whole purpose of splitting up the abseil > > > libraries. What triggered this change? Was it an actual > > > runtime error, or were you only looking at the output > > > from pkg-config --libs protobuf? > > > > > > Since the .pc files are in the protobuf-dev package, which > > > depends on the abseil-cpp-dev package, all abseil libraries > > > should be available if you are actually building on target > > > or with an SDK. > > > > Our customer reported this issue. Here is his description: > > The abseil libraries will be included in the SDK, but not on the target. > > The application is building with SDK. The yocto build is adding protobuf > > to the image (no explicit abseil libraries are added). If an application > > then are built using the SDK it can not be executed on target as > > libraries are missing. > > Can we at least change the dependency to a recommendation so > that it can be avoided via BAD_RECOMMENDATIONS? Because as it > is, it penalizes everyone that use protobuf for the case where > one provides an SDK with protobuf. > > My suggestion would be to change it to: > > # Recommend abseil-cpp so that all its libraries are installed in case > # protobuf is provided as part of an SDK. Everyone else should add > # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. > RRECOMMENDS:${PN} = "abseil-cpp" > RRECOMMENDS:${PN}-lite = "abseil-cpp" > I think that's also not the right way to do this. Adding unnecessary stuff to release images to satisfy SDK needs is wrong. I also wonder if the problem is not at the SDK level. If you build an sdk, there is explicitly installed package base and additional required dependencies of those packages. The dependencies may not be complete and fully usable, only their parts sufficient for the explicitly installed stuff is present. Are the sdk users assuming that full abseil-cpp is installed and thus using also parts not needed for protobuf? The proper solution would be probably to include abseil-cpp explicitly in SDK used in that way. Peter > > //Yi > > > > >> RDEPENDS:${PN}-compiler = "${PN}" > > >> RDEPENDS:${PN}-dev += "${PN}-compiler" > > >> RDEPENDS:${PN}-ptest = "bash > ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}" > > >> -- > > >> 2.25.1 > > > //Peter > > //Peter
Hi Khem, I wonder why this patch has been applied although there were concerns about it: regards, Christian On Sunday, 22 September 2024, 09:36:17 CET, Yi Zhao wrote: > Since commit[1], the libraries of abseil-cpp have been splitted into > separate packages. When protobuf is installed into the image, only 48 > abseil libraries are installed as runtime dependencies. But the output > of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > required. Add abseil-cpp to RDEPENDS to ensure that all required > abseil-cpp libraries are installed. > > [1] https://git.openembedded.org/meta-openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035ecc > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > --- > meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > index cc5204de4..354b3213a 100644 > --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}" > # CMake requires protoc binary to exist in sysroot, even if it has wrong architecture. > SYSROOT_DIRS += "${bindir}" > > +RDEPENDS:${PN} = "abseil-cpp" > +RDEPENDS:${PN}-lite = "abseil-cpp" > RDEPENDS:${PN}-compiler = "${PN}" > RDEPENDS:${PN}-dev += "${PN}-compiler" > RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}" > On Tuesday, 1 October 2024, 13:59:56 CET, Peter Kjellerstedt wrote: > Can we at least change the dependency to a recommendation so > that it can be avoided via BAD_RECOMMENDATIONS? Because as it > is, it penalizes everyone that use protobuf for the case where > one provides an SDK with protobuf. 2On Tuesday, 1 October 2024, 14:42:04 CET, Peter Marko via lists.openembedded.org wrote: > I think that's also not the right way to do this. > Adding unnecessary stuff to release images to satisfy SDK needs is wrong.
Hmm I thought I gave some feedback to author but there is no follow up can you send a revert or perhaps the recommends suggestion On Wed, Nov 6, 2024 at 7:54 AM Christian Eggers <ceggers@arri.de> wrote: > Hi Khem, > > I wonder why this patch has been applied although there were concerns > about it: > > regards, > Christian > > On Sunday, 22 September 2024, 09:36:17 CET, Yi Zhao wrote: > > Since commit[1], the libraries of abseil-cpp have been splitted into > > separate packages. When protobuf is installed into the image, only 48 > > abseil libraries are installed as runtime dependencies. But the output > > of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > > required. Add abseil-cpp to RDEPENDS to ensure that all required > > abseil-cpp libraries are installed. > > > > [1] > https://git.openembedded.org/meta-openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035ecc > > > > Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > --- > > meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > index cc5204de4..354b3213a 100644 > > --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > @@ -106,6 +106,8 @@ FILES:${PN}-lite = > "${libdir}/libprotobuf-lite${SOLIBS}" > > # CMake requires protoc binary to exist in sysroot, even if it has > wrong architecture. > > SYSROOT_DIRS += "${bindir}" > > > > +RDEPENDS:${PN} = "abseil-cpp" > > +RDEPENDS:${PN}-lite = "abseil-cpp" > > RDEPENDS:${PN}-compiler = "${PN}" > > RDEPENDS:${PN}-dev += "${PN}-compiler" > > RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', > 'python', 'python3-protobuf', '', d)}" > > > > > On Tuesday, 1 October 2024, 13:59:56 CET, Peter Kjellerstedt wrote: > > Can we at least change the dependency to a recommendation so > > that it can be avoided via BAD_RECOMMENDATIONS? Because as it > > is, it penalizes everyone that use protobuf for the case where > > one provides an SDK with protobuf. > > > 2On Tuesday, 1 October 2024, 14:42:04 CET, Peter Marko via > lists.openembedded.org wrote: > > I think that's also not the right way to do this. > > Adding unnecessary stuff to release images to satisfy SDK needs is wrong. > > > >
I'll explain in more details what the problem is and why this patch is
correct, given the current status of abseil-cpp recipe.
In protobuf's abseil-cpp.cmake file, there's a hardcoded list of absl
targets used by protobuf:
https://github.com/protocolbuffers/protobuf/blob/main/cmake/abseil-cpp.cmake#L56
This value in turn is parsed and used to replace the
@_protobuf_PC_REQUIRES@ in protobuf.pc.cmake.
Here's the generated probobuf.pc file's contents:
"""
$ cat ../build/protobuf.pc
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib64
includedir=/usr/include
Name: Protocol Buffers
Description: Google's Data Interchange Format
Version: 25.3.0
Requires: absl_absl_check absl_absl_log absl_algorithm absl_base
absl_bind_front absl_bits absl_btree absl_cleanup absl_cord
absl_core_headers absl_debugging absl_die_if_null
absl_dynamic_annotations absl_flags absl_flat_hash_map
absl_flat_hash_set absl_function_ref absl_hash absl_layout
absl_log_initialize absl_log_severity absl_memory absl_node_hash_map
absl_node_hash_set absl_optional absl_span absl_status absl_statusor
absl_strings absl_synchronization absl_time absl_type_traits
absl_utility absl_variant utf8_range
Libs: -L${libdir} -lprotobuf
Cflags: -I${includedir} -DPROTOBUF_USE_DLLS
Conflicts: protobuf-lite
"""
Let me use "absl_hash" and "absl_absl_log" in the above list as examples.
"""
$ find ../../../abseil-cpp/20240116.2/packa
ges-split/ -name "libabsl_hash.so.*"
../../../abseil-cpp/20240116.2/packages-split/libabsl-hash/usr/lib64/libabsl_hash.so.2401.0.0
../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0
$ find ../../../abseil-cpp/20240116.2/packa
ges-split/ -name "libabsl_absl_log.so.*"
$
"""
See? The current package splitting of abseil-cpp does not match what
protobuf assumes. And I thinks protobuf's assumption is correct, because
there's really a absl_absl_log.pc file generated by abseil-cpp.
And there's the contents and location of absl_absl_log.pc:
"""
$ cat
../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_absl_log.pc
prefix=/usr
exec_prefix=${prefix}
libdir=/usr/lib64
includedir=/usr/include
Name: absl_absl_log
Description: Abseil absl_log library
URL: https://abseil.io/
Version: 20240116
Requires: absl_log_internal_log_impl = 20240116
Libs: -L${libdir}
Cflags: -I${includedir} -DNOMINMAX
"""
And there's the contents for the absl_log_internal_log_impl:
"""
$ cat ../../../abseil-cpp/20240116.2/packag
es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_log_impl.pc
prefix=/usr
exec_prefix=${prefix}
libdir=/usr/lib64
includedir=/usr/include
Name: absl_log_internal_log_impl
Description: Abseil log_internal_log_impl library
URL: https://abseil.io/
Version: 20240116
Requires: absl_log_internal_conditions = 20240116,
absl_log_internal_message = 20240116, absl_log_internal_strip =
20240116, absl_absl_vlog_is_on = 20240116
Libs: -L${libdir}
Cflags: -I${includedir} -DNOMINMAX
"""
We can then find the absl_log_internal_conditions library:
"""
$ find ../../../abseil-cpp/20240116.2/packages-split/ -name
"*absl_log_internal_message*"
../../../abseil-cpp/20240116.2/packages-split/libabsl-log-internal-message/usr/lib64/libabsl_log_internal_message.so.2401.0.0
../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/libabsl_log_internal_message.so
../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_message.pc
../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0
"""
What does all above mean? It means that abseil-cpp is not splitted
reasonably. This could be proven by the absl_absl_log.pc file as
described above.
So, given the current situation of abseil-cpp, Yi's solution is correct.
Before abseil-cpp is re-splitted according to its pc files, which then
could possibly lead to automatic parsing of the protobuf.pc file to add
extra RDEPENDS automatically in protobuf recipe, we need to have Yi's
solution. And we need to use RDEEPNDS instead of RRECOMMENDS (see the
'Requires' directive in corresponding .pc files).
Regards,
Qi
On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote:
>> -----Original Message-----
>> From: openembedded-devel@lists.openembedded.org <openembedded-
>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via
>> lists.openembedded.org
>> Sent: Tuesday, October 1, 2024 14:00
>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao <yi.zhao@eng.windriver.com>
>> Cc: openembedded-devel@lists.openembedded.org
>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to
>> RDEPENDS
>>
>>> -----Original Message-----
>>> From: Yi Zhao <yi.zhao@windriver.com>
>>> Sent: den 25 september 2024 04:17
>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao
>> <yi.zhao@eng.windriver.com>
>>> Cc: openembedded-devel@lists.openembedded.org
>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to
>> RDEPENDS
>>>
>>> On 9/24/24 03:02, Peter Kjellerstedt wrote:
>>>>> -----Original Message-----
>>>>> From: openembedded-devel@lists.openembedded.org <openembedded-
>> devel@lists.openembedded.org> On Behalf Of Yi Zhao
>>>>> Sent: den 22 september 2024 09:36
>>>>> To: openembedded-devel@lists.openembedded.org
>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to
>> RDEPENDS
>>>>> Since commit[1], the libraries of abseil-cpp have been splitted into
>>>>> separate packages. When protobuf is installed into the image, only 48
>>>>> abseil libraries are installed as runtime dependencies. But the output
>>>>> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are
>>>>> required. Add abseil-cpp to RDEPENDS to ensure that all required
>>>>> abseil-cpp libraries are installed.
>>>>>
>>>>> [1] https://git.openembedded.org/meta-
>> openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035
>> ecc
>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
>>>>> ---
>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++
>>>>> 1 file changed, 2 insertions(+)
>>>>>
>>>>> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb
>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb
>>>>> index cc5204de4..354b3213a 100644
>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb
>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb
>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf-
>> lite${SOLIBS}"
>>>>> # CMake requires protoc binary to exist in sysroot, even if it has wrong
>> architecture.
>>>>> SYSROOT_DIRS += "${bindir}"
>>>>>
>>>>> +RDEPENDS:${PN} = "abseil-cpp"
>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp"
>>>> This defeats the whole purpose of splitting up the abseil
>>>> libraries. What triggered this change? Was it an actual
>>>> runtime error, or were you only looking at the output
>>>> from pkg-config --libs protobuf?
>>>>
>>>> Since the .pc files are in the protobuf-dev package, which
>>>> depends on the abseil-cpp-dev package, all abseil libraries
>>>> should be available if you are actually building on target
>>>> or with an SDK.
>>> Our customer reported this issue. Here is his description:
>>> The abseil libraries will be included in the SDK, but not on the target.
>>> The application is building with SDK. The yocto build is adding protobuf
>>> to the image (no explicit abseil libraries are added). If an application
>>> then are built using the SDK it can not be executed on target as
>>> libraries are missing.
>> Can we at least change the dependency to a recommendation so
>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it
>> is, it penalizes everyone that use protobuf for the case where
>> one provides an SDK with protobuf.
>>
>> My suggestion would be to change it to:
>>
>> # Recommend abseil-cpp so that all its libraries are installed in case
>> # protobuf is provided as part of an SDK. Everyone else should add
>> # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration.
>> RRECOMMENDS:${PN} = "abseil-cpp"
>> RRECOMMENDS:${PN}-lite = "abseil-cpp"
>>
> I think that's also not the right way to do this.
> Adding unnecessary stuff to release images to satisfy SDK needs is wrong.
>
> I also wonder if the problem is not at the SDK level.
> If you build an sdk, there is explicitly installed package base and additional required dependencies of those packages.
> The dependencies may not be complete and fully usable, only their parts sufficient for the explicitly installed stuff is present.
> Are the sdk users assuming that full abseil-cpp is installed and thus using also parts not needed for protobuf?
> The proper solution would be probably to include abseil-cpp explicitly in SDK used in that way.
>
> Peter
>
>>> //Yi
>>>
>>>>> RDEPENDS:${PN}-compiler = "${PN}"
>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler"
>>>>> RDEPENDS:${PN}-ptest = "bash
>> ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}"
>>>>> --
>>>>> 2.25.1
>>>> //Peter
>> //Peter
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#112631): https://lists.openembedded.org/g/openembedded-devel/message/112631
> Mute This Topic: https://lists.openembedded.org/mt/108587631/7304865
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [Qi.Chen@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Before we rework the abseil-cpp and protobuf recipes, a possible improvement for this patch is that we list the needed libraries in RDEPENDS manually. Regards, Qi On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > I'll explain in more details what the problem is and why this patch is > correct, given the current status of abseil-cpp recipe. > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of absl > targets used by protobuf: > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abseil-cpp.cmake#L56 > > > This value in turn is parsed and used to replace the > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > Here's the generated probobuf.pc file's contents: > """ > $ cat ../build/protobuf.pc > prefix=/usr > exec_prefix=/usr > libdir=/usr/lib64 > includedir=/usr/include > > Name: Protocol Buffers > Description: Google's Data Interchange Format > Version: 25.3.0 > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > absl_core_headers absl_debugging absl_die_if_null > absl_dynamic_annotations absl_flags absl_flat_hash_map > absl_flat_hash_set absl_function_ref absl_hash absl_layout > absl_log_initialize absl_log_severity absl_memory absl_node_hash_map > absl_node_hash_set absl_optional absl_span absl_status absl_statusor > absl_strings absl_synchronization absl_time absl_type_traits > absl_utility absl_variant utf8_range > Libs: -L${libdir} -lprotobuf > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > Conflicts: protobuf-lite > """ > > Let me use "absl_hash" and "absl_absl_log" in the above list as examples. > > """ > $ find ../../../abseil-cpp/20240116.2/packa > ges-split/ -name "libabsl_hash.so.*" > ../../../abseil-cpp/20240116.2/packages-split/libabsl-hash/usr/lib64/libabsl_hash.so.2401.0.0 > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > $ find ../../../abseil-cpp/20240116.2/packa > ges-split/ -name "libabsl_absl_log.so.*" > $ > """ > > See? The current package splitting of abseil-cpp does not match what > protobuf assumes. And I thinks protobuf's assumption is correct, > because there's really a absl_absl_log.pc file generated by abseil-cpp. > > And there's the contents and location of absl_absl_log.pc: > """ > $ cat > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_absl_log.pc > prefix=/usr > exec_prefix=${prefix} > libdir=/usr/lib64 > includedir=/usr/include > > Name: absl_absl_log > Description: Abseil absl_log library > URL: https://abseil.io/ > Version: 20240116 > Requires: absl_log_internal_log_impl = 20240116 > Libs: -L${libdir} > Cflags: -I${includedir} -DNOMINMAX > """ > > And there's the contents for the absl_log_internal_log_impl: > """ > $ cat ../../../abseil-cpp/20240116.2/packag > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_log_impl.pc > prefix=/usr > exec_prefix=${prefix} > libdir=/usr/lib64 > includedir=/usr/include > > Name: absl_log_internal_log_impl > Description: Abseil log_internal_log_impl library > URL: https://abseil.io/ > Version: 20240116 > Requires: absl_log_internal_conditions = 20240116, > absl_log_internal_message = 20240116, absl_log_internal_strip = > 20240116, absl_absl_vlog_is_on = 20240116 > Libs: -L${libdir} > Cflags: -I${includedir} -DNOMINMAX > """ > > We can then find the absl_log_internal_conditions library: > """ > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > "*absl_log_internal_message*" > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-internal-message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/libabsl_log_internal_message.so > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > """ > > What does all above mean? It means that abseil-cpp is not splitted > reasonably. This could be proven by the absl_absl_log.pc file as > described above. > > So, given the current situation of abseil-cpp, Yi's solution is > correct. Before abseil-cpp is re-splitted according to its pc files, > which then could possibly lead to automatic parsing of the protobuf.pc > file to add extra RDEPENDS automatically in protobuf recipe, we need > to have Yi's solution. And we need to use RDEEPNDS instead of > RRECOMMENDS (see the 'Requires' directive in corresponding .pc files). > > Regards, > Qi > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: >>> -----Original Message----- >>> From: openembedded-devel@lists.openembedded.org <openembedded- >>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via >>> lists.openembedded.org >>> Sent: Tuesday, October 1, 2024 14:00 >>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao >>> <yi.zhao@eng.windriver.com> >>> Cc: openembedded-devel@lists.openembedded.org >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to >>> RDEPENDS >>> >>>> -----Original Message----- >>>> From: Yi Zhao <yi.zhao@windriver.com> >>>> Sent: den 25 september 2024 04:17 >>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao >>> <yi.zhao@eng.windriver.com> >>>> Cc: openembedded-devel@lists.openembedded.org >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to >>> RDEPENDS >>>> >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: >>>>>> -----Original Message----- >>>>>> From: openembedded-devel@lists.openembedded.org <openembedded- >>> devel@lists.openembedded.org> On Behalf Of Yi Zhao >>>>>> Sent: den 22 september 2024 09:36 >>>>>> To: openembedded-devel@lists.openembedded.org >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to >>> RDEPENDS >>>>>> Since commit[1], the libraries of abseil-cpp have been splitted into >>>>>> separate packages. When protobuf is installed into the image, >>>>>> only 48 >>>>>> abseil libraries are installed as runtime dependencies. But the >>>>>> output >>>>>> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are >>>>>> required. Add abseil-cpp to RDEPENDS to ensure that all required >>>>>> abseil-cpp libraries are installed. >>>>>> >>>>>> [1] https://git.openembedded.org/meta- >>> openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 >>> ecc >>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> >>>>>> --- >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ >>>>>> 1 file changed, 2 insertions(+) >>>>>> >>>>>> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >>>>>> index cc5204de4..354b3213a 100644 >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf- >>> lite${SOLIBS}" >>>>>> # CMake requires protoc binary to exist in sysroot, even if it >>>>>> has wrong >>> architecture. >>>>>> SYSROOT_DIRS += "${bindir}" >>>>>> >>>>>> +RDEPENDS:${PN} = "abseil-cpp" >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" >>>>> This defeats the whole purpose of splitting up the abseil >>>>> libraries. What triggered this change? Was it an actual >>>>> runtime error, or were you only looking at the output >>>>> from pkg-config --libs protobuf? >>>>> >>>>> Since the .pc files are in the protobuf-dev package, which >>>>> depends on the abseil-cpp-dev package, all abseil libraries >>>>> should be available if you are actually building on target >>>>> or with an SDK. >>>> Our customer reported this issue. Here is his description: >>>> The abseil libraries will be included in the SDK, but not on the >>>> target. >>>> The application is building with SDK. The yocto build is adding >>>> protobuf >>>> to the image (no explicit abseil libraries are added). If an >>>> application >>>> then are built using the SDK it can not be executed on target as >>>> libraries are missing. >>> Can we at least change the dependency to a recommendation so >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it >>> is, it penalizes everyone that use protobuf for the case where >>> one provides an SDK with protobuf. >>> >>> My suggestion would be to change it to: >>> >>> # Recommend abseil-cpp so that all its libraries are installed in case >>> # protobuf is provided as part of an SDK. Everyone else should add >>> # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. >>> RRECOMMENDS:${PN} = "abseil-cpp" >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" >>> >> I think that's also not the right way to do this. >> Adding unnecessary stuff to release images to satisfy SDK needs is >> wrong. >> >> I also wonder if the problem is not at the SDK level. >> If you build an sdk, there is explicitly installed package base and >> additional required dependencies of those packages. >> The dependencies may not be complete and fully usable, only their >> parts sufficient for the explicitly installed stuff is present. >> Are the sdk users assuming that full abseil-cpp is installed and thus >> using also parts not needed for protobuf? >> The proper solution would be probably to include abseil-cpp >> explicitly in SDK used in that way. >> >> Peter >> >>>> //Yi >>>> >>>>>> RDEPENDS:${PN}-compiler = "${PN}" >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" >>>>>> RDEPENDS:${PN}-ptest = "bash >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', >>> '', d)}" >>>>>> -- >>>>>> 2.25.1 >>>>> //Peter >>> //Peter >> >> >> > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#114133): https://lists.openembedded.org/g/openembedded-devel/message/114133 > Mute This Topic: https://lists.openembedded.org/mt/108587631/3618072 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [Qi.Chen@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hi Qi, On Monday, 2 December 2024, 06:35:22 CET, Chen Qi via lists.openembedded.org wrote: > Before we rework the abseil-cpp and protobuf recipes, a possible > improvement for this patch is that we list the needed libraries in > RDEPENDS manually. my main concern was (as you already may have guessed) not having the full set of absl libraries in my rootfs, as only some of them are required. But I cannot tell which is the best option to achieve this. regards, Christian > > Regards, > Qi > > On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > > I'll explain in more details what the problem is and why this patch is > > correct, given the current status of abseil-cpp recipe. > > > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of absl > > targets used by protobuf: > > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abseil-cpp.cmake#L56 > > > > > > This value in turn is parsed and used to replace the > > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > > > Here's the generated probobuf.pc file's contents: > > """ > > $ cat ../build/protobuf.pc > > prefix=/usr > > exec_prefix=/usr > > libdir=/usr/lib64 > > includedir=/usr/include > > > > Name: Protocol Buffers > > Description: Google's Data Interchange Format > > Version: 25.3.0 > > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > > absl_core_headers absl_debugging absl_die_if_null > > absl_dynamic_annotations absl_flags absl_flat_hash_map > > absl_flat_hash_set absl_function_ref absl_hash absl_layout > > absl_log_initialize absl_log_severity absl_memory absl_node_hash_map > > absl_node_hash_set absl_optional absl_span absl_status absl_statusor > > absl_strings absl_synchronization absl_time absl_type_traits > > absl_utility absl_variant utf8_range > > Libs: -L${libdir} -lprotobuf > > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > > Conflicts: protobuf-lite > > """ > > > > Let me use "absl_hash" and "absl_absl_log" in the above list as examples. > > > > """ > > $ find ../../../abseil-cpp/20240116.2/packa > > ges-split/ -name "libabsl_hash.so.*" > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-hash/usr/lib64/libabsl_hash.so.2401.0.0 > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > > > $ find ../../../abseil-cpp/20240116.2/packa > > ges-split/ -name "libabsl_absl_log.so.*" > > $ > > """ > > > > See? The current package splitting of abseil-cpp does not match what > > protobuf assumes. And I thinks protobuf's assumption is correct, > > because there's really a absl_absl_log.pc file generated by abseil-cpp. > > > > And there's the contents and location of absl_absl_log.pc: > > """ > > $ cat > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_absl_log.pc > > prefix=/usr > > exec_prefix=${prefix} > > libdir=/usr/lib64 > > includedir=/usr/include > > > > Name: absl_absl_log > > Description: Abseil absl_log library > > URL: https://abseil.io/ > > Version: 20240116 > > Requires: absl_log_internal_log_impl = 20240116 > > Libs: -L${libdir} > > Cflags: -I${includedir} -DNOMINMAX > > """ > > > > And there's the contents for the absl_log_internal_log_impl: > > """ > > $ cat ../../../abseil-cpp/20240116.2/packag > > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_log_impl.pc > > prefix=/usr > > exec_prefix=${prefix} > > libdir=/usr/lib64 > > includedir=/usr/include > > > > Name: absl_log_internal_log_impl > > Description: Abseil log_internal_log_impl library > > URL: https://abseil.io/ > > Version: 20240116 > > Requires: absl_log_internal_conditions = 20240116, > > absl_log_internal_message = 20240116, absl_log_internal_strip = > > 20240116, absl_absl_vlog_is_on = 20240116 > > Libs: -L${libdir} > > Cflags: -I${includedir} -DNOMINMAX > > """ > > > > We can then find the absl_log_internal_conditions library: > > """ > > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > > "*absl_log_internal_message*" > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-internal-message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/libabsl_log_internal_message.so > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp-dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > > > """ > > > > What does all above mean? It means that abseil-cpp is not splitted > > reasonably. This could be proven by the absl_absl_log.pc file as > > described above. > > > > So, given the current situation of abseil-cpp, Yi's solution is > > correct. Before abseil-cpp is re-splitted according to its pc files, > > which then could possibly lead to automatic parsing of the protobuf.pc > > file to add extra RDEPENDS automatically in protobuf recipe, we need > > to have Yi's solution. And we need to use RDEEPNDS instead of > > RRECOMMENDS (see the 'Requires' directive in corresponding .pc files). > > > > Regards, > > Qi > > > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: > >>> -----Original Message----- > >>> From: openembedded-devel@lists.openembedded.org <openembedded- > >>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via > >>> lists.openembedded.org > >>> Sent: Tuesday, October 1, 2024 14:00 > >>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > >>> <yi.zhao@eng.windriver.com> > >>> Cc: openembedded-devel@lists.openembedded.org > >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > >>> RDEPENDS > >>> > >>>> -----Original Message----- > >>>> From: Yi Zhao <yi.zhao@windriver.com> > >>>> Sent: den 25 september 2024 04:17 > >>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao > >>> <yi.zhao@eng.windriver.com> > >>>> Cc: openembedded-devel@lists.openembedded.org > >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > >>> RDEPENDS > >>>> > >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: > >>>>>> -----Original Message----- > >>>>>> From: openembedded-devel@lists.openembedded.org <openembedded- > >>> devel@lists.openembedded.org> On Behalf Of Yi Zhao > >>>>>> Sent: den 22 september 2024 09:36 > >>>>>> To: openembedded-devel@lists.openembedded.org > >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > >>> RDEPENDS > >>>>>> Since commit[1], the libraries of abseil-cpp have been splitted into > >>>>>> separate packages. When protobuf is installed into the image, > >>>>>> only 48 > >>>>>> abseil libraries are installed as runtime dependencies. But the > >>>>>> output > >>>>>> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > >>>>>> required. Add abseil-cpp to RDEPENDS to ensure that all required > >>>>>> abseil-cpp libraries are installed. > >>>>>> > >>>>>> [1] https://git.openembedded.org/meta- > >>> openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > >>> ecc > >>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > >>>>>> --- > >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > >>>>>> 1 file changed, 2 insertions(+) > >>>>>> > >>>>>> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >>>>>> index cc5204de4..354b3213a 100644 > >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf- > >>> lite${SOLIBS}" > >>>>>> # CMake requires protoc binary to exist in sysroot, even if it > >>>>>> has wrong > >>> architecture. > >>>>>> SYSROOT_DIRS += "${bindir}" > >>>>>> > >>>>>> +RDEPENDS:${PN} = "abseil-cpp" > >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" > >>>>> This defeats the whole purpose of splitting up the abseil > >>>>> libraries. What triggered this change? Was it an actual > >>>>> runtime error, or were you only looking at the output > >>>>> from pkg-config --libs protobuf? > >>>>> > >>>>> Since the .pc files are in the protobuf-dev package, which > >>>>> depends on the abseil-cpp-dev package, all abseil libraries > >>>>> should be available if you are actually building on target > >>>>> or with an SDK. > >>>> Our customer reported this issue. Here is his description: > >>>> The abseil libraries will be included in the SDK, but not on the > >>>> target. > >>>> The application is building with SDK. The yocto build is adding > >>>> protobuf > >>>> to the image (no explicit abseil libraries are added). If an > >>>> application > >>>> then are built using the SDK it can not be executed on target as > >>>> libraries are missing. > >>> Can we at least change the dependency to a recommendation so > >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it > >>> is, it penalizes everyone that use protobuf for the case where > >>> one provides an SDK with protobuf. > >>> > >>> My suggestion would be to change it to: > >>> > >>> # Recommend abseil-cpp so that all its libraries are installed in case > >>> # protobuf is provided as part of an SDK. Everyone else should add > >>> # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. > >>> RRECOMMENDS:${PN} = "abseil-cpp" > >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" > >>> > >> I think that's also not the right way to do this. > >> Adding unnecessary stuff to release images to satisfy SDK needs is > >> wrong. > >> > >> I also wonder if the problem is not at the SDK level. > >> If you build an sdk, there is explicitly installed package base and > >> additional required dependencies of those packages. > >> The dependencies may not be complete and fully usable, only their > >> parts sufficient for the explicitly installed stuff is present. > >> Are the sdk users assuming that full abseil-cpp is installed and thus > >> using also parts not needed for protobuf? > >> The proper solution would be probably to include abseil-cpp > >> explicitly in SDK used in that way. > >> > >> Peter > >> > >>>> //Yi > >>>> > >>>>>> RDEPENDS:${PN}-compiler = "${PN}" > >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" > >>>>>> RDEPENDS:${PN}-ptest = "bash > >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', > >>> '', d)}" > >>>>> //Peter > >>> //Peter > >> > >> > >> > > > > > > > > > >
Hello, > -----Original Message----- > From: Christian Eggers <ceggers@arri.de> > Sent: Monday, December 2, 2024 14:39 > To: Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com>; > peter.kjellerstedt@axis.com; Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > <yi.zhao@eng.windriver.com>; openembedded- > devel@lists.openembedded.org > Cc: openembedded-devel@lists.openembedded.org; Qi.Chen@windriver.com > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > RDEPENDS > > Hi Qi, > > On Monday, 2 December 2024, 06:35:22 CET, Chen Qi via > lists.openembedded.org wrote: > > Before we rework the abseil-cpp and protobuf recipes, a possible > > improvement for this patch is that we list the needed libraries in > > RDEPENDS manually. > > my main concern was (as you already may have guessed) not having the full > set of > absl libraries in my rootfs, as only some of them are required. But I cannot tell > which is the best option to achieve this. If you want full abseil-cpp in your rootfs, please use IMAGE_INSTALL for that. Packages should pull only their real dependencies, not something you need for other reasons. You use-case seems to be to develop application with abseil-cpp library but not installing it explicitly. SDK images need to install (or depend on) additional packages compared to release images. Peter > > regards, > Christian > > > > > Regards, > > Qi > > > > On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > > > I'll explain in more details what the problem is and why this patch is > > > correct, given the current status of abseil-cpp recipe. > > > > > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of absl > > > targets used by protobuf: > > > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abseil- > cpp.cmake#L56 > > > > > > > > > This value in turn is parsed and used to replace the > > > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > > > > > Here's the generated probobuf.pc file's contents: > > > """ > > > $ cat ../build/protobuf.pc > > > prefix=/usr > > > exec_prefix=/usr > > > libdir=/usr/lib64 > > > includedir=/usr/include > > > > > > Name: Protocol Buffers > > > Description: Google's Data Interchange Format > > > Version: 25.3.0 > > > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > > > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > > > absl_core_headers absl_debugging absl_die_if_null > > > absl_dynamic_annotations absl_flags absl_flat_hash_map > > > absl_flat_hash_set absl_function_ref absl_hash absl_layout > > > absl_log_initialize absl_log_severity absl_memory absl_node_hash_map > > > absl_node_hash_set absl_optional absl_span absl_status absl_statusor > > > absl_strings absl_synchronization absl_time absl_type_traits > > > absl_utility absl_variant utf8_range > > > Libs: -L${libdir} -lprotobuf > > > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > > > Conflicts: protobuf-lite > > > """ > > > > > > Let me use "absl_hash" and "absl_absl_log" in the above list as examples. > > > > > > """ > > > $ find ../../../abseil-cpp/20240116.2/packa > > > ges-split/ -name "libabsl_hash.so.*" > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl- > hash/usr/lib64/libabsl_hash.so.2401.0.0 > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > ges-split/ -name "libabsl_absl_log.so.*" > > > $ > > > """ > > > > > > See? The current package splitting of abseil-cpp does not match what > > > protobuf assumes. And I thinks protobuf's assumption is correct, > > > because there's really a absl_absl_log.pc file generated by abseil-cpp. > > > > > > And there's the contents and location of absl_absl_log.pc: > > > """ > > > $ cat > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > dev/usr/lib64/pkgconfig/absl_absl_log.pc > > > prefix=/usr > > > exec_prefix=${prefix} > > > libdir=/usr/lib64 > > > includedir=/usr/include > > > > > > Name: absl_absl_log > > > Description: Abseil absl_log library > > > URL: https://abseil.io/ > > > Version: 20240116 > > > Requires: absl_log_internal_log_impl = 20240116 > > > Libs: -L${libdir} > > > Cflags: -I${includedir} -DNOMINMAX > > > """ > > > > > > And there's the contents for the absl_log_internal_log_impl: > > > """ > > > $ cat ../../../abseil-cpp/20240116.2/packag > > > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_log_impl.pc > > > prefix=/usr > > > exec_prefix=${prefix} > > > libdir=/usr/lib64 > > > includedir=/usr/include > > > > > > Name: absl_log_internal_log_impl > > > Description: Abseil log_internal_log_impl library > > > URL: https://abseil.io/ > > > Version: 20240116 > > > Requires: absl_log_internal_conditions = 20240116, > > > absl_log_internal_message = 20240116, absl_log_internal_strip = > > > 20240116, absl_absl_vlog_is_on = 20240116 > > > Libs: -L${libdir} > > > Cflags: -I${includedir} -DNOMINMAX > > > """ > > > > > > We can then find the absl_log_internal_conditions library: > > > """ > > > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > > > "*absl_log_internal_message*" > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-internal- > message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > dev/usr/lib64/libabsl_log_internal_message.so > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > > > > > """ > > > > > > What does all above mean? It means that abseil-cpp is not splitted > > > reasonably. This could be proven by the absl_absl_log.pc file as > > > described above. > > > > > > So, given the current situation of abseil-cpp, Yi's solution is > > > correct. Before abseil-cpp is re-splitted according to its pc files, > > > which then could possibly lead to automatic parsing of the protobuf.pc > > > file to add extra RDEPENDS automatically in protobuf recipe, we need > > > to have Yi's solution. And we need to use RDEEPNDS instead of > > > RRECOMMENDS (see the 'Requires' directive in corresponding .pc files). > > > > > > Regards, > > > Qi > > > > > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: > > >>> -----Original Message----- > > >>> From: openembedded-devel@lists.openembedded.org > <openembedded- > > >>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via > > >>> lists.openembedded.org > > >>> Sent: Tuesday, October 1, 2024 14:00 > > >>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > > >>> <yi.zhao@eng.windriver.com> > > >>> Cc: openembedded-devel@lists.openembedded.org > > >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > >>> RDEPENDS > > >>> > > >>>> -----Original Message----- > > >>>> From: Yi Zhao <yi.zhao@windriver.com> > > >>>> Sent: den 25 september 2024 04:17 > > >>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao > > >>> <yi.zhao@eng.windriver.com> > > >>>> Cc: openembedded-devel@lists.openembedded.org > > >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > >>> RDEPENDS > > >>>> > > >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: > > >>>>>> -----Original Message----- > > >>>>>> From: openembedded-devel@lists.openembedded.org > <openembedded- > > >>> devel@lists.openembedded.org> On Behalf Of Yi Zhao > > >>>>>> Sent: den 22 september 2024 09:36 > > >>>>>> To: openembedded-devel@lists.openembedded.org > > >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > >>> RDEPENDS > > >>>>>> Since commit[1], the libraries of abseil-cpp have been splitted into > > >>>>>> separate packages. When protobuf is installed into the image, > > >>>>>> only 48 > > >>>>>> abseil libraries are installed as runtime dependencies. But the > > >>>>>> output > > >>>>>> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > > >>>>>> required. Add abseil-cpp to RDEPENDS to ensure that all required > > >>>>>> abseil-cpp libraries are installed. > > >>>>>> > > >>>>>> [1] https://git.openembedded.org/meta- > > >>> > openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > > >>> ecc > > >>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > >>>>>> --- > > >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > >>>>>> 1 file changed, 2 insertions(+) > > >>>>>> > > >>>>>> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >>>>>> index cc5204de4..354b3213a 100644 > > >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf- > > >>> lite${SOLIBS}" > > >>>>>> # CMake requires protoc binary to exist in sysroot, even if it > > >>>>>> has wrong > > >>> architecture. > > >>>>>> SYSROOT_DIRS += "${bindir}" > > >>>>>> > > >>>>>> +RDEPENDS:${PN} = "abseil-cpp" > > >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" > > >>>>> This defeats the whole purpose of splitting up the abseil > > >>>>> libraries. What triggered this change? Was it an actual > > >>>>> runtime error, or were you only looking at the output > > >>>>> from pkg-config --libs protobuf? > > >>>>> > > >>>>> Since the .pc files are in the protobuf-dev package, which > > >>>>> depends on the abseil-cpp-dev package, all abseil libraries > > >>>>> should be available if you are actually building on target > > >>>>> or with an SDK. > > >>>> Our customer reported this issue. Here is his description: > > >>>> The abseil libraries will be included in the SDK, but not on the > > >>>> target. > > >>>> The application is building with SDK. The yocto build is adding > > >>>> protobuf > > >>>> to the image (no explicit abseil libraries are added). If an > > >>>> application > > >>>> then are built using the SDK it can not be executed on target as > > >>>> libraries are missing. > > >>> Can we at least change the dependency to a recommendation so > > >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it > > >>> is, it penalizes everyone that use protobuf for the case where > > >>> one provides an SDK with protobuf. > > >>> > > >>> My suggestion would be to change it to: > > >>> > > >>> # Recommend abseil-cpp so that all its libraries are installed in case > > >>> # protobuf is provided as part of an SDK. Everyone else should add > > >>> # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. > > >>> RRECOMMENDS:${PN} = "abseil-cpp" > > >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" > > >>> > > >> I think that's also not the right way to do this. > > >> Adding unnecessary stuff to release images to satisfy SDK needs is > > >> wrong. > > >> > > >> I also wonder if the problem is not at the SDK level. > > >> If you build an sdk, there is explicitly installed package base and > > >> additional required dependencies of those packages. > > >> The dependencies may not be complete and fully usable, only their > > >> parts sufficient for the explicitly installed stuff is present. > > >> Are the sdk users assuming that full abseil-cpp is installed and thus > > >> using also parts not needed for protobuf? > > >> The proper solution would be probably to include abseil-cpp > > >> explicitly in SDK used in that way. > > >> > > >> Peter > > >> > > >>>> //Yi > > >>>> > > >>>>>> RDEPENDS:${PN}-compiler = "${PN}" > > >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" > > >>>>>> RDEPENDS:${PN}-ptest = "bash > > >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', > > >>> '', d)}" > > >>>>> //Peter > > >>> //Peter > > >> > > >> > > >> > > > > > > > > > > > > > > > > > > >
Hi Peter, On Monday, 2 December 2024, 14:48:43 CET, Marko, Peter wrote: > Hello, > > > -----Original Message----- > > From: Christian Eggers <ceggers@arri.de> > > Sent: Monday, December 2, 2024 14:39 > > To: Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com>; > > peter.kjellerstedt@axis.com; Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > > <yi.zhao@eng.windriver.com>; openembedded- > > devel@lists.openembedded.org > > Cc: openembedded-devel@lists.openembedded.org; Qi.Chen@windriver.com > > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > RDEPENDS > > > > Hi Qi, > > > > On Monday, 2 December 2024, 06:35:22 CET, Chen Qi via > > lists.openembedded.org wrote: > > > Before we rework the abseil-cpp and protobuf recipes, a possible > > > improvement for this patch is that we list the needed libraries in > > > RDEPENDS manually. > > > > my main concern was (as you already may have guessed) not having the full > > set of > > absl libraries in my rootfs, as only some of them are required. But I cannot tell > > which is the best option to achieve this. > > If you want full abseil-cpp in your rootfs, please use IMAGE_INSTALL for that. > Packages should pull only their real dependencies, not something you need for other reasons. > You use-case seems to be to develop application with abseil-cpp library but not installing it explicitly. > SDK images need to install (or depend on) additional packages compared to release images. sorry, but I meant it the other way around: "I do not like having the full set of abseil-cpp libraries in my rootfs only because I use protobuf in my application". Sorry for the confusion. regards, Christian > > Peter > > > > > regards, > > Christian > > > > > > > > Regards, > > > Qi > > > > > > On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > > > > I'll explain in more details what the problem is and why this patch is > > > > correct, given the current status of abseil-cpp recipe. > > > > > > > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of absl > > > > targets used by protobuf: > > > > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abseil- > > cpp.cmake#L56 > > > > > > > > > > > > This value in turn is parsed and used to replace the > > > > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > > > > > > > Here's the generated probobuf.pc file's contents: > > > > """ > > > > $ cat ../build/protobuf.pc > > > > prefix=/usr > > > > exec_prefix=/usr > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: Protocol Buffers > > > > Description: Google's Data Interchange Format > > > > Version: 25.3.0 > > > > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > > > > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > > > > absl_core_headers absl_debugging absl_die_if_null > > > > absl_dynamic_annotations absl_flags absl_flat_hash_map > > > > absl_flat_hash_set absl_function_ref absl_hash absl_layout > > > > absl_log_initialize absl_log_severity absl_memory absl_node_hash_map > > > > absl_node_hash_set absl_optional absl_span absl_status absl_statusor > > > > absl_strings absl_synchronization absl_time absl_type_traits > > > > absl_utility absl_variant utf8_range > > > > Libs: -L${libdir} -lprotobuf > > > > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > > > > Conflicts: protobuf-lite > > > > """ > > > > > > > > Let me use "absl_hash" and "absl_absl_log" in the above list as examples. > > > > > > > > """ > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > ges-split/ -name "libabsl_hash.so.*" > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl- > > hash/usr/lib64/libabsl_hash.so.2401.0.0 > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > > > > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > ges-split/ -name "libabsl_absl_log.so.*" > > > > $ > > > > """ > > > > > > > > See? The current package splitting of abseil-cpp does not match what > > > > protobuf assumes. And I thinks protobuf's assumption is correct, > > > > because there's really a absl_absl_log.pc file generated by abseil-cpp. > > > > > > > > And there's the contents and location of absl_absl_log.pc: > > > > """ > > > > $ cat > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/pkgconfig/absl_absl_log.pc > > > > prefix=/usr > > > > exec_prefix=${prefix} > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: absl_absl_log > > > > Description: Abseil absl_log library > > > > URL: https://abseil.io/ > > > > Version: 20240116 > > > > Requires: absl_log_internal_log_impl = 20240116 > > > > Libs: -L${libdir} > > > > Cflags: -I${includedir} -DNOMINMAX > > > > """ > > > > > > > > And there's the contents for the absl_log_internal_log_impl: > > > > """ > > > > $ cat ../../../abseil-cpp/20240116.2/packag > > > > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_log_impl.pc > > > > prefix=/usr > > > > exec_prefix=${prefix} > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: absl_log_internal_log_impl > > > > Description: Abseil log_internal_log_impl library > > > > URL: https://abseil.io/ > > > > Version: 20240116 > > > > Requires: absl_log_internal_conditions = 20240116, > > > > absl_log_internal_message = 20240116, absl_log_internal_strip = > > > > 20240116, absl_absl_vlog_is_on = 20240116 > > > > Libs: -L${libdir} > > > > Cflags: -I${includedir} -DNOMINMAX > > > > """ > > > > > > > > We can then find the absl_log_internal_conditions library: > > > > """ > > > > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > > > > "*absl_log_internal_message*" > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-internal- > > message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/libabsl_log_internal_message.so > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > """ > > > > > > > > What does all above mean? It means that abseil-cpp is not splitted > > > > reasonably. This could be proven by the absl_absl_log.pc file as > > > > described above. > > > > > > > > So, given the current situation of abseil-cpp, Yi's solution is > > > > correct. Before abseil-cpp is re-splitted according to its pc files, > > > > which then could possibly lead to automatic parsing of the protobuf.pc > > > > file to add extra RDEPENDS automatically in protobuf recipe, we need > > > > to have Yi's solution. And we need to use RDEEPNDS instead of > > > > RRECOMMENDS (see the 'Requires' directive in corresponding .pc files). > > > > > > > > Regards, > > > > Qi > > > > > > > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: > > > >>> -----Original Message----- > > > >>> From: openembedded-devel@lists.openembedded.org > > <openembedded- > > > >>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via > > > >>> lists.openembedded.org > > > >>> Sent: Tuesday, October 1, 2024 14:00 > > > >>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > > > >>> <yi.zhao@eng.windriver.com> > > > >>> Cc: openembedded-devel@lists.openembedded.org > > > >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > >>> RDEPENDS > > > >>> > > > >>>> -----Original Message----- > > > >>>> From: Yi Zhao <yi.zhao@windriver.com> > > > >>>> Sent: den 25 september 2024 04:17 > > > >>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao > > > >>> <yi.zhao@eng.windriver.com> > > > >>>> Cc: openembedded-devel@lists.openembedded.org > > > >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > >>> RDEPENDS > > > >>>> > > > >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: > > > >>>>>> -----Original Message----- > > > >>>>>> From: openembedded-devel@lists.openembedded.org > > <openembedded- > > > >>> devel@lists.openembedded.org> On Behalf Of Yi Zhao > > > >>>>>> Sent: den 22 september 2024 09:36 > > > >>>>>> To: openembedded-devel@lists.openembedded.org > > > >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > >>> RDEPENDS > > > >>>>>> Since commit[1], the libraries of abseil-cpp have been splitted into > > > >>>>>> separate packages. When protobuf is installed into the image, > > > >>>>>> only 48 > > > >>>>>> abseil libraries are installed as runtime dependencies. But the > > > >>>>>> output > > > >>>>>> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > > > >>>>>> required. Add abseil-cpp to RDEPENDS to ensure that all required > > > >>>>>> abseil-cpp libraries are installed. > > > >>>>>> > > > >>>>>> [1] https://git.openembedded.org/meta- > > > >>> > > openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > > > >>> ecc > > > >>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > > >>>>>> --- > > > >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > > >>>>>> 1 file changed, 2 insertions(+) > > > >>>>>> > > > >>>>>> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> index cc5204de4..354b3213a 100644 > > > >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf- > > > >>> lite${SOLIBS}" > > > >>>>>> # CMake requires protoc binary to exist in sysroot, even if it > > > >>>>>> has wrong > > > >>> architecture. > > > >>>>>> SYSROOT_DIRS += "${bindir}" > > > >>>>>> > > > >>>>>> +RDEPENDS:${PN} = "abseil-cpp" > > > >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" > > > >>>>> This defeats the whole purpose of splitting up the abseil > > > >>>>> libraries. What triggered this change? Was it an actual > > > >>>>> runtime error, or were you only looking at the output > > > >>>>> from pkg-config --libs protobuf? > > > >>>>> > > > >>>>> Since the .pc files are in the protobuf-dev package, which > > > >>>>> depends on the abseil-cpp-dev package, all abseil libraries > > > >>>>> should be available if you are actually building on target > > > >>>>> or with an SDK. > > > >>>> Our customer reported this issue. Here is his description: > > > >>>> The abseil libraries will be included in the SDK, but not on the > > > >>>> target. > > > >>>> The application is building with SDK. The yocto build is adding > > > >>>> protobuf > > > >>>> to the image (no explicit abseil libraries are added). If an > > > >>>> application > > > >>>> then are built using the SDK it can not be executed on target as > > > >>>> libraries are missing. > > > >>> Can we at least change the dependency to a recommendation so > > > >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it > > > >>> is, it penalizes everyone that use protobuf for the case where > > > >>> one provides an SDK with protobuf. > > > >>> > > > >>> My suggestion would be to change it to: > > > >>> > > > >>> # Recommend abseil-cpp so that all its libraries are installed in case > > > >>> # protobuf is provided as part of an SDK. Everyone else should add > > > >>> # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. > > > >>> RRECOMMENDS:${PN} = "abseil-cpp" > > > >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" > > > >>> > > > >> I think that's also not the right way to do this. > > > >> Adding unnecessary stuff to release images to satisfy SDK needs is > > > >> wrong. > > > >> > > > >> I also wonder if the problem is not at the SDK level. > > > >> If you build an sdk, there is explicitly installed package base and > > > >> additional required dependencies of those packages. > > > >> The dependencies may not be complete and fully usable, only their > > > >> parts sufficient for the explicitly installed stuff is present. > > > >> Are the sdk users assuming that full abseil-cpp is installed and thus > > > >> using also parts not needed for protobuf? > > > >> The proper solution would be probably to include abseil-cpp > > > >> explicitly in SDK used in that way. > > > >> > > > >> Peter > > > >> > > > >>>> //Yi > > > >>>> > > > >>>>>> RDEPENDS:${PN}-compiler = "${PN}" > > > >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" > > > >>>>>> RDEPENDS:${PN}-ptest = "bash > > > >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', > > > >>> '', d)}" > > > >>>>> //Peter > > > >>> //Peter > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
The key problem is that the 'ldd' output does not match what the protobuf's pc says. The automatic library dependency adding thus does not give us a correct result in protobuf's case. As the list is written by protobuf's maintainers, I think we'd better trust them more than the ldd output. Let's look at a regular package's behavior. Say packageB (libB) rdepends on packageA (libA). And we want to develop an application linking to libB inside SDK. The generated application will just run fine on target. Because all actual runtime dependencies of libB are on target. Unfortunately, this is not the case for protobuf and aseil-cpp. Protobuf does NOT have all the runtime dependency it needs. Why they are special? Part of the reason is that they are both kind of development packages, I guess. Regarding aseil-cpp, as I explained in previous email, its auto splitting is not reasonable. So I think what we really should do is: 1) split aseil-cpp based on .pc files (parsing its Requires field) 2) parse the protobuf.pc file to get the actual runtime dependencies and add them to RDEPEDNS automatically. If anyone has concern for the above solution, please let me know. Thanks. Before the above is done, we need to use RDEPENDS manually. Regards, Qi -----Original Message----- From: Christian Eggers <ceggers@arri.de> Sent: Monday, December 2, 2024 9:52 PM To: peter.kjellerstedt@axis.com; Zhao, Yi <Yi.Zhao@windriver.com>; Zhao, Yi <Yi.Zhao@windriver.com>; openembedded-devel@lists.openembedded.org; Marko, Peter <Peter.Marko@siemens.com> Cc: openembedded-devel@lists.openembedded.org; Chen, Qi <Qi.Chen@windriver.com> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS Hi Peter, On Monday, 2 December 2024, 14:48:43 CET, Marko, Peter wrote: > Hello, > > > -----Original Message----- > > From: Christian Eggers <ceggers@arri.de> > > Sent: Monday, December 2, 2024 14:39 > > To: Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com>; > > peter.kjellerstedt@axis.com; Yi Zhao <yi.zhao@windriver.com>; Yi > > Zhao <yi.zhao@eng.windriver.com>; openembedded- > > devel@lists.openembedded.org > > Cc: openembedded-devel@lists.openembedded.org; Qi.Chen@windriver.com > > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > RDEPENDS > > > > Hi Qi, > > > > On Monday, 2 December 2024, 06:35:22 CET, Chen Qi via > > lists.openembedded.org wrote: > > > Before we rework the abseil-cpp and protobuf recipes, a possible > > > improvement for this patch is that we list the needed libraries in > > > RDEPENDS manually. > > > > my main concern was (as you already may have guessed) not having the > > full set of absl libraries in my rootfs, as only some of them are > > required. But I cannot tell which is the best option to achieve > > this. > > If you want full abseil-cpp in your rootfs, please use IMAGE_INSTALL for that. > Packages should pull only their real dependencies, not something you need for other reasons. > You use-case seems to be to develop application with abseil-cpp library but not installing it explicitly. > SDK images need to install (or depend on) additional packages compared to release images. sorry, but I meant it the other way around: "I do not like having the full set of abseil-cpp libraries in my rootfs only because I use protobuf in my application". Sorry for the confusion. regards, Christian > > Peter > > > > > regards, > > Christian > > > > > > > > Regards, > > > Qi > > > > > > On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > > > > I'll explain in more details what the problem is and why this > > > > patch is correct, given the current status of abseil-cpp recipe. > > > > > > > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of > > > > absl targets used by protobuf: > > > > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abse > > > > il- > > cpp.cmake#L56 > > > > > > > > > > > > This value in turn is parsed and used to replace the > > > > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > > > > > > > Here's the generated probobuf.pc file's contents: > > > > """ > > > > $ cat ../build/protobuf.pc > > > > prefix=/usr > > > > exec_prefix=/usr > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: Protocol Buffers > > > > Description: Google's Data Interchange Format > > > > Version: 25.3.0 > > > > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > > > > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > > > > absl_core_headers absl_debugging absl_die_if_null > > > > absl_dynamic_annotations absl_flags absl_flat_hash_map > > > > absl_flat_hash_set absl_function_ref absl_hash absl_layout > > > > absl_log_initialize absl_log_severity absl_memory > > > > absl_node_hash_map absl_node_hash_set absl_optional absl_span > > > > absl_status absl_statusor absl_strings absl_synchronization > > > > absl_time absl_type_traits absl_utility absl_variant utf8_range > > > > Libs: -L${libdir} -lprotobuf > > > > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > > > > Conflicts: protobuf-lite > > > > """ > > > > > > > > Let me use "absl_hash" and "absl_absl_log" in the above list as examples. > > > > > > > > """ > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > ges-split/ -name "libabsl_hash.so.*" > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl- > > hash/usr/lib64/libabsl_hash.so.2401.0.0 > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > > > > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > ges-split/ -name "libabsl_absl_log.so.*" > > > > $ > > > > """ > > > > > > > > See? The current package splitting of abseil-cpp does not match > > > > what protobuf assumes. And I thinks protobuf's assumption is > > > > correct, because there's really a absl_absl_log.pc file generated by abseil-cpp. > > > > > > > > And there's the contents and location of absl_absl_log.pc: > > > > """ > > > > $ cat > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/pkgconfig/absl_absl_log.pc > > > > prefix=/usr > > > > exec_prefix=${prefix} > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: absl_absl_log > > > > Description: Abseil absl_log library > > > > URL: https://abseil.io/ > > > > Version: 20240116 > > > > Requires: absl_log_internal_log_impl = 20240116 > > > > Libs: -L${libdir} > > > > Cflags: -I${includedir} -DNOMINMAX """ > > > > > > > > And there's the contents for the absl_log_internal_log_impl: > > > > """ > > > > $ cat ../../../abseil-cpp/20240116.2/packag > > > > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_lo > > > > g_impl.pc > > > > prefix=/usr > > > > exec_prefix=${prefix} > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: absl_log_internal_log_impl > > > > Description: Abseil log_internal_log_impl library > > > > URL: https://abseil.io/ > > > > Version: 20240116 > > > > Requires: absl_log_internal_conditions = 20240116, > > > > absl_log_internal_message = 20240116, absl_log_internal_strip = > > > > 20240116, absl_absl_vlog_is_on = 20240116 > > > > Libs: -L${libdir} > > > > Cflags: -I${includedir} -DNOMINMAX """ > > > > > > > > We can then find the absl_log_internal_conditions library: > > > > """ > > > > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > > > > "*absl_log_internal_message*" > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-intern > > > > al- > > message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/libabsl_log_internal_message.so > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > """ > > > > > > > > What does all above mean? It means that abseil-cpp is not > > > > splitted reasonably. This could be proven by the > > > > absl_absl_log.pc file as described above. > > > > > > > > So, given the current situation of abseil-cpp, Yi's solution is > > > > correct. Before abseil-cpp is re-splitted according to its pc > > > > files, which then could possibly lead to automatic parsing of > > > > the protobuf.pc file to add extra RDEPENDS automatically in > > > > protobuf recipe, we need to have Yi's solution. And we need to > > > > use RDEEPNDS instead of RRECOMMENDS (see the 'Requires' directive in corresponding .pc files). > > > > > > > > Regards, > > > > Qi > > > > > > > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: > > > >>> -----Original Message----- > > > >>> From: openembedded-devel@lists.openembedded.org > > <openembedded- > > > >>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt > > > >>> via lists.openembedded.org > > > >>> Sent: Tuesday, October 1, 2024 14:00 > > > >>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > > > >>> <yi.zhao@eng.windriver.com> > > > >>> Cc: openembedded-devel@lists.openembedded.org > > > >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add > > > >>> abseil-cpp to RDEPENDS > > > >>> > > > >>>> -----Original Message----- > > > >>>> From: Yi Zhao <yi.zhao@windriver.com> > > > >>>> Sent: den 25 september 2024 04:17 > > > >>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao > > > >>> <yi.zhao@eng.windriver.com> > > > >>>> Cc: openembedded-devel@lists.openembedded.org > > > >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add > > > >>>> abseil-cpp to > > > >>> RDEPENDS > > > >>>> > > > >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: > > > >>>>>> -----Original Message----- > > > >>>>>> From: openembedded-devel@lists.openembedded.org > > <openembedded- > > > >>> devel@lists.openembedded.org> On Behalf Of Yi Zhao > > > >>>>>> Sent: den 22 september 2024 09:36 > > > >>>>>> To: openembedded-devel@lists.openembedded.org > > > >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp > > > >>>>>> to > > > >>> RDEPENDS > > > >>>>>> Since commit[1], the libraries of abseil-cpp have been > > > >>>>>> splitted into separate packages. When protobuf is installed > > > >>>>>> into the image, only 48 abseil libraries are installed as > > > >>>>>> runtime dependencies. But the output of 'pkg-config --libs > > > >>>>>> protobuf' lists 66 abseil-cpp libraries are required. Add > > > >>>>>> abseil-cpp to RDEPENDS to ensure that all required > > > >>>>>> abseil-cpp libraries are installed. > > > >>>>>> > > > >>>>>> [1] https://git.openembedded.org/meta- > > > >>> > > openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > > > >>> ecc > > > >>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > > >>>>>> --- > > > >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > > >>>>>> 1 file changed, 2 insertions(+) > > > >>>>>> > > > >>>>>> diff --git > > > >>>>>> a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> index cc5204de4..354b3213a 100644 > > > >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = > > > >>>>>> "${libdir}/libprotobuf- > > > >>> lite${SOLIBS}" > > > >>>>>> # CMake requires protoc binary to exist in sysroot, even > > > >>>>>> if it has wrong > > > >>> architecture. > > > >>>>>> SYSROOT_DIRS += "${bindir}" > > > >>>>>> > > > >>>>>> +RDEPENDS:${PN} = "abseil-cpp" > > > >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" > > > >>>>> This defeats the whole purpose of splitting up the abseil > > > >>>>> libraries. What triggered this change? Was it an actual > > > >>>>> runtime error, or were you only looking at the output from > > > >>>>> pkg-config --libs protobuf? > > > >>>>> > > > >>>>> Since the .pc files are in the protobuf-dev package, which > > > >>>>> depends on the abseil-cpp-dev package, all abseil libraries > > > >>>>> should be available if you are actually building on target > > > >>>>> or with an SDK. > > > >>>> Our customer reported this issue. Here is his description: > > > >>>> The abseil libraries will be included in the SDK, but not on > > > >>>> the target. > > > >>>> The application is building with SDK. The yocto build is > > > >>>> adding protobuf to the image (no explicit abseil libraries > > > >>>> are added). If an application then are built using the SDK it > > > >>>> can not be executed on target as libraries are missing. > > > >>> Can we at least change the dependency to a recommendation so > > > >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it > > > >>> is, it penalizes everyone that use protobuf for the case where > > > >>> one provides an SDK with protobuf. > > > >>> > > > >>> My suggestion would be to change it to: > > > >>> > > > >>> # Recommend abseil-cpp so that all its libraries are installed > > > >>> in case # protobuf is provided as part of an SDK. Everyone > > > >>> else should add # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. > > > >>> RRECOMMENDS:${PN} = "abseil-cpp" > > > >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" > > > >>> > > > >> I think that's also not the right way to do this. > > > >> Adding unnecessary stuff to release images to satisfy SDK needs > > > >> is wrong. > > > >> > > > >> I also wonder if the problem is not at the SDK level. > > > >> If you build an sdk, there is explicitly installed package base > > > >> and additional required dependencies of those packages. > > > >> The dependencies may not be complete and fully usable, only > > > >> their parts sufficient for the explicitly installed stuff is present. > > > >> Are the sdk users assuming that full abseil-cpp is installed > > > >> and thus using also parts not needed for protobuf? > > > >> The proper solution would be probably to include abseil-cpp > > > >> explicitly in SDK used in that way. > > > >> > > > >> Peter > > > >> > > > >>>> //Yi > > > >>>> > > > >>>>>> RDEPENDS:${PN}-compiler = "${PN}" > > > >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" > > > >>>>>> RDEPENDS:${PN}-ptest = "bash > > > >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', > > > >>> 'python3-protobuf', '', d)}" > > > >>>>> //Peter > > > >>> //Peter > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
I have now sent revert as part of a larger refactoring of protobuf recipes. https://lists.openembedded.org/g/openembedded-devel/message/115221 Peter > -----Original Message----- > From: Christian Eggers <ceggers@arri.de> > Sent: Monday, December 2, 2024 14:52 > To: peter.kjellerstedt@axis.com; Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > <yi.zhao@eng.windriver.com>; openembedded-devel@lists.openembedded.org; > Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com> > Cc: openembedded-devel@lists.openembedded.org; Qi.Chen@windriver.com > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS > > Hi Peter, > > On Monday, 2 December 2024, 14:48:43 CET, Marko, Peter wrote: > > Hello, > > > > > -----Original Message----- > > > From: Christian Eggers <ceggers@arri.de> > > > Sent: Monday, December 2, 2024 14:39 > > > To: Marko, Peter (FT D EU SK BFS1) <Peter.Marko@siemens.com>; > > > peter.kjellerstedt@axis.com; Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > > > <yi.zhao@eng.windriver.com>; openembedded- > > > devel@lists.openembedded.org > > > Cc: openembedded-devel@lists.openembedded.org; Qi.Chen@windriver.com > > > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > RDEPENDS > > > > > > Hi Qi, > > > > > > On Monday, 2 December 2024, 06:35:22 CET, Chen Qi via > > > lists.openembedded.org wrote: > > > > Before we rework the abseil-cpp and protobuf recipes, a possible > > > > improvement for this patch is that we list the needed libraries in > > > > RDEPENDS manually. > > > > > > my main concern was (as you already may have guessed) not having the full > > > set of > > > absl libraries in my rootfs, as only some of them are required. But I cannot tell > > > which is the best option to achieve this. > > > > If you want full abseil-cpp in your rootfs, please use IMAGE_INSTALL for that. > > Packages should pull only their real dependencies, not something you need for > other reasons. > > You use-case seems to be to develop application with abseil-cpp library but not > installing it explicitly. > > SDK images need to install (or depend on) additional packages compared to > release images. > sorry, but I meant it the other way around: > > "I do not like having the full set of abseil-cpp libraries in my rootfs only because > I use protobuf in my application". > > Sorry for the confusion. > > regards, > Christian > > > > > > Peter > > > > > > > > regards, > > > Christian > > > > > > > > > > > Regards, > > > > Qi > > > > > > > > On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > > > > > I'll explain in more details what the problem is and why this patch is > > > > > correct, given the current status of abseil-cpp recipe. > > > > > > > > > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of absl > > > > > targets used by protobuf: > > > > > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abseil- > > > cpp.cmake#L56 > > > > > > > > > > > > > > > This value in turn is parsed and used to replace the > > > > > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > > > > > > > > > Here's the generated probobuf.pc file's contents: > > > > > """ > > > > > $ cat ../build/protobuf.pc > > > > > prefix=/usr > > > > > exec_prefix=/usr > > > > > libdir=/usr/lib64 > > > > > includedir=/usr/include > > > > > > > > > > Name: Protocol Buffers > > > > > Description: Google's Data Interchange Format > > > > > Version: 25.3.0 > > > > > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > > > > > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > > > > > absl_core_headers absl_debugging absl_die_if_null > > > > > absl_dynamic_annotations absl_flags absl_flat_hash_map > > > > > absl_flat_hash_set absl_function_ref absl_hash absl_layout > > > > > absl_log_initialize absl_log_severity absl_memory absl_node_hash_map > > > > > absl_node_hash_set absl_optional absl_span absl_status absl_statusor > > > > > absl_strings absl_synchronization absl_time absl_type_traits > > > > > absl_utility absl_variant utf8_range > > > > > Libs: -L${libdir} -lprotobuf > > > > > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > > > > > Conflicts: protobuf-lite > > > > > """ > > > > > > > > > > Let me use "absl_hash" and "absl_absl_log" in the above list as examples. > > > > > > > > > > """ > > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > > ges-split/ -name "libabsl_hash.so.*" > > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl- > > > hash/usr/lib64/libabsl_hash.so.2401.0.0 > > > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > > dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > > > > > > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > > ges-split/ -name "libabsl_absl_log.so.*" > > > > > $ > > > > > """ > > > > > > > > > > See? The current package splitting of abseil-cpp does not match what > > > > > protobuf assumes. And I thinks protobuf's assumption is correct, > > > > > because there's really a absl_absl_log.pc file generated by abseil-cpp. > > > > > > > > > > And there's the contents and location of absl_absl_log.pc: > > > > > """ > > > > > $ cat > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > > dev/usr/lib64/pkgconfig/absl_absl_log.pc > > > > > prefix=/usr > > > > > exec_prefix=${prefix} > > > > > libdir=/usr/lib64 > > > > > includedir=/usr/include > > > > > > > > > > Name: absl_absl_log > > > > > Description: Abseil absl_log library > > > > > URL: https://abseil.io/ > > > > > Version: 20240116 > > > > > Requires: absl_log_internal_log_impl = 20240116 > > > > > Libs: -L${libdir} > > > > > Cflags: -I${includedir} -DNOMINMAX > > > > > """ > > > > > > > > > > And there's the contents for the absl_log_internal_log_impl: > > > > > """ > > > > > $ cat ../../../abseil-cpp/20240116.2/packag > > > > > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_log_impl.pc > > > > > prefix=/usr > > > > > exec_prefix=${prefix} > > > > > libdir=/usr/lib64 > > > > > includedir=/usr/include > > > > > > > > > > Name: absl_log_internal_log_impl > > > > > Description: Abseil log_internal_log_impl library > > > > > URL: https://abseil.io/ > > > > > Version: 20240116 > > > > > Requires: absl_log_internal_conditions = 20240116, > > > > > absl_log_internal_message = 20240116, absl_log_internal_strip = > > > > > 20240116, absl_absl_vlog_is_on = 20240116 > > > > > Libs: -L${libdir} > > > > > Cflags: -I${includedir} -DNOMINMAX > > > > > """ > > > > > > > > > > We can then find the absl_log_internal_conditions library: > > > > > """ > > > > > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > > > > > "*absl_log_internal_message*" > > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-internal- > > > message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > > dev/usr/lib64/libabsl_log_internal_message.so > > > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > > dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > > dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > > > """ > > > > > > > > > > What does all above mean? It means that abseil-cpp is not splitted > > > > > reasonably. This could be proven by the absl_absl_log.pc file as > > > > > described above. > > > > > > > > > > So, given the current situation of abseil-cpp, Yi's solution is > > > > > correct. Before abseil-cpp is re-splitted according to its pc files, > > > > > which then could possibly lead to automatic parsing of the protobuf.pc > > > > > file to add extra RDEPENDS automatically in protobuf recipe, we need > > > > > to have Yi's solution. And we need to use RDEEPNDS instead of > > > > > RRECOMMENDS (see the 'Requires' directive in corresponding .pc files). > > > > > > > > > > Regards, > > > > > Qi > > > > > > > > > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: > > > > >>> -----Original Message----- > > > > >>> From: openembedded-devel@lists.openembedded.org > > > <openembedded- > > > > >>> devel@lists.openembedded.org> On Behalf Of Peter Kjellerstedt via > > > > >>> lists.openembedded.org > > > > >>> Sent: Tuesday, October 1, 2024 14:00 > > > > >>> To: Yi Zhao <yi.zhao@windriver.com>; Yi Zhao > > > > >>> <yi.zhao@eng.windriver.com> > > > > >>> Cc: openembedded-devel@lists.openembedded.org > > > > >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > > >>> RDEPENDS > > > > >>> > > > > >>>> -----Original Message----- > > > > >>>> From: Yi Zhao <yi.zhao@windriver.com> > > > > >>>> Sent: den 25 september 2024 04:17 > > > > >>>> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Yi Zhao > > > > >>> <yi.zhao@eng.windriver.com> > > > > >>>> Cc: openembedded-devel@lists.openembedded.org > > > > >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > > >>> RDEPENDS > > > > >>>> > > > > >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: > > > > >>>>>> -----Original Message----- > > > > >>>>>> From: openembedded-devel@lists.openembedded.org > > > <openembedded- > > > > >>> devel@lists.openembedded.org> On Behalf Of Yi Zhao > > > > >>>>>> Sent: den 22 september 2024 09:36 > > > > >>>>>> To: openembedded-devel@lists.openembedded.org > > > > >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > > > >>> RDEPENDS > > > > >>>>>> Since commit[1], the libraries of abseil-cpp have been splitted into > > > > >>>>>> separate packages. When protobuf is installed into the image, > > > > >>>>>> only 48 > > > > >>>>>> abseil libraries are installed as runtime dependencies. But the > > > > >>>>>> output > > > > >>>>>> of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are > > > > >>>>>> required. Add abseil-cpp to RDEPENDS to ensure that all required > > > > >>>>>> abseil-cpp libraries are installed. > > > > >>>>>> > > > > >>>>>> [1] https://git.openembedded.org/meta- > > > > >>> > > > openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > > > > >>> ecc > > > > >>>>>> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> > > > > >>>>>> --- > > > > >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > > > >>>>>> 1 file changed, 2 insertions(+) > > > > >>>>>> > > > > >>>>>> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > > >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > > >>>>>> index cc5204de4..354b3213a 100644 > > > > >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > > >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > > >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf- > > > > >>> lite${SOLIBS}" > > > > >>>>>> # CMake requires protoc binary to exist in sysroot, even if it > > > > >>>>>> has wrong > > > > >>> architecture. > > > > >>>>>> SYSROOT_DIRS += "${bindir}" > > > > >>>>>> > > > > >>>>>> +RDEPENDS:${PN} = "abseil-cpp" > > > > >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" > > > > >>>>> This defeats the whole purpose of splitting up the abseil > > > > >>>>> libraries. What triggered this change? Was it an actual > > > > >>>>> runtime error, or were you only looking at the output > > > > >>>>> from pkg-config --libs protobuf? > > > > >>>>> > > > > >>>>> Since the .pc files are in the protobuf-dev package, which > > > > >>>>> depends on the abseil-cpp-dev package, all abseil libraries > > > > >>>>> should be available if you are actually building on target > > > > >>>>> or with an SDK. > > > > >>>> Our customer reported this issue. Here is his description: > > > > >>>> The abseil libraries will be included in the SDK, but not on the > > > > >>>> target. > > > > >>>> The application is building with SDK. The yocto build is adding > > > > >>>> protobuf > > > > >>>> to the image (no explicit abseil libraries are added). If an > > > > >>>> application > > > > >>>> then are built using the SDK it can not be executed on target as > > > > >>>> libraries are missing. > > > > >>> Can we at least change the dependency to a recommendation so > > > > >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it > > > > >>> is, it penalizes everyone that use protobuf for the case where > > > > >>> one provides an SDK with protobuf. > > > > >>> > > > > >>> My suggestion would be to change it to: > > > > >>> > > > > >>> # Recommend abseil-cpp so that all its libraries are installed in case > > > > >>> # protobuf is provided as part of an SDK. Everyone else should add > > > > >>> # BAD_RECOMMENDATIONS += "abseil-cpp" to their configuration. > > > > >>> RRECOMMENDS:${PN} = "abseil-cpp" > > > > >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" > > > > >>> > > > > >> I think that's also not the right way to do this. > > > > >> Adding unnecessary stuff to release images to satisfy SDK needs is > > > > >> wrong. > > > > >> > > > > >> I also wonder if the problem is not at the SDK level. > > > > >> If you build an sdk, there is explicitly installed package base and > > > > >> additional required dependencies of those packages. > > > > >> The dependencies may not be complete and fully usable, only their > > > > >> parts sufficient for the explicitly installed stuff is present. > > > > >> Are the sdk users assuming that full abseil-cpp is installed and thus > > > > >> using also parts not needed for protobuf? > > > > >> The proper solution would be probably to include abseil-cpp > > > > >> explicitly in SDK used in that way. > > > > >> > > > > >> Peter > > > > >> > > > > >>>> //Yi > > > > >>>> > > > > >>>>>> RDEPENDS:${PN}-compiler = "${PN}" > > > > >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" > > > > >>>>>> RDEPENDS:${PN}-ptest = "bash > > > > >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', > > > > >>> '', d)}" > > > > >>>>> //Peter > > > > >>> //Peter > > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb index cc5204de4..354b3213a 100644 --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb @@ -106,6 +106,8 @@ FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}" # CMake requires protoc binary to exist in sysroot, even if it has wrong architecture. SYSROOT_DIRS += "${bindir}" +RDEPENDS:${PN} = "abseil-cpp" +RDEPENDS:${PN}-lite = "abseil-cpp" RDEPENDS:${PN}-compiler = "${PN}" RDEPENDS:${PN}-dev += "${PN}-compiler" RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}"
Since commit[1], the libraries of abseil-cpp have been splitted into separate packages. When protobuf is installed into the image, only 48 abseil libraries are installed as runtime dependencies. But the output of 'pkg-config --libs protobuf' lists 66 abseil-cpp libraries are required. Add abseil-cpp to RDEPENDS to ensure that all required abseil-cpp libraries are installed. [1] https://git.openembedded.org/meta-openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035ecc Signed-off-by: Yi Zhao <yi.zhao@windriver.com> --- meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ 1 file changed, 2 insertions(+)