diff mbox series

[v3] qemu: Split the qemu package

Message ID 20230602023657.309867-1-mingli.yu@eng.windriver.com
State New
Headers show
Series [v3] qemu: Split the qemu package | expand

Commit Message

mingli.yu@eng.windriver.com June 2, 2023, 2:36 a.m. UTC
From: Mingli Yu <mingli.yu@windriver.com>

Currently all files as below packaged into one package such as
qemu-7.2.0-*.rpm. After the qemu package installed on the target,
it will take up about 464M which includes not only the one matches
the arch of the target but aslo all available built qemu targets
which set by QEMU_TARGETS.

 # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
 qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
 qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
 qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
 qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
 qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
 qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
 qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
 qemu-riscv32    qemu-system-arm      qemu-system-mips64el
 qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
 qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4

Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
corresponding qemu arch package they want to install should ease the concerns
who cares much about the size in embedded device as it decreases the qemu rpm
(qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
extracted qemu RPM decreased from about 464M to about 248M.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-devtools/qemu/qemu.inc | 5 +++++
 1 file changed, 5 insertions(+)

Comments

Bruce Ashfield June 2, 2023, 1:08 p.m. UTC | #1
On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>
> From: Mingli Yu <mingli.yu@windriver.com>
>
> Currently all files as below packaged into one package such as
> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> it will take up about 464M which includes not only the one matches
> the arch of the target but aslo all available built qemu targets
> which set by QEMU_TARGETS.
>
>  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
>
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> corresponding qemu arch package they want to install should ease the concerns
> who cares much about the size in embedded device as it decreases the qemu rpm
> (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> extracted qemu RPM decreased from about 464M to about 248M.
>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..c6fd39aab6 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
>
>  FILES:${PN} += "${datadir}/icons"
>
> +python populate_packages:prepend() {
> +    archdir = d.expand('${bindir}/')
> +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> +}

I don't see a v3 changelog, but that does look like the unnecessary
default runtime dependency is gone, but what would be the method of
installing the packages if I have an image where I'd like the old
non-split functionality ?

Either a meta packages (qemu-system-all ?) or a way to override the
functionality (a variable around the do_split call ?) are options to
enable that sort of thing. As I mentioned before, there's a different
expected split of the packages in some scenarios, and it would be
ideal to not break those use cases.

Bruce

> +
>  # Put the guest agent in a separate package
>  PACKAGES =+ "${PN}-guest-agent"
>  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#182138): https://lists.openembedded.org/g/openembedded-core/message/182138
> Mute This Topic: https://lists.openembedded.org/mt/99279217/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Richard Purdie June 2, 2023, 1:19 p.m. UTC | #2
On Fri, 2023-06-02 at 09:08 -0400, Bruce Ashfield wrote:
> On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
> > 
> > From: Mingli Yu <mingli.yu@windriver.com>
> > 
> > Currently all files as below packaged into one package such as
> > qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> > it will take up about 464M which includes not only the one matches
> > the arch of the target but aslo all available built qemu targets
> > which set by QEMU_TARGETS.
> > 
> >  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
> >  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
> >  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
> >  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
> >  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
> >  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
> >  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
> >  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
> >  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
> >  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
> >  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
> > 
> > Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> > qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> > corresponding qemu arch package they want to install should ease the concerns
> > who cares much about the size in embedded device as it decreases the qemu rpm
> > (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> > extracted qemu RPM decreased from about 464M to about 248M.
> > 
> > Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> > ---
> >  meta/recipes-devtools/qemu/qemu.inc | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> > index a87dee5c99..c6fd39aab6 100644
> > --- a/meta/recipes-devtools/qemu/qemu.inc
> > +++ b/meta/recipes-devtools/qemu/qemu.inc
> > @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
> > 
> >  FILES:${PN} += "${datadir}/icons"
> > 
> > +python populate_packages:prepend() {
> > +    archdir = d.expand('${bindir}/')
> > +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> > +}
> 
> I don't see a v3 changelog, but that does look like the unnecessary
> default runtime dependency is gone, but what would be the method of
> installing the packages if I have an image where I'd like the old
> non-split functionality ?
> 
> Either a meta packages (qemu-system-all ?) or a way to override the
> functionality (a variable around the do_split call ?) are options to
> enable that sort of thing. As I mentioned before, there's a different
> expected split of the packages in some scenarios, and it would be
> ideal to not break those use cases.

This is quite a common need for things which call do_split_packages,
I'm starting to wonder if it should support some kind of "common"
package as a parameter which it would add dependencies to?

Cheers,

Richard
Yu, Mingli June 7, 2023, 3:09 a.m. UTC | #3
Hi Richard and Bruce,

On 6/2/23 21:19, Richard Purdie wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> On Fri, 2023-06-02 at 09:08 -0400, Bruce Ashfield wrote:
>> On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>>>
>>> From: Mingli Yu <mingli.yu@windriver.com>
>>>
>>> Currently all files as below packaged into one package such as
>>> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
>>> it will take up about 464M which includes not only the one matches
>>> the arch of the target but aslo all available built qemu targets
>>> which set by QEMU_TARGETS.
>>>
>>>   # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>>>   qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>>>   qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>>>   qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>>>   qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>>>   qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>>>   qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>>>   qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>>>   qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>>>   qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>>>   qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
>>>
>>> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
>>> qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
>>> corresponding qemu arch package they want to install should ease the concerns
>>> who cares much about the size in embedded device as it decreases the qemu rpm
>>> (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
>>> extracted qemu RPM decreased from about 464M to about 248M.
>>>
>>> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
>>> ---
>>>   meta/recipes-devtools/qemu/qemu.inc | 5 +++++
>>>   1 file changed, 5 insertions(+)
>>>
>>> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
>>> index a87dee5c99..c6fd39aab6 100644
>>> --- a/meta/recipes-devtools/qemu/qemu.inc
>>> +++ b/meta/recipes-devtools/qemu/qemu.inc
>>> @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
>>>
>>>   FILES:${PN} += "${datadir}/icons"
>>>
>>> +python populate_packages:prepend() {
>>> +    archdir = d.expand('${bindir}/')
>>> +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
>>> +}
>>
>> I don't see a v3 changelog, but that does look like the unnecessary
>> default runtime dependency is gone, but what would be the method of
>> installing the packages if I have an image where I'd like the old
>> non-split functionality ?
>>
>> Either a meta packages (qemu-system-all ?) or a way to override the
>> functionality (a variable around the do_split call ?) are options to
>> enable that sort of thing. As I mentioned before, there's a different
>> expected split of the packages in some scenarios, and it would be
>> ideal to not break those use cases.
> 
> This is quite a common need for things which call do_split_packages,
> I'm starting to wonder if it should support some kind of "common"
> package as a parameter which it would add dependencies to?

I'm trying to improve the package method to try to meet the need for 
both the user who cares about the rpm size and the user who want the old 
non-split functionality.

And I just did a little search for the packages which use 
do_split_packages and didn't see there is any package which keep both 
split and no-split functionality and just only split.

So I don't quite understand "This is quite a common need for things 
which call do_split_packages".

BTW, if we still use the old non-split, is there any suitable method for 
the user who cares about the rpm size? Any suggestions?

Thanks,

> 
> Cheers,
> 
> Richard
Richard Purdie June 7, 2023, 8:33 a.m. UTC | #4
On Wed, 2023-06-07 at 11:09 +0800, Yu, Mingli wrote:
> Hi Richard and Bruce,
> 
> On 6/2/23 21:19, Richard Purdie wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > 
> > On Fri, 2023-06-02 at 09:08 -0400, Bruce Ashfield wrote:
> > > On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
> > > > 
> > > > From: Mingli Yu <mingli.yu@windriver.com>
> > > > 
> > > > Currently all files as below packaged into one package such as
> > > > qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> > > > it will take up about 464M which includes not only the one matches
> > > > the arch of the target but aslo all available built qemu targets
> > > > which set by QEMU_TARGETS.
> > > > 
> > > >   # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
> > > >   qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
> > > >   qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
> > > >   qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
> > > >   qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
> > > >   qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
> > > >   qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
> > > >   qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
> > > >   qemu-riscv32    qemu-system-arm      qemu-system-mips64el
> > > >   qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
> > > >   qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
> > > > 
> > > > Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> > > > qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> > > > corresponding qemu arch package they want to install should ease the concerns
> > > > who cares much about the size in embedded device as it decreases the qemu rpm
> > > > (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> > > > extracted qemu RPM decreased from about 464M to about 248M.
> > > > 
> > > > Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> > > > ---
> > > >   meta/recipes-devtools/qemu/qemu.inc | 5 +++++
> > > >   1 file changed, 5 insertions(+)
> > > > 
> > > > diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> > > > index a87dee5c99..c6fd39aab6 100644
> > > > --- a/meta/recipes-devtools/qemu/qemu.inc
> > > > +++ b/meta/recipes-devtools/qemu/qemu.inc
> > > > @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
> > > > 
> > > >   FILES:${PN} += "${datadir}/icons"
> > > > 
> > > > +python populate_packages:prepend() {
> > > > +    archdir = d.expand('${bindir}/')
> > > > +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> > > > +}
> > > 
> > > I don't see a v3 changelog, but that does look like the unnecessary
> > > default runtime dependency is gone, but what would be the method of
> > > installing the packages if I have an image where I'd like the old
> > > non-split functionality ?
> > > 
> > > Either a meta packages (qemu-system-all ?) or a way to override the
> > > functionality (a variable around the do_split call ?) are options to
> > > enable that sort of thing. As I mentioned before, there's a different
> > > expected split of the packages in some scenarios, and it would be
> > > ideal to not break those use cases.
> > 
> > This is quite a common need for things which call do_split_packages,
> > I'm starting to wonder if it should support some kind of "common"
> > package as a parameter which it would add dependencies to?
> 
> I'm trying to improve the package method to try to meet the need for 
> both the user who cares about the rpm size and the user who want the old 
> non-split functionality.
> 
> And I just did a little search for the packages which use 
> do_split_packages and didn't see there is any package which keep both 
> split and no-split functionality and just only split.
> 
> So I don't quite understand "This is quite a common need for things 
> which call do_split_packages".
> 
> BTW, if we still use the old non-split, is there any suitable method for 
> the user who cares about the rpm size? Any suggestions?

I said we should create a package which behaves the same as the old
situation, not that we should have the same binaries in both. By that I
mean a package you can install with dependencies such that all the qemu
binaries are pulled in together. This is something which many users of
do_split_packages already do.

Taking meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-
packaging.inc for example, it has some python which is commented as:

"""
# Go through all generated packages (excluding the main package and
# the -meta package itself) and add them to the -meta package as RDEPENDS.
"""

You can see alsa-plugins does:

    packages = " ".join(do_split_packages(d, plugindir, r'^libasound_module_(.*)\.so$', 'libasound-module-%s', 'Alsa plugin for %s', extra_depends=''))
    d.setVar("RDEPENDS:alsa-plugins", packages)

i.e. it sets the alsa-plugins package to have dependencies on all the
split packages by using the return value.

Similarly, wpa-supplicant:

split_packages = do_split_packages(d, libdir, r'^(.*)\.so', '${PN}-plugin-%s', 'wpa_supplicant %s plugin', prepend=True)
split_dbg_packages = do_split_packages(d, dbglibdir, r'^(.*)\.so', '${PN}-plugin-%s-dbg', 'wpa_supplicant %s plugin - Debugging files', prepend=True, extra_depends='${PN}-dbg')

if split_packages:
    pn = d.getVar('PN')
    d.setVar('RRECOMMENDS:' + pn + '-plugins', ' '.join(split_packages))
    d.appendVar('RRECOMMENDS:' + pn + '-dbg', ' ' + ' '.join(split_dbg_packages))


So we have two options. Either we decide the return value is good
enough and just use that to create this package, or we add a new
parameter to do_split_packages so you could call something like:

do_split_packages(d, libdir, [xxx], append_split_packages_var='RRECOMMENDS:' + pn + '-plugins')

directly. We might need a better name for the new parameter but you get
the idea.

Cheers,

Richard
diff mbox series

Patch

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index a87dee5c99..c6fd39aab6 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -230,6 +230,11 @@  INSANE_SKIP:${PN} = "arch"
 
 FILES:${PN} += "${datadir}/icons"
 
+python populate_packages:prepend() {
+    archdir = d.expand('${bindir}/')
+    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
+}
+
 # Put the guest agent in a separate package
 PACKAGES =+ "${PN}-guest-agent"
 SUMMARY:${PN}-guest-agent = "QEMU guest agent"