| Message ID | 20240611141540.4209-1-twoerner@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [meta-rockchip] user-selectable wic format | expand |
Hi Trevor, On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote: > Allow the user to choose their preferred wic image format. > Can you provide some use case for this? > Signed-off-by: Trevor Woerner <twoerner@gmail.com> > --- > conf/machine/include/rockchip-wic.inc | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc > index dab61d83ed2c..eb895cd0b4ad 100644 > --- a/conf/machine/include/rockchip-wic.inc > +++ b/conf/machine/include/rockchip-wic.inc > @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc > > SPL_BINARY ?= "idbloader.img" > > -IMAGE_FSTYPES += "wic wic.bmap" > +WIC_FSTYPE ?= "wic" > +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap" Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to achieve here? If so, shouldn't we also not build wic.bmap there? Cheers, Quentin
On Tue 2024-06-11 @ 04:31:26 PM, Quentin Schulz via lists.yoctoproject.org wrote: > Hi Trevor, > > On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote: > > Allow the user to choose their preferred wic image format. > > > > Can you provide some use case for this? In order to build a wic.xz image the build has to first create a wic image, then compress it as an extra step. On a slow machine this extra step takes a noticeable amount of time. On my local, slow build machine I prefer wic images since it saves build time not having to do the compression and since everything is local, there's no over-the-internet xfer time to consider. When I build using some remote, fast build machine, the extra time spent doing the extra step of compressing is recuperated by the savings in transfer time retrieving the image from the remote. Therefore I prefer to make wic.xz images. Building wic images on remote machines would lead to very long xfer times to download the image artifact. If I simply do: IMAGE_FSTYPES += "wic.xz" in my conf/local.conf then MACHINEs that don't normally build wic images will fail. So the only way to build wic.xz images on remote builds is to tweak the meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always carrying this tweak for every remote build that I do. So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to wic.xz but that wouldn't suit my use-case since I could then have to tweak it again for any local builds. Besides, I could never guess what preferred wic version others would prefer and I don't want to be changing it every other week. So this patch maintains the existing behaviour completely, and when I do a build on a remote machine I can simply set WIC_FSTYPE = "wic.xz" in my conf/local.conf without having to tweak the layer. And if others want compression but would prefer bz2 or gzip (or whatever) then then can specify it however they wish. > > Signed-off-by: Trevor Woerner <twoerner@gmail.com> > > --- > > conf/machine/include/rockchip-wic.inc | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc > > index dab61d83ed2c..eb895cd0b4ad 100644 > > --- a/conf/machine/include/rockchip-wic.inc > > +++ b/conf/machine/include/rockchip-wic.inc > > @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc > > SPL_BINARY ?= "idbloader.img" > > -IMAGE_FSTYPES += "wic wic.bmap" > > +WIC_FSTYPE ?= "wic" > > +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap" > > Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to > achieve here? > > If so, shouldn't we also not build wic.bmap there? The goal isn't to not build a wic image, the point is to allow the user to specify which type of wic image (i.e. with or without compression and if with compression, which one?). I guess I could do: WIC_COMPRESSION ?= "" IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap" And then specify: WIC_COMPRESSION = ".xz" in my conf/local.conf?
Hi Trevor, On 6/11/24 5:13 PM, Trevor Woerner via lists.yoctoproject.org wrote: > On Tue 2024-06-11 @ 04:31:26 PM, Quentin Schulz via lists.yoctoproject.org wrote: >> Hi Trevor, >> >> On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote: >>> Allow the user to choose their preferred wic image format. >>> >> >> Can you provide some use case for this? > > In order to build a wic.xz image the build has to first create a wic image, > then compress it as an extra step. On a slow machine this extra step takes > a noticeable amount of time. > > On my local, slow build machine I prefer wic images since it saves build time > not having to do the compression and since everything is local, there's no > over-the-internet xfer time to consider. > > When I build using some remote, fast build machine, the extra time spent doing > the extra step of compressing is recuperated by the savings in transfer time > retrieving the image from the remote. Therefore I prefer to make wic.xz > images. Building wic images on remote machines would lead to very long xfer > times to download the image artifact. > > If I simply do: > > IMAGE_FSTYPES += "wic.xz" > > in my conf/local.conf then MACHINEs that don't normally build wic images will > fail. So the only way to build wic.xz images on remote builds is to tweak the > meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always > carrying this tweak for every remote build that I do. > > So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to > wic.xz but that wouldn't suit my use-case since I could then have to tweak it > again for any local builds. Besides, I could never guess what preferred wic > version others would prefer and I don't want to be changing it every other > week. > > So this patch maintains the existing behaviour completely, and when I do a > build on a remote machine I can simply set > > WIC_FSTYPE = "wic.xz" > > in my conf/local.conf without having to tweak the layer. > > And if others want compression but would prefer bz2 or gzip (or whatever) then > then can specify it however they wish. > >>> Signed-off-by: Trevor Woerner <twoerner@gmail.com> >>> --- >>> conf/machine/include/rockchip-wic.inc | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc >>> index dab61d83ed2c..eb895cd0b4ad 100644 >>> --- a/conf/machine/include/rockchip-wic.inc >>> +++ b/conf/machine/include/rockchip-wic.inc >>> @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc >>> SPL_BINARY ?= "idbloader.img" >>> -IMAGE_FSTYPES += "wic wic.bmap" >>> +WIC_FSTYPE ?= "wic" >>> +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap" >> >> Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to >> achieve here? >> >> If so, shouldn't we also not build wic.bmap there? > > The goal isn't to not build a wic image, the point is to allow the user to > specify which type of wic image (i.e. with or without compression and if with > compression, which one?). > > I guess I could do: > > WIC_COMPRESSION ?= "" > IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap" > > And then specify: > > WIC_COMPRESSION = ".xz" > > in my conf/local.conf? > Thanks for taking the time to explain your use case :) I was about to suggest this :) Should we allow multiple values in WIC_COMPRESSION? Also maybe it should be named "EXTENSION" instead since we have the dot in there? Additionally, I don't see wic.xz in IMAGE_TYPES in https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/image_types.bbclass, is xz really working? If so, why do we need to specify wic.gz and all the others in IMAGE_TYPES? Maybe we do not? Cheers, Quentin
On Tue 2024-06-11 @ 05:28:42 PM, Quentin Schulz via lists.yoctoproject.org wrote: > Hi Trevor, > > On 6/11/24 5:13 PM, Trevor Woerner via lists.yoctoproject.org wrote: > > On Tue 2024-06-11 @ 04:31:26 PM, Quentin Schulz via lists.yoctoproject.org wrote: > > > Hi Trevor, > > > > > > On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote: > > > > Allow the user to choose their preferred wic image format. > > > > > > > > > > Can you provide some use case for this? > > > > In order to build a wic.xz image the build has to first create a wic image, > > then compress it as an extra step. On a slow machine this extra step takes > > a noticeable amount of time. > > > > On my local, slow build machine I prefer wic images since it saves build time > > not having to do the compression and since everything is local, there's no > > over-the-internet xfer time to consider. > > > > When I build using some remote, fast build machine, the extra time spent doing > > the extra step of compressing is recuperated by the savings in transfer time > > retrieving the image from the remote. Therefore I prefer to make wic.xz > > images. Building wic images on remote machines would lead to very long xfer > > times to download the image artifact. > > > > If I simply do: > > > > IMAGE_FSTYPES += "wic.xz" > > > > in my conf/local.conf then MACHINEs that don't normally build wic images will > > fail. So the only way to build wic.xz images on remote builds is to tweak the > > meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always > > carrying this tweak for every remote build that I do. > > > > So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to > > wic.xz but that wouldn't suit my use-case since I could then have to tweak it > > again for any local builds. Besides, I could never guess what preferred wic > > version others would prefer and I don't want to be changing it every other > > week. > > > > So this patch maintains the existing behaviour completely, and when I do a > > build on a remote machine I can simply set > > > > WIC_FSTYPE = "wic.xz" > > > > in my conf/local.conf without having to tweak the layer. > > > > And if others want compression but would prefer bz2 or gzip (or whatever) then > > then can specify it however they wish. > > > > > > Signed-off-by: Trevor Woerner <twoerner@gmail.com> > > > > --- > > > > conf/machine/include/rockchip-wic.inc | 3 ++- > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc > > > > index dab61d83ed2c..eb895cd0b4ad 100644 > > > > --- a/conf/machine/include/rockchip-wic.inc > > > > +++ b/conf/machine/include/rockchip-wic.inc > > > > @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc > > > > SPL_BINARY ?= "idbloader.img" > > > > -IMAGE_FSTYPES += "wic wic.bmap" > > > > +WIC_FSTYPE ?= "wic" > > > > +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap" > > > > > > Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to > > > achieve here? > > > > > > If so, shouldn't we also not build wic.bmap there? > > > > The goal isn't to not build a wic image, the point is to allow the user to > > specify which type of wic image (i.e. with or without compression and if with > > compression, which one?). > > > > I guess I could do: > > > > WIC_COMPRESSION ?= "" > > IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap" > > > > And then specify: > > > > WIC_COMPRESSION = ".xz" > > > > in my conf/local.conf? > > > > Thanks for taking the time to explain your use case :) > > I was about to suggest this :) > > Should we allow multiple values in WIC_COMPRESSION? > > Also maybe it should be named "EXTENSION" instead since we have the dot in > there? Sounds good. > Additionally, I don't see wic.xz in IMAGE_TYPES in https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/image_types.bbclass, > is xz really working? I have been using "wic.xz" in IMAGE_FSTYPES for years. Works with bmaptool too. > If so, why do we need to specify wic.gz and all the > others in IMAGE_TYPES? Maybe we do not? I guess that's for others to answer/investigate.
diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc index dab61d83ed2c..eb895cd0b4ad 100644 --- a/conf/machine/include/rockchip-wic.inc +++ b/conf/machine/include/rockchip-wic.inc @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc SPL_BINARY ?= "idbloader.img" -IMAGE_FSTYPES += "wic wic.bmap" +WIC_FSTYPE ?= "wic" +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap" WKS_FILE ?= "rockchip.wks" WKS_FILE_DEPENDS ?= " \ e2fsprogs-native \
Allow the user to choose their preferred wic image format. Signed-off-by: Trevor Woerner <twoerner@gmail.com> --- conf/machine/include/rockchip-wic.inc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)