mbox series

[00/11] build-appliance-image revival

Message ID 20230221204445.23931-1-twoerner@gmail.com
Headers show
Series build-appliance-image revival | expand

Message

Trevor Woerner Feb. 21, 2023, 8:44 p.m. UTC 
Revive the build-appliance.

The build-appliance-image produces an image that contains all the necessary
pieces in order to perform a bitbake build. This patch series contains a
number of small tweaks that were needed in order for the build of the
appliance to succeed, and then for a build within the appliance to succeed as
well.

Trevor Woerner (11):
  packagegroup-self-hosted: alphabetize
  packagegroup-self-hosted: add zstd
  build-appliance-image: add support for qemu
  build-appliance-image: set TERM
  build-appliance-image kernel: linux-yocto: qemuall: add taskstats
  build-appliance-image: fix HOMEPAGE
  build-appliance-image: add /lib64 symlink
  build-appliance-image: use a real /tmp
  build-appliance-image: QB_MEM: allow user config
  QB_SMP: allow user modification
  build-appliance-image: check for xattr feature

 meta/conf/machine/include/riscv/qemuriscv.inc |  2 +-
 .../conf/machine/include/x86/qemuboot-x86.inc |  2 +-
 meta/conf/machine/qemuarm.conf                |  2 +-
 meta/conf/machine/qemuarm64.conf              |  2 +-
 meta/conf/machine/qemuppc64.conf              |  2 +-
 .../images/build-appliance-image_15.0.0.bb    | 27 ++++++++++++++-----
 .../packagegroups/packagegroup-self-hosted.bb | 11 ++++----
 meta/recipes-kernel/linux/linux-yocto.inc     |  2 +-
 8 files changed, 33 insertions(+), 17 deletions(-)

Comments

Trevor Woerner Feb. 22, 2023, 2:25 a.m. UTC | #1 
On Tue 2023-02-21 @ 03:44:42 PM, Trevor Woerner via lists.openembedded.org wrote:
> Make /tmp a real directory in the root filesystem and not a link to an
> in-memory tmpfs. Otherwise bitbake will run out of inodes on /tmp very quickly
> and stop any build.
> 
> Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> ---
>  meta/recipes-core/images/build-appliance-image_15.0.0.bb | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/meta/recipes-core/images/build-appliance-image_15.0.0.bb b/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> index 3a84de0a2b43..1f7b96a9fcf4 100644
> --- a/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> +++ b/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> @@ -113,6 +113,12 @@ fakeroot do_tweak_image () {
>  	# add a /lib64 symlink
>  	# this is needed for building rust-native on a 64-bit build appliance
>  	ln -rs ${IMAGE_ROOTFS}/lib ${IMAGE_ROOTFS}/lib64
> +
> +	# make /tmp a real directory and not a link to an in-memory tmpfs
> +	# otherwise /tmp will run out of inodes very quickly when bitbaking
> +	sed -i -e 's#l root root 1777 /tmp /var/tmp#d root root 1777 /tmp none#' ${IMAGE_ROOTFS}/etc/default/volatiles/00_core
> +	mkdir -p ${IMAGE_ROOTFS}/tmp
> +	chmod 1777 ${IMAGE_ROOTFS}/tmp
>  }

Richard, this feels rather "hacky" to me. I can't help wonder if it would be
better to implement a VOLATILE_TMP_DIR mechanism (along the same lines as
VOLATILE_LOG_DIR)?

https://docs.yoctoproject.org/dev/ref-manual/variables.html?highlight=volatile_log_dir#term-VOLATILE_LOG_DIR
https://git.openembedded.org/openembedded-core/tree/meta/recipes-core/initscripts/initscripts_1.0.bb#n107
Richard Purdie Feb. 22, 2023, 2:46 p.m. UTC | #2 
On Tue, 2023-02-21 at 21:25 -0500, Trevor Woerner wrote:
> On Tue 2023-02-21 @ 03:44:42 PM, Trevor Woerner via lists.openembedded.org wrote:
> > Make /tmp a real directory in the root filesystem and not a link to an
> > in-memory tmpfs. Otherwise bitbake will run out of inodes on /tmp very quickly
> > and stop any build.
> > 
> > Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> > ---
> >  meta/recipes-core/images/build-appliance-image_15.0.0.bb | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/meta/recipes-core/images/build-appliance-image_15.0.0.bb b/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> > index 3a84de0a2b43..1f7b96a9fcf4 100644
> > --- a/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> > +++ b/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> > @@ -113,6 +113,12 @@ fakeroot do_tweak_image () {
> >  	# add a /lib64 symlink
> >  	# this is needed for building rust-native on a 64-bit build appliance
> >  	ln -rs ${IMAGE_ROOTFS}/lib ${IMAGE_ROOTFS}/lib64
> > +
> > +	# make /tmp a real directory and not a link to an in-memory tmpfs
> > +	# otherwise /tmp will run out of inodes very quickly when bitbaking
> > +	sed -i -e 's#l root root 1777 /tmp /var/tmp#d root root 1777 /tmp none#' ${IMAGE_ROOTFS}/etc/default/volatiles/00_core
> > +	mkdir -p ${IMAGE_ROOTFS}/tmp
> > +	chmod 1777 ${IMAGE_ROOTFS}/tmp
> >  }
> 
> Richard, this feels rather "hacky" to me. I can't help wonder if it would be
> better to implement a VOLATILE_TMP_DIR mechanism (along the same lines as
> VOLATILE_LOG_DIR)?
> 
> https://docs.yoctoproject.org/dev/ref-manual/variables.html?highlight=volatile_log_dir#term-VOLATILE_LOG_DIR
> https://git.openembedded.org/openembedded-core/tree/meta/recipes-core/initscripts/initscripts_1.0.bb#n107

When I saw this, I was wondering about that too...

I think I'll hold off this patch whilst we think about that a bit but
my memory on this area is taking a bit of paging back in.

Cheers,

Richard
Trevor Woerner Feb. 22, 2023, 8:40 p.m. UTC | #3 
On Wed 2023-02-22 @ 02:46:30 PM, Richard Purdie wrote:
> On Tue, 2023-02-21 at 21:25 -0500, Trevor Woerner wrote:
> > On Tue 2023-02-21 @ 03:44:42 PM, Trevor Woerner via lists.openembedded.org wrote:
> > > Make /tmp a real directory in the root filesystem and not a link to an
> > > in-memory tmpfs. Otherwise bitbake will run out of inodes on /tmp very quickly
> > > and stop any build.
> > > 
> > > Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> > > ---
> > >  meta/recipes-core/images/build-appliance-image_15.0.0.bb | 6 ++++++
> > >  1 file changed, 6 insertions(+)
> > > 
> > > diff --git a/meta/recipes-core/images/build-appliance-image_15.0.0.bb b/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> > > index 3a84de0a2b43..1f7b96a9fcf4 100644
> > > --- a/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> > > +++ b/meta/recipes-core/images/build-appliance-image_15.0.0.bb
> > > @@ -113,6 +113,12 @@ fakeroot do_tweak_image () {
> > >  	# add a /lib64 symlink
> > >  	# this is needed for building rust-native on a 64-bit build appliance
> > >  	ln -rs ${IMAGE_ROOTFS}/lib ${IMAGE_ROOTFS}/lib64
> > > +
> > > +	# make /tmp a real directory and not a link to an in-memory tmpfs
> > > +	# otherwise /tmp will run out of inodes very quickly when bitbaking
> > > +	sed -i -e 's#l root root 1777 /tmp /var/tmp#d root root 1777 /tmp none#' ${IMAGE_ROOTFS}/etc/default/volatiles/00_core
> > > +	mkdir -p ${IMAGE_ROOTFS}/tmp
> > > +	chmod 1777 ${IMAGE_ROOTFS}/tmp
> > >  }
> > 
> > Richard, this feels rather "hacky" to me. I can't help wonder if it would be
> > better to implement a VOLATILE_TMP_DIR mechanism (along the same lines as
> > VOLATILE_LOG_DIR)?
> > 
> > https://docs.yoctoproject.org/dev/ref-manual/variables.html?highlight=volatile_log_dir#term-VOLATILE_LOG_DIR
> > https://git.openembedded.org/openembedded-core/tree/meta/recipes-core/initscripts/initscripts_1.0.bb#n107
> 
> When I saw this, I was wondering about that too...
> 
> I think I'll hold off this patch whilst we think about that a bit but
> my memory on this area is taking a bit of paging back in.

Sounds good. In that case I'll wait for the others to hit master then re-work
this patch to integrate a VOLATILE_TMP_DIR mechanism?