| Message ID | 20250325215001.643993-1-richard.purdie@linuxfoundation.org |
|---|---|
| State | Accepted, archived |
| Commit | 746dc664da9c289a3063350590d3b5aada13d8d6 |
| Headers | show |
| Series | [v3] meta/conf: Start to populate autobuilder config fragments | expand |
Thanks for the split, I'm fine with this. Alex On Tue, 25 Mar 2025 at 22:50, Richard Purdie via lists.openembedded.org <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote: > > This populates the fragments directory with: > > a) the default set of variables used in all autobuilder builds > b) the default resource related variables used in all autobuilder builds > c) three different multilib test configurations used by the autobuilder > > The aim here is to start to make some of the autobuilder configuration more > visable and patchable by users, and to allow some test confiturations to > be user selectable if appropriate and needed for debugging. > > The main aautobuilder fragment is probably not directly reusable by most > users, it contains the resource limits as used on the autobuilder itself. I > can see arguments both way for whether this should be included in OE-Core or not > but having an example of how we configure this is probably useful. > > Not all configuration in the autobuilder is being moved, this set of variables > is just a basic starting point. Some variables may ultimately make more sense > being migrated elsewhere, perhaps updating the main defaults for poky or nodistro. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > .../autobuilder-resource-constraints.conf | 26 +++++++++++++++++++ > .../yocto-autobuilder/autobuilder.conf | 13 ++++++++++ > .../multilib-mips64-n32.conf | 8 ++++++ > .../yocto-autobuilder/multilib-x86-lib32.conf | 6 +++++ > .../yocto-autobuilder/multilib-x86-lib64.conf | 6 +++++ > 5 files changed, 59 insertions(+) > create mode 100644 meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf > create mode 100644 meta/conf/fragments/yocto-autobuilder/autobuilder.conf > create mode 100644 meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf > create mode 100644 meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf > create mode 100644 meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf > > diff --git a/meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf b/meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf > new file mode 100644 > index 00000000000..58cadd92b64 > --- /dev/null > +++ b/meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf > @@ -0,0 +1,26 @@ > +BB_CONF_FRAGMENT_SUMMARY = "Resource constraint variables used on the Yocto Projects autobuilder" > +BB_CONF_FRAGMENT_DESCRIPTION = "Resource constraint variables used on the Yocto Projects autobuilder" > + > +BB_NUMBER_THREADS = '16' > +BB_NUMBER_PARSE_THREADS = '16' > +BB_SERVER_TIMEOUT = '60' > +PARALLEL_MAKE = '-j 16 -l 75' > +BB_PRESSURE_MAX_CPU = '20000' > +BB_PRESSURE_MAX_IO = '20000' > +BB_LOADFACTOR_MAX = '1.5' > +XZ_MEMLIMIT = '5%' > +XZ_THREADS = '8' > +ZSTD_THREADS = '8' > +BB_TASK_NICE_LEVEL = '5' > +BB_TASK_NICE_LEVEL:task-testimage = '0' > +BB_TASK_IONICE_LEVEL = '2.7' > +BB_TASK_IONICE_LEVEL:task-testimage = '2.1' > +TEST_QEMUBOOT_TIMEOUT = '1500' > +BB_DISKMON_DIRS ?= 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,30K HALT,${TMPDIR},100M,1K HALT,${DL_DIR},100M HALT,${SSTATE_DIR},100M HALT,/tmp,10M,1K' > +RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp' > +BB_HEARTBEAT_EVENT ?= '60' > +BB_LOG_HOST_STAT_ON_INTERVAL = '1' > +BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh -c 100 -t 3' > +BB_LOG_HOST_STAT_ON_FAILURE = '1' > +BB_LOG_HOST_STAT_CMDS_FAILURE = 'oe-time-dd-test.sh -l' > + > diff --git a/meta/conf/fragments/yocto-autobuilder/autobuilder.conf b/meta/conf/fragments/yocto-autobuilder/autobuilder.conf > new file mode 100644 > index 00000000000..e3bac343390 > --- /dev/null > +++ b/meta/conf/fragments/yocto-autobuilder/autobuilder.conf > @@ -0,0 +1,13 @@ > +BB_CONF_FRAGMENT_SUMMARY = "Default variables used on the Yocto Projects autobuilder builds" > +BB_CONF_FRAGMENT_DESCRIPTION = "Default variables used on the Yocto Projects autobuilder builds" > + > +QEMU_USE_KVM ?= 'True' > +INHERIT += 'report-error' > +PREMIRRORS = '' > +BB_GENERATE_MIRROR_TARBALLS = '1' > +IMAGE_CLASSES += 'testimage' > +SDK_EXT_TYPE = 'minimal' > +SDK_INCLUDE_TOOLCHAIN = '1' > +ESDK_LOCALCONF_REMOVE:append = 'BB_HASHSERVE' > +SDK_TOOLCHAIN_LANGS += 'rust' > + > diff --git a/meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf b/meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf > new file mode 100644 > index 00000000000..ec076b59f9f > --- /dev/null > +++ b/meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf > @@ -0,0 +1,8 @@ > +BB_CONF_FRAGMENT_SUMMARY = "Enable a MIPS64 triarch multilib" > +BB_CONF_FRAGMENT_DESCRIPTION = "Enable a MIPS64 triarch multilib" > + > +require conf/multilib.conf > +MULTILIBS = 'multilib:lib64 multilib:lib32' > +DEFAULTTUNE = 'mips64-n32' > +DEFAULTTUNE:virtclass-multilib-lib64 = 'mips64' > +DEFAULTTUNE:virtclass-multilib-lib32 = 'mips32r2' > diff --git a/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf > new file mode 100644 > index 00000000000..9b0e2578ac9 > --- /dev/null > +++ b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf > @@ -0,0 +1,6 @@ > +BB_CONF_FRAGMENT_SUMMARY = "Enable an x86 lib32 multilib" > +BB_CONF_FRAGMENT_DESCRIPTION = "Enable an x86 lib32 multilib" > + > +require conf/multilib.conf > +MULTILIBS = 'multilib:lib32' > +DEFAULTTUNE:virtclass-multilib-lib32 = 'x86' > diff --git a/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf > new file mode 100644 > index 00000000000..1986b994e4b > --- /dev/null > +++ b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf > @@ -0,0 +1,6 @@ > +BB_CONF_FRAGMENT_SUMMARY = "Enable an x86 lib64 multilib" > +BB_CONF_FRAGMENT_DESCRIPTION = "Enable an x86 lib64 multilib" > + > +require conf/multilib.conf > +MULTILIBS = 'multilib:lib64' > +DEFAULTTUNE:virtclass-multilib-lib64 = 'x86-64' > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#213620): https://lists.openembedded.org/g/openembedded-core/message/213620 > Mute This Topic: https://lists.openembedded.org/mt/111906583/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf b/meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf new file mode 100644 index 00000000000..58cadd92b64 --- /dev/null +++ b/meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf @@ -0,0 +1,26 @@ +BB_CONF_FRAGMENT_SUMMARY = "Resource constraint variables used on the Yocto Projects autobuilder" +BB_CONF_FRAGMENT_DESCRIPTION = "Resource constraint variables used on the Yocto Projects autobuilder" + +BB_NUMBER_THREADS = '16' +BB_NUMBER_PARSE_THREADS = '16' +BB_SERVER_TIMEOUT = '60' +PARALLEL_MAKE = '-j 16 -l 75' +BB_PRESSURE_MAX_CPU = '20000' +BB_PRESSURE_MAX_IO = '20000' +BB_LOADFACTOR_MAX = '1.5' +XZ_MEMLIMIT = '5%' +XZ_THREADS = '8' +ZSTD_THREADS = '8' +BB_TASK_NICE_LEVEL = '5' +BB_TASK_NICE_LEVEL:task-testimage = '0' +BB_TASK_IONICE_LEVEL = '2.7' +BB_TASK_IONICE_LEVEL:task-testimage = '2.1' +TEST_QEMUBOOT_TIMEOUT = '1500' +BB_DISKMON_DIRS ?= 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,30K HALT,${TMPDIR},100M,1K HALT,${DL_DIR},100M HALT,${SSTATE_DIR},100M HALT,/tmp,10M,1K' +RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp' +BB_HEARTBEAT_EVENT ?= '60' +BB_LOG_HOST_STAT_ON_INTERVAL = '1' +BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh -c 100 -t 3' +BB_LOG_HOST_STAT_ON_FAILURE = '1' +BB_LOG_HOST_STAT_CMDS_FAILURE = 'oe-time-dd-test.sh -l' + diff --git a/meta/conf/fragments/yocto-autobuilder/autobuilder.conf b/meta/conf/fragments/yocto-autobuilder/autobuilder.conf new file mode 100644 index 00000000000..e3bac343390 --- /dev/null +++ b/meta/conf/fragments/yocto-autobuilder/autobuilder.conf @@ -0,0 +1,13 @@ +BB_CONF_FRAGMENT_SUMMARY = "Default variables used on the Yocto Projects autobuilder builds" +BB_CONF_FRAGMENT_DESCRIPTION = "Default variables used on the Yocto Projects autobuilder builds" + +QEMU_USE_KVM ?= 'True' +INHERIT += 'report-error' +PREMIRRORS = '' +BB_GENERATE_MIRROR_TARBALLS = '1' +IMAGE_CLASSES += 'testimage' +SDK_EXT_TYPE = 'minimal' +SDK_INCLUDE_TOOLCHAIN = '1' +ESDK_LOCALCONF_REMOVE:append = 'BB_HASHSERVE' +SDK_TOOLCHAIN_LANGS += 'rust' + diff --git a/meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf b/meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf new file mode 100644 index 00000000000..ec076b59f9f --- /dev/null +++ b/meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf @@ -0,0 +1,8 @@ +BB_CONF_FRAGMENT_SUMMARY = "Enable a MIPS64 triarch multilib" +BB_CONF_FRAGMENT_DESCRIPTION = "Enable a MIPS64 triarch multilib" + +require conf/multilib.conf +MULTILIBS = 'multilib:lib64 multilib:lib32' +DEFAULTTUNE = 'mips64-n32' +DEFAULTTUNE:virtclass-multilib-lib64 = 'mips64' +DEFAULTTUNE:virtclass-multilib-lib32 = 'mips32r2' diff --git a/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf new file mode 100644 index 00000000000..9b0e2578ac9 --- /dev/null +++ b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf @@ -0,0 +1,6 @@ +BB_CONF_FRAGMENT_SUMMARY = "Enable an x86 lib32 multilib" +BB_CONF_FRAGMENT_DESCRIPTION = "Enable an x86 lib32 multilib" + +require conf/multilib.conf +MULTILIBS = 'multilib:lib32' +DEFAULTTUNE:virtclass-multilib-lib32 = 'x86' diff --git a/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf new file mode 100644 index 00000000000..1986b994e4b --- /dev/null +++ b/meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf @@ -0,0 +1,6 @@ +BB_CONF_FRAGMENT_SUMMARY = "Enable an x86 lib64 multilib" +BB_CONF_FRAGMENT_DESCRIPTION = "Enable an x86 lib64 multilib" + +require conf/multilib.conf +MULTILIBS = 'multilib:lib64' +DEFAULTTUNE:virtclass-multilib-lib64 = 'x86-64'
This populates the fragments directory with: a) the default set of variables used in all autobuilder builds b) the default resource related variables used in all autobuilder builds c) three different multilib test configurations used by the autobuilder The aim here is to start to make some of the autobuilder configuration more visable and patchable by users, and to allow some test confiturations to be user selectable if appropriate and needed for debugging. The main aautobuilder fragment is probably not directly reusable by most users, it contains the resource limits as used on the autobuilder itself. I can see arguments both way for whether this should be included in OE-Core or not but having an example of how we configure this is probably useful. Not all configuration in the autobuilder is being moved, this set of variables is just a basic starting point. Some variables may ultimately make more sense being migrated elsewhere, perhaps updating the main defaults for poky or nodistro. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- .../autobuilder-resource-constraints.conf | 26 +++++++++++++++++++ .../yocto-autobuilder/autobuilder.conf | 13 ++++++++++ .../multilib-mips64-n32.conf | 8 ++++++ .../yocto-autobuilder/multilib-x86-lib32.conf | 6 +++++ .../yocto-autobuilder/multilib-x86-lib64.conf | 6 +++++ 5 files changed, 59 insertions(+) create mode 100644 meta/conf/fragments/yocto-autobuilder/autobuilder-resource-constraints.conf create mode 100644 meta/conf/fragments/yocto-autobuilder/autobuilder.conf create mode 100644 meta/conf/fragments/yocto-autobuilder/multilib-mips64-n32.conf create mode 100644 meta/conf/fragments/yocto-autobuilder/multilib-x86-lib32.conf create mode 100644 meta/conf/fragments/yocto-autobuilder/multilib-x86-lib64.conf