| Message ID | 20250520075325.64756-3-fhoerni.opensource@witekio.com |
|---|---|
| State | New |
| Headers | show |
| Series | image_types_wic: fix dependencies on wks files | expand |
On Tue, 2025-05-20 at 09:53 +0200, fhoerni via lists.openembedded.org wrote: > From: Frederic Hoerni <fhoerni@witekio.com> > > Add dependencies on all eligible files in WKS_SEARCH_PATH. > > Before this patch, adding a wks file earlier in the search path > was not taken into account automatically. > > To reproduce the issue: > > 1. Init (use default WKS_FILE="qemux86-directdisk.wks") > > . oe-init-build-env > echo 'IMAGE_FSTYPES += "wic"' >> conf/local.conf > > 2. First clean build > > rm -f wic/qemux86-directdisk.wks > bitbake core-image-minimal -c cleansstate && bitbake core-image-minimal > > 3. Create an override for WKS_FILE (earlier in WKS_SEARCH_PATH) > and introduce an error (a very small partition size) so that we can see it happen (or not). > > mkdir -p wic > echo "part / --source rootfs --fstype=ext4 --fixed-size 1M" > wic/qemux86-directdisk.wks > > 4. Build again > > bitbake core-image-minimal > ... > NOTE: Tasks Summary: Attempted 4670 tasks of which 4670 didn't need to be rerun and all succeeded. > > => Nothing gets built, whereas we should see this error: > > ERROR: Actual rootfs size (28713 kB) is larger than allowed size 1024 kB > > Signed-off-by: Frederic Hoerni <fhoerni@witekio.com> > --- > meta/classes-recipe/image_types_wic.bbclass | 18 ++++-------------- > 1 file changed, 4 insertions(+), 14 deletions(-) > > diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass > index 61f028bd7b..33773db2e5 100644 > --- a/meta/classes-recipe/image_types_wic.bbclass > +++ b/meta/classes-recipe/image_types_wic.bbclass > @@ -45,17 +45,11 @@ inherit_defer ${@bb.utils.contains('INITRAMFS_IMAGE_BUNDLE', '1', 'kernel-artifa > WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks" > WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks" > WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" > -WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) or ''}" > +WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH'), d) or ''}" > > -def wks_search(files, search_path): > - for f in files: > - if os.path.isabs(f): > - if os.path.exists(f): > - return f > - else: > - searched = bb.utils.which(search_path, f) > - if searched: > - return searched > +def wks_search(files, search_path, d): > + from oe.searchfile import search_file > + return search_file(search_path, files, d) > > WIC_CREATE_EXTRA_ARGS ?= "" > > @@ -160,10 +154,6 @@ python () { > d.setVar('WKS_TEMPLATE_PATH', wks_file_u) > d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True') > > - # We need to re-parse each time the file changes, and bitbake > - # needs to be told about that explicitly. > - bb.parse.mark_dependency(d, wks_file) > - > try: > with open(wks_file, 'r') as f: > body = f.read() Sorry for the delay in replying to this. We need to get to the bottom of what is going on with the dependencies here. Looking at the code in the image_types_wic.bbclass, it has this: IMAGE_CMD:wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES TOPDIR" i.e. it specifically asks for there not to be a dependency on WKS_FILES and WKS_FILL_PATH. It then does: # Rebuild when the wks file or vars in WICVARS change USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}" WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}" do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}" i.e. the task should have a file-checksums entry. Looking at: "bitbake-dumpsig tmp/stamps/qemux86_64-poky-linux/core-image-minimal/1.0.do_image_wic.sigdata.XXX" I don't see a file-checksums entry or any of the right dependencies. It is that which we need to fix. To do that we need to track both the found files and the files which we looked for but did not find in file- checksums. Something like: diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass index 740ed946f8f..6180874a4cc 100644 --- a/meta/classes-recipe/image_types_wic.bbclass +++ b/meta/classes-recipe/image_types_wic.bbclass @@ -57,6 +57,16 @@ def wks_search(files, search_path): if searched: return searched +def wks_checksums(files, search_path): + ret = "" + for f in files: + found, hist = bb.utils.which(search_path, f, history=True) + ret = ret + " " + " ".join(h + ":False" for h in hist[:-1]) + if found: + ret = ret + " " + found + ":True" + return ret + + WIC_CREATE_EXTRA_ARGS ?= "" IMAGE_CMD:wic () { @@ -98,7 +108,7 @@ do_image_wic[cleandirs] = "${WORKDIR}/build-wic" # Rebuild when the wks file or vars in WICVARS change USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}" -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}" +WKS_FILE_CHECKSUM = "${@wks_checksums(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) if '${USING_WIC}' else ''}" do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}" do_image_wic[depends] += "${@' '.join('%s-native:do_populate_sysroot' % r for r in ('parted', 'gptfdisk', 'dosfstools', 'mtools'))}" I'm hoping that patch will fix some of your other corner case issues too! Cheers, Richard
diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass index 61f028bd7b..33773db2e5 100644 --- a/meta/classes-recipe/image_types_wic.bbclass +++ b/meta/classes-recipe/image_types_wic.bbclass @@ -45,17 +45,11 @@ inherit_defer ${@bb.utils.contains('INITRAMFS_IMAGE_BUNDLE', '1', 'kernel-artifa WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks" WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks" WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" -WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) or ''}" +WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH'), d) or ''}" -def wks_search(files, search_path): - for f in files: - if os.path.isabs(f): - if os.path.exists(f): - return f - else: - searched = bb.utils.which(search_path, f) - if searched: - return searched +def wks_search(files, search_path, d): + from oe.searchfile import search_file + return search_file(search_path, files, d) WIC_CREATE_EXTRA_ARGS ?= "" @@ -160,10 +154,6 @@ python () { d.setVar('WKS_TEMPLATE_PATH', wks_file_u) d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True') - # We need to re-parse each time the file changes, and bitbake - # needs to be told about that explicitly. - bb.parse.mark_dependency(d, wks_file) - try: with open(wks_file, 'r') as f: body = f.read()