| Message ID | 20260403183541.2631883-5-twoerner@gmail.com |
|---|---|
| State | Under Review |
| Headers | show |
| Series | standalone wic | expand |
On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > When "wic create ..." is invoked with a bare *wks name (i.e. without the > `.wks` extension), wic calls engine.py:find_canned_images() to find the > fully qualified *wks file. This function searches every directory formed by: > - permutating all BBLAYERS with `/wic` > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > - checking `<scripts_path>/lib/wic/canned-wks` > Where `<scripts_path>` is the directory containing the wic program. It doesn't. I just looked at the code and it uses BBPATH. That can be similar to BBLAYERS but it is different and the commit messages really should refer to BBPATH. > When wic is part of oe-core, the last search path succeeds in finding > the canned *wks files in `<topdir>/scripts/lib/wic/canned-wks` (since > the wic program is found in oe-core's `<topdir>/scripts` directory, and > `<topdir>/scripts` is not a BBLAYER). > > However, once wic is removed from oe-core, this algorithm will not find > these bare *wks files in any of the above-mentioned search paths since > the oe-core layer will no longer be the home of the wic program, and the > canned *wks files are not located in any directory relative to BBLAYERS. I'm a bit confused by this reasoning. Are you saying that wic will no longer search BBPATH (or WKS_SEARCH_PATH)? > Since these *wks files are specific to oe-core's meta layer, they should > stay with this layer. Therefore move the *wks files so they exist in one > of the locations searched relative to oe-core/meta's BBLAYERS. I don't understand why these need to move given WKS_SEARCH_PATH remains unchanged, unless wic is going to ignore WKS_SEARCH_PATH going forward? Cheers, Richard
On Fri, 2026-04-03 at 22:13 +0100, Richard Purdie via lists.openembedded.org wrote: > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > fully qualified *wks file. This function searches every directory formed by: > > - permutating all BBLAYERS with `/wic` > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > - checking `<scripts_path>/lib/wic/canned-wks` > > Where `<scripts_path>` is the directory containing the wic program. > > It doesn't. I just looked at the code and it uses BBPATH. That can be > similar to BBLAYERS but it is different and the commit messages really > should refer to BBPATH. > > > When wic is part of oe-core, the last search path succeeds in finding > > the canned *wks files in `<topdir>/scripts/lib/wic/canned-wks` (since > > the wic program is found in oe-core's `<topdir>/scripts` directory, and > > `<topdir>/scripts` is not a BBLAYER). > > > > However, once wic is removed from oe-core, this algorithm will not find > > these bare *wks files in any of the above-mentioned search paths since > > the oe-core layer will no longer be the home of the wic program, and the > > canned *wks files are not located in any directory relative to BBLAYERS. > > I'm a bit confused by this reasoning. Are you saying that wic will no > longer search BBPATH (or WKS_SEARCH_PATH)? > > > Since these *wks files are specific to oe-core's meta layer, they should > > stay with this layer. Therefore move the *wks files so they exist in one > > of the locations searched relative to oe-core/meta's BBLAYERS. > > I don't understand why these need to move given WKS_SEARCH_PATH remains > unchanged, unless wic is going to ignore WKS_SEARCH_PATH going forward? Looking at this a bit more, the issue is that you're removing the directory when you separate wic out so the files 'need' to move due to that? You could in fact just leave them there if I understand correctly? Cheers, Richard
On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote: > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > fully qualified *wks file. This function searches every directory formed by: > > - permutating all BBLAYERS with `/wic` > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > - checking `<scripts_path>/lib/wic/canned-wks` > > Where `<scripts_path>` is the directory containing the wic program. > > It doesn't. I just looked at the code and it uses BBPATH. That can be > similar to BBLAYERS but it is different and the commit messages really > should refer to BBPATH. In the oe-selftest wic tests, most of the "wic create ..." commands are called with bare *wks files (i.e. *wks files without the `.wks` extension). When this happens, as "wic create..." is called, it starts by running the code found here: https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206 which calls: https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46 which follows the algorithm that I have described above. So for the oe-selftests to pass I wanted to move them. > > When wic is part of oe-core, the last search path succeeds in finding > > the canned *wks files in `<topdir>/scripts/lib/wic/canned-wks` (since > > the wic program is found in oe-core's `<topdir>/scripts` directory, and > > `<topdir>/scripts` is not a BBLAYER). > > > > However, once wic is removed from oe-core, this algorithm will not find > > these bare *wks files in any of the above-mentioned search paths since > > the oe-core layer will no longer be the home of the wic program, and the > > canned *wks files are not located in any directory relative to BBLAYERS. > > I'm a bit confused by this reasoning. Are you saying that wic will no > longer search BBPATH (or WKS_SEARCH_PATH)? > > > Since these *wks files are specific to oe-core's meta layer, they should > > stay with this layer. Therefore move the *wks files so they exist in one > > of the locations searched relative to oe-core/meta's BBLAYERS. > > I don't understand why these need to move given WKS_SEARCH_PATH remains > unchanged, unless wic is going to ignore WKS_SEARCH_PATH going forward? As an independent tool, these *wks files are rather oe-core-specific, so I thought they should stay with the oe-core/meta layer, the same way that, say, raspberry pi-specific *wks files stay in the raspberry pi layer. My first thought was to remove them from the standalone wic repository altogether, but then decided I could keep them as examples. I thought it would be dangerous to leave them in the wic source tree where the above algorithm could find them. If I leave them in src/wic/canned-wks, someone uses the tool and creates a *wks file with a similar name to one of the canned-wks files, but they have a typo and the canned one gets used instead of theirs... ...maybe I'm overthinking it?
On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote: > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote: > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > > fully qualified *wks file. This function searches every directory formed by: > > > - permutating all BBLAYERS with `/wic` > > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > > - checking `<scripts_path>/lib/wic/canned-wks` > > > Where `<scripts_path>` is the directory containing the wic program. > > > > It doesn't. I just looked at the code and it uses BBPATH. That can be > > similar to BBLAYERS but it is different and the commit messages really > > should refer to BBPATH. > > In the oe-selftest wic tests, most of the "wic create ..." commands > are called with bare *wks files (i.e. *wks files without the `.wks` > extension). When this happens, as "wic create..." is called, it starts > by running the code found here: > > https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206 > > which calls: > > https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46 > > which follows the algorithm that I have described above. > > So for the oe-selftests to pass I wanted to move them. You are right, it is using BBLAYERS. This is very confusing since the wks files are using BBPATH, which is similar but subtly different. Is there any reason it can't use BBPATH to match? I agree the files need to move but I'd like them to end up in meta/files/wic, not meta/wic. Whilst it complicates the migration a little, I think it would be best to move them once and get them into the right place. The questions are then firstly, how to do that simply, then secondly, how to clean up the rest of this (and when to do that). Adding files/wic into the search path is trivial. Deprecating the existing search paths with appropriate warnings is a bit trickier and there is the question of when we should do that, before or after the current release. "after" would seem more obvious, but it will mean two different behaviours before and after release and with a standalone tool, this does become harder. > > I don't understand why these need to move given WKS_SEARCH_PATH remains > > unchanged, unless wic is going to ignore WKS_SEARCH_PATH going forward? > > As an independent tool, these *wks files are rather oe-core-specific, > so I thought they should stay with the oe-core/meta layer, the same way > that, say, raspberry pi-specific *wks files stay in the raspberry pi > layer. > > My first thought was to remove them from the standalone wic repository > altogether, but then decided I could keep them as examples. > > I thought it would be dangerous to leave them in the wic source > tree where the above algorithm could find them. If I leave them in > src/wic/canned-wks, someone uses the tool and creates a *wks file with a > similar name to one of the canned-wks files, but they have a typo and > the canned one gets used instead of theirs... > > ...maybe I'm overthinking it? It is definitely dangerous to leave them in the search paths so I agree with that. The files do belong in OE-Core too. It is still a little dangerous having files lying around which have the same names as the ones in core since this kind of duplication still can confuse people when they edit the wrong file. We have bigger issues I guess but some renaming of the examples might help. Cheers, Richard
On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote: > > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote: > > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > > > fully qualified *wks file. This function searches every directory formed by: > > > > - permutating all BBLAYERS with `/wic` > > > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > > > - checking `<scripts_path>/lib/wic/canned-wks` > > > > Where `<scripts_path>` is the directory containing the wic program. > > > > > > It doesn't. I just looked at the code and it uses BBPATH. That can be > > > similar to BBLAYERS but it is different and the commit messages really > > > should refer to BBPATH. > > > > In the oe-selftest wic tests, most of the "wic create ..." commands > > are called with bare *wks files (i.e. *wks files without the `.wks` > > extension). When this happens, as "wic create..." is called, it starts > > by running the code found here: > > > > https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206 > > > > which calls: > > > > https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46 > > > > which follows the algorithm that I have described above. > > > > So for the oe-selftests to pass I wanted to move them. > > You are right, it is using BBLAYERS. This is very confusing since the > wks files are using BBPATH, which is similar but subtly different. > > Is there any reason it can't use BBPATH to match? > > I agree the files need to move but I'd like them to end up in > meta/files/wic, not meta/wic. Whilst it complicates the migration a > little, I think it would be best to move them once and get them into > the right place. > > The questions are then firstly, how to do that simply, then secondly, > how to clean up the rest of this (and when to do that). > > Adding files/wic into the search path is trivial. Deprecating the > existing search paths with appropriate warnings is a bit trickier and > there is the question of when we should do that, before or after the > current release. "after" would seem more obvious, but it will mean two > different behaviours before and after release and with a standalone > tool, this does become harder. I'm only guessing here, but it looks like someone wrote "wic create" to run two different ways: with the wks file specified with or without the ".wks" extension. If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is what occurs in the oe-selftest tests, i.e. when wic is run independently without being part of a bitbake build) then wic invokes its own search algorithm to try to find the wks file using BBLAYERS and script_dir. In this case the wks file is specified without the ".wks" and no path information is given on the cmdline. It is up to wic to find the actual wks file itself. If the ".wks" is included (which is what happens when wic is called as part of a bitbake build and therefore the image_types_wic.bbclass is used) then the class adds the ".wks" at the end, and also gives wic the path to the wks file which it finds using WKS_SEARCH_PATH which is based on BBPATH and COREBASE. Assuming the persona of someone who wants to use this new, independent wic tool, and who knows nothing about The Yocto Project or bitbake, I think having wic look in "magical" places for the wks file would be hard to understand and surprising. As an independent tool I think the user should provide the path to the actual wks file (with the ".wks" extension) and if that file can't be found as specified on the cmdline, wic should simply fail with an error. As part of making wic an independent tool, I think wic's code to search for a wks file if the ".wks" is not provided should be removed. This means that, as part of the transition, I would have to modify each "wic create" test in oe-core's oe-selftests. I'm fine with that. That could be done cleanly if I write a little function to find the wks file in the wic.py oe-selftest itself using the same logic as is used in the image_types_wic.bbclass. To summarize: in my opinion, wic shouldn't have search logic. From wic's point of view the wks file should be specified on the cmdline in a way that wic will find the file the user wants to use (or not find it). The oe-selftests should be updated to use the same search logic from the image_types_wic.bbclass so that when it invokes "wic create" it is providing wic with the path to a wks file that has a ".wks" extension (which is how bitbake invokes wic) instead of specifying a bare wks file and hoping wic will find it. I'm happy to add "files/wic" as another location for image_types_wic.bbclass to search for wks files. > > > I don't understand why these need to move given WKS_SEARCH_PATH remains > > > unchanged, unless wic is going to ignore WKS_SEARCH_PATH going forward? > > > > As an independent tool, these *wks files are rather oe-core-specific, > > so I thought they should stay with the oe-core/meta layer, the same way > > that, say, raspberry pi-specific *wks files stay in the raspberry pi > > layer. > > > > My first thought was to remove them from the standalone wic repository > > altogether, but then decided I could keep them as examples. > > > > I thought it would be dangerous to leave them in the wic source > > tree where the above algorithm could find them. If I leave them in > > src/wic/canned-wks, someone uses the tool and creates a *wks file with a > > similar name to one of the canned-wks files, but they have a typo and > > the canned one gets used instead of theirs... > > > > ...maybe I'm overthinking it? > > It is definitely dangerous to leave them in the search paths so I agree > with that. The files do belong in OE-Core too. It is still a little > dangerous having files lying around which have the same names as the > ones in core since this kind of duplication still can confuse people > when they edit the wrong file. We have bigger issues I guess but some > renaming of the examples might help. > > Cheers, > > Richard > >
On Sat 2026-04-04 @ 11:46:43 AM, Trevor Woerner wrote: > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote: > > > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote: > > > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > > > > fully qualified *wks file. This function searches every directory formed by: > > > > > - permutating all BBLAYERS with `/wic` > > > > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > > > > - checking `<scripts_path>/lib/wic/canned-wks` > > > > > Where `<scripts_path>` is the directory containing the wic program. > > > > > > > > It doesn't. I just looked at the code and it uses BBPATH. That can be > > > > similar to BBLAYERS but it is different and the commit messages really > > > > should refer to BBPATH. > > > > > > In the oe-selftest wic tests, most of the "wic create ..." commands > > > are called with bare *wks files (i.e. *wks files without the `.wks` > > > extension). When this happens, as "wic create..." is called, it starts > > > by running the code found here: > > > > > > https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206 > > > > > > which calls: > > > > > > https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46 > > > > > > which follows the algorithm that I have described above. > > > > > > So for the oe-selftests to pass I wanted to move them. > > > > You are right, it is using BBLAYERS. This is very confusing since the > > wks files are using BBPATH, which is similar but subtly different. > > > > Is there any reason it can't use BBPATH to match? > > > > I agree the files need to move but I'd like them to end up in > > meta/files/wic, not meta/wic. Whilst it complicates the migration a > > little, I think it would be best to move them once and get them into > > the right place. > > > > The questions are then firstly, how to do that simply, then secondly, > > how to clean up the rest of this (and when to do that). > > > > Adding files/wic into the search path is trivial. Deprecating the > > existing search paths with appropriate warnings is a bit trickier and > > there is the question of when we should do that, before or after the > > current release. "after" would seem more obvious, but it will mean two > > different behaviours before and after release and with a standalone > > tool, this does become harder. > > I'm only guessing here, but it looks like someone wrote "wic create" to > run two different ways: with the wks file specified with or without the > ".wks" extension. > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > what occurs in the oe-selftest tests, i.e. when wic is run independently > without being part of a bitbake build) then wic invokes its own search > algorithm to try to find the wks file using BBLAYERS and script_dir. In > this case the wks file is specified without the ".wks" and no path > information is given on the cmdline. It is up to wic to find the actual > wks file itself. > > If the ".wks" is included (which is what happens when wic is called as > part of a bitbake build and therefore the image_types_wic.bbclass is > used) then the class adds the ".wks" at the end, and also gives wic the > path to the wks file which it finds using WKS_SEARCH_PATH which is based > on BBPATH and COREBASE. > > Assuming the persona of someone who wants to use this new, independent > wic tool, and who knows nothing about The Yocto Project or bitbake, I > think having wic look in "magical" places for the wks file would be hard > to understand and surprising. As an independent tool I think the user > should provide the path to the actual wks file (with the ".wks" > extension) and if that file can't be found as specified on the cmdline, > wic should simply fail with an error. > > As part of making wic an independent tool, I think wic's code to search > for a wks file if the ".wks" is not provided should be removed. This > means that, as part of the transition, I would have to modify each "wic > create" test in oe-core's oe-selftests. I'm fine with that. That could > be done cleanly if I write a little function to find the wks file in > the wic.py oe-selftest itself using the same logic as is used in the > image_types_wic.bbclass. > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > point of view the wks file should be specified on the cmdline in a way > that wic will find the file the user wants to use (or not find it). > The oe-selftests should be updated to use the same search logic from > the image_types_wic.bbclass so that when it invokes "wic create" it is > providing wic with the path to a wks file that has a ".wks" extension > (which is how bitbake invokes wic) instead of specifying a bare wks file > and hoping wic will find it. Looking at the code, it's a little deeper than that. wic also has a "wic list" subcommand to "list available canned images and source plugins" and it uses the same search logic (using BBLAYERS) to find "canned images". The existing wic documentation, help text, and code refers to these bare wks names as "images" or "canned images". If you specify a bare wks to "wic create" and it can't find the wks file, the error message says (in part): "No image named %s found..." (instead of "No wks named %s found...". Maybe I'm not understanding the history here, or maybe there's some use-case I don't understand (or has long since stopped working) but what "wic create" needs here is a wks file, not an image name, and conflating the two seems very odd. If it were up to me I'd clean the whole thing up. "wic create" needs a wks file to guide it, not an image name that wic is going to use to find a <image>.wks file using its own search logic. I already have two patches to do this: one to add the bbclass search logic to the oe-selftest wic tests, and a second one to remove wic's image search logic from wic itself. The first one is fairly easy to add and justify (I think). The second one might be harder. Maybe there is some strange use-case where we want wic to handle "wic create <bare-wks>" where <bare-wks> is thought of as an image name? Removing wic's bare-wks search logic also removes the "wic list images" subcommand since the only point of that subcommand is to invoke its search logic and find "images" (which are actually wks files). > I'm happy to add "files/wic" as another location for > image_types_wic.bbclass to search for wks files. > > > > > I don't understand why these need to move given WKS_SEARCH_PATH remains > > > > unchanged, unless wic is going to ignore WKS_SEARCH_PATH going forward? > > > > > > As an independent tool, these *wks files are rather oe-core-specific, > > > so I thought they should stay with the oe-core/meta layer, the same way > > > that, say, raspberry pi-specific *wks files stay in the raspberry pi > > > layer. > > > > > > My first thought was to remove them from the standalone wic repository > > > altogether, but then decided I could keep them as examples. > > > > > > I thought it would be dangerous to leave them in the wic source > > > tree where the above algorithm could find them. If I leave them in > > > src/wic/canned-wks, someone uses the tool and creates a *wks file with a > > > similar name to one of the canned-wks files, but they have a typo and > > > the canned one gets used instead of theirs... > > > > > > ...maybe I'm overthinking it? > > > > It is definitely dangerous to leave them in the search paths so I agree > > with that. The files do belong in OE-Core too. It is still a little > > dangerous having files lying around which have the same names as the > > ones in core since this kind of duplication still can confuse people > > when they edit the wrong file. We have bigger issues I guess but some > > renaming of the examples might help. > > > > Cheers, > > > > Richard > > > >
On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote: > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote: > > > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote: > > > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > > > > fully qualified *wks file. This function searches every directory formed by: > > > > > - permutating all BBLAYERS with `/wic` > > > > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > > > > - checking `<scripts_path>/lib/wic/canned-wks` > > > > > Where `<scripts_path>` is the directory containing the wic program. > > > > > > > > It doesn't. I just looked at the code and it uses BBPATH. That can be > > > > similar to BBLAYERS but it is different and the commit messages really > > > > should refer to BBPATH. > > > > > > In the oe-selftest wic tests, most of the "wic create ..." commands > > > are called with bare *wks files (i.e. *wks files without the `.wks` > > > extension). When this happens, as "wic create..." is called, it starts > > > by running the code found here: > > > > > > https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206 > > > > > > which calls: > > > > > > https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46 > > > > > > which follows the algorithm that I have described above. > > > > > > So for the oe-selftests to pass I wanted to move them. > > > > You are right, it is using BBLAYERS. This is very confusing since the > > wks files are using BBPATH, which is similar but subtly different. > > > > Is there any reason it can't use BBPATH to match? > > > > I agree the files need to move but I'd like them to end up in > > meta/files/wic, not meta/wic. Whilst it complicates the migration a > > little, I think it would be best to move them once and get them into > > the right place. > > > > The questions are then firstly, how to do that simply, then secondly, > > how to clean up the rest of this (and when to do that). > > > > Adding files/wic into the search path is trivial. Deprecating the > > existing search paths with appropriate warnings is a bit trickier and > > there is the question of when we should do that, before or after the > > current release. "after" would seem more obvious, but it will mean two > > different behaviours before and after release and with a standalone > > tool, this does become harder. > > I'm only guessing here, but it looks like someone wrote "wic create" to > run two different ways: with the wks file specified with or without the > ".wks" extension. > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > what occurs in the oe-selftest tests, i.e. when wic is run independently > without being part of a bitbake build) then wic invokes its own search > algorithm to try to find the wks file using BBLAYERS and script_dir. In > this case the wks file is specified without the ".wks" and no path > information is given on the cmdline. It is up to wic to find the actual > wks file itself. > > If the ".wks" is included (which is what happens when wic is called as > part of a bitbake build and therefore the image_types_wic.bbclass is > used) then the class adds the ".wks" at the end, and also gives wic the > path to the wks file which it finds using WKS_SEARCH_PATH which is based > on BBPATH and COREBASE. > > Assuming the persona of someone who wants to use this new, independent > wic tool, and who knows nothing about The Yocto Project or bitbake, I > think having wic look in "magical" places for the wks file would be hard > to understand and surprising. As an independent tool I think the user > should provide the path to the actual wks file (with the ".wks" > extension) and if that file can't be found as specified on the cmdline, > wic should simply fail with an error. > > As part of making wic an independent tool, I think wic's code to search > for a wks file if the ".wks" is not provided should be removed. This > means that, as part of the transition, I would have to modify each "wic > create" test in oe-core's oe-selftests. I'm fine with that. That could > be done cleanly if I write a little function to find the wks file in > the wic.py oe-selftest itself using the same logic as is used in the > image_types_wic.bbclass. > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > point of view the wks file should be specified on the cmdline in a way > that wic will find the file the user wants to use (or not find it). > The oe-selftests should be updated to use the same search logic from > the image_types_wic.bbclass so that when it invokes "wic create" it is > providing wic with the path to a wks file that has a ".wks" extension > (which is how bitbake invokes wic) instead of specifying a bare wks file > and hoping wic will find it. > > I'm happy to add "files/wic" as another location for > image_types_wic.bbclass to search for wks files. I guess the first question is are we going to take this for wrynose or not? I'm not particularly happy with the files a top level wic directory, I think it needs to be meta/files/wic. To make that work, we need to tweak the search paths in wic and in image_types_wic.bbclass. If we can resolve that issue, we're probably good to move forward with merging. I agree that the search path logic should be tidied up. Whether you can provide a search path to wic or whether that has to be done in metadata is an open question. I do see your perspective, equally, if you are using wic within OE, having some search knowledge may be ok. I would want to see it basically using files/wic from BBPATH with everything else removed. There is a question of how you warn users about old paths. This piece is too late for the release though. So I'd propose we add files/wic to the search path, move the files there and that lets us at least get some of the changes into the release? Cheers, Richard
On Sat 2026-04-04 @ 07:36:05 PM, Richard Purdie wrote: > On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote: > > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > > On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote: > > > > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote: > > > > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote: > > > > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the > > > > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the > > > > > > fully qualified *wks file. This function searches every directory formed by: > > > > > > - permutating all BBLAYERS with `/wic` > > > > > > - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` > > > > > > - checking `<scripts_path>/lib/wic/canned-wks` > > > > > > Where `<scripts_path>` is the directory containing the wic program. > > > > > > > > > > It doesn't. I just looked at the code and it uses BBPATH. That can be > > > > > similar to BBLAYERS but it is different and the commit messages really > > > > > should refer to BBPATH. > > > > > > > > In the oe-selftest wic tests, most of the "wic create ..." commands > > > > are called with bare *wks files (i.e. *wks files without the `.wks` > > > > extension). When this happens, as "wic create..." is called, it starts > > > > by running the code found here: > > > > > > > > https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206 > > > > > > > > which calls: > > > > > > > > https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46 > > > > > > > > which follows the algorithm that I have described above. > > > > > > > > So for the oe-selftests to pass I wanted to move them. > > > > > > You are right, it is using BBLAYERS. This is very confusing since the > > > wks files are using BBPATH, which is similar but subtly different. > > > > > > Is there any reason it can't use BBPATH to match? > > > > > > I agree the files need to move but I'd like them to end up in > > > meta/files/wic, not meta/wic. Whilst it complicates the migration a > > > little, I think it would be best to move them once and get them into > > > the right place. > > > > > > The questions are then firstly, how to do that simply, then secondly, > > > how to clean up the rest of this (and when to do that). > > > > > > Adding files/wic into the search path is trivial. Deprecating the > > > existing search paths with appropriate warnings is a bit trickier and > > > there is the question of when we should do that, before or after the > > > current release. "after" would seem more obvious, but it will mean two > > > different behaviours before and after release and with a standalone > > > tool, this does become harder. > > > > I'm only guessing here, but it looks like someone wrote "wic create" to > > run two different ways: with the wks file specified with or without the > > ".wks" extension. > > > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > > what occurs in the oe-selftest tests, i.e. when wic is run independently > > without being part of a bitbake build) then wic invokes its own search > > algorithm to try to find the wks file using BBLAYERS and script_dir. In > > this case the wks file is specified without the ".wks" and no path > > information is given on the cmdline. It is up to wic to find the actual > > wks file itself. > > > > If the ".wks" is included (which is what happens when wic is called as > > part of a bitbake build and therefore the image_types_wic.bbclass is > > used) then the class adds the ".wks" at the end, and also gives wic the > > path to the wks file which it finds using WKS_SEARCH_PATH which is based > > on BBPATH and COREBASE. > > > > Assuming the persona of someone who wants to use this new, independent > > wic tool, and who knows nothing about The Yocto Project or bitbake, I > > think having wic look in "magical" places for the wks file would be hard > > to understand and surprising. As an independent tool I think the user > > should provide the path to the actual wks file (with the ".wks" > > extension) and if that file can't be found as specified on the cmdline, > > wic should simply fail with an error. > > > > As part of making wic an independent tool, I think wic's code to search > > for a wks file if the ".wks" is not provided should be removed. This > > means that, as part of the transition, I would have to modify each "wic > > create" test in oe-core's oe-selftests. I'm fine with that. That could > > be done cleanly if I write a little function to find the wks file in > > the wic.py oe-selftest itself using the same logic as is used in the > > image_types_wic.bbclass. > > > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > > point of view the wks file should be specified on the cmdline in a way > > that wic will find the file the user wants to use (or not find it). > > The oe-selftests should be updated to use the same search logic from > > the image_types_wic.bbclass so that when it invokes "wic create" it is > > providing wic with the path to a wks file that has a ".wks" extension > > (which is how bitbake invokes wic) instead of specifying a bare wks file > > and hoping wic will find it. > > > > I'm happy to add "files/wic" as another location for > > image_types_wic.bbclass to search for wks files. > > I guess the first question is are we going to take this for wrynose or > not? I'm not particularly happy with the files a top level wic > directory, I think it needs to be meta/files/wic. To make that work, we > need to tweak the search paths in wic and in image_types_wic.bbclass. > If we can resolve that issue, we're probably good to move forward with > merging. > > I agree that the search path logic should be tidied up. Whether you can > provide a search path to wic or whether that has to be done in metadata > is an open question. I do see your perspective, equally, if you are > using wic within OE, having some search knowledge may be ok. I would > want to see it basically using files/wic from BBPATH with everything > else removed. There is a question of how you warn users about old > paths. This piece is too late for the release though. > > So I'd propose we add files/wic to the search path, move the files > there and that lets us at least get some of the changes into the > release? Sounds perfect, thanks! > Cheers, > > Richard >
On Sat, 2026-04-04 at 16:38 -0400, Trevor Woerner wrote: > On Sat 2026-04-04 @ 07:36:05 PM, Richard Purdie wrote: > > On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote: > > > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > > > > > I'm only guessing here, but it looks like someone wrote "wic create" to > > > run two different ways: with the wks file specified with or without the > > > ".wks" extension. > > > > > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > > > what occurs in the oe-selftest tests, i.e. when wic is run independently > > > without being part of a bitbake build) then wic invokes its own search > > > algorithm to try to find the wks file using BBLAYERS and script_dir. In > > > this case the wks file is specified without the ".wks" and no path > > > information is given on the cmdline. It is up to wic to find the actual > > > wks file itself. > > > > > > If the ".wks" is included (which is what happens when wic is called as > > > part of a bitbake build and therefore the image_types_wic.bbclass is > > > used) then the class adds the ".wks" at the end, and also gives wic the > > > path to the wks file which it finds using WKS_SEARCH_PATH which is based > > > on BBPATH and COREBASE. > > > > > > Assuming the persona of someone who wants to use this new, independent > > > wic tool, and who knows nothing about The Yocto Project or bitbake, I > > > think having wic look in "magical" places for the wks file would be hard > > > to understand and surprising. As an independent tool I think the user > > > should provide the path to the actual wks file (with the ".wks" > > > extension) and if that file can't be found as specified on the cmdline, > > > wic should simply fail with an error. > > > > > > As part of making wic an independent tool, I think wic's code to search > > > for a wks file if the ".wks" is not provided should be removed. This > > > means that, as part of the transition, I would have to modify each "wic > > > create" test in oe-core's oe-selftests. I'm fine with that. That could > > > be done cleanly if I write a little function to find the wks file in > > > the wic.py oe-selftest itself using the same logic as is used in the > > > image_types_wic.bbclass. > > > > > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > > > point of view the wks file should be specified on the cmdline in a way > > > that wic will find the file the user wants to use (or not find it). > > > The oe-selftests should be updated to use the same search logic from > > > the image_types_wic.bbclass so that when it invokes "wic create" it is > > > providing wic with the path to a wks file that has a ".wks" extension > > > (which is how bitbake invokes wic) instead of specifying a bare wks file > > > and hoping wic will find it. > > > > > > I'm happy to add "files/wic" as another location for > > > image_types_wic.bbclass to search for wks files. > > > > I guess the first question is are we going to take this for wrynose or > > not? I'm not particularly happy with the files a top level wic > > directory, I think it needs to be meta/files/wic. To make that work, we > > need to tweak the search paths in wic and in image_types_wic.bbclass. > > If we can resolve that issue, we're probably good to move forward with > > merging. > > > > I agree that the search path logic should be tidied up. Whether you can > > provide a search path to wic or whether that has to be done in metadata > > is an open question. I do see your perspective, equally, if you are > > using wic within OE, having some search knowledge may be ok. I would > > want to see it basically using files/wic from BBPATH with everything > > else removed. There is a question of how you warn users about old > > paths. This piece is too late for the release though. > > > > So I'd propose we add files/wic to the search path, move the files > > there and that lets us at least get some of the changes into the > > release? > > Sounds perfect, thanks! Since we're running out of time and I really don't want to have to think about this too much more/again, I've: * written a patch to add files/wic as a search path in wic * written a patch to add files/wic as a search path in the class * dropped wic and the canned-wks paths in wic * dropped wic and the canned-wks paths in the class * converted wic to use BBPATH instead of BBLAYERS * added a sanity test to detect the old paths and error * pushed a contrib/rpurdie branch into the wic repo with the patches This is going to mean people have to update layers to use "files/wic" but we can clearly tell people they need to do it. If we went with either of the other locations and changed BBLAYERS -> BBPATH, there could be corner cases we miss in the detection. Autobuilder testing shows this will break a few layers but it is a clear error which people can easily fix. How does that look to people? Cheers, Richard
On Sun 2026-04-05 @ 10:17:40 AM, Richard Purdie wrote: > On Sat, 2026-04-04 at 16:38 -0400, Trevor Woerner wrote: > > On Sat 2026-04-04 @ 07:36:05 PM, Richard Purdie wrote: > > > On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote: > > > > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > > > > > > > I'm only guessing here, but it looks like someone wrote "wic create" to > > > > run two different ways: with the wks file specified with or without the > > > > ".wks" extension. > > > > > > > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > > > > what occurs in the oe-selftest tests, i.e. when wic is run independently > > > > without being part of a bitbake build) then wic invokes its own search > > > > algorithm to try to find the wks file using BBLAYERS and script_dir. In > > > > this case the wks file is specified without the ".wks" and no path > > > > information is given on the cmdline. It is up to wic to find the actual > > > > wks file itself. > > > > > > > > If the ".wks" is included (which is what happens when wic is called as > > > > part of a bitbake build and therefore the image_types_wic.bbclass is > > > > used) then the class adds the ".wks" at the end, and also gives wic the > > > > path to the wks file which it finds using WKS_SEARCH_PATH which is based > > > > on BBPATH and COREBASE. > > > > > > > > Assuming the persona of someone who wants to use this new, independent > > > > wic tool, and who knows nothing about The Yocto Project or bitbake, I > > > > think having wic look in "magical" places for the wks file would be hard > > > > to understand and surprising. As an independent tool I think the user > > > > should provide the path to the actual wks file (with the ".wks" > > > > extension) and if that file can't be found as specified on the cmdline, > > > > wic should simply fail with an error. > > > > > > > > As part of making wic an independent tool, I think wic's code to search > > > > for a wks file if the ".wks" is not provided should be removed. This > > > > means that, as part of the transition, I would have to modify each "wic > > > > create" test in oe-core's oe-selftests. I'm fine with that. That could > > > > be done cleanly if I write a little function to find the wks file in > > > > the wic.py oe-selftest itself using the same logic as is used in the > > > > image_types_wic.bbclass. > > > > > > > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > > > > point of view the wks file should be specified on the cmdline in a way > > > > that wic will find the file the user wants to use (or not find it). > > > > The oe-selftests should be updated to use the same search logic from > > > > the image_types_wic.bbclass so that when it invokes "wic create" it is > > > > providing wic with the path to a wks file that has a ".wks" extension > > > > (which is how bitbake invokes wic) instead of specifying a bare wks file > > > > and hoping wic will find it. > > > > > > > > I'm happy to add "files/wic" as another location for > > > > image_types_wic.bbclass to search for wks files. > > > > > > I guess the first question is are we going to take this for wrynose or > > > not? I'm not particularly happy with the files a top level wic > > > directory, I think it needs to be meta/files/wic. To make that work, we > > > need to tweak the search paths in wic and in image_types_wic.bbclass. > > > If we can resolve that issue, we're probably good to move forward with > > > merging. > > > > > > I agree that the search path logic should be tidied up. Whether you can > > > provide a search path to wic or whether that has to be done in metadata > > > is an open question. I do see your perspective, equally, if you are > > > using wic within OE, having some search knowledge may be ok. I would > > > want to see it basically using files/wic from BBPATH with everything > > > else removed. There is a question of how you warn users about old > > > paths. This piece is too late for the release though. > > > > > > So I'd propose we add files/wic to the search path, move the files > > > there and that lets us at least get some of the changes into the > > > release? > > > > Sounds perfect, thanks! > > Since we're running out of time and I really don't want to have to > think about this too much more/again, I've: > > * written a patch to add files/wic as a search path in wic > * written a patch to add files/wic as a search path in the class > * dropped wic and the canned-wks paths in wic > * dropped wic and the canned-wks paths in the class > * converted wic to use BBPATH instead of BBLAYERS > * added a sanity test to detect the old paths and error > * pushed a contrib/rpurdie branch into the wic repo with the patches > > This is going to mean people have to update layers to use "files/wic" > but we can clearly tell people they need to do it. If we went with > either of the other locations and changed BBLAYERS -> BBPATH, there > could be corner cases we miss in the detection. > > Autobuilder testing shows this will break a few layers but it is a > clear error which people can easily fix. > > How does that look to people? Sounds great to me! Presumably we will move the contrib/rpurdie branch to master in git.yoctoproject.org/wic and the SRC_URI of the wic recipe in oe-core will need a tweak before making it official? > > Cheers, > > Richard > > > > >
On Sun, 2026-04-05 at 08:04 -0400, Trevor Woerner wrote: > On Sun 2026-04-05 @ 10:17:40 AM, Richard Purdie wrote: > > On Sat, 2026-04-04 at 16:38 -0400, Trevor Woerner wrote: > > > On Sat 2026-04-04 @ 07:36:05 PM, Richard Purdie wrote: > > > > On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote: > > > > > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > > > > > > > > > I'm only guessing here, but it looks like someone wrote "wic create" to > > > > > run two different ways: with the wks file specified with or without the > > > > > ".wks" extension. > > > > > > > > > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > > > > > what occurs in the oe-selftest tests, i.e. when wic is run independently > > > > > without being part of a bitbake build) then wic invokes its own search > > > > > algorithm to try to find the wks file using BBLAYERS and script_dir. In > > > > > this case the wks file is specified without the ".wks" and no path > > > > > information is given on the cmdline. It is up to wic to find the actual > > > > > wks file itself. > > > > > > > > > > If the ".wks" is included (which is what happens when wic is called as > > > > > part of a bitbake build and therefore the image_types_wic.bbclass is > > > > > used) then the class adds the ".wks" at the end, and also gives wic the > > > > > path to the wks file which it finds using WKS_SEARCH_PATH which is based > > > > > on BBPATH and COREBASE. > > > > > > > > > > Assuming the persona of someone who wants to use this new, independent > > > > > wic tool, and who knows nothing about The Yocto Project or bitbake, I > > > > > think having wic look in "magical" places for the wks file would be hard > > > > > to understand and surprising. As an independent tool I think the user > > > > > should provide the path to the actual wks file (with the ".wks" > > > > > extension) and if that file can't be found as specified on the cmdline, > > > > > wic should simply fail with an error. > > > > > > > > > > As part of making wic an independent tool, I think wic's code to search > > > > > for a wks file if the ".wks" is not provided should be removed. This > > > > > means that, as part of the transition, I would have to modify each "wic > > > > > create" test in oe-core's oe-selftests. I'm fine with that. That could > > > > > be done cleanly if I write a little function to find the wks file in > > > > > the wic.py oe-selftest itself using the same logic as is used in the > > > > > image_types_wic.bbclass. > > > > > > > > > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > > > > > point of view the wks file should be specified on the cmdline in a way > > > > > that wic will find the file the user wants to use (or not find it). > > > > > The oe-selftests should be updated to use the same search logic from > > > > > the image_types_wic.bbclass so that when it invokes "wic create" it is > > > > > providing wic with the path to a wks file that has a ".wks" extension > > > > > (which is how bitbake invokes wic) instead of specifying a bare wks file > > > > > and hoping wic will find it. > > > > > > > > > > I'm happy to add "files/wic" as another location for > > > > > image_types_wic.bbclass to search for wks files. > > > > > > > > I guess the first question is are we going to take this for wrynose or > > > > not? I'm not particularly happy with the files a top level wic > > > > directory, I think it needs to be meta/files/wic. To make that work, we > > > > need to tweak the search paths in wic and in image_types_wic.bbclass. > > > > If we can resolve that issue, we're probably good to move forward with > > > > merging. > > > > > > > > I agree that the search path logic should be tidied up. Whether you can > > > > provide a search path to wic or whether that has to be done in metadata > > > > is an open question. I do see your perspective, equally, if you are > > > > using wic within OE, having some search knowledge may be ok. I would > > > > want to see it basically using files/wic from BBPATH with everything > > > > else removed. There is a question of how you warn users about old > > > > paths. This piece is too late for the release though. > > > > > > > > So I'd propose we add files/wic to the search path, move the files > > > > there and that lets us at least get some of the changes into the > > > > release? > > > > > > Sounds perfect, thanks! > > > > Since we're running out of time and I really don't want to have to > > think about this too much more/again, I've: > > > > * written a patch to add files/wic as a search path in wic > > * written a patch to add files/wic as a search path in the class > > * dropped wic and the canned-wks paths in wic > > * dropped wic and the canned-wks paths in the class > > * converted wic to use BBPATH instead of BBLAYERS > > * added a sanity test to detect the old paths and error > > * pushed a contrib/rpurdie branch into the wic repo with the patches > > > > This is going to mean people have to update layers to use "files/wic" > > but we can clearly tell people they need to do it. If we went with > > either of the other locations and changed BBLAYERS -> BBPATH, there > > could be corner cases we miss in the detection. > > > > Autobuilder testing shows this will break a few layers but it is a > > clear error which people can easily fix. > > > > How does that look to people? > > Sounds great to me! > > Presumably we will move the contrib/rpurdie branch to master in > git.yoctoproject.org/wic and the SRC_URI of the wic recipe in oe-core > will need a tweak before making it official? I pushed onto a contrib branch so we can make a decision on this. Assuming we go ahead, we'd merge the patches to master, the update the recipe accordingly. Cheers, Richard
On Sun 2026-04-05 @ 03:16:26 PM, Richard Purdie wrote: > On Sun, 2026-04-05 at 08:04 -0400, Trevor Woerner wrote: > > On Sun 2026-04-05 @ 10:17:40 AM, Richard Purdie wrote: > > > On Sat, 2026-04-04 at 16:38 -0400, Trevor Woerner wrote: > > > > On Sat 2026-04-04 @ 07:36:05 PM, Richard Purdie wrote: > > > > > On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote: > > > > > > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote: > > > > > > > > > > > > I'm only guessing here, but it looks like someone wrote "wic create" to > > > > > > run two different ways: with the wks file specified with or without the > > > > > > ".wks" extension. > > > > > > > > > > > > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is > > > > > > what occurs in the oe-selftest tests, i.e. when wic is run independently > > > > > > without being part of a bitbake build) then wic invokes its own search > > > > > > algorithm to try to find the wks file using BBLAYERS and script_dir. In > > > > > > this case the wks file is specified without the ".wks" and no path > > > > > > information is given on the cmdline. It is up to wic to find the actual > > > > > > wks file itself. > > > > > > > > > > > > If the ".wks" is included (which is what happens when wic is called as > > > > > > part of a bitbake build and therefore the image_types_wic.bbclass is > > > > > > used) then the class adds the ".wks" at the end, and also gives wic the > > > > > > path to the wks file which it finds using WKS_SEARCH_PATH which is based > > > > > > on BBPATH and COREBASE. > > > > > > > > > > > > Assuming the persona of someone who wants to use this new, independent > > > > > > wic tool, and who knows nothing about The Yocto Project or bitbake, I > > > > > > think having wic look in "magical" places for the wks file would be hard > > > > > > to understand and surprising. As an independent tool I think the user > > > > > > should provide the path to the actual wks file (with the ".wks" > > > > > > extension) and if that file can't be found as specified on the cmdline, > > > > > > wic should simply fail with an error. > > > > > > > > > > > > As part of making wic an independent tool, I think wic's code to search > > > > > > for a wks file if the ".wks" is not provided should be removed. This > > > > > > means that, as part of the transition, I would have to modify each "wic > > > > > > create" test in oe-core's oe-selftests. I'm fine with that. That could > > > > > > be done cleanly if I write a little function to find the wks file in > > > > > > the wic.py oe-selftest itself using the same logic as is used in the > > > > > > image_types_wic.bbclass. > > > > > > > > > > > > To summarize: in my opinion, wic shouldn't have search logic. From wic's > > > > > > point of view the wks file should be specified on the cmdline in a way > > > > > > that wic will find the file the user wants to use (or not find it). > > > > > > The oe-selftests should be updated to use the same search logic from > > > > > > the image_types_wic.bbclass so that when it invokes "wic create" it is > > > > > > providing wic with the path to a wks file that has a ".wks" extension > > > > > > (which is how bitbake invokes wic) instead of specifying a bare wks file > > > > > > and hoping wic will find it. > > > > > > > > > > > > I'm happy to add "files/wic" as another location for > > > > > > image_types_wic.bbclass to search for wks files. > > > > > > > > > > I guess the first question is are we going to take this for wrynose or > > > > > not? I'm not particularly happy with the files a top level wic > > > > > directory, I think it needs to be meta/files/wic. To make that work, we > > > > > need to tweak the search paths in wic and in image_types_wic.bbclass. > > > > > If we can resolve that issue, we're probably good to move forward with > > > > > merging. > > > > > > > > > > I agree that the search path logic should be tidied up. Whether you can > > > > > provide a search path to wic or whether that has to be done in metadata > > > > > is an open question. I do see your perspective, equally, if you are > > > > > using wic within OE, having some search knowledge may be ok. I would > > > > > want to see it basically using files/wic from BBPATH with everything > > > > > else removed. There is a question of how you warn users about old > > > > > paths. This piece is too late for the release though. > > > > > > > > > > So I'd propose we add files/wic to the search path, move the files > > > > > there and that lets us at least get some of the changes into the > > > > > release? > > > > > > > > Sounds perfect, thanks! > > > > > > Since we're running out of time and I really don't want to have to > > > think about this too much more/again, I've: > > > > > > * written a patch to add files/wic as a search path in wic > > > * written a patch to add files/wic as a search path in the class > > > * dropped wic and the canned-wks paths in wic > > > * dropped wic and the canned-wks paths in the class > > > * converted wic to use BBPATH instead of BBLAYERS > > > * added a sanity test to detect the old paths and error > > > * pushed a contrib/rpurdie branch into the wic repo with the patches > > > > > > This is going to mean people have to update layers to use "files/wic" > > > but we can clearly tell people they need to do it. If we went with > > > either of the other locations and changed BBLAYERS -> BBPATH, there > > > could be corner cases we miss in the detection. > > > > > > Autobuilder testing shows this will break a few layers but it is a > > > clear error which people can easily fix. > > > > > > How does that look to people? > > > > Sounds great to me! > > > > Presumably we will move the contrib/rpurdie branch to master in > > git.yoctoproject.org/wic and the SRC_URI of the wic recipe in oe-core > > will need a tweak before making it official? > > I pushed onto a contrib branch so we can make a decision on this. > Assuming we go ahead, we'd merge the patches to master, the update the > recipe accordingly. I've looked at your patches in oe-core master-next as well as in the wic repository contrib/rpurdie and they all look fine to me. This is a nice update for wic. > Cheers, > > Richard
diff --git a/scripts/lib/wic/canned-wks/common.wks.inc b/meta/wic/common.wks.inc similarity index 100% rename from scripts/lib/wic/canned-wks/common.wks.inc rename to meta/wic/common.wks.inc diff --git a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg b/meta/wic/directdisk-bootloader-config.cfg similarity index 100% rename from scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg rename to meta/wic/directdisk-bootloader-config.cfg diff --git a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks b/meta/wic/directdisk-bootloader-config.wks similarity index 100% rename from scripts/lib/wic/canned-wks/directdisk-bootloader-config.wks rename to meta/wic/directdisk-bootloader-config.wks diff --git a/scripts/lib/wic/canned-wks/directdisk-gpt.wks b/meta/wic/directdisk-gpt.wks similarity index 100% rename from scripts/lib/wic/canned-wks/directdisk-gpt.wks rename to meta/wic/directdisk-gpt.wks diff --git a/scripts/lib/wic/canned-wks/directdisk-multi-rootfs.wks b/meta/wic/directdisk-multi-rootfs.wks similarity index 100% rename from scripts/lib/wic/canned-wks/directdisk-multi-rootfs.wks rename to meta/wic/directdisk-multi-rootfs.wks diff --git a/scripts/lib/wic/canned-wks/directdisk.wks b/meta/wic/directdisk.wks similarity index 100% rename from scripts/lib/wic/canned-wks/directdisk.wks rename to meta/wic/directdisk.wks diff --git a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in b/meta/wic/efi-bootdisk.wks.in similarity index 100% rename from scripts/lib/wic/canned-wks/efi-bootdisk.wks.in rename to meta/wic/efi-bootdisk.wks.in diff --git a/scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in b/meta/wic/efi-uki-bootdisk.wks.in similarity index 100% rename from scripts/lib/wic/canned-wks/efi-uki-bootdisk.wks.in rename to meta/wic/efi-uki-bootdisk.wks.in diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/meta/wic/mkefidisk.wks similarity index 100% rename from scripts/lib/wic/canned-wks/mkefidisk.wks rename to meta/wic/mkefidisk.wks diff --git a/scripts/lib/wic/canned-wks/mkhybridiso.wks b/meta/wic/mkhybridiso.wks similarity index 100% rename from scripts/lib/wic/canned-wks/mkhybridiso.wks rename to meta/wic/mkhybridiso.wks diff --git a/scripts/lib/wic/canned-wks/qemuloongarch.wks b/meta/wic/qemuloongarch.wks similarity index 100% rename from scripts/lib/wic/canned-wks/qemuloongarch.wks rename to meta/wic/qemuloongarch.wks diff --git a/scripts/lib/wic/canned-wks/qemuriscv.wks b/meta/wic/qemuriscv.wks similarity index 100% rename from scripts/lib/wic/canned-wks/qemuriscv.wks rename to meta/wic/qemuriscv.wks diff --git a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks b/meta/wic/qemux86-directdisk.wks similarity index 100% rename from scripts/lib/wic/canned-wks/qemux86-directdisk.wks rename to meta/wic/qemux86-directdisk.wks diff --git a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks b/meta/wic/sdimage-bootpart.wks similarity index 100% rename from scripts/lib/wic/canned-wks/sdimage-bootpart.wks rename to meta/wic/sdimage-bootpart.wks diff --git a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks b/meta/wic/systemd-bootdisk.wks similarity index 100% rename from scripts/lib/wic/canned-wks/systemd-bootdisk.wks rename to meta/wic/systemd-bootdisk.wks
When "wic create ..." is invoked with a bare *wks name (i.e. without the `.wks` extension), wic calls engine.py:find_canned_images() to find the fully qualified *wks file. This function searches every directory formed by: - permutating all BBLAYERS with `/wic` - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks` - checking `<scripts_path>/lib/wic/canned-wks` Where `<scripts_path>` is the directory containing the wic program. When wic is part of oe-core, the last search path succeeds in finding the canned *wks files in `<topdir>/scripts/lib/wic/canned-wks` (since the wic program is found in oe-core's `<topdir>/scripts` directory, and `<topdir>/scripts` is not a BBLAYER). However, once wic is removed from oe-core, this algorithm will not find these bare *wks files in any of the above-mentioned search paths since the oe-core layer will no longer be the home of the wic program, and the canned *wks files are not located in any directory relative to BBLAYERS. Since these *wks files are specific to oe-core's meta layer, they should stay with this layer. Therefore move the *wks files so they exist in one of the locations searched relative to oe-core/meta's BBLAYERS. NOTE: this patch is in preparation for removing wic from oe-core the wic oe-selftests work fine with this patch being added now Reviewed-by: Bruce Ashfield <bruce.ashfield@gmail.com> Reviewed-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Trevor Woerner <twoerner@gmail.com> --- changes in v9: - continue to search scripts/lib/wic/canned-wks for *wks files - commit message updated to reflect this change and to more clearly explain how wic finds *wks files and why these files are moved changes in v8: - (v8 was sending sector-size cmdline patch) changes in v7: - (none) changes in v6: - re-ordered so that this patch occurs before wic is removed, thus saving the commit history changes in v5: - rebase with master - split patchset out again into individual, small patches changes in v4: - (skipped) changes in v3: - squashed into 1 large commit changes in v2: - (none) --- {scripts/lib/wic/canned-wks => meta/wic}/common.wks.inc | 0 .../wic/canned-wks => meta/wic}/directdisk-bootloader-config.cfg | 0 .../wic/canned-wks => meta/wic}/directdisk-bootloader-config.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/directdisk-gpt.wks | 0 .../lib/wic/canned-wks => meta/wic}/directdisk-multi-rootfs.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/directdisk.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/efi-bootdisk.wks.in | 0 {scripts/lib/wic/canned-wks => meta/wic}/efi-uki-bootdisk.wks.in | 0 {scripts/lib/wic/canned-wks => meta/wic}/mkefidisk.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/mkhybridiso.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/qemuloongarch.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/qemuriscv.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/qemux86-directdisk.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/sdimage-bootpart.wks | 0 {scripts/lib/wic/canned-wks => meta/wic}/systemd-bootdisk.wks | 0 15 files changed, 0 insertions(+), 0 deletions(-) rename {scripts/lib/wic/canned-wks => meta/wic}/common.wks.inc (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/directdisk-bootloader-config.cfg (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/directdisk-bootloader-config.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/directdisk-gpt.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/directdisk-multi-rootfs.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/directdisk.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/efi-bootdisk.wks.in (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/efi-uki-bootdisk.wks.in (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/mkefidisk.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/mkhybridiso.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/qemuloongarch.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/qemuriscv.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/qemux86-directdisk.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/sdimage-bootpart.wks (100%) rename {scripts/lib/wic/canned-wks => meta/wic}/systemd-bootdisk.wks (100%)