diff mbox series

bitbake-layers: add/remove option to not use tinfoil

Message ID 20240330164640.7270-1-simone.p.weiss@posteo.com
State New
Headers show
Series bitbake-layers: add/remove option to not use tinfoil | expand

Commit Message

Simone Weiß March 30, 2024, 4:46 p.m. UTC
From: Simone Weiß <simone.p.weiss@posteo.com>

Fixes [YOCTO #15417]

When a layer adds a new dependency after it was added to a conf, it can not be
removed w/o this dependency in the setup. Even the dependent layer can not be
added, as the tinfoil setup will fail.
Add an option to not perform the tinfoil at all, the use will be at own risk,
i.e. the added layers might not parse properly afterwards.
This is not merged into the force options as it even changes the loading of
plugins from other layers and is hence even more invasive as force.

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
---
 bin/bitbake-layers     | 13 +++++++++----
 lib/bblayers/action.py |  6 ++++--
 2 files changed, 13 insertions(+), 6 deletions(-)

Comments

Richard Purdie April 2, 2024, 1:29 p.m. UTC | #1
On Sat, 2024-03-30 at 16:46 +0000, Simone Weiß wrote:
> From: Simone Weiß <simone.p.weiss@posteo.com>
> 
> Fixes [YOCTO #15417]
> 
> When a layer adds a new dependency after it was added to a conf, it can not be
> removed w/o this dependency in the setup. Even the dependent layer can not be
> added, as the tinfoil setup will fail.
> Add an option to not perform the tinfoil at all, the use will be at own risk,
> i.e. the added layers might not parse properly afterwards.
> This is not merged into the force options as it even changes the loading of
> plugins from other layers and is hence even more invasive as force.

Could you expand a bit more about this difference to the force option
please?

I hadn't realised there was a force option and it may make sense to
merge this into that option? I'm still not 100% sure of the
differences/risks in doing that though.

Cheers,

Richard
Simone Weiß April 2, 2024, 8:51 p.m. UTC | #2
On Tue, 2024-04-02 at 14:29 +0100, Richard Purdie wrote:
> On Sat, 2024-03-30 at 16:46 +0000, Simone Weiß wrote:
> > From: Simone Weiß <simone.p.weiss@posteo.com>
> > 
> > Fixes [YOCTO #15417]
> > 
> > When a layer adds a new dependency after it was added to a conf, it
> > can not be
> > removed w/o this dependency in the setup. Even the dependent layer can
> > not be
> > added, as the tinfoil setup will fail.
> > Add an option to not perform the tinfoil at all, the use will be at
> > own risk,
> > i.e. the added layers might not parse properly afterwards.
> > This is not merged into the force options as it even changes the
> > loading of
> > plugins from other layers and is hence even more invasive as force.
> 
> Could you expand a bit more about this difference to the force option
> please?
> 
The force option skips the recipe parse verification, this means after all
other setup is done the recipes are not parsed before whatever subcommand
is executed.

When a layer dependency is causing problems, this will not work.  bitbake-
layers first sets up tinfoil to find further plugins
for itself via getVar(BBPATH). This would already be causing issues when -
for example - a layer with dependency issues should be removed via
bitbake-layers. As this restricts also which plugins can be found for
bitbake-layers to those under the the topdir, I separated it from force. 

Best,
Simone
Simone Weiß April 10, 2024, 8:13 p.m. UTC | #3
On Tue, 2024-04-02 at 20:51 +0000, Simone Weiß wrote:
> On Tue, 2024-04-02 at 14:29 +0100, Richard Purdie wrote:
> > On Sat, 2024-03-30 at 16:46 +0000, Simone Weiß wrote:
> > > From: Simone Weiß <simone.p.weiss@posteo.com>
> > > 
> > > Fixes [YOCTO #15417]
> > > 
> > > When a layer adds a new dependency after it was added to a conf, it
> > > can not be
> > > removed w/o this dependency in the setup. Even the dependent layer
> > > can
> > > not be
> > > added, as the tinfoil setup will fail.
> > > Add an option to not perform the tinfoil at all, the use will be at
> > > own risk,
> > > i.e. the added layers might not parse properly afterwards.
> > > This is not merged into the force options as it even changes the
> > > loading of
> > > plugins from other layers and is hence even more invasive as force.
> > 
> > Could you expand a bit more about this difference to the force option
> > please?
> > 
> The force option skips the recipe parse verification, this means after
> all
> other setup is done the recipes are not parsed before whatever
> subcommand
> is executed.
> 
> When a layer dependency is causing problems, this will not work. 
> bitbake-
> layers first sets up tinfoil to find further plugins
> for itself via getVar(BBPATH). This would already be causing issues when
> -
> for example - a layer with dependency issues should be removed via
> bitbake-layers. As this restricts also which plugins can be found for
> bitbake-layers to those under the the topdir, I separated it from force.
> 
> Best,
> Simone
> 
> 
Any further thoughts/opinions?
Richard Purdie April 11, 2024, 4:11 p.m. UTC | #4
On Wed, 2024-04-10 at 20:13 +0000, Simone Weiß wrote:
> On Tue, 2024-04-02 at 20:51 +0000, Simone Weiß wrote:
> > On Tue, 2024-04-02 at 14:29 +0100, Richard Purdie wrote:
> > > On Sat, 2024-03-30 at 16:46 +0000, Simone Weiß wrote:
> > > > From: Simone Weiß <simone.p.weiss@posteo.com>
> > > > 
> > > > Fixes [YOCTO #15417]
> > > > 
> > > > When a layer adds a new dependency after it was added to a conf, it
> > > > can not be
> > > > removed w/o this dependency in the setup. Even the dependent layer
> > > > can
> > > > not be
> > > > added, as the tinfoil setup will fail.
> > > > Add an option to not perform the tinfoil at all, the use will be at
> > > > own risk,
> > > > i.e. the added layers might not parse properly afterwards.
> > > > This is not merged into the force options as it even changes the
> > > > loading of
> > > > plugins from other layers and is hence even more invasive as force.
> > > 
> > > Could you expand a bit more about this difference to the force option
> > > please?
> > > 
> > The force option skips the recipe parse verification, this means after
> > all
> > other setup is done the recipes are not parsed before whatever
> > subcommand
> > is executed.
> > 
> > When a layer dependency is causing problems, this will not work. 
> > bitbake-
> > layers first sets up tinfoil to find further plugins
> > for itself via getVar(BBPATH). This would already be causing issues when
> > -
> > for example - a layer with dependency issues should be removed via
> > bitbake-layers. As this restricts also which plugins can be found for
> > bitbake-layers to those under the the topdir, I separated it from force.
> > 
> > Best,
> > Simone
> > 
> > 
> Any further thoughts/opinions?

I'm not happy about having two very similar options but I wasn't able
to look into this in any detail and hence I've not been able to make
any sensible suggestion on how to move forward, sorry.

Cheers,

Richard
diff mbox series

Patch

diff --git a/bin/bitbake-layers b/bin/bitbake-layers
index d4b1d1aa..f5002861 100755
--- a/bin/bitbake-layers
+++ b/bin/bitbake-layers
@@ -34,6 +34,7 @@  def main():
     parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true')
     parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true')
     parser.add_argument('-F', '--force', help='Force add without recipe parse verification', action='store_true')
+    parser.add_argument('-S', '--skip_parse_tests', help='Force run without parsing the layers, this might cause later failures.', action='store_true')
     parser.add_argument('--color', choices=['auto', 'always', 'never'], default='auto', help='Colorize output (where %(metavar)s is %(choices)s)', metavar='COLOR')
 
     global_args, unparsed_args = parser.parse_known_args()
@@ -59,16 +60,20 @@  def main():
     plugins = []
     tinfoil = bb.tinfoil.Tinfoil(tracking=True)
     tinfoil.logger.setLevel(logger.getEffectiveLevel())
-    try:
+    if global_args.skip_parse_tests:
+        bbpaths = []
+    else:
         tinfoil.prepare(True)
-        for path in ([topdir] +
-                    tinfoil.config_data.getVar('BBPATH').split(':')):
+        bbpaths = tinfoil.config_data.getVar('BBPATH').split(':')
+    
+    try: 
+        for path in ([topdir] + bbpaths):
             pluginpath = os.path.join(path, 'lib', 'bblayers')
             bb.utils.load_plugins(logger, plugins, pluginpath)
 
         registered = False
         for plugin in plugins:
-            if hasattr(plugin, 'tinfoil_init'):
+            if hasattr(plugin, 'tinfoil_init') and not global_args.skip_parse_tests:
                 plugin.tinfoil_init(tinfoil)
             if hasattr(plugin, 'register_commands'):
                 registered = True
diff --git a/lib/bblayers/action.py b/lib/bblayers/action.py
index a8f26993..5aecc452 100644
--- a/lib/bblayers/action.py
+++ b/lib/bblayers/action.py
@@ -50,8 +50,8 @@  class ActionPlugin(LayerPlugin):
 
         try:
             notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None)
-            self.tinfoil.modified_files()
-            if not (args.force or notadded):
+            if not (args.force or notadded or args.skip_parse_tests):
+                self.tinfoil.modified_files()
                 try:
                     self.tinfoil.run_command('parseConfiguration')
                 except (bb.tinfoil.TinfoilUIException, bb.BBHandledException):
@@ -83,6 +83,8 @@  class ActionPlugin(LayerPlugin):
                 layerdir = os.path.abspath(item)
             layerdirs.append(layerdir)
         (_, notremoved) = bb.utils.edit_bblayers_conf(bblayers_conf, None, layerdirs)
+        if args.skip_parse_tests:
+            return 0
         self.tinfoil.modified_files()
         if notremoved:
             for item in notremoved: