| Message ID | 20240917151157.780068-1-jstephan@baylibre.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] bitbake: add documentation for "bitbake -S lockedsigs" | expand |
On Tue, 2024-09-17 at 17:11 +0200, Julien Stephan via lists.openembedded.org wrote: > The following commit added "lockedsigs" as new signature handler for the > -S option of bitbake: > 2cf13f25139 - lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs > > But it was not documented anywhere, so add some basic documentation for > it in both the .rst documentation files and the help message of bitbake > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > --- > .../bitbake-user-manual-execution.rst | 6 ++++++ > .../bitbake-user-manual-intro.rst | 13 +++++++++---- > bitbake/lib/bb/main.py | 12 +++++++----- > 3 files changed, 22 insertions(+), 9 deletions(-) Unfortunately I don't think this is quite so simple. The lockedsigs support is implemented in OE-Core and BitBake doesn't contain that code. This makes it very hard to accurately document since it is metadata dependent. I do agree we need to improve this somehow, I just don't know how. Cheers, Richard
Le mar. 17 sept. 2024 à 22:55, Richard Purdie <richard.purdie@linuxfoundation.org> a écrit : > > On Tue, 2024-09-17 at 17:11 +0200, Julien Stephan via lists.openembedded.org wrote: > > The following commit added "lockedsigs" as new signature handler for the > > -S option of bitbake: > > 2cf13f25139 - lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs > > > > But it was not documented anywhere, so add some basic documentation for > > it in both the .rst documentation files and the help message of bitbake > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > --- > > .../bitbake-user-manual-execution.rst | 6 ++++++ > > .../bitbake-user-manual-intro.rst | 13 +++++++++---- > > bitbake/lib/bb/main.py | 12 +++++++----- > > 3 files changed, 22 insertions(+), 9 deletions(-) > > Unfortunately I don't think this is quite so simple. The lockedsigs > support is implemented in OE-Core and BitBake doesn't contain that > code. This makes it very hard to accurately document since it is > metadata dependent. > > I do agree we need to improve this somehow, I just don't know how. Hi Richard, Maybe we should remove this from bitbake and implement it using bblock? :) Let me know if I can help on this! Cheers Julien > > Cheers, > > Richard > >
On Wed, 2024-09-18 at 09:05 +0200, Julien Stephan wrote: > Le mar. 17 sept. 2024 à 22:55, Richard Purdie > <richard.purdie@linuxfoundation.org> a écrit : > > > > On Tue, 2024-09-17 at 17:11 +0200, Julien Stephan via > > lists.openembedded.org wrote: > > > The following commit added "lockedsigs" as new signature handler > > > for the > > > -S option of bitbake: > > > 2cf13f25139 - lib/oe/sstatesig.py: dump locked.sigs.inc only > > > when explicitly asked via -S lockedsigs > > > > > > But it was not documented anywhere, so add some basic > > > documentation for > > > it in both the .rst documentation files and the help message of > > > bitbake > > > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > > --- > > > .../bitbake-user-manual-execution.rst | 6 ++++++ > > > .../bitbake-user-manual-intro.rst | 13 > > > +++++++++---- > > > bitbake/lib/bb/main.py | 12 > > > +++++++----- > > > 3 files changed, 22 insertions(+), 9 deletions(-) > > > > Unfortunately I don't think this is quite so simple. The lockedsigs > > support is implemented in OE-Core and BitBake doesn't contain that > > code. This makes it very hard to accurately document since it is > > metadata dependent. > > > > I do agree we need to improve this somehow, I just don't know how. > > Hi Richard, > > Maybe we should remove this from bitbake and implement it using > bblock? :) > Let me know if I can help on this! That is a direction I've been wondering about and I think it would make sense. It is one of the reasons I wanted to see the lock/unlock commands become separate tools! Cheers, Richard
Le mer. 18 sept. 2024 à 09:30, Richard Purdie <richard.purdie@linuxfoundation.org> a écrit : > > On Wed, 2024-09-18 at 09:05 +0200, Julien Stephan wrote: > > Le mar. 17 sept. 2024 à 22:55, Richard Purdie > > <richard.purdie@linuxfoundation.org> a écrit : > > > > > > On Tue, 2024-09-17 at 17:11 +0200, Julien Stephan via > > > lists.openembedded.org wrote: > > > > The following commit added "lockedsigs" as new signature handler > > > > for the > > > > -S option of bitbake: > > > > 2cf13f25139 - lib/oe/sstatesig.py: dump locked.sigs.inc only > > > > when explicitly asked via -S lockedsigs > > > > > > > > But it was not documented anywhere, so add some basic > > > > documentation for > > > > it in both the .rst documentation files and the help message of > > > > bitbake > > > > > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > > > --- > > > > .../bitbake-user-manual-execution.rst | 6 ++++++ > > > > .../bitbake-user-manual-intro.rst | 13 > > > > +++++++++---- > > > > bitbake/lib/bb/main.py | 12 > > > > +++++++----- > > > > 3 files changed, 22 insertions(+), 9 deletions(-) > > > > > > Unfortunately I don't think this is quite so simple. The lockedsigs > > > support is implemented in OE-Core and BitBake doesn't contain that > > > code. This makes it very hard to accurately document since it is > > > metadata dependent. > > > > > > I do agree we need to improve this somehow, I just don't know how. > > > > Hi Richard, > > > > Maybe we should remove this from bitbake and implement it using > > bblock? :) > > Let me know if I can help on this! > > That is a direction I've been wondering about and I think it would make > sense. It is one of the reasons I wanted to see the lock/unlock > commands become separate tools! > ok then let me try to come up with something :) By the way, my next step was to send a patch adding documentation for SIGGEN_LOCKEDSIGS* variables, but I realize that the same problem applies to these variables, right? I don't see any reference for these variables inside bitbake code.. Should I add them in yocto-docs ? > Cheers, > > Richard > >
On Wed, 2024-09-18 at 09:39 +0200, Julien Stephan wrote: > Le mer. 18 sept. 2024 à 09:30, Richard Purdie > <richard.purdie@linuxfoundation.org> a écrit : > > > > On Wed, 2024-09-18 at 09:05 +0200, Julien Stephan wrote: > > > Le mar. 17 sept. 2024 à 22:55, Richard Purdie > > > <richard.purdie@linuxfoundation.org> a écrit : > > > > > > > > On Tue, 2024-09-17 at 17:11 +0200, Julien Stephan via > > > > lists.openembedded.org wrote: > > > > > The following commit added "lockedsigs" as new signature > > > > > handler > > > > > for the > > > > > -S option of bitbake: > > > > > 2cf13f25139 - lib/oe/sstatesig.py: dump locked.sigs.inc > > > > > only > > > > > when explicitly asked via -S lockedsigs > > > > > > > > > > But it was not documented anywhere, so add some basic > > > > > documentation for > > > > > it in both the .rst documentation files and the help message > > > > > of > > > > > bitbake > > > > > > > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > > > > --- > > > > > .../bitbake-user-manual-execution.rst | 6 > > > > > ++++++ > > > > > .../bitbake-user-manual-intro.rst | 13 > > > > > +++++++++---- > > > > > bitbake/lib/bb/main.py | 12 > > > > > +++++++----- > > > > > 3 files changed, 22 insertions(+), 9 deletions(-) > > > > > > > > Unfortunately I don't think this is quite so simple. The > > > > lockedsigs > > > > support is implemented in OE-Core and BitBake doesn't contain > > > > that > > > > code. This makes it very hard to accurately document since it > > > > is > > > > metadata dependent. > > > > > > > > I do agree we need to improve this somehow, I just don't know > > > > how. > > > > > > Hi Richard, > > > > > > Maybe we should remove this from bitbake and implement it using > > > bblock? :) > > > Let me know if I can help on this! > > > > That is a direction I've been wondering about and I think it would > > make > > sense. It is one of the reasons I wanted to see the lock/unlock > > commands become separate tools! > > > > ok then let me try to come up with something :) Sounds good! > By the way, my next step was to send a patch adding documentation for > SIGGEN_LOCKEDSIGS* > variables, but I realize that the same problem applies to these > variables, right? > I don't see any reference for these variables inside bitbake code.. > Should I add them in yocto-docs ? Yes, they wouldn't be documented in bitbake but they could be documented in the main variable index. Cheers, Richard
Le mar. 17 sept. 2024 à 17:12, Julien Stephan via lists.openembedded.org <jstephan=baylibre.com@lists.openembedded.org> a écrit : > > Currently, available -S parameters are "none", "printdiff" and "lockedsigs", > but there is no check on parameter, so user can literally give anything > to -S and it will act as if "none" was specified. > > Fix this by adding the allowed values in choices for -S parameter > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > --- > bitbake/lib/bb/main.py | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py > index 1f4313fa0dd..24758646eaa 100755 > --- a/bitbake/lib/bb/main.py > +++ b/bitbake/lib/bb/main.py > @@ -213,6 +213,7 @@ def create_bitbake_parser(): > > exec_group.add_argument("-S", "--dump-signatures", action="append", > default=[], metavar="SIGNATURE_HANDLER", > + choices=["none", "printdiff", "lockedsigs"], Richard, Should I send a v2, dropping lockedsigs? Or should this patch be dropped also? Cheers Julien > help="Dump out the signature construction information, with no task " > "execution. The SIGNATURE_HANDLER parameter is passed to the " > "handler. Allowed values are none, printdiff and lockedsigs. " > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#16566): https://lists.openembedded.org/g/bitbake-devel/message/16566 > Mute This Topic: https://lists.openembedded.org/mt/108502729/4356723 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [jstephan@baylibre.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, 2024-09-18 at 09:54 +0200, Julien Stephan via lists.openembedded.org wrote: > Le mar. 17 sept. 2024 à 17:12, Julien Stephan via > lists.openembedded.org <jstephan=baylibre.com@lists.openembedded.org> > a écrit : > > > > Currently, available -S parameters are "none", "printdiff" and "lockedsigs", > > but there is no check on parameter, so user can literally give anything > > to -S and it will act as if "none" was specified. > > > > Fix this by adding the allowed values in choices for -S parameter > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > --- > > bitbake/lib/bb/main.py | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py > > index 1f4313fa0dd..24758646eaa 100755 > > --- a/bitbake/lib/bb/main.py > > +++ b/bitbake/lib/bb/main.py > > @@ -213,6 +213,7 @@ def create_bitbake_parser(): > > > > exec_group.add_argument("-S", "--dump-signatures", action="append", > > default=[], metavar="SIGNATURE_HANDLER", > > + choices=["none", "printdiff", "lockedsigs"], > > Richard, > > Should I send a v2, dropping lockedsigs? Or should this patch be dropped also? The comment applied to the series, I can't merge it as the handlers are metadata specific. I think we might need to ultimately move this functionality elsewhere and drop -S entirely. Cheers, Richard
Le mer. 18 sept. 2024 à 10:08, Richard Purdie <richard.purdie@linuxfoundation.org> a écrit : > > On Wed, 2024-09-18 at 09:54 +0200, Julien Stephan via lists.openembedded.org wrote: > > Le mar. 17 sept. 2024 à 17:12, Julien Stephan via > > lists.openembedded.org <jstephan=baylibre.com@lists.openembedded.org> > > a écrit : > > > > > > Currently, available -S parameters are "none", "printdiff" and "lockedsigs", > > > but there is no check on parameter, so user can literally give anything > > > to -S and it will act as if "none" was specified. > > > > > > Fix this by adding the allowed values in choices for -S parameter > > > > > > Signed-off-by: Julien Stephan <jstephan@baylibre.com> > > > --- > > > bitbake/lib/bb/main.py | 1 + > > > 1 file changed, 1 insertion(+) > > > > > > diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py > > > index 1f4313fa0dd..24758646eaa 100755 > > > --- a/bitbake/lib/bb/main.py > > > +++ b/bitbake/lib/bb/main.py > > > @@ -213,6 +213,7 @@ def create_bitbake_parser(): > > > > > > exec_group.add_argument("-S", "--dump-signatures", action="append", > > > default=[], metavar="SIGNATURE_HANDLER", > > > + choices=["none", "printdiff", "lockedsigs"], > > > > Richard, > > > > Should I send a v2, dropping lockedsigs? Or should this patch be dropped also? > > The comment applied to the series, I can't merge it as the handlers are > metadata specific. I think we might need to ultimately move this > functionality elsewhere and drop -S entirely. > Noted! Thank you > Cheers, > > Richard
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst index d58fbb32ea0..ac1e2a76811 100644 --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst @@ -592,6 +592,12 @@ compare the matched signatures to determine the stamps and delta where these two stamp trees diverge. This can be used to determine why tasks need to be re-run in situations where that is not expected. +The third signature handler that can be passed is "lockedsigs" that will +conveniently write the task signatures in a file called "locked-sigs.inc", +that can be included in user's configuration to lock recipes or tasks. +User can include the whole file or extract the signatures needed to lock +only selected tasks or recipes. + .. note:: It is likely that future versions of BitBake will provide other diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst index 35ffb88b026..127c662320b 100644 --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst @@ -398,10 +398,15 @@ Following is the usage and syntax for BitBake:: -S SIGNATURE_HANDLER, --dump-signatures=SIGNATURE_HANDLER Dump out the signature construction information, with no task execution. The SIGNATURE_HANDLER parameter is - passed to the handler. Two common values are none and - printdiff but the handler may define more/less. none - means only dump the signature, printdiff means compare - the dumped signature with the cached one. + passed to the handler. Allowed values are none, + printdiff and lockedsigs. none means only dump the + signature, printdiff means recursively compare the + dumped signature with the most recent one in a local + build or sstate cache (can be used to find out why + tasks re-run when that is not expected). lockedsigs + means generate a locked-sigs.inc file containing the + locked signature of all tasks needed to build the + target. -p, --parse-only Quit after parsing the BB recipes. -s, --show-versions Show current and preferred versions of all recipes. -e, --environment Show the global or per-recipe environment complete diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py index bca8ebfa090..1f4313fa0dd 100755 --- a/bitbake/lib/bb/main.py +++ b/bitbake/lib/bb/main.py @@ -215,11 +215,13 @@ def create_bitbake_parser(): default=[], metavar="SIGNATURE_HANDLER", help="Dump out the signature construction information, with no task " "execution. The SIGNATURE_HANDLER parameter is passed to the " - "handler. Two common values are none and printdiff but the handler " - "may define more/less. none means only dump the signature, printdiff" - " means recursively compare the dumped signature with the most recent" - " one in a local build or sstate cache (can be used to find out why tasks re-run" - " when that is not expected)") + "handler. Allowed values are none, printdiff and lockedsigs. " + "none means only dump the signature, printdiff means recursively " + "compare the dumped signature with the most recent one in a local " + "build or sstate cache (can be used to find out why tasks re-run " + "when that is not expected). lockedsigs means generate a locked-sigs.inc " + "file containing the locked signature of all tasks needed to build " + "the target.") exec_group.add_argument("--revisions-changed", action="store_true", help="Set the exit code depending on whether upstream floating "
The following commit added "lockedsigs" as new signature handler for the -S option of bitbake: 2cf13f25139 - lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs But it was not documented anywhere, so add some basic documentation for it in both the .rst documentation files and the help message of bitbake Signed-off-by: Julien Stephan <jstephan@baylibre.com> --- .../bitbake-user-manual-execution.rst | 6 ++++++ .../bitbake-user-manual-intro.rst | 13 +++++++++---- bitbake/lib/bb/main.py | 12 +++++++----- 3 files changed, 22 insertions(+), 9 deletions(-)