diff mbox series

[v5,2/5] prserv: move code from __init__ to bitbake-prserv

Message ID 20240429201418.657042-3-michael.opdenacker@bootlin.com
State New
Headers show
Series prserv: add support for an "upstream" server | expand

Commit Message

Michael Opdenacker April 29, 2024, 8:14 p.m. UTC 
From: Michael Opdenacker <michael.opdenacker@bootlin.com>

This script was the only user of this code.

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 bin/bitbake-prserv     |  9 ++++++++-
 lib/prserv/__init__.py | 13 -------------
 2 files changed, 8 insertions(+), 14 deletions(-)

Comments

Khem Raj May 2, 2024, 4:57 p.m. UTC | #1 
Hi Michael

I am seeing a problem with this series, specifically following
patchset is causing a regression

* ea1d7b8a36e bitbake: prserv: add bitbake selftests
* 5c66d345300 bitbake: prserv: import simplification
* c23f2c1680f bitbake: prserv: store_value() improvements
* 395b321af29 bitbake: prserv: avoid possible race condition in database code
* 8022dc1029d bitbake: prserv: enable database sharing
* b81bfdfbed8 bitbake: prserv: add "upstream" server support
* 9539ce3b14a bitbake: prserv: move code from __init__ to bitbake-prserv
* 4ec1553710b bitbake: prserv: declare "max_package_pr" client hook


error seen during packagedata task of base-files

WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to server:
Connection closed
ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object has no
attribute 'sync'
Traceback (most recent call last):
  File "/mnt/b/yoe/master/sources/poky/bitbake/lib/bb/asyncrpc/serv.py",
line 278, in PRServer._client_handler(socket=<bb.asyncrpc.connection.StreamConnection
object at 0x794b1e009110>):
                 client = self.accept_client(socket)
    >            await client.process_requests()
             except Exception as e:
  File "/mnt/b/yoe/master/sources/poky/bitbake/lib/prserv/serv.py",
line 171, in PRServerClient.process_requests():
             try:
    >            await super().process_requests()
             finally:
  File "/mnt/b/yoe/master/sources/poky/bitbake/lib/bb/asyncrpc/serv.py",
line 92, in PRServerClient.process_requests():
                     try:
    >                    response = await self.dispatch_message(d)
                     except InvokeError as e:
  File "/mnt/b/yoe/master/sources/poky/bitbake/lib/prserv/serv.py",
line 45, in PRServerClient.dispatch_message(msg={'get-pr': {'version':
'base-files-3.0.14-r0', 'pkgarch': 'imx8qm_var_som', 'checksum':
'0d287eea3da9a2573dfd3fbc709ef8697f2553563ff1211efc733f215d510663',
'history': False}}):
             except:
    >            self.server.table.sync()
                 raise
AttributeError: 'PRTable' object has no attribute 'sync'


To reproduce this you will need to add a bbappend for base-files from

https://github.com/varigit/meta-variscite-bsp-imx/blob/hardknott/recipes-core/base-files/base-files_%25.bbappend

You can make the overrides match the machine you might be building if
not one of variscite machine.


Thanks
-Khem

On Mon, Apr 29, 2024 at 1:14 PM Michael Opdenacker via
lists.openembedded.org
<michael.opdenacker=bootlin.com@lists.openembedded.org> wrote:
>
> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>
> This script was the only user of this code.
>
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> Cc: Joshua Watt <JPEWhacker@gmail.com>
> Cc: Tim Orling <ticotimo@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  bin/bitbake-prserv     |  9 ++++++++-
>  lib/prserv/__init__.py | 13 -------------
>  2 files changed, 8 insertions(+), 14 deletions(-)
>
> diff --git a/bin/bitbake-prserv b/bin/bitbake-prserv
> index ad0a069401..920663a1d8 100755
> --- a/bin/bitbake-prserv
> +++ b/bin/bitbake-prserv
> @@ -21,6 +21,13 @@ VERSION = "1.1.0"
>  PRHOST_DEFAULT="0.0.0.0"
>  PRPORT_DEFAULT=8585
>
> +def init_logger(logfile, loglevel):
> +    numeric_level = getattr(logging, loglevel.upper(), None)
> +    if not isinstance(numeric_level, int):
> +        raise ValueError("Invalid log level: %s" % loglevel)
> +    FORMAT = "%(asctime)-15s %(message)s"
> +    logging.basicConfig(level=numeric_level, filename=logfile, format=FORMAT)
> +
>  def main():
>      parser = argparse.ArgumentParser(
>          description="BitBake PR Server. Version=%s" % VERSION,
> @@ -72,7 +79,7 @@ def main():
>      )
>
>      args = parser.parse_args()
> -    prserv.init_logger(os.path.abspath(args.log), args.loglevel)
> +    init_logger(os.path.abspath(args.log), args.loglevel)
>
>      if args.start:
>          ret=prserv.serv.start_daemon(args.file, args.host, args.port, os.path.abspath(args.log), args.read_only)
> diff --git a/lib/prserv/__init__.py b/lib/prserv/__init__.py
> index 0e0aa34d0e..94658b815d 100644
> --- a/lib/prserv/__init__.py
> +++ b/lib/prserv/__init__.py
> @@ -5,16 +5,3 @@
>  #
>
>  __version__ = "1.0.0"
> -
> -import os, time
> -import sys, logging
> -
> -def init_logger(logfile, loglevel):
> -    numeric_level = getattr(logging, loglevel.upper(), None)
> -    if not isinstance(numeric_level, int):
> -        raise ValueError("Invalid log level: %s" % loglevel)
> -    FORMAT = "%(asctime)-15s %(message)s"
> -    logging.basicConfig(level=numeric_level, filename=logfile, format=FORMAT)
> -
> -class NotFoundError(Exception):
> -    pass
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#16151): https://lists.openembedded.org/g/bitbake-devel/message/16151
> Mute This Topic: https://lists.openembedded.org/mt/105808694/1997914
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Michael Opdenacker May 3, 2024, 8:04 a.m. UTC | #2 
Hi Khem

On 5/2/24 at 18:57, Khem Raj wrote:
> Hi Michael
>
> I am seeing a problem with this series, specifically following
> patchset is causing a regression
>
> * ea1d7b8a36e bitbake: prserv: add bitbake selftests
> * 5c66d345300 bitbake: prserv: import simplification
> * c23f2c1680f bitbake: prserv: store_value() improvements
> * 395b321af29 bitbake: prserv: avoid possible race condition in database code
> * 8022dc1029d bitbake: prserv: enable database sharing
> * b81bfdfbed8 bitbake: prserv: add "upstream" server support
> * 9539ce3b14a bitbake: prserv: move code from __init__ to bitbake-prserv
> * 4ec1553710b bitbake: prserv: declare "max_package_pr" client hook
>
>
> error seen during packagedata task of base-files
>
> WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to server:
> Connection closed
> ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object has no
> attribute 'sync'
> Traceback (most recent call last):


Thanks for the review. Actually, did you test with the V6 update I sent?
Cheers
Michael.
Khem Raj May 3, 2024, 2:14 p.m. UTC | #3 
On Fri, May 3, 2024 at 1:04 AM Michael Opdenacker <
michael.opdenacker@bootlin.com> wrote:

> Hi Khem
>
> On 5/2/24 at 18:57, Khem Raj wrote:
> > Hi Michael
> >
> > I am seeing a problem with this series, specifically following
> > patchset is causing a regression
> >
> > * ea1d7b8a36e bitbake: prserv: add bitbake selftests
> > * 5c66d345300 bitbake: prserv: import simplification
> > * c23f2c1680f bitbake: prserv: store_value() improvements
> > * 395b321af29 bitbake: prserv: avoid possible race condition in database
> code
> > * 8022dc1029d bitbake: prserv: enable database sharing
> > * b81bfdfbed8 bitbake: prserv: add "upstream" server support
> > * 9539ce3b14a bitbake: prserv: move code from __init__ to bitbake-prserv
> > * 4ec1553710b bitbake: prserv: declare "max_package_pr" client hook
> >
> >
> > error seen during packagedata task of base-files
> >
> > WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to server:
> > Connection closed
> > ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object has no
> > attribute 'sync'
> > Traceback (most recent call last):
>
>
> Thanks for the review. Actually, did you test with the V6 update I sent?


I think it was version in master-next that Richard has staged

>
> Cheers
> Michael.
>
> --
> Michael Opdenacker, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>
Richard Purdie May 8, 2024, 1:35 p.m. UTC | #4 
On Fri, 2024-05-03 at 07:14 -0700, Khem Raj via lists.openembedded.org
wrote:
> 
> 
> On Fri, May 3, 2024 at 1:04 AM Michael Opdenacker
> <michael.opdenacker@bootlin.com> wrote:
> > Hi Khem
> > 
> > On 5/2/24 at 18:57, Khem Raj wrote:
> > > Hi Michael
> > > 
> > > I am seeing a problem with this series, specifically following
> > > patchset is causing a regression
> > > 
> > > * ea1d7b8a36e bitbake: prserv: add bitbake selftests
> > > * 5c66d345300 bitbake: prserv: import simplification
> > > * c23f2c1680f bitbake: prserv: store_value() improvements
> > > * 395b321af29 bitbake: prserv: avoid possible race condition in
> > > database code
> > > * 8022dc1029d bitbake: prserv: enable database sharing
> > > * b81bfdfbed8 bitbake: prserv: add "upstream" server support
> > > * 9539ce3b14a bitbake: prserv: move code from __init__ to
> > > bitbake-prserv
> > > * 4ec1553710b bitbake: prserv: declare "max_package_pr" client
> > > hook
> > > 
> > > 
> > > error seen during packagedata task of base-files
> > > 
> > > WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to
> > > server:
> > > Connection closed
> > > ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object
> > > has no
> > > attribute 'sync'
> > > Traceback (most recent call last):
> > 
> > 
> > Thanks for the review. Actually, did you test with the V6 update I
> > sent?
> > 
> 
> 
> I think it was version in master-next that Richard has staged
> 

It was against v6 so I'm not able to merge these until that is fixed.

Cheers,

Richard
Khem Raj May 8, 2024, 3:13 p.m. UTC | #5 
On Wed, May 8, 2024 at 6:35 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2024-05-03 at 07:14 -0700, Khem Raj via lists.openembedded.org
> wrote:
> >
> >
> > On Fri, May 3, 2024 at 1:04 AM Michael Opdenacker
> > <michael.opdenacker@bootlin.com> wrote:
> > > Hi Khem
> > >
> > > On 5/2/24 at 18:57, Khem Raj wrote:
> > > > Hi Michael
> > > >
> > > > I am seeing a problem with this series, specifically following
> > > > patchset is causing a regression
> > > >
> > > > * ea1d7b8a36e bitbake: prserv: add bitbake selftests
> > > > * 5c66d345300 bitbake: prserv: import simplification
> > > > * c23f2c1680f bitbake: prserv: store_value() improvements
> > > > * 395b321af29 bitbake: prserv: avoid possible race condition in
> > > > database code
> > > > * 8022dc1029d bitbake: prserv: enable database sharing
> > > > * b81bfdfbed8 bitbake: prserv: add "upstream" server support
> > > > * 9539ce3b14a bitbake: prserv: move code from __init__ to
> > > > bitbake-prserv
> > > > * 4ec1553710b bitbake: prserv: declare "max_package_pr" client
> > > > hook
> > > >
> > > >
> > > > error seen during packagedata task of base-files
> > > >
> > > > WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to
> > > > server:
> > > > Connection closed
> > > > ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object
> > > > has no
> > > > attribute 'sync'
> > > > Traceback (most recent call last):
> > >
> > >
> > > Thanks for the review. Actually, did you test with the V6 update I
> > > sent?
> > >
> >
> >
> > I think it was version in master-next that Richard has staged
> >
>
> It was against v6 so I'm not able to merge these until that is fixed.
>

Right, I will try a  clean from scratch build and see if it was something
intermediate to be sure.

> Cheers,
>
> Richard
Michael Opdenacker May 8, 2024, 5:40 p.m. UTC | #6 
Richard, Khem,

On 5/8/24 at 20:43, Khem Raj wrote:
> On Wed, May 8, 2024 at 6:35 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Fri, 2024-05-03 at 07:14 -0700, Khem Raj via lists.openembedded.org
>> wrote:
>>>
>>> On Fri, May 3, 2024 at 1:04 AM Michael Opdenacker
>>> <michael.opdenacker@bootlin.com> wrote:
>>>> Hi Khem
>>>>
>>>> On 5/2/24 at 18:57, Khem Raj wrote:
>>>>> Hi Michael
>>>>>
>>>>> I am seeing a problem with this series, specifically following
>>>>> patchset is causing a regression
>>>>>
>>>>> * ea1d7b8a36e bitbake: prserv: add bitbake selftests
>>>>> * 5c66d345300 bitbake: prserv: import simplification
>>>>> * c23f2c1680f bitbake: prserv: store_value() improvements
>>>>> * 395b321af29 bitbake: prserv: avoid possible race condition in
>>>>> database code
>>>>> * 8022dc1029d bitbake: prserv: enable database sharing
>>>>> * b81bfdfbed8 bitbake: prserv: add "upstream" server support
>>>>> * 9539ce3b14a bitbake: prserv: move code from __init__ to
>>>>> bitbake-prserv
>>>>> * 4ec1553710b bitbake: prserv: declare "max_package_pr" client
>>>>> hook
>>>>>
>>>>>
>>>>> error seen during packagedata task of base-files
>>>>>
>>>>> WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to
>>>>> server:
>>>>> Connection closed
>>>>> ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object
>>>>> has no
>>>>> attribute 'sync'
>>>>> Traceback (most recent call last):
>>>>
>>>> Thanks for the review. Actually, did you test with the V6 update I
>>>> sent?
>>>>
>>>
>>> I think it was version in master-next that Richard has staged
>>>
>> It was against v6 so I'm not able to merge these until that is fixed.
>>
> Right, I will try a  clean from scratch build and see if it was something
> intermediate to be sure.


Thanks for testing.
I confirm the issue is still there. My apologies for this overlook!
This is a busy week for me, but I will submit a fix as soon as I get the 
time, and before the end of the week anyway.

Kind regards
Michael.
Khem Raj May 14, 2024, 5:03 p.m. UTC | #7 
On Wed, May 8, 2024 at 10:41 AM Michael Opdenacker
<michael.opdenacker@bootlin.com> wrote:
>
> Richard, Khem,
>
> On 5/8/24 at 20:43, Khem Raj wrote:
> > On Wed, May 8, 2024 at 6:35 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> >> On Fri, 2024-05-03 at 07:14 -0700, Khem Raj via lists.openembedded.org
> >> wrote:
> >>>
> >>> On Fri, May 3, 2024 at 1:04 AM Michael Opdenacker
> >>> <michael.opdenacker@bootlin.com> wrote:
> >>>> Hi Khem
> >>>>
> >>>> On 5/2/24 at 18:57, Khem Raj wrote:
> >>>>> Hi Michael
> >>>>>
> >>>>> I am seeing a problem with this series, specifically following
> >>>>> patchset is causing a regression
> >>>>>
> >>>>> * ea1d7b8a36e bitbake: prserv: add bitbake selftests
> >>>>> * 5c66d345300 bitbake: prserv: import simplification
> >>>>> * c23f2c1680f bitbake: prserv: store_value() improvements
> >>>>> * 395b321af29 bitbake: prserv: avoid possible race condition in
> >>>>> database code
> >>>>> * 8022dc1029d bitbake: prserv: enable database sharing
> >>>>> * b81bfdfbed8 bitbake: prserv: add "upstream" server support
> >>>>> * 9539ce3b14a bitbake: prserv: move code from __init__ to
> >>>>> bitbake-prserv
> >>>>> * 4ec1553710b bitbake: prserv: declare "max_package_pr" client
> >>>>> hook
> >>>>>
> >>>>>
> >>>>> error seen during packagedata task of base-files
> >>>>>
> >>>>> WARNING: base-files-3.0.14-r0 do_packagedata: Error talking to
> >>>>> server:
> >>>>> Connection closed
> >>>>> ERROR: Error from client ('127.0.0.1', 59846): 'PRTable' object
> >>>>> has no
> >>>>> attribute 'sync'
> >>>>> Traceback (most recent call last):
> >>>>
> >>>> Thanks for the review. Actually, did you test with the V6 update I
> >>>> sent?
> >>>>
> >>>
> >>> I think it was version in master-next that Richard has staged
> >>>
> >> It was against v6 so I'm not able to merge these until that is fixed.
> >>
> > Right, I will try a  clean from scratch build and see if it was something
> > intermediate to be sure.
>
>
> Thanks for testing.
> I confirm the issue is still there. My apologies for this overlook!
> This is a busy week for me, but I will submit a fix as soon as I get the
> time, and before the end of the week anyway.

FWIW the new patchset seems to work well for issue I was seeing

>
> Kind regards
> Michael.
>
> --
> Michael Opdenacker, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
diff mbox series

Patch

diff --git a/bin/bitbake-prserv b/bin/bitbake-prserv
index ad0a069401..920663a1d8 100755
--- a/bin/bitbake-prserv
+++ b/bin/bitbake-prserv
@@ -21,6 +21,13 @@  VERSION = "1.1.0"
 PRHOST_DEFAULT="0.0.0.0"
 PRPORT_DEFAULT=8585
 
+def init_logger(logfile, loglevel):
+    numeric_level = getattr(logging, loglevel.upper(), None)
+    if not isinstance(numeric_level, int):
+        raise ValueError("Invalid log level: %s" % loglevel)
+    FORMAT = "%(asctime)-15s %(message)s"
+    logging.basicConfig(level=numeric_level, filename=logfile, format=FORMAT)
+
 def main():
     parser = argparse.ArgumentParser(
         description="BitBake PR Server. Version=%s" % VERSION,
@@ -72,7 +79,7 @@  def main():
     )
 
     args = parser.parse_args()
-    prserv.init_logger(os.path.abspath(args.log), args.loglevel)
+    init_logger(os.path.abspath(args.log), args.loglevel)
 
     if args.start:
         ret=prserv.serv.start_daemon(args.file, args.host, args.port, os.path.abspath(args.log), args.read_only)
diff --git a/lib/prserv/__init__.py b/lib/prserv/__init__.py
index 0e0aa34d0e..94658b815d 100644
--- a/lib/prserv/__init__.py
+++ b/lib/prserv/__init__.py
@@ -5,16 +5,3 @@ 
 #
 
 __version__ = "1.0.0"
-
-import os, time
-import sys, logging
-
-def init_logger(logfile, loglevel):
-    numeric_level = getattr(logging, loglevel.upper(), None)
-    if not isinstance(numeric_level, int):
-        raise ValueError("Invalid log level: %s" % loglevel)
-    FORMAT = "%(asctime)-15s %(message)s"
-    logging.basicConfig(level=numeric_level, filename=logfile, format=FORMAT)
-
-class NotFoundError(Exception):
-    pass