mbox series

[bitbake-devel,v2,00/10] hashserv: Pipeline Upstream Queries

Message ID 20260730183254.793698-1-JPEWhacker@gmail.com
Headers show
Series hashserv: Pipeline Upstream Queries | expand

Message

Joshua Watt July 30, 2026, 6:30 p.m. UTC 
Adds a true asynchronous streaming API to the hash equivalence clients,
then uses that streaming API to pipeline requests to an upstream server.
This means that when a server wants to defer a request to an upstream,
it no longer has to wait for a complete round-trip API call. Instead, it
can use the faster streaming API to send the requests, and continue
processing other requests while a response inbound.

V2: Fixes several bugs related to when the connection between the client
and upstream server was disconnected, and added tests for them.


Joshua Watt (10):
  asyncrpc: Add Task Group
  asyncrpc: serv: Use Task Group
  asyncrpc: serv: Cancel all clients on server stop
  hashserv: tests: Improve test logging
  hashserv: client: Add asynchronous streaming API
  hashserv: server: Add queued streaming API
  hashserv: server: Use streaming and queue API for upstream unihash
    queries
  hashserv: server: Use streaming and queue API for upstream exist
    queries
  hashserv: tests: Add more upstream tests
  hashserv: tests: Add test for upstream pipelining

 bin/bitbake-hashclient       |   2 +-
 lib/bb/asyncrpc/__init__.py  |   1 +
 lib/bb/asyncrpc/serv.py      |  13 +-
 lib/bb/asyncrpc/taskgroup.py |  52 ++++++
 lib/hashserv/client.py       | 349 +++++++++++++++++++++++++++--------
 lib/hashserv/server.py       | 181 ++++++++++++++----
 lib/hashserv/tests.py        | 261 ++++++++++++++++++++++++--
 7 files changed, 723 insertions(+), 136 deletions(-)
 create mode 100644 lib/bb/asyncrpc/taskgroup.py