mbox series

[0/2] oeqa/utils/postactions: fix disk storage consumption

Message ID 20240705144638.441976-1-alexis.lothore@bootlin.com
Headers show
Series oeqa/utils/postactions: fix disk storage consumption | expand

Message

Alexis Lothoré July 5, 2024, 2:46 p.m. UTC 
Hello,
this small series aims to fix Bugzilla issue 15536 [1], which results in a
big disk storage consumption and/or inode max count issue. The main root
cause is that in its current form, the artifacts retriever convert symlinks
to the targeted files. This series changes the way those files are
retrieved to make sure the symlinks are preserved. To do so, it replaces
the series of scp performed on each file by a single tar command which
output the generated compressed archive on its stdout. It is captured and
returned as part of an ssh process, and it it then fed to another tar
command run this time on host, to extract the artifacts, at the same place
as before (tmp/log/oeqa/artifacts)

The first commit is a small update to SSH target in ssh.py to allow to
retrieve raw output (ie raw bytes, without any decoding or formatting) from
a command run on the target. This allows to get the compressed archive
directly as an output from the remote command.
The second commit uses this new feature to generate, retrieve and extract
this archive on the fly, without ever storing it onto the target nor the
host.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15536

Alexis Lothoré (2):
  oeqa/ssh: allow to retrieve raw, unformatted ouput
  oeqa/utils/postactions: transfer whole archive over ssh instead of
    doing individual copies

 meta/lib/oeqa/core/target/ssh.py   | 16 ++++++++--------
 meta/lib/oeqa/utils/postactions.py | 19 +++++++++----------
 2 files changed, 17 insertions(+), 18 deletions(-)