diff mbox series

[yocto-autobuilder-helper] clobberdir: only move dirs more than 60s old

Message ID 20230607194102.3785739-1-tgamblin@baylibre.com
State New
Headers show
Series [yocto-autobuilder-helper] clobberdir: only move dirs more than 60s old | expand

Commit Message

Trevor Gamblin June 7, 2023, 7:41 p.m. UTC
See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14952

Help avoid contention by adding a check to ensure targets are at
least 60s old when attempting to move them into a trash directory.

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
---
 janitor/clobberdir | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Comments

Richard Purdie June 13, 2023, 8:43 a.m. UTC | #1
On Wed, 2023-06-07 at 15:41 -0400, Trevor Gamblin wrote:
> See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14952
> 
> Help avoid contention by adding a check to ensure targets are at
> least 60s old when attempting to move them into a trash directory.
> 
> Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
> ---
>  janitor/clobberdir | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/janitor/clobberdir b/janitor/clobberdir
> index 16b019e..832173c 100755
> --- a/janitor/clobberdir
> +++ b/janitor/clobberdir
> @@ -52,8 +52,12 @@ trashdir = utils.getconfig("TRASH_DIR", ourconfig)
>  for x in [clobberdir]:
>      if os.path.exists(x) and os.path.exists(trashdir):
>          if (os.stat(trashdir).st_dev == os.stat(x).st_dev):
> -            trashdest = trashdir + "/" + str(int(time.time())) + '-'  + str(random.randrange(100, 100000, 2))
> -            mkdir(trashdest)
> -            subprocess.check_call(['mv', x, trashdest])
> +            x_age = time.time() - os.path.getmtime(x)
> +            if x_age > 60:
> +                trashdest = trashdir + "/" + str(int(time.time())) + '-'  + str(random.randrange(100, 100000, 2))
> +                mkdir(trashdest)
> +                subprocess.check_call(['mv', x, trashdest])
> +            else:
> +                print("Not moving '%s' - age is only %s seconds. There may be another process using it" % (x, str(int(x_age))))
>          else:
>              subprocess.check_call(['rm', "-rf", x])

I think we need to do this on the other side of the setup.

If we ask to run clobberdir, it needs to succeed and I don't think the
move causes us issues. What does cause problems is if a bitbake server
was still active on a build directory and shutting down. If we move the
directory from under it, that is ok, what causes problems is if we
actually delete it.

The age check therefore needs to be in trash_processor() in janitor/ab-
janitor.

Cheers,

Richard
diff mbox series

Patch

diff --git a/janitor/clobberdir b/janitor/clobberdir
index 16b019e..832173c 100755
--- a/janitor/clobberdir
+++ b/janitor/clobberdir
@@ -52,8 +52,12 @@  trashdir = utils.getconfig("TRASH_DIR", ourconfig)
 for x in [clobberdir]:
     if os.path.exists(x) and os.path.exists(trashdir):
         if (os.stat(trashdir).st_dev == os.stat(x).st_dev):
-            trashdest = trashdir + "/" + str(int(time.time())) + '-'  + str(random.randrange(100, 100000, 2))
-            mkdir(trashdest)
-            subprocess.check_call(['mv', x, trashdest])
+            x_age = time.time() - os.path.getmtime(x)
+            if x_age > 60:
+                trashdest = trashdir + "/" + str(int(time.time())) + '-'  + str(random.randrange(100, 100000, 2))
+                mkdir(trashdest)
+                subprocess.check_call(['mv', x, trashdest])
+            else:
+                print("Not moving '%s' - age is only %s seconds. There may be another process using it" % (x, str(int(x_age))))
         else:
             subprocess.check_call(['rm', "-rf", x])